0

How can I run this command in my Jenkins file?

sh "perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' .env"

I tried everything.

Like so:

sh """
perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' .env
"""

Or escaping the backslahes.

But I keep getting the error:

WorkflowScript: 13: unexpected char: '\' @ line 13, column 23.
FooBar
  • 5,752
  • 10
  • 44
  • 93
  • Is there a reason to run Perl inside a shell instead of just running Perl directly? The added string interpolations is probably tripping you up. Also, this will not edit the file, just print to stdout. – TLP Sep 23 '20 at 14:01
  • 1
    Your assumption is that `$ENV...` is not interpolated by the shell, which I believe it is. – TLP Sep 23 '20 at 14:11
  • @TLP Well - it doesnt work that way. I tested that as well. I tred the more simple way:`sh "cat .env.staging > .env"` (inside .env.staging I got: `DEPLOY_KEY=${DEPLOY_KEY}`) however, `.env` will end up looking like so: `DEPLOY_KEY=`. I have verified $DEPLOY_KEY is present. Running above perl script works fine. But I cannot get it to run in my Jenkinsfile – FooBar Sep 23 '20 at 14:26
  • Well, since I do not know how you run this command, I cannot say you are wrong. I ran a similar command here https://pastebin.run/McRGdDLDz5 and there it was interpolated (removed) before it reached Perl. – TLP Sep 23 '20 at 14:46

1 Answers1

2

Depending on how this command is run, the string interpolation issues can be awful to predict. Is the double quoted string interpolated by sh? Does the backslash in front of $ mean that it is escaped from sh, but not from Perl interpolation? When I ran a test string in pastebin, it simply removed the $ENV{$1}.

I'm sure there's a way to do it the hard way (this way), but an easy way is to just write the Perl code in a file instead, and run the file.

I would write your regexes like this, in a separate file, say foo.pl:

s|\${([^}]+)}|$ENV{$1} // $&|eg; 
s/\${([^}]+)}//g;

Using the logical defined-or operator // is slightly prettier than using the ternary operator. We change delimiter on the substitution operator to facilitate that.

I removed unused e modifier on second substitution.

You should note that all strings that match the regex ${....} will be removed from the input by the second substitution. So the fact that you attempt to put them back with the first substitution with $& is quite meaningless. Moreover using $& carries a notable performance reduction. Assuming that is a mistake from your side, the code can be shortened to:

s/\${([^}]+)}/$ENV{$1}/g;

Note that now you can also skip the dangerous eval modifier /e.

If you run it without warnings, which you do in your original code, you will not notice the undefined values in the %ENV hash, it will just return the empty string -- i.e. remove undefined values.

This code can now be run by your other script without interpolation issues:

sh "perl -p foo.pl .env"

Just remove the -e switch since you are no longer providing command line code.

TLP
  • 66,756
  • 10
  • 92
  • 149
  • So basically move `s/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg` inside foo.pl and then just execute `sh "perl -p foo.pl .env"`? – FooBar Sep 23 '20 at 15:18
  • @FooBar Well, yes, but there were other issues with your code, as noted in my answer. – TLP Sep 23 '20 at 15:23
  • well. the code I am trying to run with `sh` works perfectly on the server and on my local machine, but not in the jenkins pipeline; that's what I am interesting in knowing why not... – FooBar Sep 23 '20 at 15:45
  • @FooBar Well, if this is not helpful, just ignore it. I cannot help you with jenkins. The only jenkins I know is Leeroy Jenkins. ;) – TLP Sep 23 '20 at 15:57
  • hehe. I see. Anyhow, thanks for your advise to extract it to a seperate file. That seems to work. Going to test on jenkins too. However, `s/\${([^}]+)}/$ENV{$1}/g;` would unfortunately fail with: `Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/\${ <-- HERE ([^}]+)}/ at test.pl line 1.` – FooBar Sep 23 '20 at 16:49
  • @FooBar That seems to be something with new versions of Perl. https://stackoverflow.com/q/43165604/725418 Seems strange as it has not been a meta character before, not on its own anyway. But just put the backslash back, I guess. – TLP Sep 23 '20 at 19:38