No escape from the dollar sign in SED

[[e-mail] [sed-users]]

I was reading the excellent SED FAQ bud could not find an alternate solution to the following:

The script below replaces the literal string $path with the contents of variable $usb

The only way I could get this to work under busybox ash is with completely bare \\$
to escape the literal $path in two separate chunks as \\$'path' which is REALLY weird.

***@zero:~# usb='/mnt/stick'; echo 'Path $path is it!' | sed -r 's|'\\$'path'"|$usb|g"
Path /mnt/stick is it!

It's great that it works because it allows me to pre-process my here-block before I
write it out to a file, but jeepers it would be nice to understand what's happening
and whether there are any other ways to do this under busybox with ash.

Here is the slightly simplified here-doc version which I actually use:

usb='/mnt/stick'
cat <<-'EOF' | sed -r 's|'\\$"path|$usb|g"
echo and here it is:
ftpdir=$path
echo more stuff
EOF

Best regards
Stepan Novotny





[Non-text portions of this message have been removed]

[Cameron Simpson [e-mail] [sed-users]]

Post by ***@yahoo.com [sed-users]
I was reading the excellent SED FAQ bud could not find an alternate solution
The script below replaces the literal string $path with the contents of variable $usb
The only way I could get this to work under busybox ash is with completely bare \\$
to escape the literal $path in two separate chunks as \\$'path' which is REALLY weird.
Path /mnt/stick is it!
It's great that it works because it allows me to pre-process my here-block before I
write it out to a file, but jeepers it would be nice to understand what's happening
and whether there are any other ways to do this under busybox with ash.
usb='/mnt/stick'
cat <<-'EOF' | sed -r 's|'\\$"path|$usb|g"
ftpdir=$path
echo more stuff
EOF

Ok, first up I suspect busybox and ash are not special; they should cope with
standard shell syntax. So my advice here is generic.

First a nit: what you've got there is a "redundant cat". You can just write
this:

sed -r 's|'\\$"path|$usb|g" <<'-EOF'
echo and here it is:
ftpdir=$path
echo more stuff
EOF

The other thing is that I would avoid shift quotes when possible. I'd be
writing:

sed -r "s|\\\$path|$usb|g" <<'-EOF'

First plan what sed needs to receive: what is sees _after_ the shell has done
quote processing:

s|\$path|/mnt/stick|g

So you objective is to get that in from the shell. You want $usb inserted, so
naturally you want a double quoted string. Within double quote you need a
literal backslash and a literal dollar for the \$path part, so both need
escaping:

s|\\\$path|$usb|g

Then put quotes around it and you're ready.

As a final remark, I often use ^G (control-G) as a sed delimiter, just because
it isn't part of normal text.

Cheers,
Cameron Simpson <***@cskk.id.au>

[Daniel Goldman [e-mail] [sed-users]]

Do you have to use $path as the literal string to replace? If yes, why?
Could you use something simpler, like PATH_TO_REPLACE or similar?

If you have to use $path as literal string, just \ escape dollar sign in
the sed script seen by sed, as already mentioned:

$ sed --version
sed (GNU sed) 4.2.2

$ cat /tmp/sed-1.sh
usb="/mnt/stick"
echo 'ftpdir=$path' | sed "s|\$path|$usb|"

$ /tmp/sed-1.sh
ftpdir=/mnt/stick

Daniel

Post by ***@yahoo.com [sed-users]
The script below replaces the literal string $path with the contents of variable $usb
The only way I could get this to work under busybox ash is with completely bare \\$
to escape the literal $path in two separate chunks as \\$'path' which is REALLY weird.
Path /mnt/stick is it!
It's great that it works because it allows me to pre-process my here-block before I
write it out to a file, but jeepers it would be nice to understand what's happening
and whether there are any other ways to do this under busybox with ash.
usb='/mnt/stick'
cat <<-'EOF' | sed -r 's|'\\$"path|$usb|g"
ftpdir=$path
echo more stuff
EOF
Best regards
Stepan Novotny
[Non-text portions of this message have been removed]
------------------------------------
------------------------------------

[Sven Guckes [e-mail] [sed-users]]

Post by ***@yahoo.com [sed-users]
I was reading the excellent SED FAQ bud could not find
The script below replaces the literal string
$path with the contents of variable $usb
The only way I could get this to work under busybox ash
is with completely bare \\$ to escape the literal $path
in two separate chunks as \\$'path' which is REALLY weird.
echo 'Path $path is it!' | sed -r 's|'\\$'path'"|$usb|g"
Path /mnt/stick is it!

i used colons as a separator..

***@kudu:~> busybox ash
BusyBox v1.22.1 (Debian 1:1.22.0-9+deb8u1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
~ $ usb='/mnt/stick'
~ $ echo 'Path $path is it!' | sed -r s:$path:$usb:g
sed: no previous regexp
~ $ echo 'Path $path is it!' | sed -r s:\$path:$usb:g
Path $path is it!
~ $ echo 'Path $path is it!' | sed -r s:\\$path:$usb:g
sed: unmatched ':'
~ $ echo 'Path $path is it!' | sed -r s:'\$'path:$usb:g
Path /mnt/stick is it!
~ $ echo 'Path $path is it!' | sed -r s:\\\$path:$usb:g
Path /mnt/stick is it!

i expected "\$path" to suffice.. but it didnt.
likewise with "\\$path". but "'\$'" worked.
and then also "\\\$" did, too. o_O

i suppose it must be something with the minimalism of the
tools within busybox.. and maybe an older version of sed.
does anybody know about the "life of sed" within busybox?

i just downloaded busybox-1.28.3 (as of yesterday 2018-04-03):
wget http://busybox.net/downloads/busybox-1.28.3.tar.bz2
make menuconfig # i did change some options, but.. who cares?
make
make CONFIG_PREFIX=/home/user/guckes install
cd ~/bin; mv busybox busybox-1.28.3; ln -s !$ busybox

but.. no change. same results.

just for the record.. the "make menuconfig"
contains amenu for "Editors" with "sed (12 kb)".
that's only *twelve* kilobytes for sed! :-O
well, minised-1.15 had some 25.5KB. "big!" ;-)

but maybe you folks will find the solution.
here are the commands for testing again:

usb='/mnt/stick'
echo 'Path $path is it!' | sed -r s:$path:$usb:g
echo 'Path $path is it!' | sed -r s:\$path:$usb:g
echo 'Path $path is it!' | sed -r s:\\$path:$usb:g
echo 'Path $path is it!' | sed -r s:\\\$path:$usb:g
echo 'Path $path is it!' | sed -r s:'\$'path:$usb:g

enjoy!

Sven

[Tim Chase [e-mail] [sed-users]]

's|'\\$'path'"|$usb|g" Path /mnt/stick is it!

It seems like you really would be better off using `envsubst`
(available on all my linux & BSD boxes with anything that installed
the GNU gettext package):

$ path='/mnt/stick'
$ echo "Path $path is it!" | envsubst
Path /mnt/stick is it!

It's a lot cleaner and won't get hung up on things like paths that
contain pipes or other odd characters.

I'm not positive it's available in a busybox environment as I don't
have one readily at hand.

-tim

[Daniel Goldman [e-mail] [sed-users]]

Or simply:

$ cat /tmp/script-1.sh
path=/mnt/stick
echo ftpdir=$path

$ /tmp/script-1.sh
ftpdir=/mnt/stick

Daniel

Post by Tim Chase ***@thechases.com [sed-users]

's|'\\$'path'"|$usb|g" Path /mnt/stick is it!

It seems like you really would be better off using `envsubst`
(available on all my linux & BSD boxes with anything that installed
$ path='/mnt/stick'
$ echo "Path $path is it!" | envsubst
Path /mnt/stick is it!
It's a lot cleaner and won't get hung up on things like paths that
contain pipes or other odd characters.
I'm not positive it's available in a busybox environment as I don't
have one readily at hand.
-tim
------------------------------------
------------------------------------

------------------------------------

------------------------------------
--
------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/sed-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/sed-users/join
(Yahoo! ID required)

<*> To change settings via email:
sed-users-***@yahoogroups.com
sed-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
sed-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

[Jim Hill [e-mail] [sed-users]]

You seem to be using just about every conceivable variation on
quoting, and I don't see the point. I get confused trying to read your
code too. Simplify.

~ $ usb=/mnt/stick
~ $ sed "s,\$path,$usb,g" <<\EOD

Post by ***@yahoo.com [sed-users]
ftpdir=$path
echo more stuff
EOD

echo and here it is:
ftpdir=/mnt/stick
more stuff
~ $

[Rakesh Sharma [e-mail] [sed-users]]

Hi,


You could do the following also:


cat <<-'EOF' | sed -e "s|[$]path|$usb|g"
echo and here it is:
ftpdir=$path
echo more stuff
EOF

But you have to also ensure that "$usb" shell variable contents DONOT have anything which might be considered special by "sed" on it's right-hand-side of it's
"s///" command, because otherwise this will cause sed to error out.

For that you could do the following:

usb_rhs=$(printf '%s\n' "$usb" | sed -e 's:[|\&]:\\&:g;$!s/$/\\/');

Now use this variable ($usb_rhs) in your "sed" command. It properly escapes the characters that are metacharacters to sed on the rhs of s/// command.

________________________________
From: sed-***@yahoogroups.com <sed-***@yahoogroups.com> on behalf of ***@yahoo.com [sed-users] <sed-***@yahoogroups.com>
Sent: Tuesday, April 3, 2018 9:07 PM
To: sed-***@yahoogroups.com
Subject: No escape from the dollar sign in SED



I was reading the excellent SED FAQ bud could not find an alternate solution to the following:

The script below replaces the literal string $path with the contents of variable $usb

The only way I could get this to work under busybox ash is with completely bare \\$
to escape the literal $path in two separate chunks as \\$'path' which is REALLY weird.

***@zero:~# usb='/mnt/stick'; echo 'Path $path is it!' | sed -r 's|'\\$'path'"|$usb|g"
Path /mnt/stick is it!

It's great that it works because it allows me to pre-process my here-block before I
write it out to a file, but jeepers it would be nice to understand what's happening
and whether there are any other ways to do this under busybox with ash.

Here is the slightly simplified here-doc version which I actually use:

usb='/mnt/stick'
cat <<-'EOF' | sed -r 's|'\\$"path|$usb|g"
echo and here it is:
ftpdir=$path
echo more stuff
EOF

Best regards
Stepan Novotny



[Non-text portions of this message have been removed]





[Non-text portions of this message have been removed]



------------------------------------

------------------------------------
--
------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/sed-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/sed-users/join
(Yahoo! ID required)

<*> To change settings via email:
sed-users-***@yahoogroups.com
sed-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
sed-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

[Daniel Goldman [e-mail] [sed-users]]

Post by Rakesh Sharma ***@hotmail.com [sed-users]
cat <<-'EOF' | sed -e "s|[$]path|$usb|g"
ftpdir=$path
echo more stuff
EOF
usb_rhs=$(printf '%s\n' "$usb" | sed -e 's:[|\&]:\\&:g;$!s/$/\\/');

I'm sorry, but that last one-line code snippet gives me a headache!!!
It's hard to understand and fragile. I pity the maintainer. And the
first script is way more complicated than needed, and as you astutely
pointed out, subject to failure depending on the value of the usb
variable. There are *much* simpler ways to do what the OP seemed to
want, as pointed out in other replies.

The basic problem is that the OP was writing code in a very bizarre,
obscure way. :( They didn't know any better. But they need to. We are
(or should be) here to teach and learn. So here goes...

1) Do NOT use something like $path as a literal string! Only use $path
when accessing the 'path' variable. :( This kind of bizarre coding
practice is 1) totally not needed, 2) extremely confusing to others.

2) Do NOT write something "clever" that takes many minutes to decipher,
such as the usb_rhs line above. There is usually a much simpler way.
Reading one line of "tricky" code is much harder and much more
error-prone than reading many lines of well-written code. Good code
tells an easy-to-follow story. Bad code presents a puzzle.

Again, here is a very simple, fool-proof (AFAICT) way to do exactly what
the OP wanted (AFAICT) without the weird machinations:

$ cat /tmp/temp.sh
usb=/home/mac
cat << EOF
here it is:
ftpdir=$usb
more stuff
EOF

$ /tmp/temp.sh
here it is:
ftpdir=/home/mac
more stuff

Given that we have not heard back from the OP, I cannot guarantee this
is the desired behavior. So here is a final "rule":

3) Focus on test cases. Clearly define what you are trying to do. What
is the exact input (if any)? Present the input file. Does it take into
account all possibilities? What is the exact desired output / result? If
you get the test cases right, you (or someone on this group) will know
the code is right, because it will produce the correct output. If you
mess up the test cases, things will likely fail in production mode.

If the OP is still around, I would recommend UNIX Shell Programming, by
Kochan and Wood, if looking for a book to read.

Daniel


------------------------------------

------------------------------------
--
------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/sed-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/sed-users/join
(Yahoo! ID required)

<*> To change settings via email:
sed-users-***@yahoogroups.com
sed-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
sed-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

[[e-mail] [sed-users]]

OP here. Thanks everyone for the amazing suggestions, you blew the doors wide open like dynamite! Here's a summary collection of all which worked for me, extracted from the suggestions above:

usb='/mnt/stick'

sed -r 's|'\\$"path|$usb|g" > out <<'EOF'
or
sed -r 's|'\\$"path|$usb|g" > out <<-'EOF'
or
sed -r "s|\\\$path|$usb|g" > out <<-'EOF'
or
sed -r s:'\$'path:$usb:g > out <<-'EOF'
or
sed -r s:\\\$path:$usb:g > out <<-'EOF'
or
sed -e "s|[$]path|$usb|g" > out <<-'EOF'
or
sed "s|\$path|$usb|" > out <<-'EOF'
or
sed "s,\$path,$usb,g" > out <<\EOF

echo and here it is:
ftpdir=$path
echo more stuff
EOF

Special notes:
1) use of ^G as the SED delimiter, per Cameron
2) add escapes to special chars within a variable per Daneil:
usb_rhs=$(printf '%s\n' "$usb" | sed -e 's:[|\&]:\\&:g;$!s/$/\\/');

Stepan

[Non-text portions of this message have been removed]