Problems with sed in script

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Post Reply
Message
Author
2lss
Posts: 225
Joined: Sun 20 Sep 2009, 23:54

Problems with sed in script

#1 Post by 2lss »

I'm working on a little script to synchronize music between my mp3 player, external hdd and my music library on my computer. So far I've got:

Code: Select all

MUSICDIR=/root/Music

DEVDIR=/mnt/sdb1/music

diff $MUSICDIR $DEVDIR | grep "Only in $MUSICDIR" | sed -e s/"Only in $MUSICDIR: "// 
the diff piped into the grep command gives something like:

Code: Select all

Only in root/Music: Alice in Chains
Only in root/Music: Foo Fighters
etc.
so I want to strip the "Only in root/Music: " part using sed but I'm running into problems. If I try what I have above I get:

Code: Select all

sed: -e expression #1, char 17: unknown option to `s'
The man page for sed says:

Code: Select all

s/regexp/replacement/
              Attempt  to  match  regexp  against the pattern space.  If successful, replace that portion matched with replacement.  The replacement may
              contain the special character & to refer to that portion of the pattern space which matched, and the special  escapes  \1  through  \9  to
              refer to the corresponding matching sub-expressions in the regexp.
From what I can tell, sed is getting screwed up b/c the string MUSICDIR contains slashes but I'm not sure how to correct it. I tried just about every combination of sed I can find along with different styles of quoting but I still get the same error. Anyone have any ideas?

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#2 Post by big_bass »

Hey 2lss

so I want to strip the "Only in root/Music: " part using sed

Code: Select all

echo "Only in root/Music: Alice in Chains" | sed -e 's/Only in root\/Music://g' 

so you could test it with echo first

Code: Select all

| sed -e 's/Only in root\/Music://g' 
Joe

potong
Posts: 88
Joined: Fri 06 Mar 2009, 04:01

#3 Post by potong »

2lss:

The problem your having is not so much with sed but with quoting in the shell.

As a rule of thumb it's always a good idea to single quote any commands to sed on the command line.

In this case you want to interpolate a variable within the sed command; so just single quote around the bash variable like so:

Code: Select all

# a=/tmp/a b=/tmp/b; rm -rf $a $b; mkdir $a $b
# for x in {a..f};do touch $a/$x;done                #afterthought: same as 
# for x in {d..h};do touch $b/$x;done               # touch $a/{a..f} $b/{d..h}
# diff $a $b|sed -n 's|Only in '$a': ||p'
a
b
c
# 
The sed command could do with a little explanation

Code: Select all

sed -n 's|Only in '$a': ||p'
  • sed -n means turn off automatic printed output to stdout. In effect this means nothing gets printed unless you choose to print it out.

    's|Only in ' first half of sed substitute command using "|" as a delimiter.

    $a is the bash variable containing the directory name (has "/"'s in it hence the "|" delimiters)

    ': ||p' second half of substitute command which in effect wipes "Only in /tmp/a: " from input line. The p prints out to stdout whats in the pattern space if the substitute command matched.
Generally speaking you never need grep followed by sed as sed can do it all (less overhead and quicker)

Here's a link to a good sed tutorial

HTH

Potong

2lss
Posts: 225
Joined: Sun 20 Sep 2009, 23:54

#4 Post by 2lss »

big_bass:Thanks for your suggestion. I tried and it works but since I'm planning on using multiple instances of sed and using the script on different machines with different music directories I would rather change the MUSICDIR variable at the beginning than change every instance of sed in the script.

potong: I modified the script to:

Code: Select all

MUSICDIR=/root/Music

DEVDIR=/mnt/sdb1/music

diff $MUSICDIR $DEVDIR | sed -n 's|Only in '$MUSICDIR': ||p'
Everything seems to be working good, thanks

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#5 Post by jemimah »

Why not use rsync?

Puppy has the GAdmin Rsync frontend included; it's pretty nice.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#6 Post by big_bass »

2lss
big_bass:Thanks for your suggestion. I tried and it works but since I'm planning on using multiple instances of sed and using the script on different machines with different music directories I would rather change the MUSICDIR variable at the beginning than change every instance of sed in the script.
Hey 2lss
Hey potong
Hey jemimah


good to hear that the final code that potong pasted was useful
so that you could modify it to you needs

I didn't read in to what you were doing just answered the question with sed


I know that potong likes using gtkdialog so here's a code example using Xdialog that could be converted hint ,hint (gtkdialog apps are more flexible )

its a drag N drop so now it doesnt matter what folders you select

I guess that you want some sort of list generated ??





in this code snippet I dont use sed I used cut

after looking at the out put with diff

Code: Select all

| cut  -f 2 -d :
seemed easier

how to use :
1.)make it executable
2.)click on it once to launch it
3.)drag N drop your database folder in FIRST the input box
4.)drag N drop your possible new files folder in the SECOND input box



updated 5-11-2010 I added a viewer I didnt like the editor opening at such a large size the editor works well with patches but not with this

Code: Select all

#!/bin/sh


# just a simple diff tool for the lazy
# there are better but this is easy :D 
# Joe Arose 
# call this dnd_database_diff
# 
# **modifed from my dnd_diff for making patches 
# added a viewer 
# the data base folder is your grownig database 
# the  new folder directory may have new files not in the data base 
# if any new files are found they are listed in a txt file for you 

DIALOG=Xdialog

$DIALOG --title "The data base directory    " \
        --inputbox "Type in a value or Drag N drop.
         the data base directory folder 
         \n
" 0 0  2> /tmp/one.txt

retval=$?



input=`cat /tmp/one.txt`


case $retval in
  0)
    echo "Input string is '$input'";;
  1)
    echo "Cancel pressed."
    exit;;
  255)
    echo "Box closed."
    exit;;
esac

#----------------------------------
$DIALOG --title "The possible new files   " \
        --inputbox "Type in a value or Drag N drop.
        the new folder directory this may have some new files 
        the name will be given from this folder .txt
        and placed in root\n
" 0 0 2>/tmp/two.txt

retval=$?

input2=`cat /tmp/two.txt`

case $retval in
  0)
    echo "Input string is '$input2'";;
  1)
    echo "Cancel pressed."
    exit;;
  255)
    echo "Box closed."
    exit;;
esac

diff  $input $input2 | grep "$input2" | cut  -f 2 -d : >/root/`basename $input2`.txt

#or you could do this if you have another editor installed 
#$DEFAULTTEXTEDITOR  /root/`basename $input2`.txt
DIFF_LIST=/root/`basename $input2`.txt 
 
echo "`cat $DIFF_LIST`" | Xdialog --title " differences" \--backtitle "differences             "     \--textbox "-" 0 0


 
rm -f /tmp/two.txt
rm -f /tmp/one.txt







Joe

2lss
Posts: 225
Joined: Sun 20 Sep 2009, 23:54

#7 Post by 2lss »

jemimah: I have been looking at rsync and may include it somewhere else in the script. I don't know the ins and outs of it yet, hopefully I can find some tutorials on how to use it. I also didn't know about GAdmin Rsync, thanks for the tip.

big_bass: I like your script, I will incorporate it into my music sync script. The only bad part is the drag and drop only works with rox and not pcman.

I had to modify it slightly to get it to work. The three lines with Xdialog I had to change to this format without the \ :

Code: Select all

$DIALOG --title "The data base directory    " --inputbox "Type in a value or Drag N drop the data base directory folder\n" 0 0 2> /tmp/one.txt


The reason I am writing this script is b/c I have a very obsessive compulsive way I organize my music. I've tried many different music players and syncing tools but none work the way I want. With 2 computers, an ipod running rockbox, and an external backup hdd, organizing my music is taking way to long. What I have posted so far is just the tip of the iceberg. It's probably not the most efficient method but it will get me started.

Post Reply