Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 22 Jul 2014, 15:53
All times are UTC - 4
 Forum index » House Training » Beginners Help ( Start Here)
Basic Shell (Console) operation for beginners
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 16 of 24 [350 Posts]   Goto page: Previous 1, 2, 3, ..., 14, 15, 16, 17, 18, ..., 22, 23, 24 Next
Author Message
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 00:47    Post subject:  

. duplicate post deleted
Last edited by Shep on Fri 29 Apr 2011, 01:41; edited 1 time in total
Back to top
View user's profile Send private message 
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 00:53    Post subject:  

Bruce B wrote:

(2) converts tabs to four spaces
(3) then removes trailing spaces

Code:
mv $1 $1-BAK
< $1-BAK sed 's/\t*/    /g' | sed 's/ *$//' > $1


On line two, will you please show how to eliminate the re-piping to sed?


Sure, happy to.

You have it almost right. (I've changed it back to quote your first posting, as it was closer to being correct.) Laughing

The pattern \t* matches "zero or more occurrences" of the TAB. But if you think about it, you really need "one or more occurrences" of the TAB to be replaced by spaces. So change it to \t\t* which reads as "one TAB followed by zero or more TABs". The difference is hugely important, as you doubtless discovered, for one fundamental reason---in their wisdom the designers of sed declared that alongside every character in a stream is a match for "zero occurrences" of anything. Shocked

Similarly, to match your block of trailing spaces, specify "one space followed by zero or more extra spaces" by coding it as two spaces followed by an asterisk. While your original code for this step will provide the same result, it will subject the CPU to greater toil because sed finds a match alongside [u]every character[u] in the input stream and laboriously substitutes that nothing with ... a different nothing!

So the solution is:

sed -e 's/\t\t*/ /g' -e 's/ *$//'

To make sed code easier to write and understand, most implementations allow extensions to the basic commands. In addition to the * in pattern matching, we have \+ which matches "one or more occurrences of" the character that precedes it.

This means your code could also be expressed as:

sed -e 's/\t\+/ /g' -e 's/ \+$//'


HTH
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 01:24    Post subject:  

Shep,

I made a script quick and dirty and it works perfect, thanks.

Code:
<$1 cat -s | sed -e 's/\t\+/ /g' -e 's/ \+$//'
echo -n Ctrl+C to quit ; read a
<$1 cat -s | sed -e 's/\t\+/ /g' -e 's/ \+$//' > $2


Bruce

The cat -s removes extra empty lines > 2

I shoulda cat at the end and will.

The net result is a nicely cleaned file

~

PS the explanations were most helpful.

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 01:52    Post subject:  

Shep,

One reason I'm weak on sed is because I was spoiled earlier by two
excellent line modification utilities for DOS and Windows

FU & LMOD - between the two they are better and easier than sed,
maybe combined they have more functionality.

When I came to Linux, rather than learn sed, I wrote various pipes in C.
The pipes not very flexible, yet work perfect. Due to lack of flexibility, I
have various pipes.

Then little by little learning sed. I understand Linux is a Unix clone, but it
wouldn't hurt to add a few novelties.

The author of FU I can't find. I'd like to get the source code.

The author of LMOD is still around and active.

Horst Schaeffer's Software Pages

By the powerful nature of LMOD and its very small size I'd think it is
written in assembly.

On the other hand Schaeffer has a Pure Basic section. Maybe that's how it
is coded.

I wonder if you know how hard or easy it would be to port DOS Pure Basic
or Assembly Code to Linux?

Most everything with LMOD is standard I/O, it seems.

Bruce

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 02:18    Post subject:  

Programming Style

This script example is intended to be an actual practical script for making
pupsave backups.

The primary reason I may not have a current backup is because of
convenience factors. If I make backups very convenient or even
automatic, I will have current backups.

I often don't follow Linux conventions in scripting. Part this is C
conventions which I learned to like. In C, variables are conventionally
lower case and structure is function based. The way I write a program is
by making small functions. (It is easier for me and easier to test and
debug). Also, I like to use sufficient white space to make things
more readable.

The script below has been tested, I'll attach a zip file soon. I kept
everything to a character length of less than 80. To do this I used a \
which tells Bash to continue on to the next line as part of the command.

It is extremely important there be no white space following the \
otherwise, the script won't even work. Also, one reason for the scripts
posted above by me and wonderfully polished by Shep, is to remove
trailing white space.

Note the function names tend to serve as a comment giving a hint about
what the function is for.

Here is my function based script.

Code:
#!/bin/bash

main() {

    variables
    sanity
    already_bked_up
    clear_pupsave
    make_bkup
    exit 0

}

variables() {

    datestamp=`date "+_%Y_%m_%d"`

    mode=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPMODE \
    | cut -d = -f 2`

    pupdir=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
    | sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 2`

    pupfile=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
    | sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 3`

    script=`basename $0`

    xchck=`pidof  X`

    devsave="/initrd/mnt/dev_save"

    for i in `echo $pupfile | tr "." " "` ; do
        ext=$i
    done

    pupbasefn=`basename $pupfile .$ext`

}

sanity() {

    [ ! -d /initrd/pup_rw ] && echo "This is not a Puppy Frugal install, \
    goodbye" && exit

    [ "${#xchck}" -gt "1" ] && echo "X is running, goodbye" && exit

    if [ "$mode" != "12" ] ; then
        echo ; echo "$script has only been tested with Puppy's PUPMODE 12,"
        echo "due to lack of testing on other PUPMODES,this opeartion"
        echo "is terminated"
        echo ; echo "Recommendation,do your own testing and refining of"
        echo "$script for other PUPMODES" ; echo
        exit
    fi

}

already_bked_up() {

    if [ -f $devsave/$pupdir/${pupbasefn}$datestamp.zip ] ; then

        a=N

        echo "You already have a backup for today"
        echo -n "Do you want another backup (y,n)? "
        read -n 1 a

        if [ "$a" = "y" ] ; then

            for i in {a..z} ; do

                if [ ! -f $devsave/$pupdir/${pupbasefn}${datestamp}_$i.zip ]
                then
                    datestamp="${datestamp}_$i"
                    break
                fi

            done

        else

            echo ;  exit 0
       
        fi

    fi

    echo

}

clear_pupsave() {

    echo "Clearing free space in $pupfile, please wait..."
    dd if=/dev/zero of=/initrd/pup_rw/zeros.tmp bs=1M
    rm /initrd/pup_rw/zeros.tmp

}

make_bkup() {

    echo "Making $devsave/$pupdir/$pupbasefn$datestamp.zip"
    zip $devsave/$pupdir/$pupbasefn$datestamp.zip $devsave/$pupdir/$pupfile
    ls -l $devsave/$pupdir/$pupbasefn$datestamp.zip
}

main


Here is the identical script without functions, with uppercase variables and
no extra white space.

Code:
#!/bin/bash
DATESTAMP=`date "+_%Y_%m_%d"`
MODE=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPMODE \
| cut -d = -f 2`
PUPDIR=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
| sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 2`
PUPFILE=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
| sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 3`
SCRIPT=`basename $0`
XCHCK=`pidof  X`
DEVSAVE="/initrd/mnt/dev_save"
for I in `echo $PUPFILE | tr "." " "` ; do
    EXT=$I
done
PUPBASEFN=`basename $PUPFILE .$EXT`
[ ! -d /initrd/pup_rw ] && echo "This is not a Puppy Frugal install, \
goodbye" && exit
[ "${#XCHCK}" -gt "1" ] && echo "X is running, goodbye" && exit
if [ "$MODE" != "12" ] ; then
    echo ; echo "$SCRIPT has only been tested with Puppy's PUPMODE 12,"
    echo "due to lack of testing on other PUPMODES,this opeartion"
    echo "is terminated"
    echo ; echo "Recommendation,do your own testing and refining of"
    echo "$SCRIPT for other PUPMODES" ; echo
    exit
fi
if [ -f $DEVSAVE/$PUPDIR/${PUPBASEFN}$DATESTAMP.zip ] ; then
A=N
echo "You already have a backup for today"
echo -n "Do you want another backup (y,n)? "
read -n 1 A
if [ "$A" = "y" ] ; then
    for I in {a..z} ; do
        if [ ! -f $DEVSAVE/$PUPDIR/${PUPBASEFN}${DATESTAMP}_$I.zip ] ; then
            DATESTAMP="${DATESTAMP}_$I"
            break
        fi
    done
else
    echo ; exit 0
fi
echo
echo "Clearing free space in $PUPFILE, please wait..."
dd if=/dev/zero of=/initrd/pup_rw/zeros.tmp bs=1M
rm /initrd/pup_rw/zeros.tmp
echo "Making $DEVSAVE/$PUPDIR/$PUPBASEFN$DATESTAMP.zip"
zip $DEVSAVE/$PUPDIR/$PUPBASEFN$DATESTAMP.zip $DEVSAVE/$PUPDIR/$PUPFILE
ls -l $DEVSAVE/$PUPDIR/$PUPBASEFN$DATESTAMP.zip
exit 0


These scripts are much easier to work with using a text editor with syntax
highlighting.

Bruce

Any comments or questions?

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 03:24    Post subject:  

Indenting

Indenting when we use what is most commonly called statements.

Below are two examples from the post above.

Example Indented

Code:
if [ -f $devsave/$pupdir/${pupbasefn}$datestamp.zip ] ; then

    a=N

    echo "You already have a backup for today"
    echo -n "Do you want another backup (y,n)? "
    read -n 1 a

    if [ "$a" = "y" ] ; then

        for i in {a..z} ; do

            if [ ! -f $devsave/$pupdir/${pupbasefn}${datestamp}_$i.zip ] ; then
             
                datestamp="${datestamp}_$i"
                break

            fi

         done

    else

        echo ;  exit 0
       
    fi

fi


The commands are indented inside each controlling statement.

When statements are nested, each statement is indented.

In order to follow what is going on, we look at the start of the statement.
Then look straight down the column for the end instruction of the
particular statement. It could be 'done' on a 'for loop', or 'esac' for a 'case'
statement or 'fi' for an 'if' statement.

~~~

Example Not Indented (harder to follow)

Code:
if [ -f $devsave/$pupdir/${pupbasefn}$datestamp.zip ] ; then

a=N

echo "You already have a backup for today"
echo -n "Do you want another backup (y,n)? "
read -n 1 a

if [ "$a" = "y" ] ; then

for i in {a..z} ; do

if [ ! -f $devsave/$pupdir/${pupbasefn}${datestamp}_$i.zip ] ; then
             
datestamp="${datestamp}_$i"
break

fi

done

else

echo ;  exit 0
       
fi

fi


~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 08:07    Post subject:  

Bruce B wrote:

Code:
<$1 cat -s | sed -e 's/\t\+/ /g' -e 's/ \+$//'

The cat -s removes extra empty lines > 2

I shoulda cat at the end and will.

The sed script here does not create or delete any line/s, so with your cat -s filter at the input, there will be no need for an identical cat -s filter at the output. It would have nothing to do.

Quote:
Code:
echo -n Ctrl+C to quit ; read a

Small point, but you can combine the prompt and the read:

read -p 'Ctrl+C to quit, or <RET> to continue:' -n 1 a

Though neater to test the response and not rely on interrupt:

read -p 'Enter q to quit, or <RET> to continue:' -n 1 a
Back to top
View user's profile Send private message 
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 08:18    Post subject:  

Bruce B wrote:
FU & LMOD - between the two they are better and easier than sed,
maybe combined they have more functionality.

Sorry, I can't help with either. I doubt that if they were easier to learn, they could be more powerful than sed. The power of sed comes, in part, from its extended regular expression handling. Yes, it does take practise to master some of the more esoteric commands in sed, but you can still achieve much just with its common commands.

But even if you were to find a non-standard utility, the readers with a standard install would not be able to benefit from all the work you put into designing scripts with it!
Back to top
View user's profile Send private message 
SimpleWater

Joined: 19 Apr 2011
Posts: 95

PostPosted: Fri 29 Apr 2011, 08:41    Post subject:  

Quote:
"Customizing the Shell Prompt"


that sounds really intresting Cool
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 09:24    Post subject:  

Shep wrote:
The sed script here does not create or delete any line/s, so with
your cat -s filter at the input, there will be no need for an identical cat -s filter
at the output. It would have nothing to do.


True, but my after thought was, if there was any trailing white space in a
line, maybe cat wouldn't consider it an empty line. In order to qualify as an
empty line the only character would be \n (0x0a), the actual behavior would
depend on how the cat authors think. And as usual, man pages don't say
everything.

As for leading white space, the script doesn't deal with that, because I want
to preserve it.

I could test, that's for sure.

And many thanks for all the helpful information, I would never have intuited it.

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 09:28    Post subject:  

Bruce B wrote:

mode=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPMODE \
| cut -d = -f 2`

pupdir=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
| sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 2`

pupfile=`cat /initrd/pup_rw/etc/rc.d/PUPSTATE | grep PUPSAVE \
| sed "s/'//g" | tr "," " " | cut -d " " -f 3 | cut -d "/" -f 3`


Bruce, you have put a lot of thought into this script. As far as I'm concerned, if the script works and is understandable enough to be easily modified, then it's fine. Smile

There are one or two things I can point out. The code cat file | grep string can almost always be written as grep string file thereby saving one process and a pipe.

On big linux installs, tr is a small fast binary and tr -d deletes characters from a stream more effortlessly than does sed.

In the above code snippet, it would be better to save an intermediate processed value from the first `...` and just put it through one more cut to extract the value for the second and third parameters. You three times cat the same file and pass it through almost identical cut filters when there is almost certainly no need to do it more than once.

However, the block of code above is a perfect case where builtin shell commands can do the same job with a fraction of the effort. In particular, for string manipulation consider Shell Parameter Expansion techniques. ] EDIT: oops! Embarassed Embarassed While I provided that reference, and strongly recommend it, I actually didn't use that method for this task. In the end it was easy to do solely using bash's extended set command.

a=`grep PUPSAVE /initrd/pup_rw/etc/rc.d/PUPSTATE`
echo $a
IFS="[ ,/]"
set -- $a
mode=$2
pupdir=$3
pupfile=$5
echo $mode $pupdir $pupfile


IFS is the field separator (can be a single char or a group of chars) used to split the string. One thing to watch if you employ the set command is to make sure you have beforehand saved the positional parameters $1 $2 etc if they passed values (e.g., filenames) into the script when it was called. Any initial values are overwritten when you use set to reassign shell positional parameters $1 $2 etc. within your program as I illustrate above. As a technique to be applied to other puppies, this will only work if you can be sure the contents of file PUPSTATE will adhere to identical syntax for all puplets. Can you be sure of this? Otherwise, a more robust algorithm will be needed.

EDIT: mulling over bash extensions, I believe a one-liner is all that is needed for the job:

IFS="[ ,/']"; read x mode pupdir pupfile <<< `grep PUPSAVE /etc/rc.d/PUPSTATE`
echo $mode $pupdir $pupfile

To variable x I've assigned the field we aren't interested in when the line containing PUPSAVE is read in. The three arrows is a way to feed an input string to a command which expects it. IFS breaks the line into fields according to the 3 chars in the square brackets.

This discussion may be of interest to others who are following along with your explorations here.

HTH

Last edited by Shep on Sat 30 Apr 2011, 07:49; edited 4 times in total
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 09:31    Post subject:  

My standard prompt is what I call the OS2 prompt. Because that's how OS2
did it.

PS1='[\w] ' gives [~/bin]

If the path gets too long I have an alias to change it to PS1='[\W] '

It shows not the full path, just the current directory, like this:

[bin]

But don't stop there, because the sky is the limit, when it comes to custom
prompts.

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 09:38    Post subject:  

Shep,

That's why I didn't attach it. It works fine. But it need more work, it's not really
ready to deliver. It needs review and refinement. Thanks for the input. I'll study
it.

I also want to add a function, which I think I'll let the user fill in. The purpose of
the function is to delete browser cache. There is no reason to back that stuff
up.

I could find .mozilla SeaMonkey and Firefox cache with a script, but these days
people are using browsers, I'm not going to even bother with learning.

Bruce

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Fri 29 Apr 2011, 09:45    Post subject:  

Quote:
read -p 'Enter q to quit, or <RET> to continue:' -n 1 a


Maybe have I ran into a sequence or quote type problem. I'll try it your way.

Code:
read n1 a -p "Enter choice "


I think something didn't do right, so I decided to echo -n

~

_________________
New! Puppy Linux Links Page
Back to top
View user's profile Send private message 
Shep

Joined: 08 Nov 2008
Posts: 851
Location: GIRT-BY-SEA

PostPosted: Fri 29 Apr 2011, 09:46    Post subject:  

Bruce B wrote:
...but my after thought was, if there was any trailing white space in a line, maybe cat wouldn't consider it an empty line.


A perfectly logical precaution, too. However, there will be no trailing spaces because you have given sed the code to delete all trailing spaces. And if you modify it to delete trailing TABS, can there be any other trailing whitespace characters?

Just modify 's/ \+$//' to become 's/[ \t]\+$//'
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 16 of 24 [350 Posts]   Goto page: Previous 1, 2, 3, ..., 14, 15, 16, 17, 18, ..., 22, 23, 24 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Beginners Help ( Start Here)
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1720s ][ Queries: 12 (0.0366s) ][ GZIP on ]