Basic Shell (Console) operation for beginners

Booting, installing, newbie
Message
Author
Bruce B

#261 Post by Bruce B »

Self Confidence

My dad is a retired computer scientist. He started programming in the
mid fifties. This was before computers as we know them today and
before languages.

I had an excellent opportunity to get training from one of the best.
And he was more than happy to train. His skill with math is so
advanced, he can put me to sleep just talking about.

Seeing what it is like to have a real pro with C was too intimidating to
me. Do to loss of confidence, I didn't peruse programming the way I
originally intended. But with the interest I had in computers, I kept
right on writing programs within my skill level, fixing, building,
modifying and everything imaginable. And of course learning and
gaining experience.

About sed

Sed is not bash. It is a stream editor. A utility found in all our Linux
distros. I never took the time to learn it well. Lately I've been
working more with it. I thank Shep for his help, encouragement and
expertise.

I went back a few pages in this topic to study a line he posted. I
couldn't follow it. Back again to that feeling of losing of confidence.

Here is the line.

Code: Select all

ls *.mp3 | sed "h; s/['"'()]//g; :a; s/[ \.]\(.*\.\)/_\1/; ta; x; G; s/\(.*\)\n/mv "\1" /' | sh
Breaking things into small pieces

While I still can't following the line, I've learned enough to not lose the
confidence.

This is more simple than it looks. Just break it down into small pieces.
So the first part I don't understand is the first letter.

If I want to figure it out, I merely need to find a good sed page and
find out what h; means. The next full command I do understand. If I
want to train myself on it merely copy it to the screen and play with
some combinations. Just keep moving through.

Also people like Shep and myself are happy to explain things in
sufficient detail, the learning curve is easier.

Don't quit, or lose confidence. A big script like the kind Barry writes,
when taken as a whole is too much to digest. In fact you can't see
many of his scripts in one or even 10 pages. Primarily, you need to
break things down into small pieces.

A problem I have working with his scripts is keeping track of what
data various VARIABLES might contain. I can figure it out and add
comments so I don't have to wonder.

~

Bruce B

#262 Post by Bruce B »

Bash Internal Commands Help

Whoever wrote the internal help for internal commands,
speaks plain English and explains.

I put all the internal help together into one page. Formatted it stay
within eighty columns and added a table of contents.

And attached the file to this post.

~
Attachments
bash_internal_cmds.zip
(13.02 KiB) Downloaded 393 times

Bruce B

#263 Post by Bruce B »

Dirf

Dirf got its name from back in the DOS days. I combined the dir
command and the find utility to make finding files easier, more
explicit.

The command I'm posting is a part of my lstools. It finds files
according to your command line criteria. Very easy to use.

Code: Select all

#!/bin/bash
if [ $1 ] ; then
	ls -ldF * | awk '!/^d/ {print $8}' | grep -i "$1" 
else
	ls -ldF * | awk '!/^d/ {print $8}'
fi
Usage Examples

dirf tool - shows all files containing the string "tool" in them. (case insensitive)

dirf ^f - shows all files beginning with the letter f or F

dirf z$ - shows all files ending with the letter z or Z

The main command string

Code: Select all

ls -ldF * | awk '!/^d/ {print $8}' | grep -i "$1"
Broken down

Code: Select all

ls -ldF *
Shows all files in a long listing format, with the dF added
(ls --help for details)

Code: Select all

awk '!/^d/ {print $8}'
The !/^d/ filters out all entries beginning with d. In the ls -l output,
this is all directories. {print $8} prints only column 8, which is the
same as a short listing.

Code: Select all

grep -i "$1"
Searches the column 8 output for matches. The -i makes
the search 'cAsE insensitive'

Shep gave me the suggestion for an improved awk command. Put
two commands into one. Thanks.

~

Bruce B

#264 Post by Bruce B »

Ad (All Directories) (another ls tool)

It shows non-hidden directories. No files or links.
Use it the same as dirf, only this is for directories.

Here is the script.

Code: Select all

#!/bin/bash

x=`echo *`
[ "${#x}" -lt  "2" ] && exit

if [ $1 ] ; then
    ls -ldF * | awk '/^d/ {print $8}' | grep -i "$1"
else
    ls -ldF * | awk '/^d/ {print $8}'
fi

Code: Select all

x=`echo *`
[ "${#x}" -lt  "2" ] && exit
If you navigate to an empty directory, this will suppress an ls
error message.

${#x} is not the contents of x it is the byte size of x

If it is only one byte, the value in x is presumed *

That is what ad uses for an exit branch.

Enjoy!

~

Bruce B

#265 Post by Bruce B »

Ahd (displays hidden directories)

Considering the previous two posts, I don't think this requires
an explanation.

Code: Select all

#!/bin/bash

if [ $1 ] ; then
    ls -aldF .* | awk '/^d/ {print $8}' | grep -i "$1"
else
    ls -aldF .* | awk '/^d/ {print $8}'
fi
~

Bruce B

#266 Post by Bruce B »

Ahf (displays only hidden files)

All the information for this utility is contained in the three posts above.

I think it will be more fun, if you mix and match from the above code
and make your own.

~

Bruce B

#267 Post by Bruce B »

Bruce B wrote:

Code: Select all

x=`echo *`
[ "${#x}" -lt  "2" ] && exit
If you navigate to an empty directory, this will suppress an ls
error message.

${#x} is not the contents of x it is the byte size of x

If it is only one byte, the value in x is presumed *

~
Yeah, but.

But what if the directory has only a one byte directory or filename in
it? You didn't think of that?

~

Bruce B

#268 Post by Bruce B »

Introducing lsnow

Sometimes I want to know what files change or get modified when
I run a configuration tool. (my earlier approach was too
cumbersome)

For this purpose: finding files which have just been modified or
created, I wrote lsnow.

It is a newborn, only a few hours old. If anyone finds a problem with
it or can see how to improve it, please post back. Otherwise, enjoy.

Code: Select all

#/bin/bash

main() {

    variables
    cd $STARTDIR
    find_display
    exit 0

}

variables() {

    STARTDIR=/initrd/pup_rw # USER SETTING

    # if a frugal install we only want to search the
    # pupsave file, the sfs files can not change, it
    # is a "big waste" to search them.

    # your pupsave file may not be at the default
    # variable STARTDIR, if not, change it

    nowd=`date +"%Y-%m-%d"`
    ntime=`date +"%k"`

    # get rid of a leading space
    ntime=`echo $ntime`

    # if < 10 hours we need to pad with
    # a leading zero

    if [ "${#ntime}" -eq "1" ] ; then
        ntime="0$ntime"
    fi


}

find_display() {

    echo -n "Pupsave files changed on this date and hour: "
    echo "$nowd $ntime"
    # search files modified on this date
    # within this hour

    # if you modify script and search hidden files,
    # many excludes, for example:
    # found=`ls -lRa | grep -v Cache | grep \
    # "$nowd $ntime:" | awk '!/^d/ {print $8}'`

    found=`ls -lR | grep "$nowd $ntime:" | awk '!/^d/ {print $8}'`


    for i in $found ; do
        x=`find -name $i`
        ls -l --color $x
    done

}

main
Notes from after first post

Will leave code as first displayed for now and add notes

Duplicate files: if more than one file of same name which matches,
the additional file(s) display. Easy to fix by second grep
of output.

New option: Easy to add option allowing user to enter date or time
and date this way, if you want to see all modified files on a whole
day, for example, no problem.

~
Last edited by Bruce B on Wed 18 May 2011, 11:47, edited 2 times in total.

Bruce B

#269 Post by Bruce B »

Criteria

I ran alsactrl store for the purpose of saving settings, with the
intention of making the changed file immutable.

Generally, I presume somewhere in /etc, but after running alsactrl store,
the file I thought would change didn't. No files changed.

That's what gave rise to the birth of lsnow. Just were did the
settings get stored?

lsnow reported: /var/lib/alsa/asound.state

At first lsnow was just a command. If it is a command which requires
typing more than a few bytes AND I plan to run it in the future, the
criteria for writing a script is probably met.

~

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#270 Post by technosaurus »

Bruce B wrote:
Bruce B wrote:

Code: Select all

x=`echo *`
[ "${#x}" -lt  "2" ] && exit
If you navigate to an empty directory, this will suppress an ls
error message.

${#x} is not the contents of x it is the byte size of x

If it is only one byte, the value in x is presumed *

~
Yeah, but.

But what if the directory has only a one byte directory or filename in
it? You didn't think of that?

~
[ $x == "*" ] && exit
(may need a backslash for * ... on my Droid again)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Bruce B

#271 Post by Bruce B »

technosaurus wrote:[ $x == "*" ] && exit
It doesn't seem to work. The left part of the comparison might also
need quotes.

In this case, I think I'll do like the pros. Spend 60 bytes writing up a bug
report, rather than 10 bytes to fix the program.

~

Bruce B

#272 Post by Bruce B »

lspet

With this lstool you can view the contents of your pet package.
Helpful especially, if you have to manually 'unpet'.

Code: Select all

#!/bin/bash
tar -tzf 2>/dev/null "$@"
lstgz

For previewing your tar.gz and .tgz files

The code is identical to lspet, I wonder why?

Code: Select all

#!/bin/bash
tar -tzf 2>/dev/null "$@"
~

Shep
Posts: 878
Joined: Sat 08 Nov 2008, 07:55
Location: Australia

#273 Post by Shep »

Bruce B wrote:
technosaurus wrote:[ $x == "*" ] && exit
It doesn't seem to work. The left part of the comparison might also
need quotes.
Are you are testing for x being equal to an asterisk?

Code: Select all

# x=*; if [ "$x" = '*' ]; then echo xxx; fi
Instead of the old single brackets, you can use the much more-versatile double square brackets of bash's extended test function. I think you can get away without quoting the $x here because within [[.......]] there is no word splitting:

Code: Select all

x=*; if [[ $x == '*' ]]; then echo xxx; fi
An alternative: save any positional parameters, then you are free to use the super-efficient set to determine whether your directory is empty:

Code: Select all

 set -- *; if [[ $1 == '*' ]]; then echo directory is empty; fi

Bruce B

#274 Post by Bruce B »

I wrote a little script which will not work. It will work and produce the output shown in the comments after certain characters have been escaped. The escape character is used to change the meaning of the character sent to the 'for loop' is the \ (backslash). The backslash is used to give a literal meaning to the character rather then how bash uses it.

For example: echo ~ displays /root and echo \~ displays ~

Code: Select all

#!/bin/bash

for i in  ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 \
: ; < = > ? @ A B C  D E F G H I J K L M N O P Q R S T U V W \
X Y Z [ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v \
w x y z { | } ~
do
echo -n "$i "
done
echo

# ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9
# : ; < = > b @ A B C D E F G H I J K L M N O P Q R
# S T U V W X Y Z [ ] ^ _ ` a b c d e f g h i j k l
# m n o p q r s t u v w x y z { | } ~
The idea here is to self teach exactly how characters are interpreted and which need escaped.

~
Attachments
practice.zip
(372 Bytes) Downloaded 619 times

Bruce B

#275 Post by Bruce B »

Bash errors

Bash gives an error "command not found"

Sometimes you run into some though ones. This one about takes the cake.

Cross my heart - I was confident I did everything right. I looked at the code over and over and could not find the reason for the command not found.

I didn't find it with the mp editor. I can't find anything wrong with cat.

My dog didn't eat my homework but I think she messed up my script. It happens when I leave tasty stuff on the table, she sometimes jumps up on the keyboard uses it as a launch pad to the table to steal my food.

Here is the problem: She pressed the space bar to move the text to column 154, then I guess entered these characters ~V~R

I don't know this for a fact, but I confronted her and she didn't deny it.

As stated bash finds them and says command not found.

Cat don't display them. With persistence I debugged it and even wrote my own poor man's cat utility called kitty. Kitty won't display this character string on the screen either.

Echo will.

How many times I thought bash was the culprit only to find it wasn't.

Anyone else had tough times trying to figure why things just will not work right sometimes?

Anyone else started thinking the problem is Bash, because you couldn't find the problem in your source?

~~~~

Change subject

Hoping Shep or another sed expert is still around, I have a question.

I want to reduce lines empty lines more than two to double lines.

The obvious approach is to me


sed -e 's/^$^$^$^$/^$^$/' -e 's/^$^$^$/^$^$/'


It works, but I've learned enough to know that a sed expert knows the best way to do it.

Shep, anyone?

~

Shep
Posts: 878
Joined: Sat 08 Nov 2008, 07:55
Location: Australia

#276 Post by Shep »

Bruce B wrote:I want to reduce lines empty lines more than two to double lines.
See whether this is going to do what you want:

sed '/^\n*$/{N; /^\n$/N; /\n$/D;}'

reading from left to right:
if the pattern space is empty or contains only newlines then do everything inside the curly brackets, viz.,
append the next line of input to the pattern space
if the pattern space contains exactly two empty lines then append to the pattern space the next line of input
if the last line was empty, delete the first part of the pattern space and repeat until a non-blank line is encountered

Bruce B

#277 Post by Bruce B »

Introducing the file command

In its most simple form we use this external command to help figure what kind of file we are working with with.

In my example, mplayer.dsl has an unknown extension. The Linux distro Damn Small Linux uses it. I want to know what it is. I use file to look at it and report back.

I say file mplayer.dsl

file says: mplayer.dsl: gzip compressed data, from Unix

To learn more I say: gzip -d mplayer.dsl

gzip says: mplayer.dsl: unknown suffix -- ignored

I then say: mv mplayer.dsl mplayer.dsl.gz

Then I say: gzip -d mplayer.dsl.gz

Gzip decompresses it and leaves me with mplayer.dsl

Then I say: file mplayer.dsl

file then says: mplayer.dsl: POSIX tar archive (GNU)

Now I know it is a tar archive

Then I say: tar xvf mplayer.ds

Sure enough it extracts.

To learn more say file --help, for more details consult the man page.

~

Bruce B

#278 Post by Bruce B »

How to make a RAM Disk - step by step

1) Define RAM Disk size
  • Edit menu.lst and set the size of the disk on the kernel line

    ramdisk_size=192000

    This would configure to make a disk size of about 192 MB

    The 192000 is 192000k but the k is not entered
2) Format the RAM Disk in the file /etc/rc.d/rc.local
  • mkfs.ext2 -m 0 /dev/ram6

    the -m 0 says not to allocate space for the super user. It gives more space and the super user allocation is not needed.
3) Fstab
  • tell /etc/fstab about the RAM disk

    Code: Select all

    /dev/ram6 /mnt/ram ext2 rw,noatime,noauto,shortname=mixed,errors=continue 0 0
    
4) mount RAM disk
  • Use ROX-Filer mouse options on /mnt/ram

    or use the command line

    mount /dev/ram6

    or

    mount /mnt/ram

    You can unmount either on the command line or with ROX-Filer
Comments

You can of course use a different disk size, mount point and ram device.

In Puppy we have three basic places to put apps or commands for automatic run.

/etc/rc.d/rc.local
/etc/profile.local
/root/Startup


Here is a guide of what which to use in a variety of circumstances. Any graphics app needs to be put in /root/Startup

Non graphics apps or instructions you only want to run one time per session, put in /ect/rc.d/rc.local

/etc/profile.local is susceptible to being run more than once, something to consider when placing commands in this file

Many readers knew this already. But for those who didn't, now you know.

It is possible that /etc/profile.local doesn't exist. If not make it as a regular text file. It is not a shell script

I use the RAM disk when I have big files or need to batch process files. These files might get modified, converted, deleted, remodified, decoded or encoded, renamed and whatever. This could make for a lot of disk I/O for files that only live a few minutes. The RAM disk speeds things up and saves any disk wear and tear. I also compile in a RAM disk for the same benefits.

The RAM disk I present in this post is what I call - "A true formatted RAM disk"

There are other kinds of RAM disks which can be set up easier. My reason for the so-called true formatted RAM disk is ffmpeg won't write to the other formats.

Thanks

~

Indy'spup
Posts: 50
Joined: Wed 11 May 2011, 15:32
Location: SoCal

#279 Post by Indy'spup »

Ok I got to ask because this is killing me...

Does the Prefix -- used often with a command/function have any specific meaning in bash? As I understand it after the function has run the -- will disable other options pertaining to that function to run and terminate the command cleanly.

ie: --version, or --help etc. etc.

Bruce B

#280 Post by Bruce B »

Indy'spup wrote:ie: --version, or --help etc. etc.
Bash is a command interpreter. I posted the full Bash help on the internal commands and -- is not listed as an internal command.

By which I conclude only that it is not listed.

Set is an internal command and I've seen Shep post examples of set --. I would think -- is an argument the internal command set uses.

Normally we use the -- like this:

cp --help
grep --help


In this case the --help is an argument (parameter?) to the cp and grep commands.

In other words: --help is simply passed on by bash to the external command as an argument.

Bash will pass a bad argument to the cp command. In this example,
I said cp /?

cp replied

[~] cp /?
cp: missing destination file operand after `/?'
Try `cp --help' for more information.
[~]


Issue -- on the CLI and bash reports

[~] --
bash: --: command not found
[~]


This means by itself -- is neither an internal or external command.

~

Post Reply