Basic Shell (Console) operation for beginners

Booting, installing, newbie
Message
Author
Bruce B

#41 Post by Bruce B »

I have a sincere concern I want to express.

PupGeek started this topic with a great deal of interest.

Now I notice I'm the only one posting. I feel bad about that. The reason
why I feel bad is because I think unintentionally took other's interest in
posting. This is not OK with me.

I'm asking, "Can we please make this a community project. Anyone
interested please contribute and ask questions. We learn from each other
and we teach other."

If this doesn't happen, and I'm the only one, I will probably let it drop
off to page two.

Please. Our power is in community.

~

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

#42 Post by tlchost »

Bruce B wrote:I have a sincere concern I want to express.

PupGeek started this topic with a great deal of interest.

Now I notice I'm the only one posting. I feel bad about that. The reason
why I feel bad is because I think unintentionally took other's interest in
posting. This is not OK with me.
~
You want to consider that many of us value your posts....and may not want to "jump in the middle" and disrupt your train of thought.

Thanks for the information.....and please do not stop.

Thom

Bruce B

#43 Post by Bruce B »

tlchost wrote: You want to consider that many of us value your posts....and may not want
to "jump in the middle" and disrupt your train of thought.
I didn't consider that. Now I have a thought to work with I didn't have
before. To grow we need each other.

And I'd still feel better about this if PupGeek and others would resume their
interest. I'll wait and see.

Thanks

~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#44 Post by jpeps »

Bruce B wrote:
tlchost wrote: You want to consider that many of us value your posts....and may not want
to "jump in the middle" and disrupt your train of thought.
I didn't consider that. Now I have a thought to work with I didn't have
before. To grow we need each other.

And I'd still feel better about this if PupGeek and others would resume their
interest. I'll wait and see.

Thanks

~
We're sitting on the sidelines monitoring for any malicious code :)

Bruce B

#45 Post by Bruce B »

Bash programming flow

The lines are read from top to bottom and executed in that order.

Often we write our programs line1, line2, line3 and etc.

When done, if we did it right, our program works.

We write a program in order to do something for us. In most cases I write
programs because I want to solve problems. The problem might be how
to make something complex - simple.

We find in writing good programs we want to implement conditional check
and branches. Otherwise we can write a destructive program.

It is very important to know exactly what you want your program to do,
before writing it.

Think about it for a while and write notes before you program.

I'll make a demo program to show you how to make conditional checks
and branches.

Progname: bkuproot
Purpose: to make a backup of /root files on a separate partition
I need to:
make sure the partition is mounted
that there is sufficient space on the partition
I'll use the zip utility for my backup
I'll skip the space check to make my example easier
zip will bomb out with an on screen error if there wasn't enough space


My primary purpose here is actually to demonstrate how to make a
conditional check and branch, this should be easy. I'm ready to code.

Code: Select all

#!/bin/bash

DESTPART=/dev/sda2
DESTDIR=rootbkup
FILENAME=bkuproot.zip
Please note: by using variables, we make our script easily portable, and
we can make them easy to modify. We merely change the data in the
variable. We don't have to search the entire script.

Let's first make sure /dev/sda2 is mounted

Code: Select all

</proc/mounts grep $DESTPART>/dev/null
/dev/null is Linux' black hole, everything we redirect to /dev/null disappears.

By redirecting the output to /dev/null we eliminate all on screen output.

Our search criteria is /dev/sda2

Grep will not tell us if /dev/sda2 is mounted. It tells us if it found our
criteria. If it found our criteria, we know it is mounted, because grep will
give us an error code of 0 if it found the text.

What we are going to test for is grep's error code. The error code is found
in the variable $? If grep found the text then we have error 0

Now we introduce the first part of an if statement.

Code: Select all

if [ "$?" = "0" ] ; then
    [ ! -d $DESTPART/$DESTDIR ] && mkdir  $DESTPART/$DESTDIR

See how by using variables we can change the file merely by modifying
the easy to find variables at the top of the file. No searching and replace
necessary.

-d means we are looking for a directory

If a backup file already exists, let's delete it

Code: Select all

[ -f $DESTPART/$DESTDIR/$FILENAME ] && rm $DESTPART/$DESTDIR/$FILENAME


-f says we are looking for a file

Now let's make the new backup file. Note: many users make their
backups by putting dates in the file names. For our purpose, I'll keep it
more simple.

Now we are ready to make our backup

Code: Select all

zip -r9y $DESTPART/$DESTDIR/$FILENAME /root 
We don't have to check for the existence of /root, if it doesn't exist, we
have serious problems to work out.

zip is the command

-r9y means recurse /root, use maximum compression and preserve links,
don't convert to the files they reference

When you write your script you have to learn about the utility you want to
use, in this case zip

zip --help will show its options

Here is our safety branch:

Code: Select all

else
    echo "$DESTPART is not mounted, no changes made"

we terminate our if statement like this:

Code: Select all

fi
Now I'll put it all together, see if you can read it and understand it

Code: Select all

#!/bin/bash

DESTPART="/dev/sda2"
DESTDIR="rootbkup"
FILENAME="bkuproot.zip"

</proc/mounts grep $DESTPART>dev/null
if [ "$?" = "0" ] ; then
    [ ! -d $DESTPART/$DESTDIR ] && mkdir $DESTPART/$DESTDIR
    [ -f $DESTPART/$DESTDIR/$FILENAME ] && rm $DESTPART/$DESTDIR/$FILENAME
    zip -r9y $DESTPART/$DESTDIR/$FILENAME /root 
else
    echo "$DESTPART is not mounted, no changes made"
fi  
Most of my scripts look like a C programmer who escaped from a mental
ward who is trying to write in bash.

This time I think I actually followed bash conventions.

Namely using uppercase variables, quoting the variable data, and
making the script easily changed and portable.

If our partition is not mounted we have a false condition at the
beginning of our if statement. No commands execute. if looks for else and
executes it which is merely an on screen message to the user.

Also, notice the indents, they tell us what commands run within the
statement. It makes the code easier to work with and read. Especially
helpful when statements are nested and very long.

By the way this program has not been tested. If anyone can find errors,
I'll agree to owe them a nickel.

EDITED: You guys and gals are not reading well. The program contains a serious error. Please find it so you can be owed a nickel.

~

~
Last edited by Bruce B on Wed 09 Mar 2011, 04:26, edited 1 time in total.

Bruce B

#46 Post by Bruce B »

What is easier to read?

With variables

Code: Select all

#!/bin/bash

DESTPART="/mnt/sda2"
DESTDIR="rootbkup"
FILENAME="bkuproot.zip"

</proc/mounts grep $DESTPART>/dev/null
if [ "$?" = "0" ] ; then
    [ ! -d $DESTPART/$DESTDIR ] && mkdir $DESTPART/$DESTDIR
    [ -f $DESTPART/$DESTDIR/$FILENAME ] && rm $DESTPART/$DESTDIR/$FILENAME
    zip -r9y $DESTPART/$DESTDIR/$FILENAME /root
else
    echo "$DESTPART is not mounted, no changes made"
fi  
Without variables

Code: Select all

#!/bin/bash

</proc/mounts grep /mnt/sda2>/dev/null
if [ "$?" = "0" ] ; then
    [ ! -d /mnt/sda2/rootbkup ] && mkdir /mnt/sda2/rootbkup
    [ -f /mnt/sda2/rootbkup/bkuproot.zip ] && rm /mnt/sda2/rootbkup/bkuproot.zip
    zip -r9y /mnt/sda2/rootbkup/bkuproot.zip /root
else
    echo "/mnt/sda2 is not mounted, no changes made"
fi 
Obviously the one without variables is easier. But as explained earlier, the
one with variables in easily modified and more portable. Although your
text editor does accurate and fast search and replace if you didn't use
variables.

You can write your programs as you please. Either script does exactly the
same thing.

I tend to put my variables in lowercase, because it is easier for me to
read. Also, C convention uses lowercase variables.

When using variables, try and give them meaningful names.

~
Last edited by Bruce B on Thu 10 Mar 2011, 20:22, edited 1 time in total.

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#47 Post by jpeps »

maybe /dev/null. Worth a nickel?

Bruce B

#48 Post by Bruce B »

jpeps wrote:maybe /dev/null. Worth a nickel?
try again

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#49 Post by jpeps »

Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?

Bruce B

#50 Post by Bruce B »

I think I've shown how to change directories easily at lightening speed.

Now I want to show how to find files much, much, much faster than the
computer can search itself and display the results on the screen.

We do it by using a database containing a list of every file on the
machine. I don't know if this script will work on Puppy frugal, it might. It
should, but I don't recall. On second thought, it has been tested on Frugal.

There are also other support scripts which aren't yet posted. This one only
builds the database.

I leave it to the user to review the script and try and figure out what it's
doing and why I've written certain things.

If something doesn't make sense, then ask, I'll explain.

The purpose of the script is to build a database of every file on the disk.
This database can be searched much faster than a big disk with many
partitions.

Code: Select all

#!/bin/bash

main() {

	vars
	updateroot
	updatemnt
	filedatabase

}

vars() {

	dbfile=/dev/shm/update.db
	destdir=/var/log
	
	# get list of directories off /
	rootdirs=`find / -maxdepth 1 -type d | cut -c 2-100 \
	| grep -v sys | grep -v 'lost+found' | grep -v mnt \
	| grep -v proc | grep -v dev | tr "\n" " "`
#	echo $rootdirs

	# get mounted filesystems
	mntdirs=`cat /proc/mounts | grep /mnt/ \
	| cut -d " " -f 2 | tr "\n" " "`
#	echo $mntdirs
	

}

updateroot() {


	echo > $dbfile

	for i in $rootdirs ; do
		echo "updating /$i"
#		nice --adjustment=+6 find /$i -mount -type f>>$dbfile 
		find /$i -mount -type f>>$dbfile 
	done


}

updatemnt() {

	for i in $mntdirs ; do
		echo "updating $i"
#		nice --adjustment=+6 find $i -noleaf -mount -type f>>$dbfile
		find $i -noleaf -mount -type f>>$dbfile
#		break
	done

}

filedatabase() {


	mv $dbfile $destdir

}


main

Bruce B

#51 Post by Bruce B »

I can make a myriad of mistakes on English. People reading can get past
my spelling and grammar.

People are smart. They can reason and figure what I mean to say, even if
I don't say it right.

Computers have zero intelligence. Of course we have artificial
intelligence, but it's not substitute for natural stupidity.

A programming language is just that - a language. You are literally
learning a language.

I am here to help and I will if you ask questions.

Eventually, with practice, you will be able to read your new language
much the same as you can read your native language.

For a little history.

I used MS-DOS COMMAND.COM

I thought I wanted more, so I bought a much better Shell, 4DOS.COM.
It is much more powerful than COMMAND.COM

Guess what? I abandoned the use of 4DOS because I found with the use
of external utilities, I could do everything with COMMAND.COM that I
could do with 4DOS.COM

COMMAND.COM has maybe 20 to 25 internal commands.

This is all the words I needed to know to write powerful batch scripts in
DOS and Windows.

The same applies to bash, there are words you don't need to know, until
you get to advanced levels and even at that, you may not even chose to
learn the words. English is the same, I know about 5% of the words
available in a common complete dictionary.

Yes, bash is a language, but the number of words, statements and
syntax you need to know to write useful scripts is not that many.

Don't be intimidated. I'm a good teacher, for one reason, because I don't
talk over people's heads. I put myself in the shoes of a newbie and
communicate at that level.

Importantly, my goal is to teach. Hopefully aslo learn if others will
contribute. Due to tlchost, I'll continue for a while.

~
Last edited by Bruce B on Wed 09 Mar 2011, 11:05, edited 3 times in total.

Bruce B

#52 Post by Bruce B »

How I learned programming

Dad was/is a computer scientist. Now retired. Mostly he programmed.

He didn't study programming. When he went to school, there were no
programming classes.

Recruiters used top mathematicians for programmers.

Dad started programming before there were languages.

Then he learned languages. Working day in and day out, over a period of
years he became an expert programmer.

Because he was a 'scientist' he made like $70 more an hour than a
programmer.

I got my start programming at home from him. But when he'd present
math higher than my level of understanding, I couldn't keep up.

As a workaround, I bought many books on C. I would stay with one book
until it got over my head. Then I start another book until it got over my
head. Rotating the books, eventually, I was able to get deeper into th
books.

I am self taught.

I recommend the same for learning Linux and bash. C is a tough
language, but sometimes a C program is the best program to use.

Here is how I think one should learn

Use the same technicals I used with C. The Internet is full of great
learning sources.

Avoid the narcissistic teachers who are mainly interested in showing you
what they know.

Remember, none of us were born with any knowledge of programming.
We learned mostly from others and by practice.

You learn by programming and programming and programming. You
need good instructions, but you can't retain the information and learn
without writing programs.

Keep in mind, you will spend most of your time debugging your programs.
This is normal for the beginner.

Best of luck.

~

PupGeek
Posts: 353
Joined: Sun 06 Sep 2009, 11:30

#53 Post by PupGeek »

Bruce I love the test commands ([ ] -- Also the conditional execute command -- &&).... Here I have been relying on if then blocks to run simple tests with only one conditional command. This could save me some serious typing. Thank you.

Bruce B

#54 Post by Bruce B »

jpeps wrote:
Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?
Here is the error and it is serious, the program would have failed.

DESTPART=/dev/sda2

It should have read

DESTPART=/mnt/sda2

I was thinking the device. I should have been thinking the device's
mount point.

One of the problems with variables. In this case it is initialized one time,
then used repeatedly. The error in the initializing gets carried over and
you don't see it because you are referencing the variable, not the data.

Programming as I said is - a language. We need to know how to read and
write. Reading is especially important when working on community
projects.

~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#55 Post by jpeps »

Bruce B wrote:
jpeps wrote:
Bruce B wrote: try again
Script will work from "/". Guess that's good enough, right?
Here is the error and it is serious, the program would have failed.

DESTPART=/dev/sda2

It should have read

DESTPART=/mnt/sda2

I was thinking the device. I should have been thinking the device's
mount point.
Also needs to be /dev/null, or it will generate $? of 1 (error) unless you're running from /

Code: Select all

</proc/mounts grep /mnt/sda2>dev/null

Bruce B

#56 Post by Bruce B »

jpeps wrote:
Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?
My bad, jpeps gets a owed a nickel and a sincere apology.

/dev/null would have been good

but I used dev/null which is bad

So there were two errors in my untested demo script.

Will you forgive me, jpeps?

~

Bruce B

#57 Post by Bruce B »

Boo-boos

I posted a demo program to teach a few aspects of programming.

I didn't test the program because its purpose was to teach and I had no
need for the code. Because I didn't test it, I made boo-boos.

After reading my post, I saw I made a mistake. I left it there as a reader
challenge to find the mistake. Jpeps took the challenge and found
a mistake I didn't know about. I didn't give credit, because . . .(expletive)

Lessons

1) You are going to make mistakes.

2) Programming is not something to rush with.

3) You must test your programs

4) The programmer is likely to spend most of her time debugging and
refining the program.

5) When programming you can 'audit' your code by testing at various
logical increments as you write the code. This makes debugging easier
than if you try and debug a complex script.

I'll give an example of auditing from a script posted, which has been
tested and used many times.

Code: Select all

   # get mounted filesystems
   mntdirs=`cat /proc/mounts | grep /mnt/ \
   | cut -d " " -f 2 | tr "\n" " "`
#   echo $mntdirs 
The variable is called mntdirs which I intend to contain all the mounted
directories.

The test was echo $mntdirs, I wanted to see if the variable mntdirs
contained the data I expected it to contain.

Once the test succeeded I commented out the echo $mntdirs like this
# echo $mntdirs

I didn't delete the echo $mntdirs because when I want to go back and
audit the code, I can tell that portion was tested.


Save your commands and comments

I want my program to be nice - meaning giving it a low CPU priority, so
anything else running won't be slowed down much if at all by it.

Code: Select all

#      nice --adjustment=+6 find /$i -mount -type f>>$dbfile
      find /$i -mount -type f>>$dbfile 
Then I commented nice out and ran it normally. This is how I was able to
go back and know it was tested on Puppy, because Puppy's nice syntax is
not the same as the true nice command.

I could have, should have made a comment and set it to conform with
Puppy's nice command but didn't.

By not deleting the first command containing nice, the script is easily
ported back to a Linux system with the real nice command.

~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#58 Post by jpeps »

Bruce B wrote:
Will you forgive me, jpeps?

~
Only on condition that you keep up the good work :)

Bruce B

#59 Post by Bruce B »

Using too many commands (words)

I didn't know how to get the anti-theft device unstuck in my Ford. I called an expert mechanic. Have you ever met people who use too many words, to say what they want? He took about seven minutes to say what could have been said like this:
  • Close the hood and all doors. Take the key and open the driver door to defeat the alarm.
The same is true in programming. Programmers can use too many words.

After you write a successful program, go back and refine it. I give an
example from my own work.

Code: Select all

#!/bin/bash

   # get mounted filesystems 
   mntdirs=`cat /proc/mounts | grep /mnt/ \
   | cut -d " " -f 2 | tr "\n" " "`
   echo $mntdirs 
   
   # get mounted filesystems - REFINED FOR SPEED/EFFICIENCY 
   mntdirs=`</proc/mounts grep /mnt/ \
   | cut -d " " -f 2`
   echo $mntdirs 
In the refined block we have eliminated two external commands,
cat and tr.

Just because you finally wrote a script that really does what you want,
doesn't mean it's efficient. This script snippet must have been written
years ago, before I learned some of bash's nuances.

A long script which has been optimized will run much faster.

This program will show you the directories mounted on /mnt

The mount command will show all mount points

Hint: We want to give our scripts meaningful names. Don't write a script
named 'test' because test is a command.

Remember, type in the name you intend to use, before giving the script
a name.

I'll call this script deleteme.

You can download it and run it as you please.

~
Attachments
deleteme.zip
(274 Bytes) Downloaded 574 times

Bruce B

#60 Post by Bruce B »

What is a period?

Microsoft innovated. This means they looked at what others did and
copied them.

In MS-DOS a period:

* meant the current directory
* it was also used to separate the filename from the extension

--------

(Unix, of course, came long before MS)

In Linux it means the same thing and potentially much more.

To open rox from the commandline in the current directory, type

rox .

To open two instances of rox in the current directory, type

rox . .

Two periods mean the parent directory

To open rox in the current directory and another instance in the
parent directory, type

rox . ..

To make an empty file with an extension, you could type

touch filename.tmp

ls filename.tmp to verify it exists

then

rm -v filename.tmp to delete it

Linux won't let you move or copy files from directory to directory
without providing it with the source and target directories. Using the
period saves typing, because . is the current directory.

If I want to copy /mnt/sda2/foo/bar to the current directory the
command would be

cp /mnt/sda2/foo/bar .

conversely, if I want to go the other way, I type

cp ./bar /mnt/sda2/foo

same syntax with the mv command

What is a period really to the bash programmer?

Potentially, it is something which could drive her insane.

A backslash "\" primer

A typical escape character is \

If a character has a special meaning to bash or other Linux utilities,
we can often make it have a literal meaning by 'escaping' it.

The \ is a common escape character, which says treat the character
to the right literally.

Here is an example, we want to echo to the console, the text below:

You didn't enter a "filename"

The apostrophe will throw us off. We need to quote the command, like this:

echo "You didn't enter a "filename""

It won't work. We need to tell echo which "s (quotes) to treat
literally, we do this with the escape character \

This works

echo "You didn't enter a \"filename\""

Consider this a primer on periods and backslashes, it gets worse,
but at a minimum, this needs much needs understanding.

~

You can try the commands in dark blue on the command line. Except
for the substitutable ones with foo and bar.

If you enter this command which is shown above it will take you to a
PS2 prompt

echo "You didn't enter a "filename""

"Escape" it with Ctrl+C

~

Post Reply