Basic Shell (Console) operation for beginners

Booting, installing, newbie
Message
Author
Bruce B

#46 Post by Bruce B »

What is easier to read?

With variables

Code: Select all

#!/bin/bash

DESTPART="/mnt/sda2"
DESTDIR="rootbkup"
FILENAME="bkuproot.zip"

</proc/mounts grep $DESTPART>/dev/null
if [ "$?" = "0" ] ; then
    [ ! -d $DESTPART/$DESTDIR ] && mkdir $DESTPART/$DESTDIR
    [ -f $DESTPART/$DESTDIR/$FILENAME ] && rm $DESTPART/$DESTDIR/$FILENAME
    zip -r9y $DESTPART/$DESTDIR/$FILENAME /root
else
    echo "$DESTPART is not mounted, no changes made"
fi  
Without variables

Code: Select all

#!/bin/bash

</proc/mounts grep /mnt/sda2>/dev/null
if [ "$?" = "0" ] ; then
    [ ! -d /mnt/sda2/rootbkup ] && mkdir /mnt/sda2/rootbkup
    [ -f /mnt/sda2/rootbkup/bkuproot.zip ] && rm /mnt/sda2/rootbkup/bkuproot.zip
    zip -r9y /mnt/sda2/rootbkup/bkuproot.zip /root
else
    echo "/mnt/sda2 is not mounted, no changes made"
fi 
Obviously the one without variables is easier. But as explained earlier, the
one with variables in easily modified and more portable. Although your
text editor does accurate and fast search and replace if you didn't use
variables.

You can write your programs as you please. Either script does exactly the
same thing.

I tend to put my variables in lowercase, because it is easier for me to
read. Also, C convention uses lowercase variables.

When using variables, try and give them meaningful names.

~
Last edited by Bruce B on Thu 10 Mar 2011, 20:22, edited 1 time in total.

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#47 Post by jpeps »

maybe /dev/null. Worth a nickel?

Bruce B

#48 Post by Bruce B »

jpeps wrote:maybe /dev/null. Worth a nickel?
try again

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#49 Post by jpeps »

Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?

Bruce B

#50 Post by Bruce B »

I think I've shown how to change directories easily at lightening speed.

Now I want to show how to find files much, much, much faster than the
computer can search itself and display the results on the screen.

We do it by using a database containing a list of every file on the
machine. I don't know if this script will work on Puppy frugal, it might. It
should, but I don't recall. On second thought, it has been tested on Frugal.

There are also other support scripts which aren't yet posted. This one only
builds the database.

I leave it to the user to review the script and try and figure out what it's
doing and why I've written certain things.

If something doesn't make sense, then ask, I'll explain.

The purpose of the script is to build a database of every file on the disk.
This database can be searched much faster than a big disk with many
partitions.

Code: Select all

#!/bin/bash

main() {

	vars
	updateroot
	updatemnt
	filedatabase

}

vars() {

	dbfile=/dev/shm/update.db
	destdir=/var/log
	
	# get list of directories off /
	rootdirs=`find / -maxdepth 1 -type d | cut -c 2-100 \
	| grep -v sys | grep -v 'lost+found' | grep -v mnt \
	| grep -v proc | grep -v dev | tr "\n" " "`
#	echo $rootdirs

	# get mounted filesystems
	mntdirs=`cat /proc/mounts | grep /mnt/ \
	| cut -d " " -f 2 | tr "\n" " "`
#	echo $mntdirs
	

}

updateroot() {


	echo > $dbfile

	for i in $rootdirs ; do
		echo "updating /$i"
#		nice --adjustment=+6 find /$i -mount -type f>>$dbfile 
		find /$i -mount -type f>>$dbfile 
	done


}

updatemnt() {

	for i in $mntdirs ; do
		echo "updating $i"
#		nice --adjustment=+6 find $i -noleaf -mount -type f>>$dbfile
		find $i -noleaf -mount -type f>>$dbfile
#		break
	done

}

filedatabase() {


	mv $dbfile $destdir

}


main

Bruce B

#51 Post by Bruce B »

I can make a myriad of mistakes on English. People reading can get past
my spelling and grammar.

People are smart. They can reason and figure what I mean to say, even if
I don't say it right.

Computers have zero intelligence. Of course we have artificial
intelligence, but it's not substitute for natural stupidity.

A programming language is just that - a language. You are literally
learning a language.

I am here to help and I will if you ask questions.

Eventually, with practice, you will be able to read your new language
much the same as you can read your native language.

For a little history.

I used MS-DOS COMMAND.COM

I thought I wanted more, so I bought a much better Shell, 4DOS.COM.
It is much more powerful than COMMAND.COM

Guess what? I abandoned the use of 4DOS because I found with the use
of external utilities, I could do everything with COMMAND.COM that I
could do with 4DOS.COM

COMMAND.COM has maybe 20 to 25 internal commands.

This is all the words I needed to know to write powerful batch scripts in
DOS and Windows.

The same applies to bash, there are words you don't need to know, until
you get to advanced levels and even at that, you may not even chose to
learn the words. English is the same, I know about 5% of the words
available in a common complete dictionary.

Yes, bash is a language, but the number of words, statements and
syntax you need to know to write useful scripts is not that many.

Don't be intimidated. I'm a good teacher, for one reason, because I don't
talk over people's heads. I put myself in the shoes of a newbie and
communicate at that level.

Importantly, my goal is to teach. Hopefully aslo learn if others will
contribute. Due to tlchost, I'll continue for a while.

~
Last edited by Bruce B on Wed 09 Mar 2011, 11:05, edited 3 times in total.

Bruce B

#52 Post by Bruce B »

How I learned programming

Dad was/is a computer scientist. Now retired. Mostly he programmed.

He didn't study programming. When he went to school, there were no
programming classes.

Recruiters used top mathematicians for programmers.

Dad started programming before there were languages.

Then he learned languages. Working day in and day out, over a period of
years he became an expert programmer.

Because he was a 'scientist' he made like $70 more an hour than a
programmer.

I got my start programming at home from him. But when he'd present
math higher than my level of understanding, I couldn't keep up.

As a workaround, I bought many books on C. I would stay with one book
until it got over my head. Then I start another book until it got over my
head. Rotating the books, eventually, I was able to get deeper into th
books.

I am self taught.

I recommend the same for learning Linux and bash. C is a tough
language, but sometimes a C program is the best program to use.

Here is how I think one should learn

Use the same technicals I used with C. The Internet is full of great
learning sources.

Avoid the narcissistic teachers who are mainly interested in showing you
what they know.

Remember, none of us were born with any knowledge of programming.
We learned mostly from others and by practice.

You learn by programming and programming and programming. You
need good instructions, but you can't retain the information and learn
without writing programs.

Keep in mind, you will spend most of your time debugging your programs.
This is normal for the beginner.

Best of luck.

~

PupGeek
Posts: 353
Joined: Sun 06 Sep 2009, 11:30

#53 Post by PupGeek »

Bruce I love the test commands ([ ] -- Also the conditional execute command -- &&).... Here I have been relying on if then blocks to run simple tests with only one conditional command. This could save me some serious typing. Thank you.

Bruce B

#54 Post by Bruce B »

jpeps wrote:
Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?
Here is the error and it is serious, the program would have failed.

DESTPART=/dev/sda2

It should have read

DESTPART=/mnt/sda2

I was thinking the device. I should have been thinking the device's
mount point.

One of the problems with variables. In this case it is initialized one time,
then used repeatedly. The error in the initializing gets carried over and
you don't see it because you are referencing the variable, not the data.

Programming as I said is - a language. We need to know how to read and
write. Reading is especially important when working on community
projects.

~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#55 Post by jpeps »

Bruce B wrote:
jpeps wrote:
Bruce B wrote: try again
Script will work from "/". Guess that's good enough, right?
Here is the error and it is serious, the program would have failed.

DESTPART=/dev/sda2

It should have read

DESTPART=/mnt/sda2

I was thinking the device. I should have been thinking the device's
mount point.
Also needs to be /dev/null, or it will generate $? of 1 (error) unless you're running from /

Code: Select all

</proc/mounts grep /mnt/sda2>dev/null

Bruce B

#56 Post by Bruce B »

jpeps wrote:
Bruce B wrote:
jpeps wrote:maybe /dev/null. Worth a nickel?
try again
Script will work from "/". Guess that's good enough, right?
My bad, jpeps gets a owed a nickel and a sincere apology.

/dev/null would have been good

but I used dev/null which is bad

So there were two errors in my untested demo script.

Will you forgive me, jpeps?

~

Bruce B

#57 Post by Bruce B »

Boo-boos

I posted a demo program to teach a few aspects of programming.

I didn't test the program because its purpose was to teach and I had no
need for the code. Because I didn't test it, I made boo-boos.

After reading my post, I saw I made a mistake. I left it there as a reader
challenge to find the mistake. Jpeps took the challenge and found
a mistake I didn't know about. I didn't give credit, because . . .(expletive)

Lessons

1) You are going to make mistakes.

2) Programming is not something to rush with.

3) You must test your programs

4) The programmer is likely to spend most of her time debugging and
refining the program.

5) When programming you can 'audit' your code by testing at various
logical increments as you write the code. This makes debugging easier
than if you try and debug a complex script.

I'll give an example of auditing from a script posted, which has been
tested and used many times.

Code: Select all

   # get mounted filesystems
   mntdirs=`cat /proc/mounts | grep /mnt/ \
   | cut -d " " -f 2 | tr "\n" " "`
#   echo $mntdirs 
The variable is called mntdirs which I intend to contain all the mounted
directories.

The test was echo $mntdirs, I wanted to see if the variable mntdirs
contained the data I expected it to contain.

Once the test succeeded I commented out the echo $mntdirs like this
# echo $mntdirs

I didn't delete the echo $mntdirs because when I want to go back and
audit the code, I can tell that portion was tested.


Save your commands and comments

I want my program to be nice - meaning giving it a low CPU priority, so
anything else running won't be slowed down much if at all by it.

Code: Select all

#      nice --adjustment=+6 find /$i -mount -type f>>$dbfile
      find /$i -mount -type f>>$dbfile 
Then I commented nice out and ran it normally. This is how I was able to
go back and know it was tested on Puppy, because Puppy's nice syntax is
not the same as the true nice command.

I could have, should have made a comment and set it to conform with
Puppy's nice command but didn't.

By not deleting the first command containing nice, the script is easily
ported back to a Linux system with the real nice command.

~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#58 Post by jpeps »

Bruce B wrote:
Will you forgive me, jpeps?

~
Only on condition that you keep up the good work :)

Bruce B

#59 Post by Bruce B »

Using too many commands (words)

I didn't know how to get the anti-theft device unstuck in my Ford. I called an expert mechanic. Have you ever met people who use too many words, to say what they want? He took about seven minutes to say what could have been said like this:
  • Close the hood and all doors. Take the key and open the driver door to defeat the alarm.
The same is true in programming. Programmers can use too many words.

After you write a successful program, go back and refine it. I give an
example from my own work.

Code: Select all

#!/bin/bash

   # get mounted filesystems 
   mntdirs=`cat /proc/mounts | grep /mnt/ \
   | cut -d " " -f 2 | tr "\n" " "`
   echo $mntdirs 
   
   # get mounted filesystems - REFINED FOR SPEED/EFFICIENCY 
   mntdirs=`</proc/mounts grep /mnt/ \
   | cut -d " " -f 2`
   echo $mntdirs 
In the refined block we have eliminated two external commands,
cat and tr.

Just because you finally wrote a script that really does what you want,
doesn't mean it's efficient. This script snippet must have been written
years ago, before I learned some of bash's nuances.

A long script which has been optimized will run much faster.

This program will show you the directories mounted on /mnt

The mount command will show all mount points

Hint: We want to give our scripts meaningful names. Don't write a script
named 'test' because test is a command.

Remember, type in the name you intend to use, before giving the script
a name.

I'll call this script deleteme.

You can download it and run it as you please.

~
Attachments
deleteme.zip
(274 Bytes) Downloaded 574 times

Bruce B

#60 Post by Bruce B »

What is a period?

Microsoft innovated. This means they looked at what others did and
copied them.

In MS-DOS a period:

* meant the current directory
* it was also used to separate the filename from the extension

--------

(Unix, of course, came long before MS)

In Linux it means the same thing and potentially much more.

To open rox from the commandline in the current directory, type

rox .

To open two instances of rox in the current directory, type

rox . .

Two periods mean the parent directory

To open rox in the current directory and another instance in the
parent directory, type

rox . ..

To make an empty file with an extension, you could type

touch filename.tmp

ls filename.tmp to verify it exists

then

rm -v filename.tmp to delete it

Linux won't let you move or copy files from directory to directory
without providing it with the source and target directories. Using the
period saves typing, because . is the current directory.

If I want to copy /mnt/sda2/foo/bar to the current directory the
command would be

cp /mnt/sda2/foo/bar .

conversely, if I want to go the other way, I type

cp ./bar /mnt/sda2/foo

same syntax with the mv command

What is a period really to the bash programmer?

Potentially, it is something which could drive her insane.

A backslash "\" primer

A typical escape character is \

If a character has a special meaning to bash or other Linux utilities,
we can often make it have a literal meaning by 'escaping' it.

The \ is a common escape character, which says treat the character
to the right literally.

Here is an example, we want to echo to the console, the text below:

You didn't enter a "filename"

The apostrophe will throw us off. We need to quote the command, like this:

echo "You didn't enter a "filename""

It won't work. We need to tell echo which "s (quotes) to treat
literally, we do this with the escape character \

This works

echo "You didn't enter a \"filename\""

Consider this a primer on periods and backslashes, it gets worse,
but at a minimum, this needs much needs understanding.

~

You can try the commands in dark blue on the command line. Except
for the substitutable ones with foo and bar.

If you enter this command which is shown above it will take you to a
PS2 prompt

echo "You didn't enter a "filename""

"Escape" it with Ctrl+C

~

Bruce B

#61 Post by Bruce B »

Shell Expansion

an alias is written like this:

alias myname='echo Bruce'

If I type myname, it will say Bruce on the console.

If I write the alias below I don't need the apostrophe

alias cls=clear

The reason why I don't need an apostrophe is because there are no
spaces in the alias.

Spaces are command delimiters. If we don't tell bash that our
command string with spaces is to be treated as a whole, it will treat
it as individual instructions each time it encounters a space and
bomb out.

I want to demonstrate variable expansion.

1) echo '$PATH'

2) echo "$PATH"

3) echo $PATH

You can select the command, the click you mouse wheel into the
console to save typing.

~

Bruce B

#62 Post by Bruce B »

Keeping File Names Clean

My theory is many FOSS authors actually try and make Linux
friendly to Windows users. Many Linux utilities and programs now
work on 'dirty' Windows file names. Years ago, many Linux apps
would bomb out on bad file names.

Which ones and how many? I don't know. Knowing would be a
discovery process.

What I do is I clean up dirty filenames, so they will work with any
Linux utility or program and make our scripting easier.

Here is what I consider dirty filename for Linux

Elton John - I Don't Wanna Go On With You Like That.mp3

Here is the cleaned filename

Elton_John_-_I_Dont_Wanna_Go_On_With_You_Like_That.mp3

The worst part of the filename was the apostrophe in the word Don't

I think you will find the 'cleaned' file as easy to read as the 'dirty'
one.

If I keep my filenames 'clean'. I don't have to worry about what a
script will do when it processes them.

About six years ago I donated a script, one I didn't write, to Puppy.
It's name is spacereplace. Barry still includes it.
Use it to replace spaces with underscores.

For fine editing, the utility 'ytree' has been the best one I've found,
for ease of use.

Hint: Open it in the directory you want to work in. Hit enter to bring
all the files into a more full window. The rest is intuitive.

Compiling is straight forward

./configure
make install


Don't use the one hosted on Puppy's archive, something is wrong.
Also, it might be an older version.

Get it here

http://www.han.de/~werner/ytree.html


~

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#63 Post by jpeps »

Bruce B wrote:Using too many commands (words)

Code: Select all

#!/bin/bash

   # get mounted filesystems 
   mntdirs=`cat /proc/mounts | grep /mnt/ \
   | cut -d " " -f 2 | tr "\n" " "`
   echo $mntdirs 
   
   # get mounted filesystems - REFINED FOR SPEED/EFFICIENCY 
   mntdirs=`</proc/mounts grep /mnt/ \
   | cut -d " " -f 2`
   echo $mntdirs 
~
Hi Bruce,

Probably getting beyond a beginners tutorial, but....

Really, you're just substituting one command for another, "cat" for "<" ("<" is just a redirect command). Both do pretty much the same thing.

Regarding the "tr" command, I don't think it's doing anything in the first example anyway, so you could get rid of it. VAR=`cat /file` acts as a redirect with no newlines.

Code: Select all

VAR=`cat /file`
echo $VAR
echo
VAR=`</file`
echo $VAR

Bruce B

#64 Post by Bruce B »

jpeps wrote: Hi Bruce,

Probably getting beyond a beginners tutorial, but....
Hi jpeps,

You are reading and thinking, what more could I want? If everyone did
that, I'd be overjoyed.
jpeps wrote:Really, you're just substituting one command for another,
"cat" for "<" ("<" is just a redirect command). Both do pretty much the
same thing.
What I wrote was:
  • In the refined block we have eliminated two external commands,
    cat and tr.
The emphasis would have been eliminated two external commands.

You're right about the tr being useless, but obviously when I wrote the
script I thought I needed it.

If we have a choice between an internal command and an external
command - everything equal - we want the internal command. The
reason why is - it is already there. An external command has to be
located and copied to memory to be used.

I think redirection and piping may not even be commands in a
conventional sense. Rather an integral part of the kernel's I/O
(input/output) functions. I could be wrong.

I just did a little searching to determine for sure, but didn't come up with
anything conclusive or inconclusive.

I'll leave it as another unanswered ??? among my myriads of unanswered
questions, until and if I learn the answer.

~

Bruce B

#65 Post by Bruce B »

Making Life Easier

Our Linux system may contain 2 or 3 thousand executable files in our
path statement.

I don't remember the names of all the files.

I might be able to remember part of the name. If I remember the first
letters, tab completion can help me.

Otherwise?

Well these files are stored in numerous bin and sbin directories, and it is
a lot of work finding them by reading all the filenames in these directories.

Here is my idea of a helpful script

Code: Select all

#!/bin/bash

# we generally want to initialize our variables at the top of the file

[ ! $1 ] && echo "Argument missing, exiting" && exit

MODPATH=`echo /root/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:\
/usr/X11R6/bin:/usr/lib/qt/bin:/usr/local/sbin:/usr/sbin:/sbin`

BINDIRS=`echo "$MODPATH" | tr : " "`

# did it work? test

#echo $MODPATH
#echo $BINDIRS

#exit

# it worked, I now have all my bin directories stored in BINDIRS separated
# by spaces, which I will need for the for loop

for i in $BINDIRS
do
	find $i -maxdepth 1 -type f | grep $1
done

# I discovered some directories in the path statement don't exist
# /opt/bin/ /usr/local/games/ /usr/lib/java/jre/bin/
# I could modify the path statement but I'll make the mods in the script
# by removing these directories
# I added MODPATH to accomplish this

Without the comments our simple but useful script looks like this:

Code: Select all

#!/bin/bash
[ ! $1 ] && echo "Argument missing, exiting" && exit

MODPATH=`echo /root/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:\
/usr/X11R6/bin:/usr/lib/qt/bin:/usr/local/sbin:/usr/sbin:/sbin`

BINDIRS=`echo "$MODPATH" | tr : " "`

for i in $BINDIRS ; do
	find $i -maxdepth 1 -type f | grep $1
done
Remember we want to give our programs meaningful names

Linux comes with a utility called "which", but to use it, we need to know
the filename. And it's for locating the path of a known filename.

I call this script whichx, which means to me - which-extended

If I type in this command

whichx cups it will show me the full path and
name of every file containing the text "cups"

Note the use of the escape character \ at the end of the first line of
MODPATH. Bash uses the end of line to signal end of command. If we
wish to keep long lines short we can use \ to tell bash the command
continues to the next line. You cannot have any spaces after the \.


I've tested whichx and included it as a downloadable script.

You will have to adjust or at least test the variable called MODPATH to
make it right for your system.


~
Attachments
whichx.zip
(350 Bytes) Downloaded 502 times

Post Reply