The Text-User Interface

Using applications, configuring, problems
Message
Author
Bruce B

#41 Post by Bruce B »

Chapter 43 - Ispell

Ispell is an easy to use text interactive spell checker to enhance your GUI environment, making it even better than it already is.

To check a document ispell <filename>

To check spelling on a few words with keyboard input ispell

Some theory, about spell checkers.

You don't want the biggest dictionary for spell checking documents. A medium size dictionary is best because the big one has so many uncommon words, it might not flag a typo, if the typo happens to be spelled same as a rare word.

If you are good speller, a dictionary which doesn't make a lot of suggestions is also fine.

---------------

I compiled this spell checker to correspond with our directories.

Source file location was here: http://ficus-www.cs.ucla.edu/geoff/ispell.html

Should you prefer to do your own compile. The compile is easy, but there are some oddities, so read the documentation and think.

Installation of this package


mv ispell.tar.lzma.zip ispell.tar.lzma

forum won't allow the lzma extension, please consider this an exercise in frustration and I won't to it again.

lzma -d ispell.tar.lzma
tar xvf ispell.tar

install files according to the included README


Chapter 43 - Ispell

Bruce B

ls command

#42 Post by Bruce B »

Chapter 44 - ls command

Sorry, no chapters the last few days, I've been down with the flu. These chapters require more of me than a serious flu allows. I think I'm back!

Here we go; I introduce the ls command. A very basic and essential command.

The ls command without switches will give you a sorted five column directory display of all your 'not-hidden' files and directories, providing you don't have names longer than 15 on a (typical) 80 column
display. ( did you catch that? )

ls will determine the number of sorted columns displayed by the longest file name or sub-directory name in the current.

With this understanding, you might want to limit the length of your files.

Using ls' switches, I'll introduce the ones we plan to use, but there are many more, to see them;

ls --help

Code: Select all

Usage: ls [OPTION]... [FILE]...
List information about the FILEs (the current directory by default).

  -A, --almost-all           do not list implied . and ..

  -d, --directory            list directory entries instead of contents,
                                    and do not dereference symbolic links

  -F, --classify             append indicator (one of */=>@|) to entries

  -l                              use a long listing format

  -1                             list one file per line

Exit status:
 0  if OK,
 1  if minor problems (e.g., cannot access subdirectory),
 2  if serious trouble (e.g., cannot access command-line argument).

-----------------------------

Note: -A, -d and etc are command line arguments, I also call them switches because of the -, which says in effect 'switch behavior'.

When type ls, we get our wide listing, and we can't tell the difference between a directory or a file. (unless we have color enabled, but we've not got there yet)

Let's make an alias to change behavior

alias l='ls --almost all --directory --classify'
# plus user arguments appended when running above alias

You can put the new alias in /etc/profile.local or /root/.bashrc

This alias switch --classify places a / at the end of the directory name, thus differentiating between a file, as you can see by the description

~

Chapter 44 - ls command

Bruce B

user script an ls enhancement

#43 Post by Bruce B »

Chapter 46 - user script an ls enhancement


lsf is short for ls find

lsf is a user made utility for finding files in your current directory by merely typing in portions of the file name. It accepts two arguments.

In /usr/bin lsf du gives us these possible files, because each file has the text string du

aseqdump*
db_dump*
du*
gsf-vba-dump*
gtk-query-immodules-2.0*
hexdump@
isodump*
objdump*
objdump86*
pango-querymodules*
pcprofiledump*
winedump*

lsf ^b gives: this list, because they all start with b
basename@
bc*
bcc*
bdftops*
bf_compact*
bf_copy*
bf_tar*
bison*
blinky*
blinkydelayed*
bochs*
bochs-dlx*
bogofilter*
bogolexer*
bogotune*
bogoupgrade*
bogoutil*
bxcommit*
bximage*
bzcmp@
bzdiff
bzegrep@
bzfgrep@
bzgrep*
bzip2recover*
bzless@
bzmore

lsf ^b c gives:

bc*
bcc*
bf_compact*
bf_copy*
bochs*
bochs-dlx*
bxcommit*
bzcmp@
bzip2recover*

( note @ means symlink , * means real executable file )

Here is lsf, the script

Code: Select all

if [ $2 ] ; then
    ls --classify --directory * \
    | grep $1 | grep $2
    exit
elif [ $1 ] ; then
    ls --classify --directory * \
    | grep $1
else
    echo "lfs : usage up to two"
    echo "search criteria"
fi
New for us is the keyword elif. We can't use if two times as a 'keyword' in the same if statement. We can use elif multiple times after the first if.

Code: Select all

count    keywords
1)         if
many)   elif
1)         else
1)         fi
many)   then one for each if or elif
True conditions call for 'then' executions. False conditions are ignored.

Is else true of false? It can be either, depending if any conditional tests above were true or false. If any were true, else if false. If all were false, else is true.

Can you see why I tested $2 before $1?

If $2 is true, $1 will be true. This is also why I exited $2 after running the commands.

Unrelated Reminder: grep ^matches first character of line. grep $ matches last character of line.

Assignment: download, install, practice and try and understand the file. More to come!

~

Chapter 46 - user script an ls enhancement
Attachments
lsf.zip
(271 Bytes) Downloaded 1814 times

Bruce B

#44 Post by Bruce B »

Chapter 47 - formatting output

A programmer or power user (you) should know the ls command quite well. Here is a bit more training

ls outputs in columns, sorted ascending down each column from left to right. If filenames are too long, we don't get as many columns. The column view shows a lot of files for the space used, but doesn't give many file details.

Code: Select all

ash            dd          getopt      mount-FULL        stty
autologinroot  delgroup    grep        mv                su
awk            deluser     gunzip      ncurses5-config   systool
bash           df          gzip        ncursesw5-config  tack
bashnewer      df-FULL     hostname    netstat           tar
bash-puppy     dlist_test  infocmp     nice              tic
bunzip2        dmesg       infotocap   ntfs-3g           toe
busybox        dpkg-deb    ip          pidof             touch
busybox.1st    dumpkmap    ipaddr      ping              tput
busybox.bak    e3          ipcalc      ping6             true
bzcat          e3em        iplink      pipe_progress     tset
bzip2          e3ne        iproute     ps                tst
captoinfo      e3pi        iprule      psf               umount
cat            e3vi        iptunnel    ps-FULL           umount-FULL
chattr         e3ws        kill        pwd               uname
chgrp          echo        last        readlink          uncompress
chmod          ed          ln          README-mount.txt  usleep
chown          egrep       loadkeys    reset             uuidgen
clear          false       login       rm                waitmax
compile_et     fdflush     ls          rmdir             zcat
cp             fgrep       lsattr      rpm
cp-FULL        find        mk_cmds     sed
cpio           gawk        mkdir       setserial
csh            get_device  mknod       sh
ls -l, lots of file information but the number of files is limited by the number of available lines on our terminal.

Code: Select all

-rwxr-xr-x 1 root root  14996 2007-10-18 18:45 setserial
lrwxrwxrwx 1 root root      4 2009-04-29 19:29 sh -> bash
-rwxr-xr-x 1 root root  12092 2007-10-22 03:14 sleep
-rwxr-xr-x 1 root root  47728 2007-10-18 18:38 sort
lrwxrwxrwx 1 root root      7 2009-04-29 19:29 stty -> busybox
lrwxrwxrwx 1 root root      7 2009-04-29 19:29 su -> busybox
-rwxr-xr-x 1 root root  15708 2007-10-18 18:45 systool
-rwxr-xr-x 1 root root 122656 2007-10-18 18:39 tack
-rwxr-xr-x 1 root root 188208 2007-10-18 18:39 tar
-rwxr-xr-x 1 root root  39584 2007-10-18 18:39 tic
-rwxr-xr-x 1 root root  26492 2007-10-18 18:39 toe
-rwxr-xr-x 1 root root  27080 2007-10-18 18:38 touch
-rwxr-xr-x 1 root root   8356 2007-10-18 18:39 tput
lrwxrwxrwx 1 root root      7 2009-04-29 19:29 true -> busybox
-rwxr-xr-x 1 root root  30972 2007-10-18 18:39 tset
-rwxr-xr-x 1 root root    457 2008-12-27 11:48 tst
-rwxr-xr-x 1 root root   2461 2007-04-21 17:11 umount
-rwsr-xr-x 1 root root  26948 2007-10-18 18:38 umount-FULL
-rwxr-xr-x 1 root root  11944 2007-10-18 18:38 uname
lrwxrwxrwx 1 root root      7 2009-04-29 19:29 uncompress -> busybox
lrwxrwxrwx 1 root root      7 2009-04-29 19:29 usleep -> busybox
-rwxr-xr-x 1 root root   3944 2007-10-18 18:43 uuidgen
-rwxr-xr-x 1 root root  16336 2008-03-15 20:07 waitmax
lrwxrwxrwx 1 root root      4 2009-04-29 19:29 zcat -> gzip
In the example below, we simply remove some displayed data from our ls long listing, by using the cut utility. For purpose of demonstrating 'cut'.

ls -l | cut -b 30-100 gives this, instead of the listing above

Code: Select all

  14996 2007-10-18 18:45 setserial
      4 2009-04-29 19:29 sh -> bash
  12092 2007-10-22 03:14 sleep
  47728 2007-10-18 18:38 sort
      7 2009-04-29 19:29 stty -> busybox
      7 2009-04-29 19:29 su -> busybox
  15708 2007-10-18 18:45 systool
 122656 2007-10-18 18:39 tack
 188208 2007-10-18 18:39 tar
  39584 2007-10-18 18:39 tic
  26492 2007-10-18 18:39 toe
  27080 2007-10-18 18:38 touch
   8356 2007-10-18 18:39 tput
      7 2009-04-29 19:29 true -> busybox
  30972 2007-10-18 18:39 tset
    457 2008-12-27 11:48 tst
   2461 2007-04-21 17:11 umount
  26948 2007-10-18 18:38 umount-FULL
  11944 2007-10-18 18:38 uname
      7 2009-04-29 19:29 uncompress -> busybox
      7 2009-04-29 19:29 usleep -> busybox
   3944 2007-10-18 18:43 uuidgen
  16336 2008-03-15 20:07 waitmax
      4 2009-04-29 19:29 zcat -> gzip
Excercise

type cut --help (review help)

see how other programmers find usage for cut

cd /etc
grep --files-with-matches "cut " *

view files that match and note cut usage

~

Chapter 47 - formatting output

Bruce B

#45 Post by Bruce B »

Chapter 48 - more formatting

Important Lectures and Theory

Basic categories of common and core utilities:

1) full version utilities

2) cut down utilities (which Puppy uses a lot of them

3) non-existent utilities - some scripting lessons you encounter will have you use an occasional utility which Puppy doesn't have. Not a problem, because you can troubleshoot it with the which command

4) utilities some scripting lessons won't let you use, because they don't trust you as root. Not a problem for us, is it?

------------------

Puppy has for all its history used many of the cut down versions of utilities. The reason why is obvious, to keep the distribution small by saving many megabytes over the larger utilities, man and info pages.

No, you cannot simply start replacing utilities on a one for one basis for the full versions.

The BusyBox utilities may not support the input switches and features we want. Just as important, they don't output exactly the same as the full versions, the predictive text formatting Puppy uses won't be right.

And by not using the BusyBox utilities in the right places, you can cause Puppy to make dumb decisions or even fail to boot.

Remember back to an earlier lesson about the PATH statement. Search left to right. We are at the end of the search for safety. If we put full featured utilities in our personal bin directory /root/bin, they will only be used if no match was found.

Here, I show you how to use full featured utilities in your personal directory with your personal scripts.

After finishing this lesson you will have three new full featured files:

/root/bin/tr
/root/bin/cut
/root/bin/head


In order for them to be used in the script, they need to be called by their full name. Our script's purpose is to find out how much physical memory your computer, (as if you didn't know already). Nevertheless, it will display it for you.

I'm calling the script mypc01, because I'd like to evolve it numerically, into something which gives even more useful information, and provides learning opportunities for you.

Let's look at the script.

Code: Select all

#!/bin/bash
utildir="/root/bin"
echo
kb=`< /proc/meminfo ${utildir}/head -n 1 \
| ${utildir}/tr --squeeze-repeats " " \
| ${utildir}/cut --delimiter=" " --fields=2`

echo " My computer has $kb Kb fixed RAM"
echo
echo " How many megabytes is $kb Kb?"
echo

#mb=`expr $kb / 1024`
#mbsell=`expr $kb / 1000`

mb=$(($kb/1024))
mbsell=$(($kb/1000))


echo " If division by 1024 we have $mb Mb"
echo " If division by 1000 we have $mbsell Mb"
echo

Note our variable utildir, now our full featured utilities will be used by this and probably other scripts in the future.

Note the arithmetic calculations. Which are correct? You will see both in use

Assignment required

Download utils.tar.gz, extract contents to /root/bin

Practice with the script and try and understand it.

~

Chapter 48 - more formatting

Bruce B

#46 Post by Bruce B »

Chapter 49 - Eliminating trailing white space

I had a lengthy script, which ran very successfully. I modified it from time to time. Then one little change and it stopped running. I went over and over the change and couldn't find a thing.

I went over every aspect of the script and still could not find the trouble. I opened it with a hex editor to view things which a text editor wouldn't show and still didn't find the cause.

I eventually thought maybe if it wasn't anywhere in the printing characters, it was likely a problem caused by non printing characters.

In the vast majority of cases when your scripts don't do as intended the problem is in the printing characters. The problem I describe is very rare and maybe shouldn't even be mentioned . . .

. . .but I suppose if it happened to me, it might happen to you. Also, one thing we have very little use for in our scripts, text documents and web pages is trailing white space.

The only files we ever want to remove trailing white space on are text files. Scripts, and html files are text files.

For starters in building this script, let's see how to tell if the intended file is an actual text file. file will tell it if it is.

file index.html, says: HTML document text
file updatedb, says: Bourne-Again shell script text executable

we can use grep's return code to see if it is safe to remove trailing spaces

~~~~~~~~~~~~~~


line 1 : file $1 | grep " text" >/dev/null
line 2 : [ x$? = x0 ] && echo $1 is a text file


line 1 : file processes the file in $1, we pipe the output to grep, grep looks for this text string" file" and we redirect the output to the black hole >/dev/null

grep gave bash an invisible return code based on if it found the string " file", we use the return code in line 2

line 2 : we test the return code to see if it matches 0. Earlier I showed arithmetic comparison operators. The = is not for arithmetic, it's more for string comparisons. When making comparisons we never want to compare one string with a blank. By using the x, either side of the string comparison are assured of having some kind of value as a place holder.

~~~~~~~~~~~~~~

String comparison - not arithmetic

Does x0 == x0 ? If true it is a text file

Earlier I used quotes, like this:

[ "$?" = "0" ] && echo $1 is a text file


The quotes serve the same purpose as the x - you're developing you own style.

We will work more on our script in the next chapter

~

Chapter 49 - eliminating trailing white space

Bruce B

#47 Post by Bruce B »

Chapter 50 - Introducing 'notrail'

notrail strips trailing tabs and spaces from text documents. This includes scripts and html files. It also saves the originals

Code: Select all

#!/bin/bash
main() {

	var "$@"
	process "$@"

}

var() {

	[ ! $1 ] && echo "missing filespec" && exit

	savedir=/var/notrail
	[ ! -d $savedir ] && mkdir $savedir


	subdir=`date "+%Y%m%d-%H%M"`
	savedir=${savedir}/$subdir

	[ ! -d $savedir ] && mkdir ${savedir}

	[ ! -d $savedir ] && echo "$savedir doesn't exist, exiting" && exit

	logfile="${savedir}/LOG_FILE.LOG"

}

process() {

	echo "Working, please wait . . ."

	for i in "$@" ; do

		[ ! -f $i ] && continue

		file $i | grep  text >/dev/null
		if [ "$?" == "0" ] ; then
			mv $i $savedir
			<${savedir}/$i sed 's/[ \t]*$//'>$i
			echo "Cleaned $i" >> $logfile
		else
			echo "$i not a text file">>$logfile
		fi

	done


	echo
	echo "Original files saved in $savedir"
	echo "Directory contents: "
	ls $savedir


}


main "$@"
sed

A new utility introduced here is 'sed', a stream editor. You won't learn it overnight, but you will learn it in time. Try and not be overly anxious with this one.

I've provide some support material for you

A link from where I got the snippet used in this script, Handy one liners for sed

Also, use the forum search to see how others have used sed.

~

Please download in install attachment notrail

Edited: Attachment removed for a better utility, see Chapter 80 and download cleanf.zip



Chapter 50 - Introducing 'notrail'
Last edited by Bruce B on Tue 04 Aug 2009, 08:34, edited 1 time in total.

Bruce B

#48 Post by Bruce B »

Chapter 51 - I really want to keep these chapters moving. Lots more to come.

I've been down sick bad. Apart from finding energy when sick, the creative aspects needed aren't there when I want them.

I want you to know I'm still with it.

-----------------

If any have questions, or problems, please PM me.

Bruce B

#49 Post by Bruce B »

Chapter 51 - swapf

Guess what we are going to do? That's right. Make another script.

Hint: If you haven't noticed by now, the way to learn how to script and program, is by scripting and programming. You just need a few people to help you get started.

Some times we want to swap files, one file with the other. It takes about three command line operations. If for purpose of testing you swap back and forth a few times, it takes several commands.

The basic command line ( and program flow is like this )

rename file A to a non existent name
rename file B to file A's original name
rename, the renamed file A to the original name of file B


We already learned it's good to have some sanity and error checking, and it's good to have some on screen feed back.

When a command prompt merely returns, without feedback, it leaves sort of a feeling of a vacuum, did it do like we wanted? With good screen feedback, any wondering is diminished greatly.

~~~~~~~~~~

We now have a rough idea what we want our script to do. (0) give a short yet descriptive name, (1) swap file names, (2) do some error checking before doing the swap, (3) give us on screen feed back.

This is an easy script, so lets get started

SCRIPT OUTLINE

Code: Select all

main() {

}

vars() {

}

before() {

}

after() {

}
swapem() {

}

main $1 $2
Note what I did above, was put everything I want my script to do according to descriptive functions. All I have to do is fill out some simple commands in the appropriate functions, watch how easy it is. Even complex programming can be simple when broken down into small parts.

The actual script

Code: Select all

#!/bin/bash

main() {

    vars $1 $2
    before $1 $2
    swapem $1 $2
    after $1 $2

}

vars() {

    [ ! $2 ] && echo "Insufficient arguments" && exit
    [ ! -f $1 ] && echo "Filename $1 doesn't exist, exiting" && exit
    [ ! -f $2 ] && echo "Filename $2 doesn't exist, exiting" && exit
    echo $1 | grep " ">/dev/null
    [ x$? = x0 ] && echo "Come on, spaces in file names are for Windows users" && exit
    echo $2 | grep " ">/dev/null
    [ x$? = x0 ] && echo "Come on, spaces in file names are for Windows users" && exit

}

before() {

    echo ""
    echo "Before swapping filenames:"
    ls -l $1
    ls -l $2
    echo ""

}

after() {

    echo "After swapping filenames:"
    ls -l $1
    ls -l $2
    echo ""
}

swapem() {

    mv -i $1 $1.$$
    mv -i $2 $1
    mv -i $1.$$ $2

}

main $1 $2
Script notes and comments

We could name Windows spaced files, but I don't want to in this script.

$$ is the pid of the child process

If $1 is busybox, (as an example), it gets named like this: busybox.3189, odds are extremely in our favor no such file name exists. Extra insurance is the use of mv's -i (--interactive) switch to prevent an unintentional overwrite.

When we want to use a command line argument such as $1 or $2 we need to pass it to each function which uses it. Or set an internal variable, which we didn't do in this example.

With the on screen feedback, you can easily verify the files were swapped properly by comparing the before and after times, dates and sizes and relative positions on the screen.

Note: your functions can be called in any order main says. The 'after' function came before the 'swapem' : it doesn't matter.

~

Chapter 51 - swapf


Instructions: download and install the attached script, if you want it.
Attachments
swapf.zip
(445 Bytes) Downloaded 1237 times

Bruce B

#50 Post by Bruce B »

Chapter 52 - more grep

I noted this question in the beginners section,
  • "I can't find an easy way to change the computer name, I can view it in the system information viewer but cant find how to change the details."
The computer 'name' is puppypc and the GUI application hardinfo doesn't say how it arrived at that name.

If we understand that the directory /etc mainly holds system configuration information, we have a good place to start looking.

We would want to find out which files have the word puppypc in them to get an idea where to make the changes we want to make. Here is some practice using grep on the command line.

cd /etc : stay in this directory for the exercises below

grep puppypc * : look in all files, case sensitive search, display instances with file names

grep -i puppypc * : look in all files, case insensitive search, display instances with file names

grep -il * : as above but display only the file names

grep -r puppypc * : search in subdirectories also
or
grep -R puppypc * : search in subdirectories also

grep -ilr puppypc * : is valid, meaning we can mix our switches

Please practice these commands. The command line user needs to be well versed with the basics and intermediate usage of grep.

The examples I gave above use the short switches, which is what we mainly use on the command line. In scripts the long hand makes the script easier to read and we only type the script commands one time, (hopefully). Although short switches are frequently used in scripts, but it's a nice practice to use long names.

Short and long switches used in this chapter for examples

-i , --ignore-case
-r , --recursive
-R , --recursive
-l , --files-with-matches


~

Chapter 52 - more grep

Bruce B

#51 Post by Bruce B »

Chapter 53 - colorful prompts

Here's the command list for file, title: ps1colors

Code: Select all

case $1 in 

nor*) export PS1='[\w] '  ;; #  normal
bla*) export PS1='\[\e[1;30m\][\w]\e[0m\] '  ;; #  black
whi*) export PS1='\[\e[1;37m\][\w]\e[0m\] '  ;; #  white

red*) export PS1='\[\e[1;31m\][\w]\e[0m\] '  ;; #  red
gre*) export PS1='\[\e[1;32m\][\w]\e[0m\] '  ;; #  green
blu*) export PS1='\[\e[1;34m\][\w]\e[0m\] '  ;; #  blue

cya*) export PS1='\[\e[1;36m\][\w]\e[0m\] '  ;; #  cyan
mag*) export PS1='\[\e[1;35m\][\w]\e[0m\] '  ;; #  magenta
yel*) export PS1='\[\e[1;33m\][\w]\e[0m\] '  ;; #  yellow

*) echo "Usage:"
echo "ps1 <nor|bla|whi|red|gre|blu|cya|mag|yel>" ;; #  help

esac
Its obvious purpose is to change our prompt colors on the fly. A not obvious purpose is I plan do something with colorful prompts in a future lesson.

Remembering from previous lessons, there are some things we can't easily do in a 'child shell', this is one of them. If we changed our prompt colors in a child shell, the colors would change and then return to the previous state when the child exits.

Also, the 'export' command (which hasn't been discussed) doesn't work like we'd think it should in an X window terminal emulator.

FYI : The variable PS1 would mean to say - this is for our primary prompt.

The data in these variables would be hard to read without understanding bash' escape sequences and having an ASCII color chart on hand.

We can change prompt colors on the fly from the command line, but because we can't easily do it in a child shell, we run the commands by 'sourcing'.

Sourcing has been discussed earlier, but I hardly think it hurts to discuss it a little more. One reason why, is: Some of the scripting tutorials you will find online and study, barely scratch the surface of sourcing.

We use this alias: alias ps1='source /root/bin/ps1colors'

To install:

copy and paste the alias in two files, (1) /etc/profile.local , (2) /root/.bashrc
unzip ps1colors to /root/bin

To run:

I've made it pretty easy, you type lowercase ps1 and the first three letters of the color you want. If you type only ps1 you will get onscreen help

ps1 blu
ps1 cya
ps1 red


Please consider this required for an upcoming lesson. And I hope you have fun using it in the meantime.

~

Chapter 53 - colorful prompts
Attachments
ps1colors.zip
(365 Bytes) Downloaded 1142 times

Bruce B

#52 Post by Bruce B »

Chapter 54 - Minimum Profit configuration file

Optional


Puppy gives us an excellent text editor, Minimum Profit. (mp). And, as command line users, we want lots of good applications.

Minimum Profit (mp) has a highly configurable rc file. I've written an rc file which I think you will like better than the default configuration file.

The name of the file is /root/.mprcx
, but it needs to be /root/.mprc to be recognized and used by mp.

To install it download mprc.zip to /root and unzip it. You may not see the .mprcx file, because it is a 'hidden' file.

Then use your swapf utility from an earlier chapter and run this command:

swapf .mprc .mprcx

You can run the same command over and over to swap the configuration files to see which you like best. Also, you can modify the files.

If you want to use mc as your default Midnight Commander editor

F9 -> Options -> Configuration . . . -> uncheck 'use internal edit', then Save

~

Chapter 54 - Minimum Profit configuration file
Attachments
mprc.zip
(3.91 KiB) Downloaded 1166 times

Bruce B

#53 Post by Bruce B »

Chapter 55 - an automated e2fsck (part 1)

If you don't have Linux partitions, you can't do this exercise. I intend it to become a Puppy portable script for automatically checking ext2 and ext3 partitions.

Here is a portion.

Code: Select all

#!/bin/bash
main() {

	# getlist_of_partitions
	# getlist_of_linux_partitions
	# cut_first_field_from_list
	# put_list_into_one_line
	# put_one_line_list_into_variable

}


getlist_of_partitions() {

	
	echo "function: getlist_of_partitions"
	echo ""
	fdisk -l 

}

getlist_of_linux_partitions() {

	echo ""
	echo "function: getlist_of_linux_partitions"
	echo ""
	fdisk -l | grep "83  Linux"

}

cut_first_field_from_list() {

	echo ""
	echo "function: cut_first_field_from_list"
	echo ""
	fdisk -l | grep "83  Linux" | cut -d " " -f 1

}


put_list_into_one_line() {

	echo ""
	echo "function: put_list_into_one_line"
	echo ""
	fdisk -l | grep "83  Linux" | cut -d " " -f 1 | tr "\n" " "
	echo " " # for on screen formatting only

}

put_one_line_list_into_variable() {

	echo ""
	echo "function: put_one_line_list_into_variable"
	echo ""
	partitions=`fdisk -l | grep "83  Linux" | cut -d " " -f 1 | tr "\n" " "`
	# display data in variable
	echo "variable \"partitions\" contents: $partitions"
}


main
Notes:

This script will do nothing, at first, because the commands in main have # before the commands. To run a function or multiple of functions remove the #

The only function we need is the last one, it does it all. But in the event it is not apparent to you want it does, the logic which builds up to it is displayed in the previous functions.

Exercise:

Copy and paste the code to a practice script. Experiment with it by removing and reinserting the comments in main. Try and understand exactly what we are doing. Namely piping utilities to utilities, until we have exactly the data we need for the latter parts of or our script.

Note also, the script is (and will be) portable, it should run on any Puppy computer. The reason I say 'Puppy' is because some Linux don't let us run root easily and fdisk is a root command.

Note: although not common practice, you can be very descriptive with function names.

We'll work on the script some more, in upcoming lessons.

~

Chapter 55 - an automated e2fsck (part 1)

Bruce B

#54 Post by Bruce B »

Chapter 56 - hexedit

Part of building a good text user interface is having plenty of good TUI applications. Attached to this post is [N]Curses Hexedit 0.9.7 by Adam Rogoyski

There might be a slightly newer version available, than the one attached. (It compiled very easily in Puppy.)

http://www.rogoyski.com/adam/programs/hexedit

http://packages.qa.debian.org/n/ncurses-hexedit.html

Code: Select all

Archive:  hexedit.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
    24180  07-18-09 14:05   hexedit
    12256  07-18-09 14:04   hexedit.1.html
    27013  07-18-09 14:03   hexedit.info
 --------                   -------
    63449                   3 files

The man page is converted to html
The info page can be read with a text editor

----

Puppy has an editor of the same name, rename it to a different name. The reason for the replacement is because this editor is so much easier to work with. Until you reboot, you may have to run it by full path.

Enjoy

~

Chapter 56 - hexedit
Attachments
hexedit.zip
(25.5 KiB) Downloaded 1161 times

Bruce B

#55 Post by Bruce B »

Chapter 57 - more about loops

I knew a fellow who while studying beginning C, couldn't for the life of him grasp the reason for using loops in programming.

In this example, I hope to make it very clear why we may want to use loops. As an additional bonus, I wish to show how we can make it look as if we are doing arithmetic when all we are doing is echoing arguments.

Code: Select all

#!/bin/bash
for i in 0 1 2 3 4 5 6 7 8 9 ; do
	for x in 0 1 2 3 4 5 6 7 8 9 ; do
		for z in 0 1 2 3 4 5 6 7 8 9 ; do
			echo $i$x$z
		done
	done
done
What we are looking at are three 'for loops' - nested. Nesting is a word and a concept the new programmer needs to become familiar with.

The indents help us easily identify where the nesting occurs.

This simple routine will count from 000 to 999 much, much faster, than if we used arithmetic to do it.

If we wanted to do the same thing without loops we would need 1000 lines to do what we can do with 8 lines.

Main thing in this chapter is understand what is presented, and of course feel free to copy, paste, play and practice.

~

Chapter 57 - more about loops

PS If you didn't catch the flow in the loops, try these:

Code: Select all

for i in a b c d ; do
    for x in 1 2 3 4 ; do
        echo $i$x
    done
done
and

Code: Select all

for x in 1 2 3 4 ; do
    for i in a b c d ; do
        echo $x$i
    done
done
And your own variants, until you really do understand.

Bruce B

#56 Post by Bruce B »

Chapter 58 - guess

Thousands of files, I forget the name, but I'm going to use tab completion to help remember. I remembered it started with the letters g-u-e-s-s

Here is where tab completion comes in and makes life easier for us command line users. I typed in the letters I remembered then hit the tab key which filled in the rest, the filename I want is guess_fstype.

We can almost tell what this file does by its name, which is a bit unusual for Linux. Let's see what more we can learn, before putting it to work.

which guess_fstype tells me it's at /sbin/guess_fstype

How does an sbin directory differ from a bin directory? It is a clue that only root can use the file. Think supervisorbin

Let's learn more

cd /sbin

file guess_fstype tells me it's an ELF 32-bit file. Meaning it's compiled and not human readable. So, next step is check help;

guess_fstype --help (results are nothing)
guess_fstype -h (results are nothing)

I want to see the text inside this non-human readable file

<guess_fstype strings

I've just learned a lot, by using strings, among other things, the filesystems it's designed to guess. I won't post the results in this post, because of lots of output.

Do it as your own exercise please, you want to know and understand the file strings.

I searched linux.die.net for a man page and found nothing.

I typed in;
guess_fstype /dev/hda2 and it returned ext3

This is about how I'd have guessed it would work and all I really need to know.

In chapter 55 we came up with this pipe string to put all our Linux partitions into a variable.

partitions=`fdisk -l | grep "83 Linux" | cut -d " " -f 1 | tr "\n" " "`

Linux data partitions are type 83, but not all Linux data partitions are ext2 or ext3. We will use guess_fstype to verify that the partition we intend to check is indeed an ext2 or ext3.

Our testing and repair tool e2fsck has its own built in error checking, so we won't get into trouble as far as data damage.

Nevertheless, I want us to do our own conditional checking and branching when we know we cannot predict the conditions and some checking is in order. In this case, we can't predict our 'customer' isn't using reisferfs, ext4, or something else. Let's do appropriate checking for her.

Code: Select all

#!/bin/bash
main()
{

    variables
    partition_types

}

variables()
{

    partitions=`fdisk -l | grep "83  Linux" \
    | cut -d " " -f 1 | tr "\n" " "`

}

partition_types()
{

	for i in $partitions ; do
		echo -n " This Linux partition $i is type: "
		guess_fstype $i
	done

}

main
Script notes: Please note how I placed the curly braces {}, in a different position than in previous lessons.

Code: Select all

before() {

	echo

}

Code: Select all

this_example() 
{

	echo

}
It is important? The answer is; potentially very important. We you start programming on teams with another or others, do it the way they do it, if you can. You will get along better when working on the same programs with others, and some programmers have very strong feelings about style.


I made a script called showtype and attached it to this post. If you have Linux partitions, please download it, run it and study it. If you don't have Linux partitions, the script is of no value to you.

~

Chapter 58 - guess
Attachments
showtype.zip
Shows Linux partitions and format type
(329 Bytes) Downloaded 1144 times

Bruce B

#57 Post by Bruce B »

Chapter 59 - more building our Linux partition checking utility

Puppy doesn't come with much for automatically checking our filesystems. This is cool, because it makes for a good exercise and gives an opportunity to build some useful utilities.

I want to build this utility with you step-by-step, so when we are done, we really understand what we have done.

So far, we've learned (1) how to locate Linux partitions and (2) how to identify the partition type.

We don't want to run our filesystem repair utility on a mounted partition, in this chapter we will learn how to tell if a partition is mounted.

Two files carry this information, (1) /etc/mtab and (2) /proc/mounts.

I use /proc/mounts to test for mounted filesystems.

I'd like you first to look at /proc/mounts; this command;

cat /proc/mounts

Suppose we want to know if /dev/hda1 or /dev/sda1 is mounted, we could use grep capabilities. Here's a fun little script.

Your arguments will be the device name or the mount point, and if you give bad arguments, you can get false positives. But we won't do that in our final script.

Code: Select all

#!/bin/bash
[ ! $1 ] && echo "no arguments, exiting" && exit
</proc/mounts grep $1 >/dev/null
[ x$? = x0 ] && echo $1 is mounted
We could modify the script to say if or if not mounted, using the if;then statement

Code: Select all

#!/bin/bash
[ ! $1 ] && echo "no arguments, exiting" && exit
</proc/mounts grep $1 >/dev/null
if [ x$? = x0 ] ; then
	echo "$1 is mounted"
else
	echo "$1 is not mounted"
fi
We could modify our script to accept more arguments

Code: Select all

[ ! $1 ] && echo "no arguments, exiting" && exit
for i in "$@" ; do
	</proc/mounts grep $i >/dev/null
	if [ x$? = x0 ] ; then
		echo "$i is mounted"
	else
		echo "$i is not mounted"
	fi
done
The purpose of this lesson is to introduce you to how we will check if a partition to be checked is mounted. That is all.

If you have grasped the file /proc/mounts, grep and its return code and how we will use it, you have grasped the meaning of this lesson.

Test and use the scripts as you please.

~

Chapter 59 - more building our Linux partition checking utility

Bruce B

#58 Post by Bruce B »

Chapter 60 - various topics

A rant

I have about 30 books on C. Do we need that many? We shouldn't, but in a way maybe we do. I'll explain why, according to my opinion.

Often the author starts off with an easy teaching gradient, after reading a few chapters, the gradient becomes too steep. If it is genuinely too steep, we set the book down. Maybe pick up another. This cycle seems to repeat itself, starting easy, then getting too steep.

The saving grace is; after reading many beginning chapters, there is enough learning under our belts, the steeper chapters are not so steep. By rotating books, maybe we can make progress.

Why this series

Simply stated, it is a response to the wishes of a few people. Frankly, all the audience I need to motivate me is one person.

A lot to learn

If you have noticed, the chapters are not presented in an A-Z fashion. Even if one chapter is too hard, the next chapter may not even be related to the previous chapter. I hope there is no chance of the student running into the steep learning gradient I referred to in 'my rant'

A different approach

I've studied some excellent online lessons. I see no reason for me to reinvent the wheel or even compete. I'm in a position to give exercises very Puppy centric and other authors likely aren't in this position.

Who you are in relationship to your computer

In my opinion we wear (at least) three hats;
  • 1) computer operator
    2) system administrator
    3) network administrator
This is the default for us - who don't have a professional administrator. I point this out because - I don't know if it occurs to many people that they are indeed admin.

The value of scripting

Let's take Puppy as an example. Puppy is a carefully thought out hodgepodge, a literal hodgepodge of mostly open source applications.

Scripting is the 'glue' that puts the hodgepodge together and makes Puppy run.

Many of Puppy's homegrown applications are merely scripts hosting a graphical front-end and running back-end programs.

Making our scripts more attractive

I left you with 'cecho' to easily color your text output, which adds a nice finishing touch.

There are also programs we can use to enhance our programs, such as dialog. I leave this chapter with a picture of a project I'm working on for us. The code isn't ready to upload, but you can see by the picture how we can make professional, easy to use interfaces and menu items for our scripts.

~

Chapter 60 - various topics
Attachments
init-editor.png
(6.52 KiB) Downloaded 1710 times

Bruce B

#59 Post by Bruce B »

Chapter 61 - setting date and time on the command line

setdate

Code: Select all

#!/bin/bash

[ ! $1 ] && echo "Usage example: setdate 20090721" && exit

date +%Y%m%d -s "$1" >/dev/null 2>&1
date

settime

Code: Select all

#!/bin/bash
if [ ! $1 ] ; then

    echo "Usage examples:"
    echo " settime 06:25:30PM"
    echo " or"
    echo " settime 18:25:30"
    exit

fi

echo "$1" | grep M >/dev/null
if [ x$1 = x0 ] ; then
    date +%T%p -s "$1" >/dev/null 2>&1
else
    date +%T -s "$1" >/dev/null 2>&1
fi

date
Scripts attached to post

~

Chapter 61 - setting date and time on the command line
Attachments
time-and-date.zip
setdate + settime scripts
(543 Bytes) Downloaded 1152 times

Bruce B

#60 Post by Bruce B »

Chapter 62 - the init editor

The init editor code is ready. This utility should handle all the initrd.gz chores. It also makes backups.

The code is posted, but be sure and use the attached file, there are too many \s and maybe line wrapping for copy and paste.

This is also our first example of using ComeOn Dialog!, by Thomas E. Dickey

Code: Select all

#!/bin/sh

main() {

    check_for_file
    options
    process_answer
    [ -f /tmp/initedit.tmp ] && rm /tmp/initedit.tmp

}

check_for_file() {

    if [ ! -f initrd.gz ] ; then
        dialog --backtitle 'File not found!' --msgbox "The required file: \
        initrd.gz does not exist in this directory \
        please change to the directory which has initrd.gz" 7 55 
    clear
    exit
    fi

}

options() {

    dialog --backtitle "Init Editor Main Menu - Puppy Versions 3.xx and 4.xx" \
    --nocancel --menu "Please choose" 0 35 8 \
    "x" "eXit" \
    "1" "Extract initrd.gz" \
    "2" "Delete extracted tree" \
    "3" "Compress initrd.gz" \
    "4" "Edit init - geany" \
    "5" "Edit init - medit" \
    "6" "Edit init - mp" \
    "7" "Midnight Commander" \
    2>/tmp/initedit.tmp    

}

make_initdir() {

    cd initdir
    zcat ../initrd.gz | cpio -i -d
    cd ..

}


process_answer() {

    answer=`cat /tmp/initedit.tmp`

    case $answer in
        1) 

           [ ! -d initdir ] && make_initdir

        ;;   

        2) 

         [ -d initdir ] && rm -rf initdr

        ;;
        
        3) 
           cnt=2
           if [ -d initdir ] ; then
           cp initrd.gz initrd1.gz

              for i in 1 2 3 4 5 ; do

                  cp initrd${i}.gz initrd${cnt}.gz >/dev/null 2>&1
                  cnt=`expr $cnt - 1`    
                   
              done

           fi

           cd initdir
           find . | cpio -o -H newc | gzip -9 > ../initrd.gz               
           cd ..
        ;;

        4)             

            [ ! -d initdir ] && make_initdir
            geany initdir/init &

        ;;

        5)

            [ ! -d initdir ] && make_initdir
            medit initdir/init &

        ;;

        6) 

            [ ! -d initdir ] && make_initdir
            mp initdir/init

        ;;            

        7)
            mc initdir .

        ;;

        *) 

            [ -f /tmp/initedit.tmp ] && rm /tmp/initedit.tmp
            clear ; exit

        ;;

    esac            


    exec $0

}

main

Script notes: I introduced an internal command 'exec'. In bash we can run other shells from our shell. In a practical sense, when we use exec, we stop the shell we are in and use a new shell, which is often a new script.

exec $0 means stop the script and run it again - in this case from the beginning. When using initedit, you will find it more convenient to have it open until your work is done, so that's why the exec $0. Remember $0 is the variable name for the program itself.

We also used cnt=`expr $cnt - 1`, unlike our other example this is arithmetic. It simply adds one to the value in cnt each time it runs, using the expr statement. You will use it and see it frequently.

As far as the dialog interface, I won't go into detail on this, we want an easy introduction and gradient which can hopefully come later.

~

Chapter 62 - the init editor
Attachments
initedit.zip
init editor script
(895 Bytes) Downloaded 1171 times

Post Reply