Puppy Linux Discussion Forum
"THE" alternative forum : puppylinux.info

 The time now is Sun 18 Feb 2018, 05:22 All times are UTC - 4
 Moderators: Flash, Ian, JohnMurga
 Page 1 of 8 [110 Posts] Goto page: 1, 2, 3, ..., 6, 7, 8 Next
Author Message
Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 06:19    Post subject:  The Text-User InterfaceSubject description: In-house training

Introduction

Notice - I removed the new version of bash. I had some odd problems
with it running Midnight Commander and never resolved the issue, and I
tried. I'll leave the post intact minus the attachment. Sorry.

Some users are interested in learning Text User Interface (TUI). I enjoy learning, teaching and helping. I particularly enjoy the TUI and all that can be done with it. By knowing it well you can add a lot of extensibility to your Linux system.

The emulator and shell

If a person opens up rxvt, Puppy's default terminal emulator, you will be working with the bash shell. You will see Puppy's default prompt, the #.

The need to learn

There is nothing and I mean nothing intuitive about the bash shell with a # sign. For this reason, the TUI has to be learned. Once a certain undefinable point in learning curve has been achieved, the user has enough know how and confidence to take it from there and work on his own.

The curriculum

I'll try and present easy chapters with a balance of theory and practical application. If you're like me, you'll be tempted to not follow instructions to the letter, to personalize things. I ask you not to do this under certain circumstances. I'll explain why.

With computers there are various ways to accomplish the same end. For the purpose of these chapters, I'll state which is mandatory. Items will be mandatory on the basis that I expect to use them later, and in my future writing I expect things to be in certain places.

The conclusion

When the chapters reach a logical conclusion, I'll conclude. At the conclusion, you'll be able to put everything back just how it was if you want. You then will also be able to set things up in very customized fashion.

Optional Exercise

This is strictly optional. The idea is to update bash. Attached to this post is:

GNU bash, version 4.0.10(1)-release (i686-pc-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.

I've used this bash for a while now with Puppy version 4.00 and it works swell.

Let's consider a few things. (1) if this version is not newer than your existing version, it won't be an update. (2) if it doesn't work right, you will want to restore your old version.

Basically, what I really want to do is have you type in some commands and learn some things, and make a modification to your system.

Open your terminal emulator - rxvt

bash --version

Check if it's lower than 4.0.10

Where is bash? The which command will tell you.

which bash

cd to that directory, if /bin then

cd /bin

Check if bash and the download is there with the ls command

ls bash*

If you don't see bash and bash.bz2, STOP something isn't right

Make a backup copy of bash

cp bash bash.bak

We are using bash to replace bash, but the bash we are using should be in memory.

Unpack bash.bz2

bzip2 -d -f bash.bz2

-d for decompress -f for force overwrite

Close rxvt

exit

Open rxvt and check the bash version

-----------------

Edit attachment temporarily removed for changes in instructions.

_________________

Last edited by Bruce B on Mon 10 Jan 2011, 12:34; edited 4 times in total
Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 07:39    Post subject: profile.localSubject description: chapter 1

profile.local

The easy life - hopefully soon you will see the command line can be an easy way to do things, even mundane tasks.

Here is one example:

rox /usr/bin

If you open rxvt and type the above command, you should note everything happened much faster and easier, than if you clicked to open ROX-Filer and navigated up and down to /usr/bin.

In this chapter we are going to work on a file called 'profile.local'. The main purpose is preparing to make our life on the TUI easier. The file itself probably doesn't exist, but it will by the end of this chapter.

This will be the contents:

export PATH="$PATH:/root/bin" export PS1="[\w] " alias cp='cp --interactive' alias mv='mv --interactive' alias rm='rm --interactive' alias bin='cd /root/bin' alias path='echo$PATH'

Explained somewhat

PATH (uppercase, always upper case) is a list of all the directories Linux will use to search for executable files. I included an alias called 'path', which you can use to see the PATH displayed on screen. Linux will look for executable files in all these directories and only these directories. The search is left to right, the first match is the one Linux uses.

/root/bin will become our working directory. We will make our scripts and other files in this directory. Doing it this way, our scripts and other related files will never be mixed in with files in other directories.

If you make a script, you give it a name, good practice is make sure the name doesn't already exist in the PATH statement. Simply type in the intended name before assigning it to a script.

Never make a script called 'test' because test is an actual command.

Also note we put /root/bin at the end of the search chain. Later, if you are crazy like me you can put it first, but not in the course of this study.

PS1 is how we define main prompt. In this case, it will show your full working directory. Having the prompt display your full working directory is more meaningful than the # sign. Also, there are entire web pages about various ways to display the prompt. Although this exercise is mandatory - change the PS1 as you please

cp, mv, rm - these are the commands to copy, move and remove files. The default Puppy doesn't make them an "Are you sure?" They obey at the enter key.

The aliases above have redefined these commands to get your permission on certain command line actions. ( we added a safety factor )

The alias 'bin' makes it easy to change directories to our work directory /root/bin

The mandatory assignment

1) make a text file /etc/profile.local - copy and paste the commands in darkred from above to that file, and of course save the changes.

The contents of this file will probably show effect the next time you open rxvt.

2) make a directory /root/bin, this might be tricky. Historically there never was such a directory in Puppy. The command below would be all you need:

mkdir /root/bin

I noticed in newer Puppy versions we have a symlink like this

/root/bin -> /root/my-applications/bin

I trust you can solve the riddle and make the directory with the tools at hand.

~

Chapter 1 - profile.local

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 09:08    Post subject:

The Source Command

In Chapter 1, we made a file - profile.local

* it is not a shell script - no #/bin/sh
* it is not in the PATH
* it doesn't have an executable bit set, but somehow;
* it is a text file that runs commands

Absent all the qualities we normally associate with an executable file, every command is executed. I'll explain.

The source command is what executes it. Also, in Linux we often have short commands and long commands to do the same thing.

Two examples:

source is the long command : . is the short command
test is the long command : [ is the short command

How /etc/profile.local works in Puppy

Considering that /etc/profile might change with Puppy versions, I otherwise expect you will find a line like the one below, which I will explain.

[ -r /etc/profile.local ] && . /etc/profile.local

There it is - our shorthand, I'll show what it would look like in long hand

test -r /etc/profile.local && source /etc/profile.local

English is the harder of the languages. In English it is saying

[ ( means test )
-r ( defines our test - if the file exists and is readable )
/etc/profile.local ( the name of the test file )
&& ( means if test is true, do next command otherwise do nothing )

Before we installed /etc/profile.local, the test would have been false and nothing would have happened. After making the file, the outcome of the test is true and profile.local gets sourced, meaning all the commands it contains get executed, using this command:

. /etc/profile.local

Shells, shells and more shells

The human eye will not discern this, but when we run a bash script, the shell we are looking at and possibly interacting with is not the same shell that ran the script.

Perhaps well to conceptualize a parent shell spawning a child shell. The parent shell passes most of its environment to the child shell. The child, however does not return with the environment it set. ( as a rule )

Here is an example which should be easy to follow:

The parent shell has this variable EDITOR=mp

The child sets EDITOR=nano

nano will be the value for the variable EDITOR during the execution of the script, but when the script finishes, the value of EDITOR is mp, as it was before the script started.

Sourcing, as opposed to running a shell script is more the equivalent of typing commands into the existing shell. A child process is not spawned when sourcing.

When we sourced /etc/profile.local, we changed the environment in the same shell which sourced it. And that shell was before X ran, and before the terminal emulator hosted another shell.

The environment get passed up fairly easily.

For a related exercise for this chapter, open rxvt and run this command:

env

It will show a lot of your Linux environment. It is also a command which can be helpful in writing some scripts and troubleshooting.

~

Chapter 2 - The Source Command

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 11:59    Post subject:

Enhancing our TUI

This chapter is for the purpose of improving your Text User Interface, making it an easier, more pleasant and powerful working environment.

We will install two applications from Puppy's official repository

mrxvt-0.5.2.pet

The reason why is, because its better than rxvt, in a number of areas. Of particular importance to me is the handling of F keys and the eol (end of line) and bol (beginning of line).

Puppy includes a great TUI text editor 'mp' ( Minimum Profit ), but rxvt doesn't handle the bol and eol as it should for some reason, thus making working with mp quite difficult.

mc-4.6.1.pet
If mc doesn't run - maybe you need gpm-1.20.1.pet

Run Midnight Commander by typing mc in terminal emulator. It is a very powerful, extensible and time honored file manager for the TUI.

Look for these packages in pet_packages 3 and 4

Please consider this an essential assignment.

~

Chapter 3 - Enhancing our TUI

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 13:37    Post subject:

First script

We will make a script for making scripts.

Making a script might take two or three steps. If we can do it in one step it's easier.

This is the top line of our script. It tells the Linux that it's a Bourne shell script text executable

#!/bin/bash

We want to make sure the user enters an argument, so we make a test. Testing was already touched on in Chapter 2. Remember '[' is shorthand for test. We signal the end of the test with ]

The explanation point ! means NOT

$1 is the first argument. [ !$1 ] && echo "usage: newsh <filename> && exit

Explained in English: If there is no argument, show the user how to use our utility and exit.

We are using two &&, this means that two conditions must be true in order for our line to fully execute. If the first && is false nothing at all happens.

Using && moves left to right and if any conditions are not true the remaining && commands do run.

In order for an execution, it must be true - there is no argument.

#!/bin/bash
[ ! $1 ] && echo "usage: newsh <filename>" && exit ------------------------ Bash executes in a top to bottom manner. If we get past the conditional test in line two, we can execute other commands. Here I introduce an if/else statement combined with a test. If the 'text' in our argument$1 matches an existing file, we will do one thing, otherwise we will do something else.

a) if [ -f $1 ] ; then b) leafpad$1 &
c) else
d) echo "#!/bin/bash">$1 e) chmod 755$1
f) leafpad $1 & g) fi Explained a) if argument$1 is -f (a existing file) ; then

b) open it with leafpad in the background. a single & means background. the practical effect of using the single & is our command prompt will return, without the & our command prompt hangs until leafpad is closed

c) means means otherwise (else) do the next commands. (d),(e) and (f) only get executed if (a) is false, which means there is no file matching the name in $1 d) is the echo command, meaning display the text entered to standard output, our monitor typically. in this case we use the greater than symbol > to redirect the output to a non existent file of the name in$1 - which actually makes the file by the redirection.

e) chmod is an external command assigning attributes to the new file. As far as what the numbers mean, I think it would be better at a later time to simply refer you to a web page explaining it.

For now 755 is what we will use to make our files executable

f) same as (b)

g) the end of the if statement, note 'if' spelled backward is 'fi'

Let's put it together and have a look at it.

 Code: #!/bin/bash [ ! $1 ] && echo "usage: newsh " && exit if [ -f$1 ] ; then    leafpad $1 & else echo "#!/bin/bash">$1    chmod 755 $1 leafpad$1 & fi

How to make the first script

Open the terminal emulator

bin # you did fill out /etc/profile.local

Select the code from the forum post and copy to leafpad, close leafpad

chmod 755 newsh

If all went according to plan, we have a new utility we didn't have before

~

Chapter 4 - First script

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 14:22    Post subject:

Words

I want to keep our work separate from the other Puppy directories and files. At the end of the training, we will know right where everything is. You can clean up and start over, keep your work - do as you wish.

This is required for future lessons

Make a directory called /root/doc
Make a directory called /root/doc/words

mkdir /root/doc
mkdir /root/doc/words

The file is 3mb and the contents are 10mb - make sure you have the space.

I want this file because we can use its contents in a variety of ways. And, you will end up with some really nice utilities, which otherwise don't even exist. I'd rather us build meaningful utilities, when possible, than merely demonstrate various bash features.

I hope I'm also demonstrating that using the TUI need not be substitute for the GUI, rather a way of enhancing your operating system, making it more fun and powerful.

Maybe, you've noted by now, the text in darkblue is reserved for commands you will actually type in. We accomplish some learning simply by doing and repetition. After enough repetition, the commands become second nature, and it doesn't take all that much either.

Suppose we download the zip file to /tmp, here's how to extract it to the directory we need the extracted files to be in;

cd /root/doc/words
unzip /tmp/mword10.zip

That's it for this chapter, we will use these files in later lessons.

~

Chapter 5 - Words

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 19:24    Post subject: Enhancing mrxvtSubject description: Chapter 6

Enhancing mrxvt

The exercises in this chapter are optional, but likely desirable.

You installed mrxvt. I noted the rc file; /root/.mrxvtrc is very empty of the commands available. I've included some commands to add to the rc file and mrxvt documentation to put in your doc directory.

Step 1)

Open the rc file

Hint: You can copy the command from the forum post and paste it to the command line. To do so, highlight the command, activate mrxvt and push the mouse wheel. (The TUI doesn't necessarily involve a lot of typing, sometimes very little.)

Copy and paste this text into leafpad /root/.mrxvtrc

mrxvt*geometry: 80x25
mrxvt*bottomTabbar: true
mrxvt*mouseWheelScrollPage: true
mrxvt*tabTitle: bash
mrxvt*scrollbarRight: true
mrxvt*foreground: white
mrxvt*background: black
mrxvt*font: 10x20
!mrxvt*autohideTabbar: true
!mrxvt*scrollBar: false

Among other things, these rc instructions will give you a larger font.

Note how you can change colors, I set it for black background and white text. Change it to your preferences.

Step 2)

I put together two text files. One containing the short commands for mrxvt and another containing the long commands.

 Code: Archive:  mrxvt-text.zip   Length     Date   Time    Name  --------    ----   ----    ----      5504  07-01-09 12:04   mrxvt-long.txt      7060  07-01-09 12:02   mrxvt-short.txt  --------                   -------     12564                   2 files

Step 3)

Open your favorite browser, navigate to /root/doc and make a bookmark for the directory. With the bookmark, you will be able to easily find any documentation pertaining to this course and view it in the browser.

~

Chapter 6 - Enhancing mrxvt

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Wed 01 Jul 2009, 20:21    Post subject: A Modest Introduction to VariablesSubject description: Chapter 7

A Modest Introduction to Variables

A variable is a way of storing data in a name. The data is accessed by referencing the name.

The construct of a variable is like this

VARIABLE_NAME="this text is the data stored in the variable VARIABLE_NAME"

The left side of the construct is the name of the variable
The = defines the end of the name and begins the data portion
The right side is the data the variable holds

We set a variable exactly as shown above. No spaces allowed before or after the = sign.

We access the data by prepending the variable name with a $sign When we refer to the variable with the$ sign we are referencing the data.

Command line example and exercise;

myname="first name last name"
echo $myname Variables have many, many uses. As we begin building our scripts, the uses, as well as the necessity of variables will become more apparent. ------------------------ Earlier we made a shell script - newsh. In the script we used leafpad as our GUI editor. Suppose we build 50 scripts using leafpad and then decide we'd rather use geany. We would have to go back and change every instance of leafpad to geany. We can easily use one variable and effect the change in all 50 scripts, merely by changing the data in one variable We can revisit profile.local and add this variable: guiedit="leafpad" Our first script originally said; leafpad$1 &

With the variable guiedit the script would read like this:

$guiedit$1 &

It would do exactly the same thing, except if we changed the data in the variable to something like 'geany' or 'medit' the same script will open a different editor.

For the purpose of exercise and learning, let's make the change. At this point, you should be able to do this without instruction, nevertheless I'll provide the commands to run.

(remember about how to copy and paste from forum to the command line and to the text editor)

$guiedit$1 &

~

Chapter 7 - A Modest Introduction to Variables

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 02:59    Post subject: AliasesSubject description: Chapter 8

Aliases

If you notice, we've already opened /etc/profile.local a few times. When you discover you are performing long commands on the command line repetitively, it's a good indicator you might want to make an alias for the command.

We want to make it easy to add aliases - and we will.

We can edit /etc/profile.local with a short command - an alias. An alias is constructed like this:

alias profilel='$guiedit /etc/profile.local ; source /etc/profile.local' ( note the variable, the source command and the command separator ';' ) We will add the alias above and three more; alias bashrc='$guiedit /root/.bashrc ; source /root/.bashrc'
alias rclocal='$guiedit /etc/rc.d/rc.local' alias mrxvtrc='$guiedit /root/.mrxvtrc'

rc.local

Note we didn't source rc.local, we simply edited it. rc.local is a file we place non X commands we want to run only one time per session.

This file will contain things like mounting devices we want mounted, such as;

mount -t ext2 /dev/sda3 /mnt/sda3
modprobe usbcore

We only want to do these actions one time, rc.local runs only one time, so it is the appropriate file for such commands.

.bashrc

Is for commands we 'source' over and over. /root/.bashrc gets sourced each time we open a terminal instance of bash. This is a default.

profile.local

We can control if it runs each time an instance of bash runs.

Conventions

rc.local and .bashrc are conventional Linux files. profile.local isn't particularly conventional with Linux distributions. Rather a 'feature' included with Puppy.

An important consideration for all scripting

Do not refer to any variable before it has been defined. Our files read from top to bottom. We defined a variable guiedit=leafpad

Now we are about to add some aliases which use the variable. They must be inserted below the definition of variable, otherwise we get errors.

Assignment (required)

Open

alias bashrc='$guiedit /root/.bashrc ; source /root/.bashrc' alias profilel='$guiedit /etc/profile.local ; source /etc/profile.local'
alias rclocal='$guiedit /etc/rc.d/rc.local' alias mrxvtrc='$guiedit /root/.mrxvtrc'

---------------------------------------------

You should be able to remember the names of your own made aliases. We also have the alias command, which if typed with no arguments will display your aliases.

If for some reason you don't want an alias for a bash session you can unalias it. Maybe you are removing a lot of files with the 'rm' command and you don't want the interactive portion.

Merely type: unalias rm

If you want to bypass an alias for just a few times, but not for your entire bash session, then type the following;

\rm <arguments>

The \ will cause the rm command to behave as if there was no 'rm' alias.

------------------------------------------------

Hereafter you will simply type the alias name to open the files we made editing aliases for. (thus making life easier)

~

Chapter 8 - Aliases

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 04:35    Post subject:

grep

grep is search filter utility and commonly used in scripting.

In this chapter we will make two simple scripts using grep.

------------

We can run most any application on the CLI, including those in our Puppy menu. But the name in the menu doesn't say what the name of the executable file is.

Suppose we want to know the file name for the menu item Wizard Wizard. We can make a simple script which will make it easy to discover the executable name for Wizard Wizard, as well as any other menu item.

First script

[ ! $1 ] && echo "Missing argument" && exit </root/.jwmrc grep --ignore-case "$1"

Let's make another script

bin ; newsh grepalias

[ ! $1 ] && echo "Missing argument" && exit alias | grep --ignore-case "$1"

That's it! Not so hard.

But I do need to explain the scripts. The first lines have been covered. I'll explain the second lines of each script.

------------

</root/.jwmrc grep --ignore-case "$1" The < and > are our redirection symbols. The < directs the content of the file /root/.jwmrc to the command grep. As you can see, grep is instructed to ignore case. The menu item says Wizard Wizard. The "$1" says to use argument one.

Our command line can say:

or with quotes

-------------

alias | grep --ignore-case "$1" The alias command is standard output to the monitor, we use this symbol: '|' to pipe the output to grep, which will filter the output to match our search criteria. Pipes and redirection are very important in constructing scripts. You've been shown some examples, if you don't fully understand, just hang in. < | > will be repeated over and over many times. Our example scripts show a very simple usage for grep. Grep is a very powerful utility and capable of doing complex chores. Included with this chapter is the man page for grep in text format. I recommend downloading it and unpacking it to your /root/doc directory. When you are writing scripts you can open it in your browser and use it for reference. If you want to get an overview of the many ways grep is used, you can click on the forum search link at the bottom of the post and enter the search criteria grep. Chapter 9 - grep  grep.zip Description grep man page - text format Download Filename grep.zip Filesize 10.54 KB Downloaded 1585 Time(s) _________________ Bruce B Joined: 18 May 2005 Posts: 11488 Location: The Peoples Republic of California  Posted: Thu 02 Jul 2009, 10:34 Post subject: Comments, for loop, read command, string comparisons and more Our utility lower renames files to the lower case. Linux convention is lowercase names without spaces. The word list files you downloaded and unzipped are all uppercase. I wrote a utility to make easy work of changing the filename case to lower case.  Code: # lower converts files names to lower case # without overwriting conflicting names # known bug doesn't rename files with spaces # feature: doesn't rename directories # feature: preview changes # feature: interactive for f in * do if [ -f$f ] ; then         g=echo -n "$f" | tr '[A-Z]' '[a-z]' if [ "$f" !=  "$g" ] ; then echo mv "$f" "$g" fi fi done # get user permission read -n 1 -p "Continue with conversion (y,n) " ans # if yes make changes if [ "$ans" = "y" -o "$ans" = "y" ] ; then for f in * do if [ -f$f ] ; then         g=echo -n "$f" | tr '[A-Z]' '[a-z]' if [ "$f" !=  "$g" ] ; then mv --verbose --interactive "$f" "$g" fi fi done fi echo ls Explanations # comments You will hear the term comment the line or comment out. Most computer languages have a way of adding comments to the code - a way of signaling the processor or compiler to ignore the commented text. Bash uses #. If someone tells you to comment out some lines in bash, they mean to say put a # sign in front of the code. We can use a comment to communicate our thoughts in the code and to block code without deleting it. The idea being everything on that line after the # is ignored. # for loop The for loop will be used over and over in our scripting. If you don't understand it now, you will soon enough. Here is the construct: for myvar in args do my_commands$myvar
done

In English it says, for my variable (myvar), process all my arguments according to my commands. When the last one is processed, finish.

If / else / conditional tests

In the second set we have three if / else constructs

get user input

read -n 1 -p "Continue with conversion (y,n) " ans

Construct 1

Test user input, if true run the commands

if [ "$ans" = "y" -o "$ans" = "y" ] ; then

Construct 2

If our variable represents a file then do the work, otherwise don't, meaning we aren't renaming directories

if [ -f $f ] ; then Construct3 We process the variable f converting any uppercase letters to lower case and store the information in a variable called g The use of the backtick  is necessary to move the output into the variable, you will learn when to use the  g=echo -n "$f" | tr '[A-Z]' '[a-z]'

Here we make a comparison to see if they do not match, not matching makes for a true condition, which we need for execution. Meaning any files in the directory which are not already lower case will be converted

if [ "$f" != "$g" ] ; then
mv --verbose --interactive "$f" "$g"

The mv command is also the rename command. We rename the files which match our criteria, with verbosity, meaning the users sees what goes on. Interactive means if a conflict occurs the user is notified and asked what to do.

More

This is an example of an interactive shell script. An important aspect of the interactivity is using the read command to get an answer from the user. We want to user to give an informed answer, so we display the results of the proposed changes in the top routines.

Please note the indentations - they are very helpful in reading the file. And are standard good coding practice. You will want to learn when and where to indent.

This may be the last time I show you how to make the script, using our existing system, like this;

bin ; newsh lower

(copy and paste the text, save and close the editor and the script is done)

Then run the script

cd /root/doc/words
lower

Optional exercise - make a script called upper which converts file names to upper case

bin ; cp lower upper ; newsh upper

Then figure out what to modify, test the script in /root/doc/words

~

Chapter 10 - Comments, for loop, read command, string comparisons and more

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 11:33    Post subject: Line endings, file, dos2unix, hexdumpSubject description: Chapter 11

Line endings, file, dos2unix, hexdump

Line endings

With our text files we don't see the line with in most editors. (non visible printing characters)

Different operating systems use different line endings, probably for the purpose of being different.

Mac : uses carriage return
MS-DOS : uses carriage return plus line feed
Unix : uses linefeed

Puppy gives us a utility called file, it's purpose is to help us learn the characteristics and type of file.
The way to use file is:file<filespec>

I ran 'file' on our word lists and this is the output:

aareadme.txt: ASCII English text, with CRLF line terminators
acronyms.txt: \012- Sendmail frozen configuration \012- - version 4GL\015
common.txt: ASCII English text, with CRLF line terminators
compound.txt: Non-ISO extended-ASCII English text, with CRLF line terminators
crosswd.txt: ASCII C program text, with CRLF line terminators
crswd-d.txt: ASCII text, with CRLF line terminators
fiction.txt: ASCII English text, with CRLF line terminators
freq-int.txt: ASCII English text, with CRLF line terminators
freq.txt: ASCII English text, with CRLF line terminators
kjvfreq.txt: ASCII English text, with CRLF line terminators
names-f.txt: ASCII text, with CRLF line terminators
names-m.txt: ASCII text, with CRLF line terminators
names.txt: Non-ISO extended-ASCII English text, with CRLF line terminators
oftenmis.txt: ASCII text, with CRLF line terminators
places.txt: ASCII text, with CRLF line terminators
single.txt: \012- Sendmail frozen configuration \012- - version \015
usaconst.txt: ISO-8859 English text, with CRLF line terminators

I wasn't sure what acronyms.txt and single.txt are in terms of line endings, I order to get a better look, I ran hexdump -C single.txt.

I saw plenty of the hexadecimal 0x0D and 0x0A - which are the messy dos line endings.

I then opened it with geany and it looked like a long list of single words. It looked fine, I felt safe in fixing the line endings on these two supposed Sendmail files.

The tool we use is fix the messy line endings is dos2unix, also included with Puppy. It accepts wildcards, so there won't be a need to make a script.

After running dos2unix my output from the file utility is:
acronyms.txt: \012- Sendmail frozen configuration \012- - version 4H
common.txt: ASCII English text
compound.txt: Non-ISO extended-ASCII English text
crosswd.txt: ASCII C program text
crswd-d.txt: ASCII text
fiction.txt: ASCII English text
freq-int.txt: ASCII English text
freq.txt: ASCII English text
kjvfreq.txt: ASCII English text
names-f.txt: ASCII text
names-m.txt: ASCII text
names.txt: Non-ISO extended-ASCII English text
oftenmis.txt: ASCII text
places.txt: ASCII text
single.txt: \012- Sendmail frozen configuration \012- - version
usaconst.txt: ISO-8859 English text

Notice everyone of the with CRLF line terminators are gone. We now have genuine Unix files, conforming to the proper standards and conventions for our operating system.

cd /root/doc/words

lower (if you haven't already ran it)

file *.txt

hexdump -C single.txt and view the output especially looking for Od Oa - the messy dos
line terminators - Ctrl + C to break

dos2unix *.txt

hexdump -C single.txt and note there are no more 0d characters

file *.txt and note there are no reports of CRLF in the files

------------------------

Summary, you ran your new utility 'lower', learned a little about other utilities; hexdump, dos2unix and file

The skill learned is how to test and clean up text files to make them truly Linux files.

~

Chapter 11 - Line endings, file, dos2unix, hexdump

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 14:22    Post subject: LookSubject description: Chapter 12

Look

This chapter is very short. I finished the script for using the various word lists. The file name is called look

Usage:

look arg1
or
look arg1 arg2

Where the arguments represent a portion of text in a word you want to lookup.

To search the beginning of the text, use the ^ sign

look ^arg1

For example: to look up Bruce and other words which start with br, I would use this syntax

look ^br

The $sign represents the end of the word, I could enter arguments like this look ^b e$

The output would show every word that starts with b and ends with e in the list I'm searching.

------------------------

Other word lists from different languages could also be added to /root/doc/words. The program is very extensible.

I won't try and explain the program in this post, because it will be much easier if I wait a couple days, until you've had a chance to interface with it, inspect it and see how it works. This way the explanation will have more meaning.

-------------------------

Installation instructions, unzip look from the package look.zip in the /root/bin directory

~

Chapter 12 - look

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 15:43    Post subject:

Midnight Commander

I made change to Puppy's official Midnight Commander's mc.ext file. It will now open both .pup and .pet packages in a virtual filesystem.

Simply highlight the .pup or .pet with the arrow keys and hit the enter key to open the archive.

This makes it very easy to inspect contents and/or perform manual installs.

Installation instructions: Download the attached zip package and extract it in /usr/local/share/mc

OR;

In order to demonstrate just how easy mc makes things, you can do like this;

* open mc
* highlight the file
* tap the enter key to open it
* in the other panel, navigate to /usr/local/share/mc
* copy mc.ext from one panel to the other

It can even open iso files with the tap of an enter key

- Enjoy

~

Chapter 13 - Midnight Commander

_________________

Bruce B

Joined: 18 May 2005
Posts: 11488
Location: The Peoples Republic of California

 Posted: Thu 02 Jul 2009, 21:23    Post subject:

updatedb

I wrote three scripts and packaged them as 'findutils.zip'. There are binary packages available which do about the same thing and share similar names as these scripts. If you happen to have these already installed, I'll leave it to you which to use.

In this chapter, I present updatedb

The style and program flow I use here, is not conventional for bash scripting, in the following ways;
* everything is done in functions

* the program flow is the function main

* the variables are in lowercase

I've never actually seen this type of programming and in your other courses, you probably won't either. But you might like it because;
* its very easy to write complex programs by breaking things down to small and simple functions

* it's easier for me to read lowercase text variables

* the program flow is very easy to follow, we have only one command, main

* main controls other commands

* if we look at main we can see the entire flow

* after each function, I return to main for the next command;

* no jumping around between functions

* if you are having problems, it's very easy to comment out commands in main and debug the troublesome function

* the style is quite similar to how C is written, it's from C I got the idea to do bash in a similar way

Ultimately, you'll develop your style. I suppose it's good to be exposed to different ways of doing things. Don't think my style is the right style, it's just a style.

 Code: #!/bin/bash main() {     var     filesdb     dirsdb     showfiles } var() {     file="/var/log/updatedbf"     dirf="/var/log/updatedbd"     dirs="archive bin etc lib opt root sbin usr var" } filesdb() {     [ -f $file ] && >$file     for i in $dirs ; do echo "Processing files in /$i . . ."         find /$i -noleaf -mount -type f >>$file     done } dirsdb() {     [ -f $dirf ] && >$dirf     for i in $dirs ; do echo "Processing directories in /$i . . ."         find /$i -noleaf -mount -type d >>$dirf     done } showfiles() {     echo     ls -l --color=always $dirf ls -l --color=always$file     echo } main

( don't copy and paste this code, I have a package prepared for download, it will be attached in the next post)

Explaining updatedb

Compared to searching all of Puppy for a file, it's lightening fast to simply search a database. We build two databases, one for directories and one for files.

var: we set our variables, which include the directories we plan to catalog, this can be modified

filesdb: the instructions for making the files database and echoing the progress on screen

dirsdb: instructions for making the directories database and echoing the progress on screen

showfiles: we simply display the two databases

---------------

In the next chapter I'll show a find utility and upload the three combined files

~

Chapter 14 - updatedb

_________________

 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year  Sort by: Post TimePost SubjectAuthor AscendingDescending
 Page 1 of 8 [110 Posts] Goto page: 1, 2, 3, ..., 6, 7, 8 Next
 Jump to: Select a forum ------------------- Forum index |___ House Training |   |___ Beginners Help ( Start Here) |   |___ Users ( For the regulars ) |   |   |___ Usuarios de habla Hispana |   |   |___ Für deutschsprachige Anhänger |   |   |___ Pour les francophones |   |     |   |___ HOWTO ( Solutions ) |   |___ Bugs ( Submit bugs ) |     |___ Advanced Topics |   |___ Additional Software (PETs, n' stuff) |   |   |___ Package Collections / Repositories |   |   |___ REQUESTS |   |   |___ Browsers and Internet |   |   |___ Business |   |   |___ Compiling |   |   |___ Desktop |   |   |___ Documents |   |   |___ Drivers |   |   |___ Educational |   |   |___ Engineering/Science/Simulation |   |   |___ Eye Candy |   |   |___ Filesystem |   |   |___ Games |   |   |___ Graphics |   |   |___ Multimedia |   |   |___ Network |   |   |___ Security/Privacy |   |   |___ System |   |   |___ Utilities |   |   |___ Virtualization |   |   |___ Unsorted |   |     |   |___ Cutting edge |   |   |___ Multi-session live-CD/DVD |   |     |   |___ Hardware |   |   |___ Audio |   |   |___ Networking |   |   |   |___ Dialup |   |   |   |___ Ethernet |   |   |   |___ Wireless |   |   |     |   |   |___ Printers |   |   |___ Video |   |     |   |___ Puppy Derivatives |   |___ Puppy Projects |       |___ Next Puppy Development |       |   |___ 4.x |       |   |   |___ Bugs (4.x dev) |       |   |   |___ Usability Issues (4.x dev) |       |   |     |       |   |___ 5.x |       |       |___ Bugs (5.x dev) |       |       |___ Usability Issues (5.x dev) |       |     |       |___ Localization Project |       |___ Documentation Project |     |___ Taking the Puppy out for a walk |   |___ Announcements |   |___ Puppy Power |   |___ Suggestions |   |___ Misc |     |___ Off-Topic Area     |___ Programming     |___ Security

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum