Canned Bash lessons - learn to use Bash commands

Programs intended to teach, or to help one learn or study a specific subject - this includes educational games.
Message
Author
User avatar
ttuuxxx
Posts: 11171
Joined: Sat 05 May 2007, 10:00
Location: Ontario Canada,Sydney Australia
Contact:

Canned Bash lessons - learn to use Bash commands

#1 Post by ttuuxxx »

Ok I came across this How to learn Bash commands, its full of scripts text, it looks pretty good and when I get some free time I'll read and do the lessons, anyways here it is :)

To start it, extract the package into a folder and go to ... and click on it :)
I also made a menu item shortcut in /menu/document :)

start the course
/usr/doc/bash/advanced/HTML/index.html

scripts
/usr/doc/bash/advanced
/usr/local/bin/doc/bash2_doc

ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)

elraven
Posts: 82
Joined: Wed 04 Apr 2007, 03:20
Location: wnc-usa

Bash Lessons

#2 Post by elraven »

wow ! thanks tuuxx this is cool beans. :D
elraven

User avatar
ttuuxxx
Posts: 11171
Joined: Sat 05 May 2007, 10:00
Location: Ontario Canada,Sydney Australia
Contact:

Re: Bash Lessons

#3 Post by ttuuxxx »

elraven wrote:wow ! thanks tuuxx this is cool beans. :D
elraven
Yes elraven I hope Lobster puts this link in a how to Wiki for people wanting to learn Bash scripting. This will increase people learning bash language and it will help puppy grow :)
ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

Re: Bash Lessons How to ??

#4 Post by tlchost »

ttuuxxx wrote:Ok I came across this How to learn Bash commands, its full of scripts text, it looks pretty good and when I get some free time I'll read and do the lessons, anyways here it is :)

ttuuxxx
Thanks....I extracted it all and uploaded it to my account at one of the local colleges....that way I can learn and play with the scripts via ssh from anywhere.

Thom

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#5 Post by Lobster »

Thanks for providing this Ttuuxxx :)
Puppy is at present largely customised Bash scripts

Combine a bit of bash with gtkdialog3 and you can create simple but immensely useful customized programs
Even I have done it with YAP for example . . .
http://puppylinux.org/wikka/BuddhistYAP
Last edited by Lobster on Mon 05 Dec 2011, 18:35, edited 1 time in total.
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

DMcCunney
Posts: 889
Joined: Tue 03 Feb 2009, 00:45

#6 Post by DMcCunney »

People interested in the topic should visit Heiner's Shelldorado, at http://www.shelldorado.com/

It's one of the best resources I know for shell scripting, and covers a lot more than just bash.

(Note that bash derives from the AT&T System V Bourne shell, with enhancements made on the Korn shell and some stuff first implemented in the C shell created by Bill Joy for BSD Unix. It tries to be a combination of all of them. Unfortunately, the Bash scripting guide doesn't specify which bits are specific to bash.)
______
Dennis

User avatar
pri
Posts: 342
Joined: Fri 09 Oct 2009, 18:31
Location: Bandung Indonesia
Contact:

#7 Post by pri »

hey guys, try this link :

http://www.linuxconfig.org/Bash_scripting_Tutorial

i think thats tutorial is good for newbie on linux,
Learning by Doing

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#8 Post by amigo »

There is a program -perl script actually, called 'bashisms' which will purify scripts of bashisms. Very handy for creating more portable scripts. It is part of debian devtools, but has its' own project page on sourceforge.

You can also search for a web-page called 'lintsh' which lists some other quirks of many different shells.

I've been doing a lot of work lately on optimizing scripts to use the dash shell instead of bash because it runs 25-40% faster. But I have always been a 'basher', so my normal way of coding includes lots of 'bashisms'. The bashisms program and the notes on lintsh make it easier to find the parts that need to be changed to use dash, ash or other mostly POSIX-compliant shell. There are a couple of other things to watch for to be able to run scripts using a Bourne-compatible shell. There is a modernized version of the Bourne shell available from the 'heirloom' tools site on source forge -search for 'heirloom-sh'. It runs compliant scripts another 10-15% faster than dash.
Recently I wanted alter a script to not use an awk construct:
| awk ' { print $1 } '
My first attempt using shell code ran three times slower than calling awk. I went at it again and came up with a routine that runs 10 times faster than calling awk! I usually try to 'factor out' all calls to sed or awk, unless it is clear that using the external tools is faster.

DMcCunney
Posts: 889
Joined: Tue 03 Feb 2009, 00:45

#9 Post by DMcCunney »

amigo wrote:There is a program -perl script actually, called 'bashisms' which will purify scripts of bashisms. Very handy for creating more portable scripts. It is part of debian devtools, but has its' own project page on sourceforge.
The page is here: http://sourceforge.net/projects/checkbaskisms/
I've been doing a lot of work lately on optimizing scripts to use the dash shell instead of bash because it runs 25-40% faster.
Other distros use "ash" for the same purpose. As far as I can tell, ash is bash without all the code meant for interactive use at a command line. It's just an interpreter for the shell language, and preferable for things like install scripts.
There is a modernized version of the Bourne shell available from the 'heirloom' tools site on source forge -search for 'heirloom-sh'. It runs compliant scripts another 10-15% faster than dash.
It's part of http://heirloom.sourceforge.net

The trick is "compliant" scripts. The Bourne shell is not POSIX compliant (as the POSIX specs evolved well after the Bourne shell was written.) Hopefully, the heirloom shell is based on SysVR2 or later code, when shell functions got added to the Bourne shell.
Recently I wanted alter a script to not use an awk construct:
| awk ' { print $1 } '
My first attempt using shell code ran three times slower than calling awk. I went at it again and came up with a routine that runs 10 times faster than calling awk! I usually try to 'factor out' all calls to sed or awk, unless it is clear that using the external tools is faster.
What did your revised routine look like?

The original Bourne shell language was minimal, intended to be used as glue to tie together external processes like awk or sed, and was mostly flow-of-control constructs. I'm an old Korn shell user, and one of the enhancements the Korn shell made was making things like "echo" a built-in rather than calling an external program, and adding built-in integer arithmetic so you didn't have to call eval. There's a sample Korn shell script that implements grep in the Korn shell. It's mostly a proof-of-concept - calling the real grep will almost certainly be much faster for any non-trivial search - but it was a neat example of what could be done entirely in the shell.

Versions of ksh after ksh88 could be successfully installed as /bin/sh on Sun systems (and I did so).

Ubuntu includes ksh, and I looked at trying to move it to Puppy, but there are library dependencies I didn't have time to track down when I looked.

There is also the Z shell - zsh - with a PET available for Puppy.

See http://en.wikipedia.org/wiki/Z_shell and the Sourceforge page at http://zsh.sourceforge.net/

(I have a Win32 port of zsh as part of the UNXUTILS package.)
______
Dennis

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#10 Post by amigo »

Thanks for supplying those links. Sometimes I use the Alfred Stieglitz approach and only give a few clues and let the reader do the searching -when things aren't hard to find.
Alfred Stieglitz was a famous turn-of-the-century(20th) photgrapher who was married to Geogia O'Keefe. He was the first person to exhibit Matisse's work in the USA, at his studio in New York. He had no business cards for the studio, and when asked about it he said: "If a man wants something, he'll find it".

Anyway,
heirloom-sh is SVR4 compliant and includes functions and other 'goodies'. if invoked as 'jsh', it includes job-control, so it could be used as a login shell -but I don't even set it up as /bin/sh. There are too many scripts out there which use /bin/sh, but only if /bin/sh is a link to bash.

About dash/ash. ash is mostly POSIX-compliant, but dash claims to be fully POSIX-compliant. You'll find that the busybox version of ash is not the same as the real ash -you need to write Bourne-compliant code for busybox ash (especially regarding the use of `code` backticks instead of $(code) constructs. Another difference between POSIX and Bourne is that Bourne lacks the 'test -e file' (or [ -e file ])check.

So, for installers and initrd's which use busybox, it is best to use only Bourne-compliant syntax.

Here's my fake awk implementation:

Code: Select all

fake_awk_print() {
	CNT=1
	while [ "$CNT" -le "$CHOICE" ]; do
		#echo $1
		#echo $CNT
		#CNT=$(( $CNT+1 ))
		case $1 in
			' '|'	') true ;;
			*) 
				if [ $CNT = $CHOICE ] && [ $1 != '' ] ; then
				#if [ $CNT = $CHOICE ] ; then
					echo $1 && break 
				else
					CNT=$(( $CNT+1 )) ; shift
				fi
				;;
		esac
		#CNT=$(( $CNT+1 ))
		#[ $CNT -le $(($ARGS)) ] && shift
		#shift
		
	done

}

# call the funtion like this:
fake_awk_print $@

# original code like this:
# echo "$(for i in $(ldd "$FILE" | grep 'not found' | sort | uniq | awk '{print $1}'); do echo "3:> Missing: $i"; done)" >> "$RESULT"

#can be replaced with this:
#echo "$(for i in $(ldd "$FILE" | grep 'not found' | sort | uniq | awk_print 1 2> /dev/null); do echo "3:> Missing: $i"; done)" >> "$RESULT"

I seem to have deleted my original version which was slower than the real thing, or I'd post that, too.
The original script I was modifying used awk like that in 2 or 3 places and also sed. By rewriting those sections and getting rid of useless sort |uniq statements and making everything Bourne-compliant, I was able to make the script run about 40% faster using dash and around 50% faster using heirloom-sh.
Here's the sed trick:
Original code:

Code: Select all

BINDIRS="$(echo ${PATH} | sed 's/:/ /g;s/\.$//')"
New code:

Code: Select all

BINDIRS="`IFS=: ; echo ${PATH}`"
Here'S another nice trick for speeding up a common routine -to check if a file is an ELF binary. Original:

Code: Select all

if [ $(file "$FILE" | grep -q 'ELF') ]; then
New code:

Code: Select all

is_elf()
{
	[ "`dd if=$1 bs=1 skip=1 count=3 2> /dev/null`" = "ELF" ]
}
# call like this:
if is_elf $file ; then
Using dd to check for the file signature is safer than running file, and is much faster than calling a file|grep construction -really noticeable where the rouitn gets called 100's/1,000's of times haven't checked specifically, but probably also at least 50% faster...

So, have fun with that!

DMcCunney
Posts: 889
Joined: Tue 03 Feb 2009, 00:45

#11 Post by DMcCunney »

amigo wrote:Thanks for supplying those links. Sometimes I use the Alfred Stieglitz approach and only give a few clues and let the reader do the searching -when things aren't hard to find.
Alfred Stieglitz was a famous turn-of-the-century(20th) photgrapher who was married to Geogia O'Keefe. He was the first person to exhibit Matisse's work in the USA, at his studio in New York. He had no business cards for the studio, and when asked about it he said: "If a man wants something, he'll find it".
I know who Stieglitz was. I also live in walking distance of the site of the famous "Armory Show" in 1913
Using dd to check for the file signature is safer than running file, and is much faster than calling a file|grep construction -really noticeable where the rouitn gets called 100's/1,000's of times haven't checked specifically, but probably also at least 50% faster...
dd is bigger than file, but using file requires calling grep, too, so I see where the speedup comes from.

Have you experimented with setting the sticky bit on something like dd before doing this?
So, have fun with that!
At some point, I suspect I will.
______
Dennis

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#12 Post by amigo »

Have you experimented with setting the sticky bit on something like dd before doing this?
I don't understand what you mean -I know about the sticky bit, but don't understand how it relates here...

Using dd may also be safer than file -I have heard of security risks using file, but haven't seen full info.

I also wrote and use a clone of ldd which uses objdump and is lots safer than ldd itself which actually executes a binary to check its' deps.

DMcCunney
Posts: 889
Joined: Tue 03 Feb 2009, 00:45

#13 Post by DMcCunney »

amigo wrote:
Have you experimented with setting the sticky bit on something like dd before doing this?
I don't understand what you mean -I know about the sticky bit, but don't understand how it relates here...
I'm thinking about repeatedly calling a utility in a script.

In the old days, I set the sticky bit on various things I'd have occasion to call repeatedly in a script, like echo. Setting the sticky bit forced the executable to be kept in swap, where it could be reloaded more quickly, and helped execution times on scripts that called things repeatedly. (Back then, /bin/sh was the Bourne shell, and echo was an external command, not a shell built-in. Having echo as a built in in the Korn shell was one reason I clutched it happily to my chest.)

I haven't really played with doing it in Puppy, and don't know how it might affect things. (And thinking about it, I might copy stuff used like that in the script to /dev/shm and run it from there.)
Using dd may also be safer than file -I have heard of security risks using file, but haven't seen full info.
I'd be curious to know more. File is looking at the file header and using a "magic number" to decide what type of file it is. I fail to see how simply opening a local file and reading the header constitutes a security risk.
I also wrote and use a clone of ldd which uses objdump and is lots safer than ldd itself which actually executes a binary to check its' deps.
Is there some concern that the binary it executes might not be the one you expect, and may have malicious effects?
______
Dennis

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#14 Post by musher0 »

Hello, Bash fans!

Maybe you can dish out some pocket money and find "The Linux Shell Handbook", a special issue Linux Pro magazine, published in Spring 2010.
Might still be on the shelves of magazine outlets as of this writing.

When you're finished with it, you'll have earned a "Bash Baccalaureate"! Or so the cover says! :D
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
plankenstein
Posts: 120
Joined: Sun 16 Nov 2008, 00:49
Location: Arkansas, USA

#15 Post by plankenstein »

Downloading pet, will check out these other sites later. Sounds like great fun. I've done some reading on bash before, but not really done alot with it yet.

Thanks
I carefully plan ALL my random acts! :lol:

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#16 Post by amigo »

DMcCunney, I use the is_elf as a function in any script where it is needed, though I see what you mean now about using the sticky bit for external programs.

As I said, I'm unsure about the mechansim whereby file might be less-than-safe -maybe it only has to do with it getting the file-type wrong -there were/are issues with /etc/mailcap since it relys on file to decide the filetype.

As for ldd, it is a script which runs '/lib/ld-linux.so.2 --verify $@' to determine dependencies. Obviously, that could be disastrous when run on a malicious ELF binary... My replacement uses 'objdump $@ |grep NEEDED' to avoid that. It also provides a more accurate list of dependencies -ldd lists everything linked to by an object -even if ti doesn't really use them all- sometimes Makefiles link libraries which are not needed.

HiDeHo
Posts: 311
Joined: Wed 16 Mar 2011, 09:57

#17 Post by HiDeHo »

if the pet is not an application they y is it a .pet adn not a .tar.gz

seems weird to extract a pet and not install it

User avatar
ttuuxxx
Posts: 11171
Joined: Sat 05 May 2007, 10:00
Location: Ontario Canada,Sydney Australia
Contact:

#18 Post by ttuuxxx »

HiDeHo wrote:if the pet is not an application they y is it a .pet adn not a .tar.gz

seems weird to extract a pet and not install it
ummm yes it has a menu list under /documents
ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)

User avatar
tallboy
Posts: 1760
Joined: Tue 21 Sep 2010, 21:56
Location: Drøbak, Norway

#19 Post by tallboy »

It sometimes surprises me that so many Linux users are unfamiliar with one of the oldest sources for Linux info. Check out http://tldp.org - The Linux Documentation Project. Look under 'guides', there you'll find the latest version of Wendell Cooper's superb 'Advanced Bash-Scripting Guide'.

There are other useful guides there as well, like 'Introduction to Linux - A Hands on Guide' (don't laugh - read it and discover the 'white spots' on your Linux knowledge map!), and the extremely useful 'GNU/Linux Command-Line Tools Summary'; that one will definitely teach you new ways to live your life in the Linux world!

Tallboy.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#20 Post by technosaurus »

I wish someone would modify the advance BASH scripting guide to be the advanced SHELL scripting guide and point out the "bashisms"

perhaps it already exists?

re: file and dd
I have used read <file with a case $REPLY ...statement for this sort of thing it just reads the first line and is a builtin, but I hope it is safe?

you get []ELF for binary, shared libs and their symlinks
png yields PMG
xpm yields /* XPM */
scripts yield the shabang

here is an example - I wanted to find all occurences of dialog in all scripts so I hacked this together

usage
findusage string
output is a list of all files:linenumbers where $1 occurs

Code: Select all

#!/bin/sh
for FILE in ${PATH//://* }/*;  do
	[ ! -f ${FILE} ] && continue
	read HEAD <${FILE} 2>/dev/null
	case ${HEAD} in
	*/bin*sh*)
		i=0
		while read LINE; do
			i=$(($i+1))
			case $LINE in
				*${1}*)echo ${FILE}:$i;;
			esac
		done <${FILE}
	;;
	esac
done
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Post Reply