Canned Bash lessons - learn to use Bash commands
- ttuuxxx
- Posts: 11171
- Joined: Sat 05 May 2007, 10:00
- Location: Ontario Canada,Sydney Australia
- Contact:
Canned Bash lessons - learn to use Bash commands
Ok I came across this How to learn Bash commands, its full of scripts text, it looks pretty good and when I get some free time I'll read and do the lessons, anyways here it is
To start it, extract the package into a folder and go to ... and click on it
I also made a menu item shortcut in /menu/document
start the course
/usr/doc/bash/advanced/HTML/index.html
scripts
/usr/doc/bash/advanced
/usr/local/bin/doc/bash2_doc
ttuuxxx
To start it, extract the package into a folder and go to ... and click on it
I also made a menu item shortcut in /menu/document
start the course
/usr/doc/bash/advanced/HTML/index.html
scripts
/usr/doc/bash/advanced
/usr/local/bin/doc/bash2_doc
ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
Bash Lessons
wow ! thanks tuuxx this is cool beans.
elraven
elraven
- ttuuxxx
- Posts: 11171
- Joined: Sat 05 May 2007, 10:00
- Location: Ontario Canada,Sydney Australia
- Contact:
Re: Bash Lessons
Yes elraven I hope Lobster puts this link in a how to Wiki for people wanting to learn Bash scripting. This will increase people learning bash language and it will help puppy growelraven wrote:wow ! thanks tuuxx this is cool beans.
elraven
ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
Re: Bash Lessons How to ??
Thanks....I extracted it all and uploaded it to my account at one of the local colleges....that way I can learn and play with the scripts via ssh from anywhere.ttuuxxx wrote:Ok I came across this How to learn Bash commands, its full of scripts text, it looks pretty good and when I get some free time I'll read and do the lessons, anyways here it is
ttuuxxx
Thom
- Lobster
- Official Crustacean
- Posts: 15522
- Joined: Wed 04 May 2005, 06:06
- Location: Paradox Realm
- Contact:
Thanks for providing this Ttuuxxx
Puppy is at present largely customised Bash scripts
Combine a bit of bash with gtkdialog3 and you can create simple but immensely useful customized programs
Even I have done it with YAP for example . . .
http://puppylinux.org/wikka/BuddhistYAP
Puppy is at present largely customised Bash scripts
Combine a bit of bash with gtkdialog3 and you can create simple but immensely useful customized programs
Even I have done it with YAP for example . . .
http://puppylinux.org/wikka/BuddhistYAP
Last edited by Lobster on Mon 05 Dec 2011, 18:35, edited 1 time in total.
People interested in the topic should visit Heiner's Shelldorado, at http://www.shelldorado.com/
It's one of the best resources I know for shell scripting, and covers a lot more than just bash.
(Note that bash derives from the AT&T System V Bourne shell, with enhancements made on the Korn shell and some stuff first implemented in the C shell created by Bill Joy for BSD Unix. It tries to be a combination of all of them. Unfortunately, the Bash scripting guide doesn't specify which bits are specific to bash.)
______
Dennis
It's one of the best resources I know for shell scripting, and covers a lot more than just bash.
(Note that bash derives from the AT&T System V Bourne shell, with enhancements made on the Korn shell and some stuff first implemented in the C shell created by Bill Joy for BSD Unix. It tries to be a combination of all of them. Unfortunately, the Bash scripting guide doesn't specify which bits are specific to bash.)
______
Dennis
hey guys, try this link :
http://www.linuxconfig.org/Bash_scripting_Tutorial
i think thats tutorial is good for newbie on linux,
http://www.linuxconfig.org/Bash_scripting_Tutorial
i think thats tutorial is good for newbie on linux,
Learning by Doing
There is a program -perl script actually, called 'bashisms' which will purify scripts of bashisms. Very handy for creating more portable scripts. It is part of debian devtools, but has its' own project page on sourceforge.
You can also search for a web-page called 'lintsh' which lists some other quirks of many different shells.
I've been doing a lot of work lately on optimizing scripts to use the dash shell instead of bash because it runs 25-40% faster. But I have always been a 'basher', so my normal way of coding includes lots of 'bashisms'. The bashisms program and the notes on lintsh make it easier to find the parts that need to be changed to use dash, ash or other mostly POSIX-compliant shell. There are a couple of other things to watch for to be able to run scripts using a Bourne-compatible shell. There is a modernized version of the Bourne shell available from the 'heirloom' tools site on source forge -search for 'heirloom-sh'. It runs compliant scripts another 10-15% faster than dash.
Recently I wanted alter a script to not use an awk construct:
| awk ' { print $1 } '
My first attempt using shell code ran three times slower than calling awk. I went at it again and came up with a routine that runs 10 times faster than calling awk! I usually try to 'factor out' all calls to sed or awk, unless it is clear that using the external tools is faster.
You can also search for a web-page called 'lintsh' which lists some other quirks of many different shells.
I've been doing a lot of work lately on optimizing scripts to use the dash shell instead of bash because it runs 25-40% faster. But I have always been a 'basher', so my normal way of coding includes lots of 'bashisms'. The bashisms program and the notes on lintsh make it easier to find the parts that need to be changed to use dash, ash or other mostly POSIX-compliant shell. There are a couple of other things to watch for to be able to run scripts using a Bourne-compatible shell. There is a modernized version of the Bourne shell available from the 'heirloom' tools site on source forge -search for 'heirloom-sh'. It runs compliant scripts another 10-15% faster than dash.
Recently I wanted alter a script to not use an awk construct:
| awk ' { print $1 } '
My first attempt using shell code ran three times slower than calling awk. I went at it again and came up with a routine that runs 10 times faster than calling awk! I usually try to 'factor out' all calls to sed or awk, unless it is clear that using the external tools is faster.
The page is here: http://sourceforge.net/projects/checkbaskisms/amigo wrote:There is a program -perl script actually, called 'bashisms' which will purify scripts of bashisms. Very handy for creating more portable scripts. It is part of debian devtools, but has its' own project page on sourceforge.
Other distros use "ash" for the same purpose. As far as I can tell, ash is bash without all the code meant for interactive use at a command line. It's just an interpreter for the shell language, and preferable for things like install scripts.I've been doing a lot of work lately on optimizing scripts to use the dash shell instead of bash because it runs 25-40% faster.
It's part of http://heirloom.sourceforge.netThere is a modernized version of the Bourne shell available from the 'heirloom' tools site on source forge -search for 'heirloom-sh'. It runs compliant scripts another 10-15% faster than dash.
The trick is "compliant" scripts. The Bourne shell is not POSIX compliant (as the POSIX specs evolved well after the Bourne shell was written.) Hopefully, the heirloom shell is based on SysVR2 or later code, when shell functions got added to the Bourne shell.
What did your revised routine look like?Recently I wanted alter a script to not use an awk construct:
| awk ' { print $1 } '
My first attempt using shell code ran three times slower than calling awk. I went at it again and came up with a routine that runs 10 times faster than calling awk! I usually try to 'factor out' all calls to sed or awk, unless it is clear that using the external tools is faster.
The original Bourne shell language was minimal, intended to be used as glue to tie together external processes like awk or sed, and was mostly flow-of-control constructs. I'm an old Korn shell user, and one of the enhancements the Korn shell made was making things like "echo" a built-in rather than calling an external program, and adding built-in integer arithmetic so you didn't have to call eval. There's a sample Korn shell script that implements grep in the Korn shell. It's mostly a proof-of-concept - calling the real grep will almost certainly be much faster for any non-trivial search - but it was a neat example of what could be done entirely in the shell.
Versions of ksh after ksh88 could be successfully installed as /bin/sh on Sun systems (and I did so).
Ubuntu includes ksh, and I looked at trying to move it to Puppy, but there are library dependencies I didn't have time to track down when I looked.
There is also the Z shell - zsh - with a PET available for Puppy.
See http://en.wikipedia.org/wiki/Z_shell and the Sourceforge page at http://zsh.sourceforge.net/
(I have a Win32 port of zsh as part of the UNXUTILS package.)
______
Dennis
Thanks for supplying those links. Sometimes I use the Alfred Stieglitz approach and only give a few clues and let the reader do the searching -when things aren't hard to find.
Alfred Stieglitz was a famous turn-of-the-century(20th) photgrapher who was married to Geogia O'Keefe. He was the first person to exhibit Matisse's work in the USA, at his studio in New York. He had no business cards for the studio, and when asked about it he said: "If a man wants something, he'll find it".
Anyway,
heirloom-sh is SVR4 compliant and includes functions and other 'goodies'. if invoked as 'jsh', it includes job-control, so it could be used as a login shell -but I don't even set it up as /bin/sh. There are too many scripts out there which use /bin/sh, but only if /bin/sh is a link to bash.
About dash/ash. ash is mostly POSIX-compliant, but dash claims to be fully POSIX-compliant. You'll find that the busybox version of ash is not the same as the real ash -you need to write Bourne-compliant code for busybox ash (especially regarding the use of `code` backticks instead of $(code) constructs. Another difference between POSIX and Bourne is that Bourne lacks the 'test -e file' (or [ -e file ])check.
So, for installers and initrd's which use busybox, it is best to use only Bourne-compliant syntax.
Here's my fake awk implementation:
I seem to have deleted my original version which was slower than the real thing, or I'd post that, too.
The original script I was modifying used awk like that in 2 or 3 places and also sed. By rewriting those sections and getting rid of useless sort |uniq statements and making everything Bourne-compliant, I was able to make the script run about 40% faster using dash and around 50% faster using heirloom-sh.
Here's the sed trick:
Original code:
New code:
Here'S another nice trick for speeding up a common routine -to check if a file is an ELF binary. Original:
New code:
Using dd to check for the file signature is safer than running file, and is much faster than calling a file|grep construction -really noticeable where the rouitn gets called 100's/1,000's of times haven't checked specifically, but probably also at least 50% faster...
So, have fun with that!
Alfred Stieglitz was a famous turn-of-the-century(20th) photgrapher who was married to Geogia O'Keefe. He was the first person to exhibit Matisse's work in the USA, at his studio in New York. He had no business cards for the studio, and when asked about it he said: "If a man wants something, he'll find it".
Anyway,
heirloom-sh is SVR4 compliant and includes functions and other 'goodies'. if invoked as 'jsh', it includes job-control, so it could be used as a login shell -but I don't even set it up as /bin/sh. There are too many scripts out there which use /bin/sh, but only if /bin/sh is a link to bash.
About dash/ash. ash is mostly POSIX-compliant, but dash claims to be fully POSIX-compliant. You'll find that the busybox version of ash is not the same as the real ash -you need to write Bourne-compliant code for busybox ash (especially regarding the use of `code` backticks instead of $(code) constructs. Another difference between POSIX and Bourne is that Bourne lacks the 'test -e file' (or [ -e file ])check.
So, for installers and initrd's which use busybox, it is best to use only Bourne-compliant syntax.
Here's my fake awk implementation:
Code: Select all
fake_awk_print() {
CNT=1
while [ "$CNT" -le "$CHOICE" ]; do
#echo $1
#echo $CNT
#CNT=$(( $CNT+1 ))
case $1 in
' '|' ') true ;;
*)
if [ $CNT = $CHOICE ] && [ $1 != '' ] ; then
#if [ $CNT = $CHOICE ] ; then
echo $1 && break
else
CNT=$(( $CNT+1 )) ; shift
fi
;;
esac
#CNT=$(( $CNT+1 ))
#[ $CNT -le $(($ARGS)) ] && shift
#shift
done
}
# call the funtion like this:
fake_awk_print $@
# original code like this:
# echo "$(for i in $(ldd "$FILE" | grep 'not found' | sort | uniq | awk '{print $1}'); do echo "3:> Missing: $i"; done)" >> "$RESULT"
#can be replaced with this:
#echo "$(for i in $(ldd "$FILE" | grep 'not found' | sort | uniq | awk_print 1 2> /dev/null); do echo "3:> Missing: $i"; done)" >> "$RESULT"
The original script I was modifying used awk like that in 2 or 3 places and also sed. By rewriting those sections and getting rid of useless sort |uniq statements and making everything Bourne-compliant, I was able to make the script run about 40% faster using dash and around 50% faster using heirloom-sh.
Here's the sed trick:
Original code:
Code: Select all
BINDIRS="$(echo ${PATH} | sed 's/:/ /g;s/\.$//')"
Code: Select all
BINDIRS="`IFS=: ; echo ${PATH}`"
Code: Select all
if [ $(file "$FILE" | grep -q 'ELF') ]; then
Code: Select all
is_elf()
{
[ "`dd if=$1 bs=1 skip=1 count=3 2> /dev/null`" = "ELF" ]
}
# call like this:
if is_elf $file ; then
So, have fun with that!
I know who Stieglitz was. I also live in walking distance of the site of the famous "Armory Show" in 1913amigo wrote:Thanks for supplying those links. Sometimes I use the Alfred Stieglitz approach and only give a few clues and let the reader do the searching -when things aren't hard to find.
Alfred Stieglitz was a famous turn-of-the-century(20th) photgrapher who was married to Geogia O'Keefe. He was the first person to exhibit Matisse's work in the USA, at his studio in New York. He had no business cards for the studio, and when asked about it he said: "If a man wants something, he'll find it".
dd is bigger than file, but using file requires calling grep, too, so I see where the speedup comes from.Using dd to check for the file signature is safer than running file, and is much faster than calling a file|grep construction -really noticeable where the rouitn gets called 100's/1,000's of times haven't checked specifically, but probably also at least 50% faster...
Have you experimented with setting the sticky bit on something like dd before doing this?
At some point, I suspect I will.So, have fun with that!
______
Dennis
I don't understand what you mean -I know about the sticky bit, but don't understand how it relates here...Have you experimented with setting the sticky bit on something like dd before doing this?
Using dd may also be safer than file -I have heard of security risks using file, but haven't seen full info.
I also wrote and use a clone of ldd which uses objdump and is lots safer than ldd itself which actually executes a binary to check its' deps.
I'm thinking about repeatedly calling a utility in a script.amigo wrote:I don't understand what you mean -I know about the sticky bit, but don't understand how it relates here...Have you experimented with setting the sticky bit on something like dd before doing this?
In the old days, I set the sticky bit on various things I'd have occasion to call repeatedly in a script, like echo. Setting the sticky bit forced the executable to be kept in swap, where it could be reloaded more quickly, and helped execution times on scripts that called things repeatedly. (Back then, /bin/sh was the Bourne shell, and echo was an external command, not a shell built-in. Having echo as a built in in the Korn shell was one reason I clutched it happily to my chest.)
I haven't really played with doing it in Puppy, and don't know how it might affect things. (And thinking about it, I might copy stuff used like that in the script to /dev/shm and run it from there.)
I'd be curious to know more. File is looking at the file header and using a "magic number" to decide what type of file it is. I fail to see how simply opening a local file and reading the header constitutes a security risk.Using dd may also be safer than file -I have heard of security risks using file, but haven't seen full info.
Is there some concern that the binary it executes might not be the one you expect, and may have malicious effects?I also wrote and use a clone of ldd which uses objdump and is lots safer than ldd itself which actually executes a binary to check its' deps.
______
Dennis
Hello, Bash fans!
Maybe you can dish out some pocket money and find "The Linux Shell Handbook", a special issue Linux Pro magazine, published in Spring 2010.
Might still be on the shelves of magazine outlets as of this writing.
When you're finished with it, you'll have earned a "Bash Baccalaureate"! Or so the cover says!
Maybe you can dish out some pocket money and find "The Linux Shell Handbook", a special issue Linux Pro magazine, published in Spring 2010.
Might still be on the shelves of magazine outlets as of this writing.
When you're finished with it, you'll have earned a "Bash Baccalaureate"! Or so the cover says!
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
- plankenstein
- Posts: 120
- Joined: Sun 16 Nov 2008, 00:49
- Location: Arkansas, USA
DMcCunney, I use the is_elf as a function in any script where it is needed, though I see what you mean now about using the sticky bit for external programs.
As I said, I'm unsure about the mechansim whereby file might be less-than-safe -maybe it only has to do with it getting the file-type wrong -there were/are issues with /etc/mailcap since it relys on file to decide the filetype.
As for ldd, it is a script which runs '/lib/ld-linux.so.2 --verify $@' to determine dependencies. Obviously, that could be disastrous when run on a malicious ELF binary... My replacement uses 'objdump $@ |grep NEEDED' to avoid that. It also provides a more accurate list of dependencies -ldd lists everything linked to by an object -even if ti doesn't really use them all- sometimes Makefiles link libraries which are not needed.
As I said, I'm unsure about the mechansim whereby file might be less-than-safe -maybe it only has to do with it getting the file-type wrong -there were/are issues with /etc/mailcap since it relys on file to decide the filetype.
As for ldd, it is a script which runs '/lib/ld-linux.so.2 --verify $@' to determine dependencies. Obviously, that could be disastrous when run on a malicious ELF binary... My replacement uses 'objdump $@ |grep NEEDED' to avoid that. It also provides a more accurate list of dependencies -ldd lists everything linked to by an object -even if ti doesn't really use them all- sometimes Makefiles link libraries which are not needed.
- ttuuxxx
- Posts: 11171
- Joined: Sat 05 May 2007, 10:00
- Location: Ontario Canada,Sydney Australia
- Contact:
ummm yes it has a menu list under /documentsHiDeHo wrote:if the pet is not an application they y is it a .pet adn not a .tar.gz
seems weird to extract a pet and not install it
ttuuxxx
http://audio.online-convert.com/ <-- excellent site
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
http://samples.mplayerhq.hu/A-codecs/ <-- Codec Test Files
http://html5games.com/ <-- excellent HTML5 games :)
It sometimes surprises me that so many Linux users are unfamiliar with one of the oldest sources for Linux info. Check out http://tldp.org - The Linux Documentation Project. Look under 'guides', there you'll find the latest version of Wendell Cooper's superb 'Advanced Bash-Scripting Guide'.
There are other useful guides there as well, like 'Introduction to Linux - A Hands on Guide' (don't laugh - read it and discover the 'white spots' on your Linux knowledge map!), and the extremely useful 'GNU/Linux Command-Line Tools Summary'; that one will definitely teach you new ways to live your life in the Linux world!
Tallboy.
There are other useful guides there as well, like 'Introduction to Linux - A Hands on Guide' (don't laugh - read it and discover the 'white spots' on your Linux knowledge map!), and the extremely useful 'GNU/Linux Command-Line Tools Summary'; that one will definitely teach you new ways to live your life in the Linux world!
Tallboy.
- technosaurus
- Posts: 4853
- Joined: Mon 19 May 2008, 01:24
- Location: Blue Springs, MO
- Contact:
I wish someone would modify the advance BASH scripting guide to be the advanced SHELL scripting guide and point out the "bashisms"
perhaps it already exists?
re: file and dd
I have used read <file with a case $REPLY ...statement for this sort of thing it just reads the first line and is a builtin, but I hope it is safe?
you get []ELF for binary, shared libs and their symlinks
png yields PMG
xpm yields /* XPM */
scripts yield the shabang
here is an example - I wanted to find all occurences of dialog in all scripts so I hacked this together
usage
findusage string
output is a list of all files:linenumbers where $1 occurs
perhaps it already exists?
re: file and dd
I have used read <file with a case $REPLY ...statement for this sort of thing it just reads the first line and is a builtin, but I hope it is safe?
you get []ELF for binary, shared libs and their symlinks
png yields PMG
xpm yields /* XPM */
scripts yield the shabang
here is an example - I wanted to find all occurences of dialog in all scripts so I hacked this together
usage
findusage string
output is a list of all files:linenumbers where $1 occurs
Code: Select all
#!/bin/sh
for FILE in ${PATH//://* }/*; do
[ ! -f ${FILE} ] && continue
read HEAD <${FILE} 2>/dev/null
case ${HEAD} in
*/bin*sh*)
i=0
while read LINE; do
i=$(($i+1))
case $LINE in
*${1}*)echo ${FILE}:$i;;
esac
done <${FILE}
;;
esac
done
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].