How to put a progress bar in dir2sfs? (Solved)

[stemsee]

Now I made the script give defaults for overwrite and compression options. And in case of using Stop to interrupt, the partially created sfs is removed.

Code: Select all

if [ -f "$1.sfs" ];then
  DO=$(yad --window-icon="application-x-squashfs-image" --title="Overwrite" --text="<b>$(basename $1).sfs</b> already exists.\n Would you like to overwrite or rename it?\n" \
  --text-align="center" --borders="10" --buttons-layout="center" --button="gtk-yes:1" --button="gtk-no:0" --center --timeout 2)

  case $DO in
    0|252) exit ;;
    70|1) rm -f "$1.sfs" "$1.sfs-md5.txt" ;;
  esac
fi

Code: Select all

--button="GZIP:0" --button="XZ:1" --button="XZ-HC:2" --center --timeout 2
[[ $? == "" ]] && $?="1"
case $? in
  0) COMP="-comp gzip" ;;
  1) COMP="-comp xz" ;;
  2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
  *) exit ;;
esac

Code: Select all

if [ $? -ne 0 ]; then
  rm -f "$1.sfs"
  killall mksquashfs
  kill $PID
  rm -f typescript
  exit
fi

However killall mksquashfs is not ideal in case you have more than one mksquashfs job running! We need to target the specific mksquashfs process id! Or by getting the time each process started we can use ps -e | grep mksquashfs, time running to determine which process to kill.

EDIT: Maybe we can assign the pid for each mksquashfs process.

Code: Select all

echo "$((RANDOM))" >/tmp/$1

Code: Select all

cat  /tmp/$1 > /proc/sys/kernel/ns_last_pid; mksquashfs

Code: Select all

kill $(cat /tmp/$1)

[rufwoof]

I'd suggest including lz4 compression also. Very fast (but relatively poor compression). lz4 -Xhc tends to be more like gzip for compression, but is slower (relatively speaking), so not really worth including that IMO when gzip is available (unless you want very fast decompression also), just the standard lz4 ... for its speed. lzo -Xcompression-level 1 is a alternative, not as quick, but relatively quick ... and more commonly available (lzop is in busybox for instance).

Re killing. I know you can get a process name for a pid using the comm= parameter

# ps -p 2523 -o comm=
geany

maybe something along those lines might be used??

[stemsee]

Hi rufwoof

can you provide a scripted example of your ps * comm= command? I could not use it as is.

Added lz4.

I found a way to catch the mksquashfs pid by time, but I cannot now get the script to kill it.

Code: Select all

time=$(date | awk '{print $4}')  # unique time label

( sleep 1; ps -e | grep mksquashfs | tail -1 | awk '{print $1}' >/tmp/$time; exit ) & # delay and catch the last mksquashfs pid and store in file

(script -q -c "stty rows 40 cols 100; mksquashfs "$1" "$1.sfs" $COMP" | \
while read -n 100 LINE; do echo $LINE | busybox strings | egrep '[0-9]\%' | \
awk '{print $NF}'; done ) | yad --progress --center --undecorated --on-top \
--skip-taskbar --progress-text="Building $1.sfs"  --borders="10" \
--text-align="center" --button="gtk-stop:0" \
--buttons-layout="center" --auto-close

if [[ $? -eq 0 ]]; then
  pd=$(cat /tmp/$time) # read file for pid
  kill $pd
  rm -f "$1.sfs"
  rm -f "$1.sfs-md5.txt"
  rm -f typescript
  exit
fi

[rufwoof]

Maybe its different under Fatdog (proper/full Linux) ??

For instance if I run galculator and put it into the background, it shows pid 4550 was used. If I then run ps -p 4550 -o comm= ... then it returns the name i.e. galculator.

# galculator &
[1] 4550
# ps -p 4550 -o comm=
galculator
#

Does that do similar for you?

If you use args= instead, then it shows both the command and arguments. Starting geany /mnt/sdb1/menu.lst for instance

# geany /mnt/sdb1/menu.lst &
[1] 4899
# ps -p 4899 -o comm=
geany
# ps -p 4899 -o args=
geany /mnt/sdb1/menu.lst
#

Another approach might be to bring the relevant backgrounded task to the foreground and kill that (i.e. normal fg, bg, jobs commands)

# jobs
[1]+ Running geany /mnt/sdb1/menu.lst &
#
# fg %1
... and geany becomes the active window/task

or kill %1 ... to kill it.

With mksquashfs it does use multi-cores, but running mksquashfs /usr usr.sfs & and then I can run jobs and see its [2] and fg %2 brings that to the front and I can ctrl-c end it and all cores (child) tasks also closed. When doing that interactively however you get the mksquashfs progress bars ticking away between running the commands unless you direct that to /dev/null.

You can select jobs by the first few characters of the string

galculator &
fg %gal

or via a string within the command line, so if you were mksquashfs /usr ... and mksquashfs /lib64 ... then you could foreground (or kill) the mksquashfs /lib one by using fg %?lib ... i.e. ?lib matches the substring lib contained within the command line.

[stemsee]

Cracked it!

I got it working with specific mksquashfs process IDs. And lz4 option.

Probably there is a better way.

stemsee

[technosaurus]

I'd suggest including lz4 compression also. Very fast (but relatively poor compression). lz4 -Xhc tends to be more like gzip for compression, but is slower (relatively speaking), so not really worth including that IMO when gzip is available (unless you want very fast decompression also), just the standard lz4 ... for its speed. lzo -Xcompression-level 1 is a alternative, not as quick, but relatively quick ... and more commonly available (lzop is in busybox for instance).

For mksquashfs slow compression isn't that big of a deal since it only compresses once but gets decompressed thousands of times over. Using lz4 with high compression can give comparable compression to gzip but decompresses 10s of times faster - kind of important for read only filesystems especially during boot. zstd is another one that got added in recent years that is better than deflate (gzip) in compression speed, compression ratio and decompression speed - with the possibility of approaching xz levels of compression with minimal effects on speed. There was another one added for small data (4096 bytes) to better compress swap memory called zBeWalgo or 842 - looks like it would be horrible for squashfs

[stemsee]

ZSTD and LZ4 -Xhc options available.

also tidied up ending. md5 was not created!

[step]

Cracked it!

I got it working with specific mksquashfs process IDs. And lz4 option.

Probably there is a better way.

stemsee

Code: Select all

# original
( sleep 0.3; ps -e | grep mksquashfs | tail -1 | awk '{print $1}' >/tmp/$time ) &

# replace tail -1 with awk:
# when awk enters END $0 is the last input line
( sleep 0.3; ps -e | grep mksquashfs | awk 'END {print $1}' >/tmp/$time ) &

A shorter way to get the pid of the lastly-executed mksquashfs command is pgrep -n mksquashfs

Code: Select all

( sleep 0.3; pgrep -n mksquashfs >/tmp/$time ) &

(pgrep must be installed)

Concerning the initialization of variable time, it could be simplified this way
time=$(date +%T). You could replace %T with %s, too.

SIDE NOTE

A bigger change (but worth the learning exercise, and the improvement it would make to script robustness) is to replace the whole idea of writing a temporary script and attaching it to the Quit button, is to set an exit trap.

Code: Select all

trap 'my commands here' EXIT

Then, when the script is about to exit it runs 'my commands here'. Your commands could be the stuff that currently goes into the temporary script.

Two gotchas concerning traps:

1/
You need to set the trap in the subshell that is supposed to run it.

Code: Select all

# main script
trap 'echo BYE' EXIT

prints BYE when main script exits.

Code: Select all

# main script
trap 'echo BYE' EXIT
# subshell
(trap 'echo HI' EXIT)

prints HI --from the subshell-- then BYE --from main script.

Any command placed to the right of a pipe character runs in a subshell.

2/
trap "echo "$VAR"" substitutes $VAR when the *trap* command is executed, while trap 'echo "$VAR"' substitutes $VAR when echo is executed (on EXIT). In the latter case $VAR must be defined in the process that *executes* echo.

In the latter case, if the trap runs in a subshell, the main script needs to export VAR otherwise the subshell will find $VAR == "".

[stemsee]

Excellent intro to trap, thanks step. It would be appreciated if you implemented your suggestions.

At first I was using a function. But the behaviour of yad buttons and the syntax kept changing....

Code: Select all

--button="Stop:1"

Code: Select all

--button="Stop:bash -c 'endit'"

Code: Select all

--button="Stop":"bash -c 'endit'"

Code: Select all

--button="Stop" "bash -c 'endit'"

Code: Select all

--button="Stop" "echo 1"

Code: Select all

--button="Stop":"echo 1"

Code: Select all

--button="Stop!gtk-stop!Stop operation:1"

Can anyone offer guidance on the various behaviours??

[stemsee]

This version with some of step's optimisations. I left the exit script for now.

[step]

Excellent intro to trap, thanks step. It would be appreciated if you implemented your suggestions.

At first I was using a function. But the behaviour of yad buttons and the syntax kept changing....

--button=BUTTON:ID
Add the dialog button. May be used multiple times. ID is an exit code or a command. BUTTON may be gtk stock item name for predefined
buttons (like gtk-close or gtk-ok) or text in a form LABEL[!ICON[!TOOLTIP]] where `!' is an item separator. Full list of stock items
may be found in gtk-demo program, in snippet called "Stock Items and Icon Browser". If no buttons specified OK and Cancel buttons used.
See EXIT STATUS section for more. If ID have a non-numeric value it treats like a command and click on such button doesn't close the
dialog.

First you can notice that there's only one argument, BUTTON:ID. Therefore, however one liberally adds quotes inside and outside this option, the result must *a single string* equivalent to what one gets from this minimal and legit quoting style:

Code: Select all

"--button=whatever I need to put in here"

for the whole option.

Then you notice that man yad says BUTTON and ID and the separator in between ":". Therefore the rightmost ":" in the "whatever I need to put in here" is taken to split BUTTON and ID. The rightmost ":" is. Can one include a ":" in BUTTON? Sure, but there must be another ":" further to the right to indicate where ID starts.

We'll get back to ID but for the moment being let's say that it can be null, therefore ":" can be omitted. Let's be clear about this. You can omit ":" in the sense that you can omit the last character of the entire option string when that character is a ":" because ID is null.

All right, now let's dissect BUTTON. LABEL[!ICON[!TOOLTIP]] means that
* there must be a LABEL (which can be "" but we'll go back to that)
* if there is an ICON it must be preceded by "!" to separate it from the label
* if there is a tooltip it must be preceded by "!"ICON

ICON is either an icon name, e.g., gtk-yes (for a stock icon), yad (for a theme icon either under your current GTK there, failing that under /usr/share/icons/hicolor), and /path/to/my/icon.png (for a free-standing icon file). Detail: you can put hicolor also under /usr/local/share/icons and ~/.local/share/icons.

I mentioned LABEL can be null (even though man yad doesn't mention it). So the absolute minimum custom button spec is, e.g.,

Code: Select all

--button=gtk-stop

Implicitly it means

Code: Select all

--button=""gtk-stop":"0

where "" is LABEL and 0 is ID and I had to add ":" to separate ID from BUTTON.

Try adding a LABEL to the above example.
Try adding a tooltip.
Try a null LABEL but keep the tooltip.
Try a null ICON.

Finally, let's dissect ID.
Man yad says that ID is either a numeric return code or a command, of course subject to the constraint that "BUTTON:your command here" must boil down to a single string, not multiple ones. It becomes an exercise in shell quoting syntax, which can be complicated at times. You know where you want to go (to a single string), try quoting in various ways till you get there.

Can anyone offer guidance on the various behaviours??

See above. Now you can study your examples, and eliminate the ones that are simply wrong (because they violate the very first rule). The behavior of the remaining examples can be explained following the points I discussed above.

HTH

[stemsee]

buttons fixed following step's lesson!

Code: Select all

#!/bin/bash
#
[ ! -d "$1" ] && exit

if [ "$(echo "$1" | wc -w)" -gt 1 ]; then
  yad --window-icon="application-x-squashfs-image" --title="Rename Folder" \
  --text="Remove blank spaces from\n<b>$(basename "$1")</b> and try again." \
  --text-align="center" --borders="10" --width="220" --no-buttons --timeout="3" --center
exit
fi

if [ -f "$1.sfs" ];then
  yad --window-icon="application-x-squashfs-image" --title="Overwrite" \
  --text="<b>$(basename $1).sfs</b> already exists.\n Would you like to overwrite it?\n" \
  --text-align="center" --borders="10" --buttons-layout="center" --button="gtk-yes:1" --button="gtk-no:0" \
  --center --timeout-indicator=top --timeout 7

  case $? in
    0|252) exit ;;
    70|1) rm -f "$1".sfs
    rm -f "$1".sfs-md5.txt;;
  esac
fi
yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type: (default xz)" \
--text="LZ4 is fastest; GZIP for older machines
XZ-HC is the highest compression; ZSTD is latest tech." --text-align="center" --borders="10" --buttons-layout="center" \
--columns=2 --button="GZIP:0" --button="XZ:1" --button="XZ-HC:2" \
--button="LZ4:3" --button="LZ4-HC:4" --button="ZSTD:5" --center --timeout-indicator=top --timeout 7
case $? in
  0) COMP="-comp gzip" ;;
  1) COMP="-comp xz" ;;
  2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
  3) COMP="-comp lz4" ;;
  4) COMP="-comp lz4 -Xhc" ;;
  5) COMP="-comp zstd" ;;
  *) exit ;;
esac
( sleep 0.5; pgrep -n mksquashfs >/tmp/$$ ) &
(script -q -c "stty rows 40 cols 100; mksquashfs "$1" "$1.sfs" $COMP" | \
while read -n 100 LINE; do  echo $LINE | busybox strings | egrep '[0-9]\%' | \
awk '{print $NF}'; done ) | yad --progress --center --undecorated \
--skip-taskbar --progress-text="Building $1.sfs $COMP" --borders="10" \
--text-align="center" --button="Stop:1'" \
--buttons-layout="center" --auto-close
case $? in
1) kill $(cat /tmp/$$)
[[ -f "$1.sfs" ]] && rm -f "$1.sfs"
;;
esac
[[ -f "$1.sfs" ]] && md5sum "$1.sfs" > "$1.sfs-md5.txt"
rm -f typescript
rm -f /tmp/$$
sync