How to put a progress bar in dir2sfs? (Solved)

[rg66]

Hi all,

I'm trying to get dir2sfs to have a working progress bar. I think it was Jemimah who originally added some code to the script but that doesn't seem to work too well with current puppies, it only captures % a few times. Any help would be much appreciated.

Old code:

Code: Select all

mksquashfs "$t" "$z.sfs" |while read -n 50 LINE ; do 
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0

[sc0ttman]

Hi mate, sorry I can't be more helpful than this, but IIRC (it was about 4 years ago), one of the Woofy versions I did (not latest) had SFS progress bars, but it actually slowed down the speed of making the SFS, so I removed it..

But it should be in *one* of the woofy versions I did...

Sorry - I know that i more a chore/errand than actual help...

[fredx181]

Hi rg66

When I do e.g:

Code: Select all

mksquashfs "squashfs-root" "new.sfs" > file

Or:

Code: Select all

mksquashfs "squashfs-root" "new.sfs" | tee file

There are only a few (progress) lines in "file"
I think that explains why you get so little change of % from Xdialog
So I'm afraid it's just not possible to capture mksquashfs output in % by % precisely. Hope I'm wrong, though, I'd like Xdialog progress to work also.

Fred

[musher0]

Guys?

mksquasfs has a progress bar by default.
Please see attached.

BFN.

[rufwoof]

Remember that mksquashfs will use multiple cores if available/specified so its buffering is 'unusual'. By running it through script (a old command so commonly available) and line buffering grep will help, something like ....

Code: Select all

#!/bin/bash
script -q -c "mksquashfs /home /tmp/test.sfs -info -progress -noappend \
-processors 4" | \
grep --line-buffered '\[' | \
grep --line-buffered '\]' | \
grep --line-buffered '\%' | \
sed -u 's/.*\(....\)/\1/'

For large sized mksquashfs however, it will stick on 0% for quite a while and will tend to jump quickly towards the end, perhaps going from 49% to 99% in a quick jump despite having taken a while to get from 0% to 1%. That's all part of compression (takes a while to deduce the codebits that will be used to represent character sequences, but once those codebits are in place can encode quickly).

PS that last sed command just prints out the last 4 (count the dots) characters of a line (again using no buffering (-u parameter)).

[fredx181]

Thanks rufwoof,

Using the 'script' command makes it work nicely, so modified code from first post:

Code: Select all

script -q -c "mksquashfs "$t" "$z.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0

@musher0

mksquasfs has a progress bar by default.

Yes, but only from terminal, it's nice to be able to use Xdialog gui progress bar also.

Fred

[rg66]

Thanks rufwoof,

Using the 'script' command makes it work nicely, so modified code from first post:

Code: Select all

script -q -c "mksquashfs "$t" "$z.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0

@musher0

mksquasfs has a progress bar by default.

Yes, but only from terminal, it's nice to be able to use Xdialog gui progress bar also.

Fred

Cheers rufwoof and Fred, exactly what I was after.

[rg66]

Hmmmmm, it only seems to work when run from terminal. Dropping a dir on the script or running from thunar custom action shows no progress.

[fredx181]

Hmmmmm, it only seems to work when run from terminal. Dropping a dir on the script or running from thunar custom action shows no progress.

Yes indeed, I see also now, pity !

EDIT:
This works for me by dropping a dir on the script or running from thunar custom action:

Code: Select all

t=$1
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "new.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0

Fred

[rg66]

Works for me too, thanks Fred. Here's my dir2sfs which has compression options.

Code: Select all

#!/bin/bash -x

t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`

if [ ! -d "$t" ];then

  echo "error: no valid folder specified!"
  exit 0

fi

### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`

#echo --- $check


#echo $t - $z


if [ -f "$z.sfs" ];then

yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="$z.sfs already exists, refusing to overwrite it!" --button="gtk-ok" \
--borders="10" --buttons-layout="center"
  exit 0

fi

yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" --button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
echo $LINE | egrep '[0-9]\%'| cut -f1 -d% | awk '{print $NF}'| grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync


echo
echo
s=`du -m "$z.sfs" | sed "s/\s.*//"`
echo "created: $z.sfs ( $s MB )"
echo "created: $z.sfs-md5.txt"
echo
echo "...byebye..."
echo

[phat7]

Works for me too, thanks Fred. Here's my dir2sfs which has compression options.

Code: Select all

#!/bin/bash -x

t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`

if [ ! -d "$t" ];then

  echo "error: no valid folder specified!"
  exit 0

fi

### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`

#echo --- $check


#echo $t - $z


if [ -f "$z.sfs" ];then

yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="$z.sfs already exists, refusing to overwrite it!" --button="gtk-ok" \
--borders="10" --buttons-layout="center"
  exit 0

fi

yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" --button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
echo $LINE | egrep '[0-9]\%'| cut -f1 -d% | awk '{print $NF}'| grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync


echo
echo
s=`du -m "$z.sfs" | sed "s/\s.*//"`
echo "created: $z.sfs ( $s MB )"
echo "created: $z.sfs-md5.txt"
echo
echo "...byebye..."
echo

Some elements now GUI only, some CLI only

[fredx181]

Works for me too, thanks Fred. Here's my dir2sfs which has compression options.

Thanks, I made for myself small change, to be able to cancel the mksquashfs process by closing the progressbar window:

Code: Select all

(
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done &
) | Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait... \n To cancel: Close this window" 7 70 0

if [ $? -ne 0 ]; then    # progressbar closed by user, so kill mksquashfs process
SQUASHPID=$(ps -eo pid,cmd | grep -v grep | grep "mksquashfs $t $z.sfs" | awk '{ print $1 }')
kill $SQUASHPID
yad --center --borders=10 --title="SFS creation canceled" --window-icon="application-x-squashfs-image" --width=300 --text="  SFS creation canceled " --timeout=5 --button="gtk-close:0" &
exit
fi
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync

Fred

[rg66]

Some elements now GUI only, some CLI only

There is no interaction with the command line. All the echo lines were there originally and I don't really see a need to remove them.

[rg66]

Thanks, I made for myself small change, to be able to cancel the mksquashfs process by closing the progressbar window:

Nice, very handy. Here's one using only yad
(I've noticed that both yad and Xdialog reach %100 before the sfs is actually finished using xz compression)

Code: Select all

#!/bin/bash 

t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`

if [ ! -d "$t" ];then
  echo "error: no valid folder specified!"
  exit 0
fi

### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`

if [ -f "$z.sfs" ];then
  yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="<b>$z.sfs already exists, refusing to overwrite it!</b>" --button="gtk-ok" \
  --text-align="center" --borders="10" --buttons-layout="center"
  exit 0
fi

yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" \
--button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
  0) COMP="-comp gzip" ;;
  1) COMP="-comp xz" ;;
  2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
  *) exit ;;
esac

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG 
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close  
case $? in
0) while [ "`pidof mksquashfs`" ]; do
   sleep .2
   done ;;
*) killall mksquashfs
   rm -f typescript
   exit ;;  
esac

md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync 

I've even removed all the CLI echo lines

[phat7]

I've even removed all the CLI echo lines

Almost all

Code: Select all

if [ ! -d "$t" ];then 
   echo "error: no valid folder specified!" 
   exit 0 
 fi

[belham2]

Hi rg66 & Fred,

Ok, kind of a dunce question: rg66, can I just copy your script, put it in a blank text file, name it whatever (i.e. "squashingit.sh"), then "chmod +x ..." it, then grab an icon for it, and save it to a handy place (desktop?). Thus I'll then be able to drag ANY directory I was getting ready to squash and place it on top of the script, and PRESTO..the script will come to life squashing that directory without me doing anything else?

Sorry if the above is a noob question; I just want to make sure I understand so i can use it. I've been unsquashing and especially squashing a lot of stuff this past week (modifying sfs files), and this script would save me the headache of pounding the long command I use into the terminal.

Only thing I wonder, is this script or can this script, be modified to compress at max "xz"? I use (what you taught me, Fred) this following command always when I am re-squashing a SFS and I am looking for smallest size possible:

Code: Select all

# mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86

.....and for x86_64, I just add "_64" to the "x86" above (hope I am doing that correct)


Thanks, guys, for any feedback.

[rg66]

Almost all

Code: Select all

if [ ! -d "$t" ];then 
   echo "error: no valid folder specified!" 
   exit 0 
 fi

Yep, definitely missed that one. Here's a new copy with option to over write existing sfs.

Code: Select all

#!/bin/bash

t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`

[ ! -d "$t" ] && exit

if [ -f "$z.sfs" ];then
  yad --window-icon="application-x-squashfs-image" --title="Overwrite" --text="<b>$(basename $z).sfs</b> already exists.\n Would you like to overwrite it?\n" \
  --text-align="center" --borders="10" --buttons-layout="center" --button="gtk-yes:0" --button="gtk-no:1"
  case $? in
    0) rm -f "$z.sfs" "$z.sfs-md5.txt" ;;
    *) exit ;;
  esac
fi

yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" \
--button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
  0) COMP="-comp gzip" ;;
  1) COMP="-comp xz" ;;
  2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
  *) exit ;;
esac

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG 
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close  
case $? in
0) while [ "`pidof mksquashfs`" ]; do
   sleep .2
   done ;;
*) killall mksquashfs
   rm -f typescript
   exit ;;  
esac

rm -f typescript
md5sum "$z.sfs" > "$z.sfs-md5.txt"
sync 

[rg66]

Hi rg66 & Fred,

Ok, kind of a dunce question: rg66, can I just copy your script, put it in a blank text file, name it whatever (i.e. "squashingit.sh"), then "chmod +x ..." it, then grab an icon for it, and save it to a handy place (desktop?). Thus I'll then be able to drag ANY directory I was getting ready to squash and place it on top of the script, and PRESTO..the script will come to life squashing that directory without me doing anything else?

Sorry if the above is a noob question; I just want to make sure I understand so i can use it. I've been unsquashing and especially squashing a lot of stuff this past week (modifying sfs files), and this script would save me the headache of pounding the long command I use into the terminal.

Only thing I wonder, is this script or can this script, be modified to compress at max "xz"? I use (what you taught me, Fred) this following command always when I am re-squashing a SFS and I am looking for smallest size possible:

Code: Select all

# mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86

.....and for x86_64, I just add "_64" to the "x86" above (hope I am doing that correct)


Thanks, guys, for any feedback.

I'm testing with the script on the desktop and dragging dirs onto it so it definitely works in Xfce.

Changing

Code: Select all

2) COMP="-comp xz -b 1024k -Xbcj x86" ;;

to

Code: Select all

2) COMP="-all-root -noappend -comp xz -b 1048576 -Xbcj x86" ;;

should do the trick.

I don't think x86_64 is an option.

"Available filters: x86, arm, armthumb, powerpc, sparc, ia64"

I think ia64 is for Intel Itanium and x86 works for both 32 and 64bit

[misko_2083]

Code: Select all

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close

^this part has so many pipes.
Not sure what is invert match at the end grep -v '\.' doing.
Awk can do all of that.

Code: Select all

script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" |
awk '/\[*\]*[0-9]\%/{print "#",$NF;gsub("%","");print $NF; fflush(stdout)}' |
yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close

[fredx181]

mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86

Hi Belham,
As far as I understand, the "-all-root" option makes all files owned by root.
If so, I wouldn't recommend that option, because there may be sometimes (executable) files that needs to have other ownership, e.g. messagebus.
Better leave ownership of files as they are, I'd say.
(specially when re-squashing filesystem of Dog based distributions as they have /home/<user> owned by 'normal' user)

Fred