Page 1 of 3
How to put a progress bar in dir2sfs? (Solved)
Posted: Sat 30 Sep 2017, 03:30
by rg66
Hi all,
I'm trying to get dir2sfs to have a working progress bar. I think it was Jemimah who originally added some code to the script but that doesn't seem to work too well with current puppies, it only captures % a few times. Any help would be much appreciated.
Old code:
Code: Select all
mksquashfs "$t" "$z.sfs" |while read -n 50 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
Posted: Sat 30 Sep 2017, 17:38
by sc0ttman
Hi mate, sorry I can't be more helpful than this, but IIRC (it was about 4 years ago), one of the Woofy versions I did (not latest) had SFS progress bars, but it actually slowed down the speed of making the SFS, so I removed it..
But it should be in *one* of the woofy versions I did...
Sorry - I know that i more a chore/errand than actual help...
Posted: Sat 30 Sep 2017, 18:07
by fredx181
Hi rg66
When I do e.g:
Code: Select all
mksquashfs "squashfs-root" "new.sfs" > file
Or:
Code: Select all
mksquashfs "squashfs-root" "new.sfs" | tee file
There are only a few (progress) lines in "file"
I think that explains why you get so little change of % from Xdialog
So I'm afraid it's just not possible to capture mksquashfs output in % by % precisely. Hope I'm wrong, though, I'd like Xdialog progress to work also.
Fred
Posted: Sun 01 Oct 2017, 01:00
by musher0
Guys?
mksquasfs has a progress bar by default.
Please see attached.
BFN.
Posted: Sun 01 Oct 2017, 02:30
by rufwoof
Remember that mksquashfs will use multiple cores if available/specified so its buffering is 'unusual'. By running it through script (a old command so commonly available) and line buffering grep will help, something like ....
Code: Select all
#!/bin/bash
script -q -c "mksquashfs /home /tmp/test.sfs -info -progress -noappend \
-processors 4" | \
grep --line-buffered '\[' | \
grep --line-buffered '\]' | \
grep --line-buffered '\%' | \
sed -u 's/.*\(....\)/\1/'
For large sized mksquashfs however, it will stick on 0% for quite a while and will tend to jump quickly towards the end, perhaps going from 49% to 99% in a quick jump despite having taken a while to get from 0% to 1%. That's all part of compression (takes a while to deduce the codebits that will be used to represent character sequences, but once those codebits are in place can encode quickly).
PS that last sed command just prints out the last 4 (count the dots) characters of a line (again using no buffering (-u parameter)).
Posted: Sun 01 Oct 2017, 10:15
by fredx181
Thanks rufwoof,
Using the 'script' command makes it work nicely, so modified code from first post:
Code: Select all
script -q -c "mksquashfs "$t" "$z.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
@musher0
mksquasfs has a progress bar by default.
Yes, but only from terminal, it's nice to be able to use Xdialog gui progress bar also.
Fred
Posted: Sun 01 Oct 2017, 11:40
by rg66
fredx181 wrote:Thanks rufwoof,
Using the 'script' command makes it work nicely, so modified code from first post:
Code: Select all
script -q -c "mksquashfs "$t" "$z.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
@musher0
mksquasfs has a progress bar by default.
Yes, but only from terminal, it's nice to be able to use Xdialog gui progress bar also.
Fred
Cheers rufwoof and Fred, exactly what I was after.
Posted: Sun 01 Oct 2017, 12:40
by rg66
Hmmmmm, it only seems to work when run from terminal. Dropping a dir on the script or running from thunar custom action shows no progress.
Posted: Sun 01 Oct 2017, 19:33
by fredx181
rg66 wrote:Hmmmmm, it only seems to work when run from terminal. Dropping a dir on the script or running from thunar custom action shows no progress.
Yes indeed, I see also now, pity !
EDIT:
This works for me by dropping a dir on the script or running from thunar custom action:
Code: Select all
t=$1
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "new.sfs"" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
Fred
Posted: Mon 02 Oct 2017, 05:22
by rg66
Works for me too, thanks Fred. Here's my dir2sfs which has compression options.
Code: Select all
#!/bin/bash -x
t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`
if [ ! -d "$t" ];then
echo "error: no valid folder specified!"
exit 0
fi
### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`
#echo --- $check
#echo $t - $z
if [ -f "$z.sfs" ];then
yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="$z.sfs already exists, refusing to overwrite it!" --button="gtk-ok" \
--borders="10" --buttons-layout="center"
exit 0
fi
yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" --button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
echo $LINE | egrep '[0-9]\%'| cut -f1 -d% | awk '{print $NF}'| grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync
echo
echo
s=`du -m "$z.sfs" | sed "s/\s.*//"`
echo "created: $z.sfs ( $s MB )"
echo "created: $z.sfs-md5.txt"
echo
echo "...byebye..."
echo
Posted: Mon 02 Oct 2017, 05:46
by phat7
rg66 wrote:Works for me too, thanks Fred. Here's my dir2sfs which has compression options.
Code: Select all
#!/bin/bash -x
t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`
if [ ! -d "$t" ];then
echo "error: no valid folder specified!"
exit 0
fi
### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`
#echo --- $check
#echo $t - $z
if [ -f "$z.sfs" ];then
yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="$z.sfs already exists, refusing to overwrite it!" --button="gtk-ok" \
--borders="10" --buttons-layout="center"
exit 0
fi
yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" --button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
echo $LINE | egrep '[0-9]\%'| cut -f1 -d% | awk '{print $NF}'| grep -v '\.' ;done | \
Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait..." 0 0
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync
echo
echo
s=`du -m "$z.sfs" | sed "s/\s.*//"`
echo "created: $z.sfs ( $s MB )"
echo "created: $z.sfs-md5.txt"
echo
echo "...byebye..."
echo
Some elements now GUI only, some CLI only
Posted: Mon 02 Oct 2017, 09:47
by fredx181
rg66 wrote:Works for me too, thanks Fred. Here's my dir2sfs which has compression options.
Thanks, I made for myself small change, to be able to cancel the mksquashfs process by closing the progressbar window:
Code: Select all
(
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" |while read -n 99 LINE ; do
echo $LINE |egrep '[0-9]\%'| cut -f1 -d% |awk '{print $NF}'|grep -v '\.' ;done &
) | Xdialog --title "Building Squashfile " --gauge "Building Squashfile. Please Wait... \n To cancel: Close this window" 7 70 0
if [ $? -ne 0 ]; then # progressbar closed by user, so kill mksquashfs process
SQUASHPID=$(ps -eo pid,cmd | grep -v grep | grep "mksquashfs $t $z.sfs" | awk '{ print $1 }')
kill $SQUASHPID
yad --center --borders=10 --title="SFS creation canceled" --window-icon="application-x-squashfs-image" --width=300 --text=" SFS creation canceled " --timeout=5 --button="gtk-close:0" &
exit
fi
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync
Fred
Posted: Mon 02 Oct 2017, 09:49
by rg66
phat7 wrote:Some elements now GUI only, some CLI only
There is no interaction with the command line. All the echo lines were there originally and I don't really see a need to remove them.
Posted: Mon 02 Oct 2017, 10:59
by rg66
fredx181 wrote:Thanks, I made for myself small change, to be able to cancel the mksquashfs process by closing the progressbar window:
Nice, very handy. Here's one using only yad
(I've noticed that both yad and Xdialog reach %100 before the sfs is actually finished using xz compression)
Code: Select all
#!/bin/bash
t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`
if [ ! -d "$t" ];then
echo "error: no valid folder specified!"
exit 0
fi
### if no _versionnummer in the end, get it from system
check=`echo $t | sed "s/\(.*\)_[0-9][0-9][0-9]$/\1/"`
if [ -f "$z.sfs" ];then
yad --window-icon="application-x-squashfs-image" --title="sfs exists" --text="<b>$z.sfs already exists, refusing to overwrite it!</b>" --button="gtk-ok" \
--text-align="center" --borders="10" --buttons-layout="center"
exit 0
fi
yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" \
--button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close
case $? in
0) while [ "`pidof mksquashfs`" ]; do
sleep .2
done ;;
*) killall mksquashfs
rm -f typescript
exit ;;
esac
md5sum "$z.sfs" > "$z.sfs-md5.txt"
rm -f typescript
sync
I've even removed all the CLI echo lines
Posted: Mon 02 Oct 2017, 11:10
by phat7
rg66 wrote:I've even removed all the CLI echo lines
Almost all
Code: Select all
if [ ! -d "$t" ];then
echo "error: no valid folder specified!"
exit 0
fi
Posted: Mon 02 Oct 2017, 11:16
by belham2
Hi rg66 & Fred,
Ok, kind of a dunce question: rg66, can I just copy your script, put it in a blank text file, name it whatever (i.e. "squashingit.sh"), then "chmod +x ..." it, then grab an icon for it, and save it to a handy place (desktop?). Thus I'll then be able to drag ANY directory I was getting ready to squash and place it on top of the script, and
PRESTO..the script will come to life squashing that directory without me doing anything else?
Sorry if the above is a noob question; I just want to make sure I understand so i can use it. I've been unsquashing and especially squashing a lot of stuff this past week (modifying sfs files), and this script would save me the headache of pounding the long command I use into the terminal.
Only thing I wonder, is this script or can this script, be modified to compress at max "xz"? I use (what you taught me, Fred) this following command always when I am re-squashing a SFS and I am looking for smallest size possible:
Code: Select all
# mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86
.....and for x86_64, I just add "_64" to the "x86" above (hope I am doing that correct)
Thanks, guys, for any feedback.
Posted: Mon 02 Oct 2017, 11:33
by rg66
phat7 wrote:
Almost all
Code: Select all
if [ ! -d "$t" ];then
echo "error: no valid folder specified!"
exit 0
fi
Yep, definitely missed that one. Here's a new copy with option to over write existing sfs.
Code: Select all
#!/bin/bash
t=`echo "$1" | sed "s/\/$//"`
z=`echo "$1" | sed "s/\/$//"`
[ ! -d "$t" ] && exit
if [ -f "$z.sfs" ];then
yad --window-icon="application-x-squashfs-image" --title="Overwrite" --text="<b>$(basename $z).sfs</b> already exists.\n Would you like to overwrite it?\n" \
--text-align="center" --borders="10" --buttons-layout="center" --button="gtk-yes:0" --button="gtk-no:1"
case $? in
0) rm -f "$z.sfs" "$z.sfs-md5.txt" ;;
*) exit ;;
esac
fi
yad --window-icon="application-x-squashfs-image" --title "Choose Compression Type" --text "Choose which algorthim to compress the sfs with.
Choosing XZ will give you a smaller iso but may
be slower than GZIP on very lowspec machines.
XZ-HC is a higher compression xz squashfile." --text-align="center" --borders="5" --buttons-layout="center" \
--button="GZIP:0" --button="XZ:1" --button="XZ-HC:2"
case $? in
0) COMP="-comp gzip" ;;
1) COMP="-comp xz" ;;
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
*) exit ;;
esac
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close
case $? in
0) while [ "`pidof mksquashfs`" ]; do
sleep .2
done ;;
*) killall mksquashfs
rm -f typescript
exit ;;
esac
rm -f typescript
md5sum "$z.sfs" > "$z.sfs-md5.txt"
sync
Posted: Mon 02 Oct 2017, 12:26
by rg66
belham2 wrote:Hi rg66 & Fred,
Ok, kind of a dunce question: rg66, can I just copy your script, put it in a blank text file, name it whatever (i.e. "squashingit.sh"), then "chmod +x ..." it, then grab an icon for it, and save it to a handy place (desktop?). Thus I'll then be able to drag ANY directory I was getting ready to squash and place it on top of the script, and
PRESTO..the script will come to life squashing that directory without me doing anything else?
Sorry if the above is a noob question; I just want to make sure I understand so i can use it. I've been unsquashing and especially squashing a lot of stuff this past week (modifying sfs files), and this script would save me the headache of pounding the long command I use into the terminal.
Only thing I wonder, is this script or can this script, be modified to compress at max "xz"? I use (what you taught me, Fred) this following command always when I am re-squashing a SFS and I am looking for smallest size possible:
Code: Select all
# mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86
.....and for x86_64, I just add "_64" to the "x86" above (hope I am doing that correct)
Thanks, guys, for any feedback.
I'm testing with the script on the desktop and dragging dirs onto it so it definitely works in Xfce.
Changing
Code: Select all
2) COMP="-comp xz -b 1024k -Xbcj x86" ;;
to
Code: Select all
2) COMP="-all-root -noappend -comp xz -b 1048576 -Xbcj x86" ;;
should do the trick.
I don't think x86_64 is an option.
"Available filters: x86, arm, armthumb, powerpc, sparc, ia64"
I think ia64 is for Intel Itanium and x86 works for both 32 and 64bit
Posted: Mon 02 Oct 2017, 15:11
by misko_2083
Code: Select all
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" | while read -n 99 LINE ; do
PROG=`echo $LINE | egrep '[0-9]\%' | cut -f1 -d% | awk '{print $NF}'| grep -v '\.'`
echo "#$PROG%"
echo $PROG
done | yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close
^this part has so many pipes.
Not sure what is invert match at the end grep -v '\.' doing.
Awk can do all of that.
Code: Select all
script -q -c "stty rows 40 cols 100; mksquashfs "$t" "$z.sfs" $COMP" |
awk '/\[*\]*[0-9]\%/{print "#",$NF;gsub("%","");print $NF; fflush(stdout)}' |
yad --window-icon="application-x-squashfs-image" --progress --borders="10" --skip-taskbar --text="<b>Building $t.sfs</b>\n" \
--text-align="center" --width="250" --button="Stop!gtk-stop!Stop operation:1" --buttons-layout="center" --auto-close
Posted: Mon 02 Oct 2017, 16:42
by fredx181
belham2 wrote:mksquash dir-example 'new-sfs-name' -all-root -noappend -comp xz -b 1048576 -Xbcj x86
Hi Belham,
As far as I understand, the "-all-root" option makes all files owned by root.
If so, I wouldn't recommend that option, because there may be sometimes (executable) files that needs to have other ownership, e.g. messagebus.
Better leave ownership of files as they are, I'd say.
(specially when re-squashing filesystem of Dog based distributions as they have /home/<user> owned by 'normal' user)
Fred