sfs_load-2.4 on-the-fly

[jemimah]

There is code in the init script that does the whiteout cleaning. Unfortunately it doesn't work with newer aufs versions because the name of the opaque whiteout files changed and the script has not been updated (Barry has fixed it in woof now). I think it should be safe to move the code to sfs-load and remove it from the init script. I intend to try it anyway.

[shinobar]

Major update:
# 14 Feb 2011 v0.8:

• warning excessive extra files
• restart main dialog(thanks to Bert)
• cleanup whiteout at unload(thanks to jpeps and jemimah)
• mkfontscale, mkfontdir

Treating the whiteout may reduce the problem as jpeps reported, but note that sfs can not be the same as pet.
Suppose:

1. Load a sfs
2. Delete some of the files or directories in the sfs
3. Unload the sfs by the sfs_load-0.8, which cleans up the whiteouts
4. Load again the same sfs

Most of case, the once deleted files or directories revive with the last re-load.
But the files or the directories conflict (duplicated) with other layers shall not come back.

[jpeps]

Treating the whiteout may reduce the problem as jpeps reported, but note that sfs can not be the same as pet.

As I recall, whiteouts were successfully removed with a pet install, but not with SFS.

[jemimah]

Installing a pet writes to the top layer whereas installing an sfs puts the data on the bottom layer. That's why there's a difference,

[pemasu]

New usage for sfs load on the fly. When using woof and ./3builddistro. When it is time to create devx.sfs, so far the Barry`s bacon basic compiler installing hasnt been succesful. Devx.sfs not loaded has been the error message (it has been always loaded with bootmanager)

Now I have twice used sfs load on the fly for devx.sfs. Both times bacon basic compiler and support files have been succesfully added to the devx.sfs

[jpeps]

Installing a pet writes to the top layer whereas installing an sfs puts the data on the bottom layer. That's why there's a difference,

If SFS's become used more, it might be worth the hassle to touch files on the top layer before mounting (or something to that effect )

Edit: Tried it...unfortunately, mounted files won't clobber the touch files. Think it's easier just to delete the whiteouts before mounting.

[shinobar]

Installing a pet writes to the top layer whereas installing an sfs puts the data on the bottom layer. That's why there's a difference,

If SFS's become used more, it might be worth the hassle to touch files on the top layer before mounting (or something to that effect )

Edit: Tried it...unfortunately, mounted files won't clobber the touch files. Think it's easier just to delete the whiteouts before mounting.

Nice idea, jpeps.
You can copy the files AFTER mounting.
To do so, you need to know to which layer the sfs is mounted.
You can make a script under /root/Startup or under /etc/init.d, then the sfs_load finds out the script and runs it after mounted.
Note that the script under /etc/init.d is called with a parameter 'start'.

The following code is to see where the sfs is mounted:

Code: Select all

MYLAYER=""
for L in $(LANG=C df| cut -d'%' -f2| tr -d ' '  | grep  '/initrd/pup_ro'| grep -vw '/initrd/pup_ro[12]'| sort -r); do
 [ -s $L$0 ] && MYLAYER=$L && break
done
echo $MYLAYER >&2

Next code is an example lifting up the files to the top layer:

Code: Select all

D=SOME_DIRECTORY_PATH
S=$MYLAYER$D
if [ "$S" != "$D" -a  -d "$S" ]; then
  mkdir -p $D
  for P in in $(find $S -maxdepth 1 -type f -printf '%P '); do
    [ -e "$D/$P" ] || cp -p "$S/$P" "$D/$P"
  done
fi

You can see what i do in the lang_pack_all-0.3-w5.sfs.
http://shino.pos.to/party/bridge.cgi?pu ... languages/

Note that this way consumes pupsave and the files are left even after unloading the sfs.

[jpeps]

Running this script first targeted matching whiteouts left after removing a PET with the same files, so the SFS loaded correctly.

Edit: Looks like I'll need to add files as well...
Done


USAGE: sfs-prep [myfile.sfs]

Code: Select all

#!/bin/sh 

[ -f /tmp/dir ] && rm /tmp/dir
[ -d /tmp/work ] && rm -r /tmp/work 
mkdir /tmp/work

mount "$1" /tmp/work -o loop
cd /tmp/work 
find .  >>/tmp/dir

cd /
while read line; do
 [ "$line" -a "$line" != "." ] && WO=".wh.$(basename $line)"
 [ "$WO" ] && DEL="$(find /initrd/pup_rw | grep "$WO")"
  [ -f  "$DEL" ] && rm  "$DEL" 
done </tmp/dir 

## Cleanup
umount /tmp/work
rmdir /tmp/work
rm /tmp/dir 

[shinobar]

Running this script first targeted matching whiteouts left after removing a PET with the same files, so the SFS loaded correctly.

@jpeps
Are you aware the the sfs_load-0.8 has similar code at unloading sfs?
Note that it does not at loading sfs but at unloading.

[jpeps]

@jpeps
Are you aware the the sfs_load-0.8 has similar code at unloading sfs?
Note that it does not at loading sfs but at unloading.

I did notice that I could no longer find whiteouts to test after unloading the SFS's Problem, though, is getting them to load in the first place.

I'm using my own pet uninstaller which actually deletes all the files (unlike the GUI PPM), so there are more whiteouts. For example with Xorg_High-1.1-Lucid.pet, the PPM leaves (3) /usr/lib/xorg/modules/extensions. Deleting them results in three whiteouts. Reinstalling the PET removes them, but the SFS won't. Similar story for others.

[shinobar]

Deleting them results in three whiteouts. Reinstalling the PET removes them, but the SFS won't.

Then, load the SFS, unload the SFS, and re-load the SFS

[jpeps]

Deleting them results in three whiteouts. Reinstalling the PET removes them, but the SFS won't.

Then, load the SFS, unload the SFS, and re-load the SFS

Didn't work, since the files initially didn't load and continue not to load. Removing them prior to mounting works.

[jemimah]

If the user says no they do not want to copy the SFS to /mnt/home, does it still get added to the boot list?

The reason I ask is some users never want to load the SFS permanently. But if you say "don't copy", the next time you boot, sfs-load makes you unload the sfs before you can load it again.

I think it would make more sense not to add temporary SFSes to the boot list. Or am I missing something?

[shinobar]

If the user says no they do not want to copy the SFS to /mnt/home, does it still get added to the boot list?

The reason I ask is some users never want to load the SFS permanently. But if you say "don't copy", the next time you boot, sfs-load makes you unload the sfs before you can load it again.

I think it would make more sense not to add temporary SFSes to the boot list.

Ah, right jemimah, good point!

EDIT: Refined at sfs_load-0.9.

[jpeps]

Think I found the bug:

Code: Select all

812  for W in $(find /initrd$SLAYER -mindepth 2 -type f -name .wh.* -not -name .wh..wh.* -printf '/%P '); do

Try -name ".wh*"

[shinobar]

Try -name ".wh*"

Thanks, but I cannot see any difference between .wh* or ".wh*"...

[jpeps]

Try -name ".wh*"

Thanks, but I cannot see any difference between .wh* or ".wh*"...

Interesting..doesn't work without the quotes for me. For example:

Code: Select all

~ $ find /initrd/pup_rw  -mindepth 2 -type f -name .wh* 
~ $  find /initrd/pup_rw  -mindepth 2 -type f -name ".wh*" 
/initrd/pup_rw/etc/.wh.windowmanager.openbox
/initrd/pup_rw/usr/lib/xorg/modules/extensions/.wh.libdri2.so
~ $ 

To be safe, "\.wh*"

[shinobar]

Code: Select all

~ $ find /initrd/pup_rw  -mindepth 2 -type f -name .wh* 
~ $  find /initrd/pup_rw  -mindepth 2 -type f -name ".wh*" 
/initrd/pup_rw/etc/.wh.windowmanager.openbox
/initrd/pup_rw/usr/lib/xorg/modules/extensions/.wh.libdri2.so
~ $ 

May depends on the version(of what?).
On wary-500m08:

Code: Select all

# find /initrd/pup_rw  -mindepth 2 -type f -name .wh* 
/initrd/pup_rw/dev/.wh.usbdev8.4
/initrd/pup_rw/dev/vboxusb/008/.wh.004
/initrd/pup_rw/dev/bus/usb/008/.wh.004
# find /initrd/pup_rw  -mindepth 2 -type f -name ".wh*" 
/initrd/pup_rw/dev/.wh.usbdev8.4
/initrd/pup_rw/dev/vboxusb/008/.wh.004
/initrd/pup_rw/dev/bus/usb/008/.wh.00

I agree, to be safe, "\.wh*"

[jpeps]

Apparently bash shell requires it, sh doesn't. Must be another reason its not removing the whiteouts....looking at it now.

Edit: even if script is /bin/sh, needs the quotes if main shell is bash.

[mavrothal]

v.08 fails to find the sfs files although they are detected and present in the dropdown list. This in pupmode6, because /mnt/home is missing the trailing slash so the file /mnt/homeinitrd/pup_rw/name.sfs can not be found
This little patch solves the issue

Code: Select all

915,916c915,916
< HOMELINK=$(readlink /mnt/home/)
< [ "$HOMELINK" != "" -a "$PUPHOME" = "$HOMELINK" ] && PUPHOME=/mnt/home/
---
> HOMELINK=$(readlink /mnt/home)
> [ "$HOMELINK" != "" -a "$PUPHOME" = "$HOMELINK" ] && PUPHOME=/mnt/home
990c990
<   DIRNAME=$(echo $DIRNAME| sed -e "s,$HOMELINK,/mnt/home/,")
---
>   DIRNAME=$(echo $DIRNAME| sed -e "s,$HOMELINK,/mnt/home,")
1159c1159
<   /mnt/home/) FILEISAT="home";;
---
>   /mnt/home) FILEISAT="home";;

PS: Also some sfs (wine-1.3.6-i486-sfs4.sfs is an example) fail to unload on the fly because they are "busy" and the `remount,del' step fails. Would it possible to to detect and kill associated open files before the removal from the union?