Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 25 Jan 2020, 07:56
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Filesystem
rw sfs (read write squashfs)
Post new topic   Reply to topic View previous topic :: View next topic
Page 2 of 2 [19 Posts]   Goto page: Previous 1, 2
Author Message
rufwoof


Joined: 24 Feb 2014
Posts: 3717

PostPosted: Sat 19 Oct 2019, 20:03    Post subject:  

Packaged the individual draft scripts up into a single script and attached that (actual gzip'd) file to the first post.
_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3717

PostPosted: Tue 22 Oct 2019, 19:06    Post subject:  

mikeslr's idea here (nice one Mike Smile) is food for thought about extending rw-sfs to incorporate encryption.

Something like cryptsetup can be used to encrypt entire folders (as available in Fatdog), or images ...
Code:
dd if=/dev/zero of=a.img bs=1M count=20
# Can't be too small, less than 20MB and seem to hit problems
cryptsetup -y luksFormat a.img
... needs uppercase YES and enter password ... twice

cryptsetup luksOpen a.img a
... enter password
mkfs.ext2 /dev/mapper/a
mount /dev/mapper/a /mnt/a

... use as desired

umount /mnt/a
cryptsetup close a

If the sfs content is a copy of the encrypted folder content (two folders i.e. root and .root .. assuming a entire main sfs type content), then that's a much much higher hurdle for any attempted tampering.

If the tarball (changes/save folder) content is also piped through something like openssl enc before being appended to the end of the sfs, then the entire 'sfs' content is encrypted. The sfs can be mounted/loaded, but the content is encrypted. Entering a correct password opens up the clear text (unencrypted) content and if the same password also applies to the tarball (changes) then that keeps interaction low/simple (could however have two passwords needing to be entered, one to open up the sfs, the other to open up the changes).

Initially I was looking at storing the rw-sfs tarball content in compressed form, but for speed reasons opted not to. Which if encryption is being used is a good thing as typically encrypted data has low compressibility.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 3620
Location: 500 seconds from Sol

PostPosted: Wed 23 Oct 2019, 10:31    Post subject:  

I'm following rufwoof's reasoning but not yet ready to attempt to internalize it. Just wanted to mention that cryptsetup --which rufwoof mentioned above-- if not already present in your Puppy, appears to be generally available without a great many dependencies. cf., https://pkgs.org/download/cryptsetup
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3717

PostPosted: Wed 23 Oct 2019, 10:48    Post subject:  

Another variation of storing changes back to the same sfs

A large sfs, such as a Puppy (or in my case Fatdog) main sfs, can take a modest amount of time to be rebuilt (reformed using mksquashfs). Appending additional (or changed) content however is a relatively quick operation. If for instance my main sfs (fd64.sfs) is 1GB in size and already contains a /bin folder within that, then if I have another bin folder content that I would like to add to fd64.sfs then

mksquashfs bin fd64.sfs -keep-as-directory

will add that bin folder into fd64.sfs and if the bin folder being added contains relatively little mksquashfs tends to add that very quickly. As there was already a /bin folder inside the sfs however, that operation adds the additional bin folder to the sfs with a bin_1 folder name. If we add yet another bin folder to the sfs then that is stored inside the sfs as bin_2 ...etc.

What then if instead of adding a bin folder, we add a folder called 'changes' i.e. save folder content. Well again that's quick to mksquashfs add if the content of changes (save) is relatively small. And we can repeat that multiple times, so that fd64.sfs contains changes_1 changes_2 changes_3 ...etc. Where (in this case) changes_3 is the most recent 'save' and changes_1 is the oldest save.

I'm using overlayfs style layering here, and that supports up to 128 folders being layered as the 'lowerdir'. That's actually more like a 125 upper limit as the 128 figure officially documented also includes the work folders. When layering it layers top layer to bottom layer in the left to right order so the sort of overlayfs syntax we should use for lowerdir looks like

mkdir sfs
mount fd64.sfs sfs
mkdir work top changes
mount -t overlay overlay -o \
lowerdir=s/changes_3:s/changes_2:s/changes_1:s,\
upperdir=changes,workdir=work top

so the rightmost lowerdir value (s) is the deepest layer, and the leftmost (s/changes_3) is the highest layer (most recent).

work and changes folders both have to be new/empty when doing that mount for the setup to work correctly. The work folder also needs to be on the same filesystem as the upperdir.

With that mounted, we can view the content via the 'top' folder, and any changes we make are recorded in the 'changes' folder. So when done we might umount that
umount top
umount s
... and add the content of the changes folder to the sfs
mksquashfs changes fd64.sfs -keep-as-directory

the -keep-as-directory ensures the content of changes are added as a folder, and not just the content of the changes folder being added as-is to the sfs.

After adding that changes folder to the sfs (that generally runs through very quickly) the fd64.sfs will now contain yet another changes folder, in this case named changes_4. So the next time we open the sfs we need to extend the mount command to include that

mount -t overlay overlay -o \
lowerdir=s/changes_4:s/changes_3:s/changes_2:s/changes_1:s,\
upperdir=changes,workdir=work top

... and so on. Up to around a limit of changes_125 maximum. Way before then however (125 saves), we might reform the sfs, which in effect defrag's it. i.e. mount the sfs and create a new sfs from the 'top' folder excluding the changes folders and then replace the main sfs with that new version that includes all of the changes. That is a slow(ish) task but being performed once after every 100 or so saves is a tolerable task/delay.

With up to 125 saves each being recorded separately, we also have a audit trail of changes.

For tidiness reasons, rather than saving changes as /changes /changes_1 /changes_2 ... etc inside the sfs, using a .changes folder 'hides' those files. Otherwise in plain (not viewing hidden files/folders) view, the root folder might be filled with changes_xxx folders and look messy. That still uses the same arrangement, but just different folder names

mount -t overlay overlay -o \
lowerdir=s/.changes/changes_3:s/.changes/changes_2:s/.changes/changes_1:s,\
upperdir=.changes/changes,workdir=.changes/work top

and where to add that to the fd64.sfs we
rm -rf .changes/work
mksquashfs .changes fd64.sfs -keep-as-directory

Of course if you don't want to save changes, then its just a matter of leaving things as they are, close the top layer, close the sfs, remove the changes (and work) folders and the sfs remains as-before.

This is all described as for overlayfs, but I guess the same could be used with aufs, just requiring aufs style layer mount commands instead of the above overlayfs mount commands.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 2 of 2 [19 Posts]   Goto page: Previous 1, 2
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Filesystem
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0533s ][ Queries: 12 (0.0229s) ][ GZIP on ]