Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 22 Dec 2014, 18:47
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
A SAVE-session to directory option added for PUPs [CLOSED]
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 24 [353 Posts]   Goto page: 1, 2, 3, ..., 22, 23, 24 Next
Author Message
gcmartin


Joined: 14 Oct 2005
Posts: 4507
Location: Earth

PostPosted: Sat 03 May 2014, 01:05    Post subject:  A SAVE-session to directory option added for PUPs [CLOSED]  

After the thread's efforts were accepted, GCMartin wrote:
This addition to PUPPY allows session changes to be saved to a directory (folder). Its primary advantage over the older "save to a file" method is that the size restriction of saving to a file disappears.

Thanks to everyone who have participated in bringing this option to PUPPY Linux from this time forward.

This thread is requested closed

Over the years our systems have grown in every dimension: RAM size, disk size, disc size, USB size, SD technology, etc. not to mention that other factors like LAN and WAN speed increases have occurred in the home and workplace. Puppy got it start in an age when existing resources at the time were extremely constrained.

The current save-session strategy is based upon a very good strategy on one hand, but, I feel it can be improved. Here's an advantage I see:
Instead of the save-session being "stuffed" into a "disk mimicking" container file as it is done for frugal/Live running systems, could it be created, unstuffed, in a single partition instead of the "constained container" of today? It is constrained because it is "LOCKED" at a selected size. Removal of this locked constraint in today's system would allow users greater flexibility in use of their system(s).

If this is allowed, one MAJOR problem will be averted. Today, that major problem is expanding a save-session disk image, on the fly. If partition save-session technology is offered, many/most users will NEVER need to do any save-session expansion. Further, if a partition need expansion, gParted does this in non-destructive ways as well as the ability to copy/clone a partition is simple and understood by most.

My questions
  • Can this option to users be done today for a save-session?
  • Is there anything missing in the logic for such an option to be available for save-session use?
    Note: Further, allowing a save-session to a folder on permanent storage versus a dedicated partition, would also accomplish the same objective if I am correct.
Any guidance, understanding, or demonstration that may exist where a booting PUP distro would see and use a partition/folder instead of a file (as is done today *.4fs) to present a RAM based operation as it does today for frugal installations would be helpful. This intends NOT to be a full-install approach, rather it intends to continue Frugal/LiveDVD RAM system operations, as it occurrs today; but, without the mentioned constraint.

Please offer any idea(s) you feel are helpful.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile

Last edited by gcmartin on Tue 17 Jun 2014, 04:49; edited 2 times in total
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3398
Location: Oregon

PostPosted: Sat 03 May 2014, 01:39    Post subject:  

When one first gets the message to set up a pupsave file, they are given options as to where to save the file and even if they want to use a file with an option of using the entire partition.
I do not know if the partition selection is restricted to the partition Puppy is installed on or one can select any partition on any drive.
In effect, this comes very close to being like a full install.
But when booting, the personal information, additions, etc is loaded into memory from the selected partition rather than from a pupsave file and then the Puppy SFS is loaded and merged with it.

Is this similar to what you are requesting?
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 392

PostPosted: Sat 03 May 2014, 02:52    Post subject:  

The savefile needs to be a ext format (symbolic links etc). Creating a sfs savefile enables that savefile to be stored on other file system types (NTFS etc).

If you have a ext partition, then one way (the way I do it) is to be selective as to what gets saved or not. I boot from a LiveCD in ram mode (no savefile, so 'savefile' space is all of RAM + swap (1.7GB in my case)), and have a script that creates symbolic links to copies of the directories/files I want to persist across reboots that are stored on the HDD ext partition (linked to the directories/files that are inside Puppy).

For example maybe have a copy of /root/.mozilla on HDD and rm /root/.mozilla and ln -s /mnt/home/.mozilla /root/.mozilla

I wrap a simple test around that to see if the symbolic link already exists or not (in case accidentally run twice) - something like (where curdir is set to /mnt/home)
Code:

cd /root
echo l >./.mozilla/loadedtest
if [ -f $curdir/.mozilla/loadedtest ];
then
   # looks like already previously been loaded
   # just remove the test file
   rm $curdir/.mozilla/loadedtest
else
   cd /root
   rm -r -f .mozilla
   ln -s $curdir/.mozilla .mozilla
   cd $curdir
fi

i.e. it creates a file in /root/.mozilla and if that file appears in /mnt/home/.mozilla then the symbolic link is already in place, otherwise it creates the symbolic link.

So I boot from LiveCD in RAM only mode (puppy pfix=ram pmedia=cd, HDD's remain totally untouched and it all runs totally from RAM/CD i.e. online banking mode (fresh opsys and browser)). Or I run a script after having booted which loads some sfs's and pet's and creates all of the desired symbolic links (general usage desktop and (specific/targeted) changes persist across reboots).

Mikeb put me onto yet another way to do it - he copies /init/pup_pw (again selected parts (not tmp, dev, proc ..etc)) to a sfs and copies that back in again after the next reboot (changes persist).

Puppy's great for its flexibility and choices, and IMO the best choice of LiveCD with optional persistence (small size = speed and range of choices for persistence (none, some, all)).
Back to top
View user's profile Send private message 
mavrothal


Joined: 24 Aug 2009
Posts: 1904

PostPosted: Sat 03 May 2014, 04:50    Post subject: Re: Request for SAVE-session option to remove a constraint  

gcmartin wrote:

My questions.


The option to save to partition while booting with an SFS is there.
However, you can not do that when booting from a CD (Does not make sense).

To implement this do the following.
- Boot from a CD and Install in an (empty) partition.
- Run grub4dos
- Shutdown without saving
- Remove CD
- Boot from partition (will be a first run)
- On shutdown you are given the option to save to partition

See sceeny
save_options.png
 Description   
 Filesize   28.78 KB
 Viewed   1072 Time(s)

save_options.png


_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9070

PostPosted: Sat 03 May 2014, 06:02    Post subject:  

Not make sense from a CD??? no more or less than a save file really...odd statement.

Actually its not quite as neat as it could be... ie the often cited use of a save folder rather than the root of a partition.... that then does allow coexistance with the puppy sfs AND other semi frugal installs.

You get the same freedom of space without hogging the partition.

My save folder ala slax is now 5 years old.... never had a problem generally and definitely never ran out of space.

The adjustment even to puppies init script is tiny compared to other funky options that could exist.... (hint ..use a bind mount)

Note for earlier...the partition like the save file is mounted and added to the union layer..nothing is loaded to ram.

The save folder option is what I use for my older desktops which have less ram to play with....I gave up on save files some years ago.

mike
Back to top
View user's profile Send private message 
mavrothal


Joined: 24 Aug 2009
Posts: 1904

PostPosted: Sat 03 May 2014, 06:32    Post subject:  

mikeb wrote:
Not make sense from a CD??? no more or less than a save file really...odd statement.

The reason that does not make sense (to me) to have a partition (as suggested) as a "savefile" to removable media is that is a source of trouble when you use different puppies. A savefile on the other hand can indeed accommodate several different puppies on the same partition without a problem. And this is something that can be done with current puppies.

mikeb wrote:
Actually its not quite as neat as it could be... ie the often cited use of a save folder rather than the root of a partition....
.....
The adjustment even to puppies init script is tiny compared to other funky options that could exist.... (hint ..use a bind mount)

Savefolder would make sense of course (as in the case of different savefiles).
Since you apparently know how to implement multiple savefolders in a partition please share your experience and provide some working code for puppy.

Edited for clarity

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy

Last edited by mavrothal on Sat 03 May 2014, 06:50; edited 2 times in total
Back to top
View user's profile Send private message 
gcmartin


Joined: 14 Oct 2005
Posts: 4507
Location: Earth

PostPosted: Sat 03 May 2014, 06:48    Post subject:  

The idea of folders for a given PUP is a better one. Instead of having the system compressing and decompressing SFSs, I would like to have session saves direct to folders (even encriypted in some case if necessary).

With all the HDD space, a SFS compressed file solution is probably unnecessary for many of us (when PUPPY got started years ago, HDD size-cost was an issue. Most PCs built since 2000+ have ample space for PUP's meager save-sessions needs) and the overhead, although, minor is some cases, is unnecessary given a choice to the user.

Thanks @Mavrothal as that picture you show is similar to what is asked.

After thought about what each of you have shared, a folder selection on a Linux partition would provide a better solution where folder could exist for each PUP one want to create.

Today, PUPs "key" on some prefix followed by 'save' (L64save.4fs for example). Could PUP's save session name and key of a folder with such naming instead of a SFS file? This seemingly would preserve the logic in save-session processing while extending the option for user selection.

Lastly, if the forum perfect this, what chance is there that it would land in WOOF-CE so that future PUPs would have a consistent offering to users at shutdown processing?

Here to help

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2232
Location: The Blue Marble

PostPosted: Sat 03 May 2014, 07:35    Post subject:  

Save folder is implemented in Fatdog, since 620beta1 (over a year ago ) http://www.murga-linux.com/puppy/viewtopic.php?t=84372. We did get the idea from slax. For this to work, the filesystem of the partition must be a compatible Linux filesystem.
_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
mavrothal


Joined: 24 Aug 2009
Posts: 1904

PostPosted: Sat 03 May 2014, 08:17    Post subject:  

gcmartin wrote:
if the forum perfect this, what chance is there that it would land in WOOF-CE so that future PUPs would have a consistent offering to users at shutdown processing?


The chances are excellent, particularly if mikeb, jamesbond or anyone else provides any patches Wink
You see, nobody "owns" woof-CE. If people contribute will grow, if not will fade away. Is a "democracy" now... Laughing

Edit: Added second line/paragraph.

Last edited by mavrothal on Sat 03 May 2014, 08:27; edited 3 times in total
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9070

PostPosted: Sat 03 May 2014, 08:20    Post subject:  

Yeah it was slax that gave me the idea too.. I first tested it out on puppy 2.12 and then added it to others....so its not like this is cutting edge.

It makes double sense since most users now use a puppy folder anyway so in slax style it just needs a folder added in there really.
In my case I just made a folder called {$PUPPYVERSION}_save/
but since many pups now share the same version numbers (!) something more verbose makes sense if NOT using the pup folder.

Removable media and storage on permanent...well thats a whole can of worms really...I tend to see things as either / or ... either a live CD with perhaps persistence to itself OR install to a medium be it usb or hard drive....certainly would simplify the boot script.
If a specifically named folder is used then same rules apply as to a save file...ie multiple saves should not be a problem.... now as to why we all have multiple pups thats another story Very Happy

As to script to mount a save folder... I am not on puppy but will post it when I am... basically added the detection of a version named folder and then bind mounted it to the union. Like with my other save options there is more code involved to add the option to the shutdown choices than to implement the actual save method. This one slipped into the code for a save file detection.

mike
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3398
Location: Oregon

PostPosted: Sat 03 May 2014, 08:39    Post subject:  

mavrothal wrote:
gcmartin wrote:
if the forum perfect this, what chance is there that it would land in WOOF-CE so that future PUPs would have a consistent offering to users at shutdown processing?


The chances are excellent, particularly if mikeb, jamesbond or anyone else provides any patches Wink
You see, nobody "owns" woof-CE. If people contribute will grow, if not will fade away. Is a "democracy" now... Laughing

.


That is true to an extent.
For it to be implemented in Woof-CE, would it have to be approved by our benevolent dictator (maybe retired), Barry.

And just being curious, how much different is Woof-CE from Barry's official Woof?
Back to top
View user's profile Send private message 
mavrothal


Joined: 24 Aug 2009
Posts: 1904

PostPosted: Sat 03 May 2014, 09:21    Post subject:  

8-bit wrote:
mavrothal wrote:

You see, nobody "owns" woof-CE. If people contribute will grow, if not will fade away. Is a "democracy" now... Laughing


That is true to an extent.
For it to be implemented in Woof-CE, would it have to be approved by our benevolent dictator (maybe retired), Barry.

Actually is absolutely true!
Barry, although very welcome, has nothing to do with woof-CE. Looks like he is indeed retiring.
"You are on your own kids" Laughing

8-bit wrote:
And just being curious, how much different is Woof-CE from Barry's official Woof?

About 60.000 lines of code...

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 392

PostPosted: Sat 03 May 2014, 09:52    Post subject:  

gcmartin wrote:
The idea of folders for a given PUP is a better one. Instead of having the system compressing and decompressing SFSs, I would like to have session saves direct to folders (even encriypted in some case if necessary).

With all the HDD space, a SFS compressed file solution is probably unnecessary for many of us (when PUPPY got started years ago, HDD size-cost was an issue. Most PCs built since 2000+ have ample space for PUP's meager save-sessions needs) and the overhead, although, minor is some cases, is unnecessary given a choice to the user.

Thanks @Mavrothal as that picture you show is similar to what is asked.

After thought about what each of you have shared, a folder selection on a Linux partition would provide a better solution where folder could exist for each PUP one want to create.

Today, PUPs "key" on some prefix followed by 'save' (L64save.4fs for example). Could PUP's save session name and key of a folder with such naming instead of a SFS file? This seemingly would preserve the logic in save-session processing while extending the option for user selection.

Lastly, if the forum perfect this, what chance is there that it would land in WOOF-CE so that future PUPs would have a consistent offering to users at shutdown processing?

Why the need for change?

Currently the choice spans from saving nothing (ram boot with no save) to everything (full install), with any other combination between that you choose. Personally I prefer ram boot with no savefile, but do symbolic link a few files/directories for where I want persistence. Others are content with full installs. Frugal is somewhere in the middle (like having a difference file where only changed files/directories are recorded). And you can save to a wide range of medium - usb, cd, hdd.
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2232
Location: The Blue Marble

PostPosted: Sat 03 May 2014, 10:26    Post subject:  

mavrothal wrote:
8-bit wrote:
mavrothal wrote:

You see, nobody "owns" woof-CE. If people contribute will grow, if not will fade away. Is a "democracy" now... Laughing


That is true to an extent.
For it to be implemented in Woof-CE, would it have to be approved by our benevolent dictator (maybe retired), Barry.

Actually is absolutely true!
Barry, although very welcome, has nothing to do with woof-CE. Looks like he is indeed retiring.
"You are on your own kids" Laughing


In other words, Barry has nothing to do with Woof-CE. Woof-CE has its own gatekeepers, which I assure you, is more generous when accepting contributions (that being its lifeblood, you see) Very Happy

@rufwoof - because it happens too frequent that people setup a small savefile and before you know it, it gets full. Save-to-directory "saves" you from this problem but it doesn't consume a whole partition like save-to-partition. And you can still have a full backup easily by copying the entire save directory, if you wish. But save-to-directory only makes sense for others who are already running Linux since it requires a Linux-compatible partition (not only ext2/3/4, you can have it with f2fs, xfs, jfs, reiserfs, btrfs, zfs, nfs, whatever-fs you can think of which is an Unix-compatible partition --- ntfs, vfat and cifs are specifically *excluded*).

Quote:
The chances are excellent, particularly if mikeb, jamesbond or anyone else provides any patches
Mav, you won't like my patch. It's about 1000 lines long and replaces /sbin/init entirely - it's called "Fatdog's /sbin/init" Laughing And then we'll have a bigger problem Wink Mikeb's one has better chance because it comes from puppy's /sbin/init, at least.
_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9070

PostPosted: Sat 03 May 2014, 11:03    Post subject:  

Quote:
Mav, you won't like my patch. It's about 1000 lines long and replaces /sbin/init entirely - it's called "Fatdog's /sbin/init" And then we'll have a bigger problem Mikeb's one has better chance because it comes from puppy's /sbin/init, at least.

Actually no ... its not really a case of patches but sorely in need of a full rewrite... look at what it needs to achive and write code to achieve that tidilly.

60,000 lines of code (bash/ash script I presume) suggests a very messy system that has far too many patches and cludges already... should not woof ce be an opportunity to look at things afresh... cleaner code, better features, removal of obsolete or unuseful ones.
Check out the slax boot/system wrappers...they have hardly changed at all over 9 years ...sort of got it right in the first place...i digress..back to puppy.

I threw out the puppy 4+ init that most are using as after finding some unpleasant bugs I also realised that modifying it to add features was a simply an unfeasible nightmare. In my case I took the puppy 2 init...cleaned it out and took out the snapmerge stuff and then added my options. Modifying is pretty easy as the source it was built from had a pretty logical modular approach. So I can only say such and such works... sfs to ram save, save folder, multiple sfs and so on ... other derivatives have added similar... and as these mods work well then perhaps main stream puppy could benefit ... since woof is the ONLY way anything like that is allowed in...here we are.
I tested loading the full system to ram for Quirky ... the actual implentation will not be the same as I did the test using puppy 4.12 but it showed that it was feasible plus gave a line or three of code of the core of the technique. For quirky thats up to the writer to adapt the technique into his code.

So I have an init of 22k .... works for me...tested my experiments and of no use as it is for anyone else most likely being an individual of one I can only test on what I have around me. Although init is fairly portable they do normally have to be tweaked to match each kernel's release's quirks.

Ok the save folder is detected in the save detection code looking for the name pattern
Code:
   if [ -d /mnt/data/${PUPPYVERSION}_save ]; then
      PUPSAVE="${1},${2},/${PUPPYVERSION}_save"  #use folder..still mode 12
      PUPMODE=12
   fi


and in the load save file section we have...
Code:
   if [ -d $SMNTPT$SAVEFILE ]; then #using a folder so no loop needed
      echo -n "Mounting ${SAVEFILE} on ${1}..." >/dev/console
      mount -o bind $SMNTPT$SAVEFILE $1;check_status $? 


then the rest is the original save file mount script
note later busybox uses --bind and ${1} is normally /pup_rw.
Bind is used as the dev_save is already mounted plus we need to effectively mount our save folder.

mike
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 24 [353 Posts]   Goto page: 1, 2, 3, ..., 22, 23, 24 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1314s ][ Queries: 12 (0.0056s) ][ GZIP on ]