Yet another basic remaster script for frugal install

How to do things, solutions, recipes, tutorials
Message
Author
User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

Yet another basic remaster script for frugal install

#1 Post by nic007 »

Last edited by nic007 on Wed 16 Aug 2017, 12:33, edited 31 times in total.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#2 Post by musher0 »

HI nic007.

On your copy line, wouldn't

Code: Select all

cp -a /initrd/pup_ro?/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
be ok?

Or does the pup_ro1 need to be copied after the others?

But very nice. Simpler remastering than this, you die! ;)

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#3 Post by nic007 »

musher0 wrote:HI nic007.

On your copy line, wouldn't

Code: Select all

cp -a /initrd/pup_ro?/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
be ok?

Or does the pup_ro1 need to be copied after the others?

But very nice. Simpler remastering than this, you die! ;)

BFN.
pup_ro1 needs to be copied after the others as it holds the contents of the savefile when booting from usb flashdrive. pup_rw records the latest changes during a session so needs to be copied last. Where pup_ro1 is not in use your suggestion should work.

belham2
Posts: 1715
Joined: Mon 15 Aug 2016, 22:47

Re: Yet another basic remaster script for frugal install

#4 Post by belham2 »

nic007 wrote:I use this quick and very basic script to remaster the base sfs sometimes so thought I'd just share it anyway. This should capture the contents of your savefile/folder (ie.programs installed, etc.) as well as the contents of extra sfs files loaded at the time. The working directory must be in a linux filesystem environment (a linux partition works best). Run the script immediately at desktop after booting with savefile/folder and loading any extra sfs files you want to include with your remaster before doing anything else. Note: this is an example script and you may need to change the paths in the script according to your setup. After the new sfs has been created, rename it to the correct name of your distribution's base sfs and move it to where your puppy files reside. Rename the old base sfs and the savefile to something like 1xxxxx (as backup should things not work out as expected). Reboot (without your savefile) and see if your remaster has been successful (afterwhich you can delete the old base sfs and savefile). I tested this with Racy. Puppy's newer than that will probably have the file .XLOADED in /root instead of /etc so you need to change that in the script if so. Hope this is useful to some.

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
wait
cd /mnt/home/puppyfilesystem
rm -r ./initrd ./sys ./etc/.XLOADED
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Press OK when ready" 0 0
mksquashfs /mnt/home/puppyfilesystem /mnt/home/new.sfs -noappend
wait 
rm -r /mnt/home/puppyfilesystem
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
   
exit
Note
The above code is for frugal install to harddrive. If you are booting and running puppy from flashdrive, you need to replace /initrd/pup_rw/* with /initrd/pup_ro1/* in the script (in the same order).
Hi nic007,

I just wanted to post and say 'thank you' for this script. I use it on everything, from old pups up to the very newest (just changing that part you noted in the script for .XLOADED). For me, since I do everything via "frugal" installs, your simple script allowing me to remaster the 'main-puppy.sfs' only is much better than trying to fully remaster a puppy. It s quicker, cleaner and gives me simple flexibility when setting other frugal installs/folders up.

Thank again. :wink:

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#5 Post by nic007 »

Glad you find it useful belham2. BTW - you can actually just add ./root/.XLOADED to the rm line of the script. The script will delete the applicable file depending on the Puppy you are running and the non-applicable one will not hinder the running of the script.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

Re: Yet another basic remaster script for frugal install

#6 Post by greengeek »

nic007 wrote:

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem....
I always run my puppy without any savefile and I don't have any /mnt/home - everything is done in ram.

Can you suggest any modification of this script to cover my situation? I would like to be able to grab any sfs files that are loaded.

Thus far I have been using an earlier version of one of your remaster scripts and it grabs pets that are loaded but not sfs files.

cheers!

EDIT : I just had a go at making some changes and came up with the following:

Code: Select all

#!/bin/sh 
mkdir /tmp/puppyfilesystem 
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /tmp/puppyfilesystem 
wait 
cd /tmp/puppyfilesystem 
rm -r ./initrd ./sys ./etc/.XLOADED 
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Then please wait for 'success'. message. Press OK when ready." 0 0 
mksquashfs /tmp/puppyfilesystem /tmp/new.sfs -noappend 
wait 
rm -r /tmp/puppyfilesystem 
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
exit
Seems to work - I hope to test it in more depth.

User avatar
bigpup
Posts: 13886
Joined: Sun 11 Oct 2009, 18:15
Location: S.C. USA

#7 Post by bigpup »

Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package.

If the original SFS package is present.
After you do the remaster, just load it when you want to use it.

The idea of a remaster is to get the contents of the save added as part of the main Puppy SFS.
The custom setup, added installed programs, etc.....

You can get the main Puppy SFS package too big, where it will take a lot of memory to load into memory.
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected :shock:
YaPI(any iso installer)

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#8 Post by greengeek »

bigpup wrote:To me that seems to negate the purpose of SFS packages you load.The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package.
.
Possibly - but what if you wanted to load (and grab) the devx and remaster it into your new puppy?

There are times where grafting an sfs might be useful.

User avatar
bigpup
Posts: 13886
Joined: Sun 11 Oct 2009, 18:15
Location: S.C. USA

#9 Post by bigpup »

Sure, anything is what you may want to do. :lol:

However, in your example. The devx is only needed when compiling.
The stuff in it is not needed, normally, to use Puppy and run programs.
Python, may be the only thing in it, you may need to run some programs.
In that situation, install Python.
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected :shock:
YaPI(any iso installer)

mfb

#10 Post by mfb »

Hi greengeek,

The screenshot below shows the code I use to remaster from tahr-64-6.0.5 running entirely in ram including two loaded sfs files (one of which is its 100+MB 32 bit compatibility sfs), but I only load the devx.sfs (from the normally unmounted sda2) when I want to use it.

I boot from a CD (with a copy of the main sfs on my hard drive partition sda2 - to bypass the main sfs file on the CD and reduce elapsed boot time from some 200+ seconds to about 65 seconds).

I like gzip level 1 compression and with 3GB total ram (1GB used) it takes 32 seconds to complete the remastered 640MB main sfs (there are many installed pets as well as the loaded sfs files).

The actual script is also available for download. After download, rename it deleting the final three (dot gz) characters and make it executable.

importantly, before running the script you may wish to edit:

* the name of any current main sfs you want to keep (if it might otherwise be overwritten by the remastered version)

* /mnt/sda2/puppy_tahr64_6.0.5.sfs - to reflect your personal choice of main sfs location and/or the exact name as preset for your Pup distribution's main sfs

* gzip1 compression to xz or perhaps lz4 if available (Packit from SFR, though not used to remaster, will show your options if you view its "sfs" drop-down-list).

Extra notes:

* the script can be run from anywhere, but may be best placed on the same partition where the remastered sfs will be located - so the partition is mounted before clicking to run the script.

* The script just runs, without any further action, so just wait for the beep and the centre screen message showing the time taken - then mouse over the message to exit - do not save before rebooting to test.
Attachments
ram2sfs.png
(59.24 KiB) Downloaded 1338 times
ram2sfs.gz
(331 Bytes) Downloaded 170 times

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

Re: Yet another basic remaster script for frugal install

#11 Post by nic007 »

greengeek wrote:
nic007 wrote:

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem....
I always run my puppy without any savefile and I don't have any /mnt/home - everything is done in ram.

Can you suggest any modification of this script to cover my situation? I would like to be able to grab any sfs files that are loaded.

Thus far I have been using an earlier version of one of your remaster scripts and it grabs pets that are loaded but not sfs files.

cheers!

EDIT : I just had a go at making some changes and came up with the following:

Code: Select all

#!/bin/sh 
mkdir /tmp/puppyfilesystem 
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /tmp/puppyfilesystem 
wait 
cd /tmp/puppyfilesystem 
rm -r ./initrd ./sys ./etc/.XLOADED 
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Then please wait for 'success'. message. Press OK when ready." 0 0 
mksquashfs /tmp/puppyfilesystem /tmp/new.sfs -noappend 
wait 
rm -r /tmp/puppyfilesystem 
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
exit
Seems to work - I hope to test it in more depth.
That's doing it in RAM, should work if you have enough RAM. The following thread may be of interest to you: http://www.murga-linux.com/puppy/viewtopic.php?t=109745
I keep the original base sfs intact, load sfs's for my applications and record any personal settings also in an sfs file.

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

Unload SFSes before Remaster

#12 Post by mikeslr »

[quote="bigpup"]Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package./quote]

Seems to me that this objection could be overcome if, before running remaster, unwanted SFSes are unloaded. Or am I missing something? Coding isn't my strong suit. But, as I read it, this line

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*

copies loaded SFSes into the remaster. If any of the "ro"s are not in use, noting from that "ro" gets copied.

Since I always run portable-wine (a 32-bit application) from Tahrpup64 and Xenialpup64, part of my initial setup routine is to load the 32-bit compatibility SFS.* On the other hand, I prefer to use the most current web-browsers and LibreOffice SFSes. If I understand the script correctly, a remaster would accomplish my goal if, before remastering, I unloaded Libreoffice and webbrowsers but left the 32-bit compatibility SFS loaded.

mikesLr

* Actually, I've long since converted the compatibility SFS to an installable pet. Consider the statement for the purpose of illustration. Perhaps core Qt4 and Qt5 SFSes would have been better. Both are relatively small, and some applications I use need one, and some the other.

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

Re: Unload SFSes before Remaster

#13 Post by nic007 »

mikeslr wrote:
bigpup wrote:Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package./quote]

Seems to me that this objection could be overcome if, before running remaster, unwanted SFSes are unloaded. Or am I missing something? Coding isn't my strong suit. But, as I read it, this line

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*

copies loaded SFSes into the remaster. If any of the "ro"s are not in use, noting from that "ro" gets copied.

Since I always run portable-wine (a 32-bit application) from Tahrpup64 and Xenialpup64, part of my initial setup routine is to load the 32-bit compatibility SFS.* On the other hand, I prefer to use the most current web-browsers and LibreOffice SFSes. If I understand the script correctly, a remaster would accomplish my goal if, before remastering, I unloaded Libreoffice and webbrowsers but left the 32-bit compatibility SFS loaded.

mikesLr

* Actually, I've long since converted the compatibility SFS to an installable pet. Consider the statement for the purpose of illustration. Perhaps core Qt4 and Qt5 SFSes would have been better. Both are relatively small, and some applications I use need one, and some the other.
Correct. The script will only include the contents of those extra sfs's which are loaded at the time of the remaster. The installed pets, loaded sfs's and configuration settings will be included in a new base sfs. Some people prefer to have everything included in their base sfs and it is probably the way to go if you want to make your own puplet for sharing.

User avatar
Moat
Posts: 955
Joined: Tue 16 Jul 2013, 06:04
Location: Mid-mitten

#14 Post by Moat »

Plus, it seems some programs are not readily available as .pet packages, but are available as an SFS - and testing a program by running it for a while as an SFS before deciding to include it in the base/main SFS sounds to me like a really great option. Kudos to nic007 for this! 8)

I admit to being one who prefers everything in the base SFS (within reason) - sure, can take longer to boot, but it's always there at the ready when needed, and no fiddling with loading/unloading and interrupting the workflow.

Bob

ndujoe1
Posts: 851
Joined: Mon 05 Dec 2005, 01:06

#15 Post by ndujoe1 »

Question: so if I remove all the default puppy applications I don't wish to use, the will remaster everything I do use, personal files and sfs's?

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#16 Post by nic007 »

ndujoe1 wrote:Question: so if I remove all the default puppy applications I don't wish to use, the will remaster everything I do use, personal files and sfs's?
It will include all loaded sfs's, pets you have installed and personal files that are stored in the savefile/folder (not personal files that are stored elsewhere on your drive). As far as removal of default built-in applications goes: - this will depend how you remove it. If you use some application that only white-out (mark them that they can not be seen by the operating system) files, I don't think this basic script will actually delete them from the base sfs (but it should be masked so that there won't be a menu entry for them, etc.). If you physically remove them by editing the base sfs, it will definitely be gone.
Last edited by nic007 on Fri 02 Jun 2017, 07:01, edited 1 time in total.

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

#17 Post by mikeslr »

Repeating my previous caution "coding is not my strong suit", as I understand nic007's command line:

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/* /mnt/home/puppyfilesystem

whatever you have in /root will be copied. /root is where all application settings are stored.

The builtin remaster application will not include in your custom remastered version the applications you removed using Menu>Setup>Remove builtin packages. The way this works --when the builtin remaster works-- is that remove builtin doesn't actually remove the package, only "white's out" the link to it. So the builtin remaster, doesn't have a link to follow and, consequently, doesn't include the application. nic007's remaster simply copies what's there including both the applications and the "white outs".

Frankly, removing applications IMHO is a waste of time (unless, of course, you're involved in some grand project, like taking a Puppy which uses JWM as its Window Manager and substituting XFCE as Window Manager). Storage space is cheap. Recently I purchased a package of 3 8Gbs USB-Keys for $12. Hard-drives can be found for as little as $.04 per Gb*. Most Devs creating Puppies don't include large applications --such as LibreOffice-- in their creation; and a Frugally installed Puppy doesn't load an entire builtin** application into RAM, but rather creates links to it which are followed as needed. The only downside of builtin applications is that they appear on the Menu delaying your selection of those applications you actually want to use. Newer Puppies have builtin Menu>Setup>Menu Manager --edit the Menu, which can remove an application's appearance from the Menu. In older Puppies, you can edit their desktop files, /usr/share/applications/xxx.desktop, so that the category argument is one which will not produce a display, e.g Categories=Calculator to Categories=ZCalculator. Both techniques would, I believe, be carried over into your remaster using nic007's remaster script.

mikesLr

* If it takes me an hour to remove 25 Mbs of applications, conserving the space worth 4 cents per Gb, the value of my time is .1 cent per hour. Would you work for that?

** When you install a new version of an application or load an SFS, say Palemoon, into a Puppy where the Dev has builtin a version of Palemoon, both versions continue to exist. But the links Puppy uses will now point to the new version rather than the old.

nic007 can explain it better. I've used the term "links" which I, and I think most of us, understand. But nic007 can probably explain how inodes function.

User avatar
smokey01
Posts: 2813
Joined: Sat 30 Dec 2006, 23:15
Location: South Australia :-(
Contact:

#18 Post by smokey01 »

I always have the devx loaded as I do a lot of compiling. This method would allow me to include the devx in the main base so I could compile in a virtual environment like Vbox and Qemu. Nice.

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#19 Post by nic007 »

mikeslr wrote:Repeating my previous caution "coding is not my strong suit", as I understand nic007's command line:

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/* /mnt/home/puppyfilesystem

whatever you have in /root will be copied. /root is where all application settings are stored.

The builtin remaster application will not include in your custom remastered version the applications you removed using Menu>Setup>Remove builtin packages. The way this works --when the builtin remaster works-- is that remove builtin doesn't actually remove the package, only "white's out" the link to it. So the builtin remaster, doesn't have a link to follow and, consequently, doesn't include the application. nic007's remaster simply copies what's there including both the applications and the "white outs".

Frankly, removing applications IMHO is a waste of time (unless, of course, you're involved in some grand project, like taking a Puppy which uses JWM as its Window Manager and substituting XFCE as Window Manager). Storage space is cheap. Recently I purchased a package of 3 8Gbs USB-Keys for $12. Hard-drives can be found for as little as $.04 per Gb*. Most Devs creating Puppies don't include large applications --such as LibreOffice-- in their creation; and a Frugally installed Puppy doesn't load an entire builtin** application into RAM, but rather creates links to it which are followed as needed. The only downside of builtin applications is that they appear on the Menu delaying your selection of those applications you actually want to use. Newer Puppies have builtin Menu>Setup>Menu Manager --edit the Menu, which can remove an application's appearance from the Menu. In older Puppies, you can edit their desktop files, /usr/share/applications/xxx.desktop, so that the category argument is one which will not produce a display, e.g Categories=Calculator to Categories=ZCalculator. Both techniques would, I believe, be carried over into your remaster using nic007's remaster script.

mikesLr

* If it takes me an hour to remove 25 Mbs of applications, conserving the space worth 4 cents per Gb, the value of my time is .1 cent per hour. Would you work for that?

** When you install a new version of an application or load an SFS, say Palemoon, into a Puppy where the Dev has builtin a version of Palemoon, both versions continue to exist. But the links Puppy uses will now point to the new version rather than the old.

nic007 can explain it better. I've used the term "links" which I, and I think most of us, understand. But nic007 can probably explain how inodes function.
Unfortunately this does NOT seem to be the case. I "removed" Seamonkey using "Remove builtin Packages" and then ran the builtin remaster script. Only the menu entries were deleted, Seamonkey was still there in the remastered sfs. My script also only removes the menu entries. I remember pup412 had an application called pet-be-gone which worked but Remove builtin packages doesn't seem to.

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

#20 Post by mikeslr »

Another reason not to actually remove "builtins" unless you really know what you are doing. With the "remove builtin" application so limited, it would seem that in order to actually remove an application and all of its files, you'd have to open its listing in /root/.packages in a text viewer, then manually delete the folders, and in some cases, the specific files shown on that list. But to safely do so, you have to be certain that every file on that list isn't also used by some application you still want to be functional.*

Perhaps the coding requirement of a program that would keep track of all files within each system and which applications make use of each file explains why "pet-be-gone" is no longer supported, and "remove builtins" has limited value.

mikesLr

* and if you possess that knowledge, its probably faster to woof your own Puppy version than to strip someone else's version of what you don't want.

Post Reply