Modpup? Please!

A home for all kinds of Puppy related projects
Message
Author
blubb_fallo
Posts: 19
Joined: Tue 20 Feb 2007, 16:51

Modpup? Please!

#1 Post by blubb_fallo »

I believe that puppy could profit from becoming more modular. Puppy unleashed gives wonderful freedom to everyone who wants to make his/her own customized distro. On the other hand, that procedure mostly is a one way street.

Unfortunately, I'm much worse at implementing ideas than at having them. This one is inspired by dsl - I think it is ok to learn from competitors: ;)

I definitely find global saving via pup_save.xfs more elegant than dsl's plain /root backup. But in my view, dsl is more flexible towards performing specific tasks on limited hardware, due to "mydsl extensions" that can be tagged as "regular" or "optional" (by putting them in the corresponding folder). Such adaptability would be a nice-to-have in puppy, too, wouldn't it? How about a large collection of specialized sfs modules that could be added or skipped, just like devx...sfs, but based on a really slim core system (perhaps similar to barebones)?

Why load planmaker, why load abiword, let alone azureus, Gimp or wine, when you want nothing but to surf and listen music? No need to let them bloat your ramdisk for nothing, I think. On the other hand, there can be situations where you want (or have) to use a particular biggie that actually would paralyze your poor old machine if its ram was grabbed by all the internet stuff, flash and codecs in pup_ro2. Loading nothing but what you need might still work. Currently, you'd have to get through unleashed to achieve this.

Wait, even better: imagine a simple bootup dialog starting from within initrd.gz that provides one checkbox for each module that is present in the boot directory, so you can decide at boot time which applications are worth to be loaded into ramdisk? Well, forcing such a choice every time would be rather inconvenient for sure. But if we made the dialog remember our last time's choices and auto-confirm them after a short timeout (say, 3 or 5 s), the startup would be smooth as usual, while offering great flexibility.

That way, we could solve the same tasks on smaller machines (or alternatively, more advanced tasks on the same hw). The only precondition is that they don't require the whole puppy system at once. I believe that holds for most.

What do you think?
Last edited by blubb_fallo on Thu 01 Mar 2007, 17:20, edited 2 times in total.

amish
Posts: 615
Joined: Sun 24 Sep 2006, 23:15

#2 Post by amish »

i can't make a distro yet, but let me know if you need anything.

grafpup 104 inspires a similar archetecture http://grafpup.com

whodo of the revived puppy ce and others are talking about this: http://murga-linux.com/puppy/viewtopic. ... 585#100585

this could lead to creating a barebones version as easy as deleting a core.sfs file. instead of a bloated medium puppy, you could have small 40-50mb, medium (default) puppy less than 100mb, and large less that 400mb puppy: http://murga-linux.com/puppy/viewtopic. ... 330#100330

see the example diagram there. i'm going to be putting a lot of stuff about a sort of modpup on my wikipage: http://puppylinux.org/wikka/MramisH#grandunifiedpup <- link works, section in question doesn't exist yet.
sadly, it is not possible to separate politics from free software. free software - politics = unfree software.

erikkire
Posts: 31
Joined: Thu 09 Nov 2006, 18:01
Location: cebu

#3 Post by erikkire »

it would be very convenient if those LARGE programs are installed as modules rather than installed it and save it in pup_save.sfs. in that way you can save some space on your pup_save file. or if a new version arrives you can easily delete or overwrite the old one. puppy supports modules, but i think its limited to only 4 sfs modules? am i right guys?

i have made a module myself. i have downloaded a slax wine and an mplayer module and a firefox.pup installer, i have unsquashed both wine and mplayer and merged them together in a directory with firefox. make squash the directory and i have a module. renaming the module to mymodule1_213.sfs and installed at /mnt/home and thats it! ive been using firefox,wine and mplayer frequently and it goes well.

it woud be better if puppy could support large numbers of modules, in that way it will be convenient, saves space on pup_save, and what else?

however, too many modules loaded means youve got to have some Mb of memory and some Mb of swap partition? an i right?

well, modules are cool. :)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#4 Post by sunburnt »


blubb_fallo
Posts: 19
Joined: Tue 20 Feb 2007, 16:51

#5 Post by blubb_fallo »

erikkire wrote:i have made a module myself. i have downloaded a slax wine and an mplayer module and a firefox.pup installer, i have unsquashed both wine and mplayer and merged them together in a directory with firefox. make squash the directory and i have a module. renaming the module to mymodule1_213.sfs and installed at /mnt/home and thats it! ive been using firefox,wine and mplayer frequently and it goes well.
Interesting, that's a bit similar already. :)

I am thinking of modules which contain a specialized, preinstalled application each (or at most, a couple of closely related, interdependent apps/libs). Of course, there could also be some predefined "group selectors" that allow to include/exclude several modules with a single click, like "core" or "multimedia-light", as well as saved (customized) patterns.

See armish's grande unified puppy proposal for an (arbitrary) example of a collection that might be labelled "core" (it's the middle column of the table). Other than suggested for grand unified puppy though, I am thinking of comparatively large numbers of modules. The modpup chooser could easily provide, say, 10 to 15 group checkboxes on the first screen, and many more detailed ones on deeper levels. The interface should provide a sensible default selection that is automatically confirmed on timeout.

Modules that are loaded through this hypothetical boot interface would correspond with mandatory mydsl modules, whilst dotpups etc. (installed into pup_save.xfs like usual) would behave about as dsl's optional modules.

As far as I can see, such a modular system could do a job similar to the one of conventional package managment, but adjustable for each session at boot time. If everything beyond the "barebones" was handled that way, pup_save.xfs would be entirely dedicated to personal data.

Eww, of course, the modpup selector would have to resolve dependencies between the modules ... Well, long way to go.

erikkire wrote:however, too many modules loaded means youve got to have some Mb of memory and some Mb of swap partition? an i right?
I suppose, merely distributing the packages over separated sfs modules won't increase the total ramdisk size requirements all that much, but I haven't actually measured the overhead yet. If you deactivate modules you don't need, ramdisk can be made smaller anyway.
Last edited by blubb_fallo on Sat 10 Mar 2007, 20:45, edited 1 time in total.

amish
Posts: 615
Joined: Sun 24 Sep 2006, 23:15

#6 Post by amish »

for an (arbitrary) example of a collection that might be labelled "core"
it's not the least bit arbitrary. it's an Example, to be sure, but a carefully weighed one. all the apps shown (except the larger .sfs files on the last column) are in puppy 2.11. the heavier ones are in the core. so that when you jettison the core you end up with instant barebones. but you still have some things like leafpad and file managers, and dillo is needed for the help system.

the idea is that by default you get the barebones plus core. at your option, remove the "core" and at your option, add the .sfs files on the right column.
Eww, of course, the modpup selector would have to resolve dependencies between the modules ... Well, long way to go
i promise you're overcomplicating this. your intentions are noble but you're starting farther from puppy's current design than i am, and you're straying farther from its simplicity than i am. trust me, there are enough headaches in the one i'm talking about. the one you're talking about is a dream in a pipedream- but more power to you. the kind of modularity you're talking about already exists in slax. the kind i'm talking about is more simple- like puppy. with all due respect, of course!

erikkire
Posts: 31
Joined: Thu 09 Nov 2006, 18:01
Location: cebu

#7 Post by erikkire »

software modules are a great idea for puppy. we dont want large programs like OpenOffice to be installed in pup_save.sfs. im still new to linux and to puppy, but as far as i know, pup_save is limited to about 1Gb? with large programs like OO, pup_save fills up easily. so, the idea of modules is VERY GREAT, like sunburnt said.

about a program that loads and unloads a mod i think this can be easily made as puppy has many enthusiasts, fans and great programmers. if many see this as a great advantage and a practical way of saving space on pup_save, then this feature will certainly be available on the next release. :)
I suppose, merely distributing the packages over separated sfs modules won't increase the total ramdisk size requirements all that much, but I haven't actually measured the overhead yet.
i think the modules that are loaded at bootup will certainly be eating away memory. this is right, right? itll be worse if you dont have a swap partition or swap drive then all those modules will allocate a larger chunk of your memory available. i have observed my memory and swap drive meter fills up after loading my mymodule1_213.sfs on the next bootup.

the above problem can be eliminated if no modules are loaded during startup. if a software module is needed, one can run a special program that mounts and unmounts a module and thus eliminating memory usage.

about dependencies, well, i guess all libs will be stored on a single file module that can be squashed and unsquashed for additional libs.

this idea of modules, are used by many linux distributions. like slax. hope this will fully be integrated into puppy. :)


pardon on my english.



go puppy!

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

#8 Post by WhoDo »

erikkire wrote:im still new to linux and to puppy, but as far as i know, pup_save is limited to about 1Gb?
Actually, I don't believe anyone knows the theoretical limit of pup_save. Mine is 2.25Gb on one machine and I have no problems. That said, I'd rather it was MUCH smaller and applications were contained in an sfs file instead.
erikkire wrote:if many see this as a great advantage and a practical way of saving space on pup_save, then this feature will certainly be available on the next release. :)
Oh, given the interest here I can all but guarantee it will make it into the next release of Puppy 2.15CE, after that it's up to BarryK.
erikkire wrote:this idea of modules, are used by many linux distributions. like slax. hope this will fully be integrated into puppy. :)
I liked Slax, too, but it wasn't what could be called "light" on RAM if you wanted plenty of modules loaded. Puppy will be light on memory simply because only the core will get loaded there.

We also need to consider a special edition for the LiveCD, since that would load from sfs files more slowly than hard disk on older hardware. That said, it is ALL possible, one way or another. I'm starting on some sfs modules today! First will be Firefox/Thunderbird, so I can have Opera with its inbuilt email client in the core instead of seamonkey. :P

blubb_fallo
Posts: 19
Joined: Tue 20 Feb 2007, 16:51

#9 Post by blubb_fallo »

amish wrote:
for an (arbitrary) example of a collection that might be labelled "core"
it's not the least bit arbitrary. it's an Example, to be sure, but a carefully weighed one. all
I notice that my remark may have sounded somewhat dismissive. Sorry - it was not meant that way.

I "arbitrarily" had chosen an example colletion a group selector could be based on. Since I was not concerned with the content of any particular collection (aside from barebones), I could likewise have chosen another "arbitrary" example for "core". Yours happened to be the first around. Also, I could have chosen any other label in the first place, like, "snore", "windows media", "crypto", whatever. Neither label nor actual content really mattered here to me; my point was and is about fine grained modularity as such.

---------

I haven't tried slax on my own, but the documentation suggests that the X versions are nowhere as light as Puppy. The CLI onebone there is about as big as the whole PuppyOpera I am currently using. Also, it does not offer anything like pup_save.xfs. The closest Puppy alternative I know is dsl. Again, there is nothing like pup_save.xfs, but it is as small as PuppyOpera and uses modules. That's where I started above...
Last edited by blubb_fallo on Thu 01 Mar 2007, 17:53, edited 2 times in total.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#10 Post by sunburnt »

amish; If by core you mean the main base sfs file that has Xwin in it,
it can't be unloaded with out crashing Xwin.
UnionFS allows a maximum of 5 unions at any one spot (mount point).
So the base sfs file is 1 & the Save files is 2, you can can load 3 more sfs files.
This is the idea behind it, the base sfs file has Xwin & libraries needed,
then just the VERY needed & used core apps. (browser & media) are added to it.
Xwin makes for a fixed base sfs file, so add just what you commonly need.
But the core could be bigger, even full Puppy versions... anyway you want it.

amish
Posts: 615
Joined: Sun 24 Sep 2006, 23:15

#11 Post by amish »

okay, so unionfs has a limit of 5... and the kernel can only do about 10 .sfs because of loops, so even puppy 2 can only do 10 but 5 can be swapped in realtime.

i'm talking about the xwin staying in barebones, firstly. so that's the easy part. the tricky part maybe is a core that can be on the cd and load by default. then to make it not run, you delete it.

then the other items i've suggested in my diagram can be swapped, but i was thinking then the os could reboot. does that make a real difference?

the idea of swapping in realtime is interesting and could add flexibility, but also limitations that perhaps a "best of both worlds" could be reached, even if it means recompiling unionfs or something. this is a long term plan, you could start something with puppy 1 right now i guess, but it's still different than what i'm talking about.
sadly, it is not possible to separate politics from free software. free software - politics = unfree software.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#12 Post by sunburnt »

amish; I must have missed your post outlining what your talking about.
Give me a short story version? I'm interested in what others are thinking on this.

The loops are no problem, sfsManager will mount & union each SFS file loaded.
Then when unloading an SFS file, it'll ununion & unmount it freeing the loops.
So the SFS files that're unioned will only take up 5 loop devices total & no more.

NOTE: I just got my utility: mksfs working & I made a PupServer SFS file.
............... If someone want's to try it, I've posted it below with a pix of it.
Also is a pix of the sfsManager that loads & unloads SFS files.
Attachments
sfsManager.png
This is the sfsManager for swapping SFS files. ... TESTING
(20.99 KiB) Downloaded 1241 times
mksfs.png
This is the mksfs app., for making SFS files. ... TESTING
(10.56 KiB) Downloaded 1502 times
mksfs.tar.gz
It's a GUI app. with 2 files: mksfs &amp; mksfs.gtk, run: mksfs for GUI above.
(2.14 KiB) Downloaded 988 times

amish
Posts: 615
Joined: Sun 24 Sep 2006, 23:15

#13 Post by amish »

I notice that my remark may have sounded somewhat dismissive. Sorry - it was not meant that way.
no worries :)

sunburnt: those are fantastic gui's! the labels on the buttons... add "source"... maybe a readme file would be helpful? and the short story version is:

once upon a time, amish tried an os called puppy. eventually he started looking at the different things people wanted and tried to find a way that puppy could retain outward simplicity, and small size, while becoming more scalable.

(unbeknownst to amish at first, everyone else was talking about it too.) so he made a diagram of a relatively barebones, relatively normal or medium, and relatively huge puppy, all of which were less than 400mb, was normally under 90 or 80, and which could quickly become barebones with a few useful features in less than 50mb by deleting a single file from the .iso. it was based on puppy 2, but had better .sfs handling inspired by puppy 1, and grafpup 104. also it was based on a version of puppy that hadn't been released yet: one with lgpl scripts :)

the end?
sadly, it is not possible to separate politics from free software. free software - politics = unfree software.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#14 Post by sunburnt »

amish; "Add Source" a dir. or file with apps. in it, that's one of the sources
used to make the new Squash file, several source dirs. or files can be used.

I don't know what "lgpl scripts" are... enlighten me ???


The real picture here is this:

The "boot sfs" file with X & maybe a browser & some core apps. is made small.
Probably about 30MB to 40MB for the "core" Puppy version with MU's DRI Xorg.
Core apps. would be: Geaney, burner SW, & maybe an IM app. (Gaim, etc.).
This because Web access & Media are the REAL backbone of most PC usage.
Media apps. (XMMS & Mplayer) would be in a "extra sfs" file that's ~ 20MB.
The trick is that custom "boot sfs" files can be made & swap the extra ones.
So there would be lots of "boot sfs" files also, so many custom core Puppies.

The "extra sfs" files are groups of like apps., many of them so they're small,
less than 100MB, most less than 50MB, so they can be loaded into memory.
Commonly used extra sfs files could be joined into larger ones & added to
boot sfs files, thus allowing 1 more sfs to be loaded within the total limit of 5.

blubb_fallo
Posts: 19
Joined: Tue 20 Feb 2007, 16:51

#15 Post by blubb_fallo »

That GUI looks really promising! I didn't comment on it yet just because I haven't found the time to try anything. Does it also append to existing sfs, and if so, in which directory-keep mode?

On another matter: I am still puzzled by your repeated statements that there is a limit of 5 mounts to unionfs. All I know about unionfs is some vague ideas, but I'm quite sure dsl allows more than 5 mydsl modules to be loaded (at boot time). They are also squashfs, superposed into one big unionfs.

amish
Posts: 615
Joined: Sun 24 Sep 2006, 23:15

#16 Post by amish »

lgpl is a gpl-compatible license for software. currently the scripts that make puppy startup the way it does are copyright+all rights reserved... however

1. pretty much the only thing that is protected (i.e. not already permitted by the other notes barry has left around) is the name puppylinux, you're free to pretty much do what you want provided credit is given and you call your version something else...

2. barry is considering releasing the scripts under the lgpl.

i think it will make puppy easier to take seriously in the linux community, and i wouldn't go out of my way to make a derivative until that point. it makes creating derivatives much easier from a license standpoint.

for this reason, the thing i'm talking about is much more likely to be based on a newer version of puppy than an older one, but- the way it's structured with .sfs should make it easier to scale back up or down, not to mention easier to fix bugs in.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#17 Post by sunburnt »

I haven't had much luck appending to a Squash file, but it can be done.
For this existing SFS files would just be one of the source files for a new sfs file.

DSL uses Cloop files, not Squash files... DSL is based on Debian.
5 image files / dirs. is the limit according to the UnionFS site, & many tests.
But this may have changed, I don't know. AUFS may be much more capable.

sccat
Posts: 160
Joined: Mon 22 Aug 2005, 04:22

#18 Post by sccat »

Very Good Ideas.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#19 Post by Flash »

Would you guys like me to move this thread to Cutting Edge? Or maybe Projects? It seems to have gone far beyond a mere suggestion. :)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#20 Post by sunburnt »

Blubb_fallo ? ...Other wise this thread seems to be waining...

Post Reply