SFS-Combiner - merge some squashfs addons

Stuff that has yet to be sorted into a category.
Message
Author
User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#46 Post by MU »

great, my congratulations :)

Many thanks for not giving up quickly.
Now with your help, we have found an easy to use solution 8)

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#47 Post by sunburnt »

MU; The GUI I made used directories, partitions, or SFS & image files for sources.
It would be easy to add this to your GUI... I'll post a pix of my GUI.

Also, mksquashfs can handle multiple sources, so there's no need to copy
temp. files to a Linux partition or an image file, this is how mine worked.

Let me know what you think.
PS; It's good to hear from you too, I've had personal stuff to do for awhile.
Attachments
mksfs_gui.png
Simple, the [Add source to list] button opened a file browser.
It would check if image or SFS files were mounted & mount them if not.
(16.5 KiB) Downloaded 2315 times

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#48 Post by MU »

looks good, was it working already?
We had to find the code again buried in the forum...

It would be an advantage to use no temp-folders, so even people without Linux-partitions could use it.

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#49 Post by sunburnt »

I can't remember just what it's condition was, it's been a year or so.
I'll take a look at it tomorrow, I seem to recall there was a top dir. problem.

This apps. an important part of a no union & small main SFS file Puppy setup.
Instead of a normal package install, it's installed into a new main SFS file.
So there's no Save file needed unless Linux files are handled on a FAT fs.
As always... Puppy needs more loop devices, 16 would be good, 32 maybe?

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#50 Post by MU »

here is a small tool to add PETs to a SFS:
http://www.murga-linux.com/puppy/viewto ... 076#198076

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#51 Post by sunburnt »

MU; This is the usage from help, just use the sources in a line.

mksquashfs source1 source2 ... dest [options] [-e list of exclude dirs/files]

The sources can be: directories, partitions, & image or SFS files.
The new SFS file should contain the contense of the directories or mounts.

Having the app. check if an item is mounted & use it is good way to go,
as someone may want to put part of the save file in an SFS file.
I started writing Bash libraries to do stuff like this as I'm sick of rewriting.

Or better... Just mount the SFS file or partition again at your mount point,
it won't hurt anything, it's accessible from both mount points.
I commonly forget this little trick, doing double mounting.
You can also mount over another mount point, it just covers it up.
Dougal suggested this (I think) for doing a full install to an image file.

Could you integrate the .pet to SFS into the app.?
Maybe add .deb, .tar.gz & .tgz to it also?
.deb may need small changes to the dir. structure to be compatable.
If the packages can be extracted directly to mksquashfs it'd be fantastic!
Though I don't know if that'd be possable... I'm sure you would know.
This'd over come the same problem, the need for a temp. ext2.

Your app. could use most of the common package types for sources!

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#52 Post by sunburnt »

Mark; A few Qs about your SFS app. if I may...
1) Does it make a manifest file of all the /paths/files contained in the new SFS file?
2) Does it have a GUI for creation of menu files for the apps. in the new SFS file?
I ask because I've been using Puppy-400-NOP, & I noticed the goffice-400.sfs
addon file for it didn't have them. But they're made to be a pair so it's handled somehow.

I'm working on a no union setup & I'm trying to resolve issues, the SFS files are one.
Another is the PATH thing we talked about, to add to the paths makes them too long.
I'd like to be able to addon lots of SFS files (10 or 20), so adding to the paths is out.
Sadly there's over a dozen places to look for files... ( There's gotta be a better way )
My current idea is for 2 dirs., 1 in PATH & 1 in LD_LIBRARY_PATH with links in them
to all the files inside the SFS files. This is a really sucky way to do it.
The only other way I see is to make the SFS files differently by combining dirs. & paths.
Needless to say I'm open to better ideas for doing this, please add your thoughts...

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#53 Post by MU »

1) Does it make a manifest file of all the /paths/files contained in the new SFS file?
2) Does it have a GUI for creation of menu files for the apps. in the new SFS file?
1) no

2) no, but if applications include .desktop files, they are added to the menu automatically.
You just had to run "fixmenus" in Puppy.
In Muppy, the new .desktop files are detected automatically.

To create .desktop-files for an installed pup/pet without one, you can use:
http://dotpups.de/dotpups/System_Utilit ... -packages/

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#54 Post by sunburnt »

Thanks Mark, I'll write the code line to create a manifest file, "find ....".
I'll have a look at the app.: XDG-menus-for-alien-packages to see if it'll do the job.
I've got the code written to scan drives for SFS addon dirs., just XDG & paths are left.

Any thoughts on a better solution for paths into the SFS files?
If there's a better solution I'd like to work on it rather than wasting time on a dead end.

User avatar
dinky
Posts: 699
Joined: Sat 19 Jan 2008, 23:39

#55 Post by dinky »

Hi MU, great program! Thanks so much for creating it. Couple questions:

1. Does the pup_save file load into ram? I.e., if I have 512 Mb of ram, and a pupsave_file of 1Gb, will this effect my speed? So if I merge .sfs files with the pup_save, will I run into problems in a frugal install?
2. Do .sfs files effect speed at all? If they don't get copied to the ram on bootup, do massive sizes of them effect the speed at which puppy runs, assuming it's being booted into ram?
3. I love .sfs Is it possible to create a database, much like ttuuxxx's lib's page where they are all in the one place? This would make it much easier for newcomers to puppy.

Thanks mate.

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#56 Post by MU »

1. Does the pup_save file load into ram?
no.
2. Do .sfs files effect speed at all? If they don't get copied to the ram on bootup, do massive sizes of them effect the speed at which puppy runs, assuming it's being booted into ram?
In general no.
In practice:
yes.
SFS files are managed by a Kernelmodule, the unionfs-driver.
It must check, if files changed.
This has some effect.
Rebuilding menus can take longer, if you use KDE.sfs, that has dozends of menu-entries.

If you use a sfs without menus (like devx.sfs), you of course will not notice a delay.
3. I love .sfs Is it possible to create a database, much like ttuuxxx's lib's page where they are all in the one place? This would make it much easier for newcomers to puppy.
Until now, they are "cluttered around".
For Muppy, we list tested sfs-files on the Muppy-website.
A collection of various contributions is here:
http://puppyisos.org/files/sfs/
Password see:
http://www.murga-linux.com/puppy/viewtopic.php?t=28930

The Wiki should list several, too.
Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
dinky
Posts: 699
Joined: Sat 19 Jan 2008, 23:39

#57 Post by dinky »

Thanks for that. Have been fun with your pet2sfs converter, and the sfs combiner. There seems alot less need for an sfs file database now. Incidentally, when setting sfs files at bootup, can there be a problem with file clashes? I.e., if one sfs file has the same file as another, and they are not combined but mounted as one of the three you can boot up from, can there be issues? Also, if two sfs files with the same file on them are combined, does this duplicate the file? I'm guessing not, but thought it a good idea to check.

Anyway, thanks again matey.

~dinky

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#58 Post by MU »

dinky wrote:Thanks for that. Have been fun with your pet2sfs converter, and the sfs combiner. There seems alot less need for an sfs file database now. Incidentally, when setting sfs files at bootup, can there be a problem with file clashes? I.e., if one sfs file has the same file as another, and they are not combined but mounted as one of the three you can boot up from, can there be issues?
yes, if they are different...
Puppy mounts them in the priority as they are listed in the bootmanager.
If you add:
a.sfs
b.sfs
c.sfs
and all have /usr/bin/playme.sh
then only playbe.sh from c.sfs is used.
dinky wrote:Also, if two sfs files with the same file on them are combined, does this duplicate the file? I'm guessing not, but thought it a good idea to check.
As above.
Here, files are overwritten with the ones from the last sfs added to the combiner.
Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#59 Post by sunburnt »

Hi; I found the post again finally... Disregard the PM. I'll send what I write.
If you have code to feed .tar.gz or .pet files directly to mksquashfs, tell me.

Yep, each source in the mksquashfs command is over written by the next.

NOTE: I've read on the web it's a bad idea to use ext3 for flash devices.
The journal gets written to lots, so it tends to wear out the flash drive.
BUT... For this purpose it won't hurt, because the SFS build is a short use.

So instead of: mkfs.ext3 /dev/sda1

For most uses: mkfs.ext2 /dev/sda1

User avatar
dinky
Posts: 699
Joined: Sat 19 Jan 2008, 23:39

#60 Post by dinky »

Hi sunburnt, how close are you to having a test version of your project with making sfs files without a linux partition? Think it's a great idea, and would love to try it. Cheers!
~dinky

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#61 Post by sunburnt »

Hi dinky; With any luck MU will have the time to modify his great app.

Last night I simplified the identifing of image file types & their mounting.
Today I'll be testing that work, I'll try a few SFS file builds & check them.
If all looks good I'll repost "mksfs" here in the Software forum for testing.

User avatar
dinky
Posts: 699
Joined: Sat 19 Jan 2008, 23:39

#62 Post by dinky »

Look forward to it matey, will be happy to test things. Cheers.
~dinky

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#63 Post by sunburnt »

After much work...
It seems the top mount directory being saved to the SFS file is a real mksquashfs problem.
I looked at the docs & there's no way to control it to keep from doing it...
Explanation:
If there's just one dir., then it's contense is put in the SFS file.
If there's several dirs., then the selected dirs. & their contenses are put in the SFS file.
Some times this is exactly what you want, like what I was doing, but sometimes it's not.

What this means for mounted iso, sfs. ext2, & ext3 files is the last mount point dir. is included.
Like this mount point for an SFS file: /mnt/sfs, so /sfs is stored in the SFS file as top dir.

Sadly... The only option left is to do one source (dir. or file), & then append the next, etc.
This way all the sources will have their contenses stored in the SFS file.
I didn't have any luck appending to a SFS file before but I'll try it again, maybe this'll work.

ADDENDUM:
Success at appending each dir & file to the SFS file, just the contenses were stored.
We need a way to control this, sometimes you want the dirs. stored & sometimes not.
All the mounted files definitely need their contenses stored, but dirs. need it both ways.
To get the top dir. stored, all I can think of is to do an empty dir. with the source one.
Several sources store the top dirs., & just reuse the same empty /0 dir. over & over.

Using an empty dummy dir. to force storing of the top dir. works, but...
Each time it's used a number is appended to it, like this: /0, /0_1, /0_2, etc.
The same thing happens to all dirs. stored several times, like: /bin, /bin_1, etc.
This seems acceptable to me to be able to control the building of SFS files.
Otherwise, copying files into a ext2-3 image file or partition is the only other way.
Again... If anyone else has any info or ideas to offer on this, speak up!

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#64 Post by sunburnt »

Got it working (I think), posting it in Software forum.
MU; I'll send the bits & pieces of code if you want, or you can DnLd & pick it apart.
Let me know what you want & what you think... Terry

User avatar
dinky
Posts: 699
Joined: Sat 19 Jan 2008, 23:39

#65 Post by dinky »

Hi Sunburnt, can I get a link to your post? I can't find it. I don't understand much of what you wrote, but am eager to try it. I'm off to Sydney for the next week, and will hopefully get a chance to try it up there, if not will give you my feedback when I return. Cheers!
~dinky

Post Reply