GtkDialog - Make Image / Save / SFS files utility.

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#31 Post by sunburnt »

Yes Debian is bloated, but many offshoots slim it down nicely. I tried Feather Linux and I liked it a lot.
Software is layers, so we must always be cautious about each of them, and in particular the lower ones.
# I`m making up a Calculator AppPkg using a union for you to look at...

### I realized last night that I didn`t include a Swap file in my app. It`s an image file also.
.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#32 Post by musher0 »

sunburnt wrote:musher; As I`ve said, I use to write a diddy in V.B. and show it to my neighbor and it`d crash his PC.
Coding is hazardous at best. That`s why I opt for simple languages, Bash, BaCon, etc.

# I don`t think there`s a problem with my image file maker, but I haven`t tried every possibility. (...)
.
Hi, sunburnt.

My bad... I re-read your initial post and realized that you've designed your utility
for the new Devian-live Puppy-compatible distro.

Forget what I said. Sorry for any confusion. I've got to get myself some new
eyeglasses...

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#33 Post by sunburnt »

Not really, it has a "Puppy extension" checkbox now. So for many distros.

And I think I`ll add a swap file to it also, a swap file is an image file too.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#34 Post by mikeb »

@amigo
ok finally got to test in Lucid pup. Not the latest but does give a broader view of the feasibility.

Firstly added the missing bits... unionfs-fuse and fakechroot.
ran the installer but lacked ar...unpacked the deb manually to app.
fakechroot from ubuntu wants full getopt... and -- needed adding.
chrome bin path was wrong :D (debian sid packages would not unpack???)
Ok this is all minor stuff plus it does not need to be rox/appdir centric to work so I basically did everything by hand following the script.
But files would need adding and a suitable system for configuring. I would say a gui to do it locally to encompass a choice of sources...hard drive, sfs and so on...its flexible in that respect.

In doing so I did create the union.
In my layman's terms it builds a union using the normal puppy file system as the bottom layer and a folder of choice for the read write...the app itself is sandwiched in between. The app itself could be a mounted sfs/image file or a folder anywhere.

It effectively sandboxes the app and does indeed leave the main file system untouched apart from where the user chooses to put it.

Ok onto running chrome.
Well a that was a non starter since it wants newer libstdc++ than lucid can run. (also wanted mozilla libs and at 150MB installed its quite a behemoth..not sure what the attraction is..firefox all is forgiven.. but I digress) It also seemed a poor test since chrome like firefox can run from anywhere so the only 'benefit' would be the relocation of its profile. I then extracted a package of vlc to the app folder . It did appear in the fake union ...but when I ran vlc it said it could not find its plugins...they did exist in the same fake union ..message was
cannot read (null)/plugins/plugins-04041e-28.dat

So all looked correct yet had a problem finding itself. I also noticed ldd did not work either. Am I missing something in running it perhaps or does this mean each app would need specially configuring to work this way?

Is union-fuse used as we are mounting on an already existing kernel unionfs? Does the fuse version have much impact on performance?...I did not really get a chance to test that aspect.

Advantage I can see... apps can be run independently so conflicting libs do not matter.... A neater, if it can be implemented simply, way than using LD..variables plus sfs can be taken advantage of.

It could be cited it leaves the system clean after use but that equally applies to kernel inserted sfs. If unioned outside the puppy filesystem it would be kept out of remasters. Might be a good choice for such as a dev toolchain too.

Not sure if configs and such being written elsewhere is an advantage but it would group together these items to give the option of a total clean up after usage....eg union to /tmp . Good for the paranoid :)

There is a warning that dbus connection will be killed...does that mean of the whole system?

Performance.... hmm fuse union layer on unionlayer inserting possible a loop mounted device... . a system in a system

Ok that's it for now.... interesting stuff

mike

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#35 Post by musher0 »

Hi, sunburnt.

I was finally able to create a good usable pupsave with your script, when I added the
-m 0 -F -F parameters, like so, to the mkfs command.

Code: Select all

echo y | mkfs.$FS -m 0 -F -F $P/$F
Maybe a single -F is be enough, but I did read on the mkfs man page that doubling it
may be useful, solving problems at times.

I suppose the other types of files don't need the -F parameter? A swap file, probably
not, it being necessarily a raw, unformatted file in which bytes come and go. Debian
live-rw savefiles and such, I don't know if they need their file system formatted. (Can
someone answer that please? It would be helpful, I think. Thanks in advance.)

What I am trying to say, sunburnt, is that perhaps the mkfs command you use in your
script needs to be further differentiated per savefile type? Just a thought.

In any case... the command

Code: Select all

echo y | mkfs.$FS -m 0 -F -F $P/$F
produces a valid pupsave file, into which you can really save stuff, it's not a Teflon
savefile.

However, on first use of this pupsave file, in the Puppy desktop, the icons appeared as
danger road signs, and the ROX pinboard had no background. To get a responsive
ROX pinboard, I had to use the command

Code: Select all

gdk-pixbuf-query-loaders --update-cache
suggested by watchdog here:
http://www.murga-linux.com/puppy/viewto ... ost#722565

I know this last paragraph is outside the subject of this thread, but I thought I'd mention
it anyway, in case somebody got stuck with that bug.

Back on the dot, pupsave files created with the above command can be opened and
minimally populated. For example, in the file that I created with the edited sunburnt
script, I was able to create a /root/Startup structure and copy the gdk command in that
folder. It didn't execute on first boot, as a script in /root/Startup normally does on
subsequent boots, but it was there and usable. So I was able to correct my pix-buffers
problem immediately simply by clicking on it.

Another thing which I noticed is that the pupsave file created with the "-m 0 -F -F"
parameters had a "Lost and Found" folder, which the usual pupsave files do not have.

I hope my little test report was clear enough. Any questions, please ask.

BFN.

musher0

~~~~~~~~~~~~~~;
PS. My test Puppy was pemasu's raringPup 3992.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#36 Post by amigo »

mikeb, glad you had some success -chrome is really a bad example because it needs some extra steps mount-binding of /tmp for one thing -which may fix the dbus errors.

The main little seed I wanted to sow was the mechanism for creating the union/chroot which is private and self-destructive once the program exits.

unionfs-fuse is completely independent of aufs -although it would not surprise me if there *is* a way that they get mixed-up.

About using such AppDirs with non-rox filers, all you need is a link from the AppRun script into your normal path(name the link after the program). This way, clicking on that link will do the same as clicking on the AppDir under rox.

If you want to play with that method, then you'll find it works without extra stuff (mount ---bind) for lots of stuff -no LD_LIBRARY_PATH mods needed and accesses to /etc /usr/share work as expected.

All chroots may suffer from strangeness similar to what chrome is doing particularly regarding underlying system mounts. mount --binding them in the union fixes it.

The process does look a bit complicated, but it's less messy than loads of links, and inside the chroot everything is transparent, while being sandboxed.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#37 Post by mikeb »

yes I have one of those handles on this.. its often clearer when doing rather than when reading about it on a forum. I have a mental picture of the process and that's usually a good sign. :)

Only outstanding question is can you shed any light on why vlc could not find its plugin folder?.
I will test out more apps this way but if you have any pointers on the subject.

I also did not try out a mounted image or sfs yet so will play with that too...at least I have a working method now.

Note to general...this mounts the app in whatever form it takes on top of the puppy filesystem like good bunnies need to :)

mike

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#38 Post by sunburnt »

Mike & amigo; Yes, Chrome is a bad example.

unionfs-fuse is really touchy as to what options are used.
I found mhddfs works without any fuss, but as amigo said: "it`s not made for the purpose".

Like aufs, each running union instance is a cpu + ram load, but added layers not so much.
User space unionfs-fuse is slow compared to aufs, no doubt about it.
This is my reasoning for worrying about using many of these = many instances.
A few to augment a normal app install system or lots of apps in one would be no problem.

# Mike; Perhaps you could test the speed / sys. load effects of a few of these.?
.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#39 Post by mikeb »

Hmm as mentioned it may particularly be useful for dev tool chains or sandboxing for the nervous.... at least it might save the reboot cycle some seem to go through.

One I will play with ..... just visiting lucid shutdown loops and its installer in general....qemu fun which is a little slow....and a general tidy round.

I think chrome should be called lead.... more appropriate metal. :D

mike

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#40 Post by sunburnt »

Chrome is actually fairly fast.
Firefox needed nothing, it runs from anywhere as-is. Very well written app.
The newer Chrome needed only one link because they put it in /lib ( Duh...).
But this is a no-brainer, to add and remove a link is far better than any union.


### MIke & amigo; A Q if I may...

I`ve got Rox installed on Debian Wheeezy and I have the Rox desktop line in:

$HOME/user/.bash_aliases

Line = rox -p /(path)/PinBd &

### This should start the Rox desktop at boot, but it doesn`t.
But if a Vterm is started then the Rox desktop starts, close the term. and Rox is gone.
# I figure this behavior is the key to getting it to start properly.

Any ideas that may help guys.?
.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#41 Post by amigo »

.bash_* files are only read when a login terminal is started.
If you want to autostart rox, then you need to put the command elsewhere. Which WM are you using?

All tools which use FUSE are, in theory, slower than the direct access afforded by normal filesystem drivers. However, at least for unionfs-fuse, it is still faster than disk I/O so now slowness can be measured.

Mounts, in and of themselves, do not impose any RAM penalty at all. It is unlikely that a user would have many of these private union/chroots running at once -and if they did it still poses no speed/memory disadvantages.

The fact that this method *can* be used for anything should not lead one to overuse the trick. All the ideas about sfs/appdir are great for *applications*, but one should not use these methods for libraries. Libs should always be in a normal installed package so they can be accounted for by the package manager.

The Chakra project put it nicely in their specs for bundle management: a bundle may depend on any package, but may not depend on any other bundle.

mikeb, I use unionfs-fuse chroots as part of src2pkg package creation - a quick, easy, dependable way to isolate package content-creation for those builds which don't support DESTDIR or other such methods. Been using it for years without any problems.

The example AppRun for chrome is really still very basic -I tried to keep it that way so that the couple of lines which are the gist of the mechanism stand out.

Firefox is really not a good example either -exactly because it is made to run from 'anywhere'. Most apps are not.

One thing to keep in mind, under Apple OSX, everything which doesn't come included with the OS is a bundle. Any bundle which needs libs which are not part of the original OS, or needs to replace those, comes with those libs inside the bundle. Trying to reference another bundle as a dependency poses lots of problems, so the use of some intermediate sfs with a bunch of extra libs is not really viable.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#42 Post by sunburnt »

Yep, if other folks try to use a "lib sfs file" it`s bound to have bad results.
If the lib sfs was compiled on the os, and all apps that use it were complied with it... Maybe.?

# I like the idea of one or two union AppPkgs with most of the "stubborn" apps in them.
The rest: "Easy to get to run", or "exec. modded", or apps compiled "relocated or relative".

amigo; I got Saintless`s Wheezy booting to the Rox desktop with icons. Many thanks my friend...
Strangely many web pages said .bashrc was the place to start apps. I didn`t think so...

Debian is a real wake-up call for my many easy years here at Puppy. But I knew that...
Doing the simplest stuff is a struggle, many would quit. But this is a real opportunity for me.
The folks at Debian really have made a can of worms. Barry did a good "simplification job".


# Now I need to learn how to add JWM tray apps, and where to get them.
.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#43 Post by mikeb »

Barry simplified slackware...you can see traces of it...slackware is simpler I find and from the world of puppy you would find it the most familiar...main difference would be the init run levels.

When I make an sfs I include any libs that are not in the core / host system. In other words any package can run without the need for any other. Seems sensible and duplicate libs are not a problems or a major size penalty. I have one exception and that is QT. Its chunky and versions change often and its not part of the core on anything I run.(bear in mind a 100MB core is retained for easy usb stick or cd usage for debugging/fixing/testing) I also have wxwidgets as a module but that's only for testing..I add the needed libs if I keep the app. In fact when trying a new app each additional dependancy is quickly deb2lzm and added until all works...I then have a bundle to add to the app sfs. Non destructive testing is my want :)
Of course goes without saying so i will that such libs should be built or chosen for that same core system. Thats roughly the slax approach too.

Bad is the app that does require oddball libs... I remember xaralx being like that years ago and its not a good situation.
There is a good reason the likes of debian catagorise their packages by version.

Isolation...definately a bonus for building . I used to boot pfix=ram with the dev on puppy.... now I just don't save with my archive save. On slax I have got lazy and just add the dev file and build...and thats with a save folder....but only for standard stuff. If its mingw or tinylibc for example I go into no save mode. Adding a dev package should not affect the running of the system but life is not always perfect which is another one to consider.
I also like to test new apps before adding and easy removal / isolation is a must for me.

Performance... well anything outside the core is an sfs in these parts...some are just a few hundred k and all seems happy...a heavy window manager would on the other hand be noticed.

have fun

mike

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#44 Post by mikeb »

chrome...well I question a web browser thats larger than some of my puppies...and thats comparing compressed to compressed.

mike

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#45 Post by amigo »

I like the way chromium starts and displays, but I'm doing a long penance for using Opera so many years...
I don't really like 'predictive' features unless I'm writing them myself.

Any decent browser is gonna be huge with whatever it needs -I worry more about the experience than the size of software. And I think that most users are perfectly willing to 'pay' in disk-space for any application they really want or need.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#46 Post by mikeb »

Webkit was always chunky... wonder what all that code is for. I remember the days when konqueror crashed all over the place and was as slow as hell... so credit to google for taking their software and improving it. (or whoever first wrote it... nokia?)
I suppose using free software as your base makes good sense in terms of profit :)

Much of the rendering in firefox seems to be handled in raw javascript..opera compiles it in some way.....webkit..some other binary method.... for FF that must be a source of slowdown. When I was younger and bored I de-commented and de-whitespeced all the code in firefox 1.5... its was considerably more responsive for it. Using gtk2 must have a speed penalty too. Qt is large but give better dynamic performance I find.

mike

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#47 Post by sunburnt »

Mike; Do you happen to know how JWM loads the apps into the tray.?
I know about the <Swallow > tag in .jwmrc, but it says nothing about 4 items in the tray.

See here: http://www.murga-linux.com/puppy/viewtopic.php?t=90778

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#48 Post by mikeb »

Mike; Do you happen to know how JWM loads the apps into the tray.?
Badly I suspect ... :D

Actually not a clue...its on my top 10 lists of things to remove from puppy which apparently makes them no longer puppies. I will have to think of a new name... Guinea Piggies seems apt and we have an extremely cute long haired one here.
Would I have to call 'pets' 'family'?

Michael

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#49 Post by jpeps »

mikeb wrote: When I was younger and bored I de-commented and de-whitespeced all the code in firefox 1.5... its was considerably more responsive for it.
mike
Maybe you didn't know that code gets compiled into machine code like you do now, and that it was only your imagination. Comments and whitespaces are considered good coding practices to make it understandable for later editing.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#50 Post by mikeb »

Code: Select all

Maybe you didn't know that code gets compiled into machine code like you do now, and that it was only your imagination. 
Ah you saw an opportunity to troll again ...welcome to the thread. :)

Now the xul interpreter is compiled but take a stroll into firefox/chrome and you will see some jar files. Look inside and you will see a pile of uncompressed (and uncompiled) javascript. This is the code that handles the html... running on xul but its raw script in this bunny..

Thanks for the clarification ..i never realised when I write in C that it is compiled into machine code. When I did program directly in machine code to make drivers many moon ago I obviously did not understand that either.

So desparate to try and discredit...you need a hobby...one I can understand with my limited intelligence.

mike

Post Reply