The State of Package Management

What features/apps/bugfixes needed in a future Puppy

Should Puppy's package format be changed?

Yes, without backwards compatibility.
11
28%
Yes, with backwards compatibility.
10
26%
No, but the PET format should be standardized/stricter.
8
21%
No, the PET format works fine.
10
26%
 
Total votes: 39

Message
Author
disciple
Posts: 6984
Joined: Sun 21 May 2006, 01:46
Location: Auckland, New Zealand

#166 Post by disciple »

Moose on the loose wrote:Will AUFS allow you to have more than one layered file system?
The whole point in it is that it has multiple layers - and the overhead of that is what some people don't like. e.g. every squashfile is in a separate layer.
I think you are trying to ask if more than one layer can be writable, and no, I think the way a layered filesystem works is that you can only write into the top layer.
What I am thinking is making a mount point and mounting a second layered file system there. This way, we could get the effect of being able to write into lower layers. The "lower layer" really would be the top layer of the main file system. The "root" could be the mount point for the second system.
You can mount something (e.g. a partition or another save file) and write into it. You could keep your home directory outside of the save file this way if you want. It is traditional to keep various parts of the Unix / Linux filesystem on different partitions.
I'm a little unclear what you are trying to achieve though...
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#167 Post by Q5sys »

disciple wrote:
sunburnt wrote:Q5sys; Yes, the long version of what I was saying. Well put...

disciple; The main bulk of files in Puppy are in the SFS file.
So most all of the files are more secure, mainly the critical exec.s and libs.
But loose file installs are open to attack ( .pet ) as they go in the Save file.
But adding SFS apps. does not have the loose file problem and is secure.
The Save file`s mostly config. files ( or it should be...), less damage done.
The save file is anything that something tries to change. So if the hypothetical linux virus wants to modify or delete a program, it does so, regardless of whether you have a frugal or full install. Your ability with a frugal install to get the original program back because it is in the .sfs is not really different from getting the original back from the install CD / live CD / repository with a traditional full install.
Squashfiles are great, but they are not a security feature. If you are paranoid about security you should look at other things, like for starters:
1) never creating a save file
2) not running everything as root
Or are you guys not actually talking about squashfiles, but squashfiles that are only mounted, not unioned? Isn't that pretty impractical for libs, as you'd end up including duplicates in every squashfile which has an app that depends on them?
Isn't it simpler just to make places like /usr read-only (hint: don't run as root)*.

* Note that I am a "root forever" guy... but if someone else wants to be paranoid about security, that's their choice.
The method I explained above would allow for the editing of additional SFS files, which would not be reflected in a safe file. Its totally impractical, but possible. Fact is, you can edit the system without those changes being reflected in the safe file.
2nd, I know you're repeating the mantra of 'dont run as root to be secure', but really thats not entirely accurate. Doesnt matter if the user is root or not. A properly written virus or rootkit will use system files that have root auth to do the work. The user not being root doesnt matter one bit, if the attack is utilizing something that is running on the system as root regardless of the user. Example I had a discussion the other day with someone about the possibility of using xorg itself as the attack vector. Doesnt matter what your user is... xorg runs as root. Another example would be utilizing a bug that exists within /usr/sbin/irqbalance or a bug in an console-kit-daemon.
Not running as root protects a user from user stupidity or user ignorance about proper web saftey.

Like all things, there is no one security fix. Its layers upon layers of security that help make you secure. Any single layer will always fail.
disciple wrote:
What I am thinking is making a mount point and mounting a second layered file system there. This way, we could get the effect of being able to write into lower layers. The "lower layer" really would be the top layer of the main file system. The "root" could be the mount point for the second system.
You can mount something (e.g. a partition or another save file) and write into it. You could keep your home directory outside of the save file this way if you want. It is traditional to keep various parts of the Unix / Linux filesystem on different partitions.
I'm a little unclear what you are trying to achieve though...
That is a feature that TazOC implemented in LightHousePup. Upon first boot and creation of save file the user could decide if programs would install to the save file OR would be placed into a unique folder in the /mnt/home directory.
This way the user could (if they wanted) keep all their programs seperate, while their config settings were preserved in the safe file. I forget the exact version. I think it was the last 32 bit release.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#168 Post by sunburnt »

Yes; I said a properly setup standard loose file install would have read-only and read-write partitions ( or dirs. if installed to a single partition ).
And yes, a new virused file can be made or copied over one in the SFS file. That`s not the SFS files fault, it`s the loose file system of the Save file.
You`re sort of mixing apples with oranges, but you did say "in Puppy". So "in Puppy" you are correct of course, the SFS file only makes for quick recovery.

When I say Squash files I don`t mean SFS necessarily. I usually specify. Though SFS files are one type and could be included in some cases.

Consider... If the Squash file is not unioned with a Save file or any loose file system then it`s hard to hack. But access to the partition could.

disciple
Posts: 6984
Joined: Sun 21 May 2006, 01:46
Location: Auckland, New Zealand

#169 Post by disciple »

2nd, I know you're repeating the mantra of 'dont run as root to be secure', but really thats not entirely accurate. Doesnt matter if the user is root or not. A properly written virus or rootkit will use system files that have root auth to do the work. The user not being root doesnt matter one bit, if the attack is utilizing something that is running on the system as root regardless of the user. Example I had a discussion the other day with someone about the possibility of using xorg itself as the attack vector. Doesnt matter what your user is... xorg runs as root.
I don't pay much attention to the mantra, but I was pretty sure a big focus of it is that xorg should never be run as root. It's not a simple matter of "the user" not being root, it's about everything being run with appropriate permissions.
sunburnt wrote:When I say Squash files I don`t mean SFS necessarily. I usually specify. Though SFS files are one type and could be included in some cases.
Do you mind defining what you mean by "Squash files" then? Are some of them functionally different from SFS?
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#170 Post by Q5sys »

disciple wrote:
2nd, I know you're repeating the mantra of 'dont run as root to be secure', but really thats not entirely accurate. Doesnt matter if the user is root or not. A properly written virus or rootkit will use system files that have root auth to do the work. The user not being root doesnt matter one bit, if the attack is utilizing something that is running on the system as root regardless of the user. Example I had a discussion the other day with someone about the possibility of using xorg itself as the attack vector. Doesnt matter what your user is... xorg runs as root.
I don't pay much attention to the mantra, but I was pretty sure a big focus of it is that xorg should never be run as root. It's not a simple matter of "the user" not being root, it's about everything being run with appropriate permissions.
attached is a quick capture of htop from 3 systems, opensuse, arch, and slacko. Xorg has to run as root. because the GDM, KDM, whatever has to start up to allow a user to log in and choose their Desktop manager. Since all that happens before a user logs in, it has to be run as root.
Attachments
slacko.png
htop capture from slacko
(105.93 KiB) Downloaded 324 times
arch.png
htop capture from arch
(164.69 KiB) Downloaded 321 times

User avatar
Moose On The Loose
Posts: 965
Joined: Thu 24 Feb 2011, 14:54

#171 Post by Moose On The Loose »

disciple wrote:
Moose on the loose wrote:Will AUFS allow you to have more than one layered file system?
The whole point in it is that it has multiple layers - and the overhead of that is what some people don't like. e.g. every squashfile is in a separate layer.
I think you are trying to ask if more than one layer can be writable, and no, I think the way a layered filesystem works is that you can only write into the top layer.
What I am thinking is making a mount point and mounting a second layered file system there. This way, we could get the effect of being able to write into lower layers. The "lower layer" really would be the top layer of the main file system. The "root" could be the mount point for the second system.
You can mount something (e.g. a partition or another save file) and write into it. You could keep your home directory outside of the save file this way if you want. It is traditional to keep various parts of the Unix / Linux filesystem on different partitions.
I'm a little unclear what you are trying to achieve though...
Imagine the case where I have two wildly different computers. They require different drivers for the hardware and the screen's can't be set to the same resolution. If I want to be able to quickly move my work back and forth between these, I need to keep my work and all the settings separate.

Basically, I am trying to come up with a way to do that, and also to have the same programs installed on both. The ideal case would be to have a memory stick with two save files for hardware issues (one fore each computer) and a third save file for all of my work. This would allow my work to be compressed also.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#172 Post by sunburnt »

Moose On The Loose; Yes, aufs will run more than one union at a time. Config. files and app. installs should be separate. Put work in a dir.
I see what you`re getting at here and I`ve thought the same in the past. Then I realized that the union is an unnecessary complexity to be rid of.

disciple; As I said, a non-unioned Squash file is safe from the union`s effects. It`s compiled with --prefix (/path/mountPoint) so it works when mounted only.
ChoicePup did this and it worked very well. As typical with Puppy variants it`s no longer with us, but it`s "proof of concept" is what`s important.
Now if Puppy package builders ( and Barry? ) would use it... Problems solved.

amigo; I agree again. A CD / DVD is a horrible boot / run device for an O.S. And the H.D. is the old standard, a USB flash drive is a more modern device.
A PC with 2GB or more of ram doesn`t need a swap, so it`s suited for flash. Flash isn`t perfect, but... Mass storage is still the realm of a H.D. of course.
PXE booting`s great for large groups of PCs, but not so much for 1 to 3 PCs.

disciple
Posts: 6984
Joined: Sun 21 May 2006, 01:46
Location: Auckland, New Zealand

#173 Post by disciple »

Moose On The Loose wrote:Imagine the case where I have two wildly different computers. They require different drivers for the hardware and the screen's can't be set to the same resolution. If I want to be able to quickly move my work back and forth between these, I need to keep my work and all the settings separate.
Have you actually tried using the same USB stick Puppy install on both? I thought that it already handles this, by creating a separate xorg configuration for each.
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#174 Post by amigo »

Something is wrong here. Two or three people are starting to agree with me sometimes... Was it something I said?

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#175 Post by Aitch »

:lol: :lol:

nice, amigo...

Aitch :)

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#176 Post by amigo »

:D :D + :D
Not used to getting much love around here...

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#177 Post by greengeek »

.
Interesting topic. Some heated debate.

I vote for two things:

1) A continuation of the current fragmented status quo.
2) A newly defined package management/control system that satisfies the devs who see a need for change.

Why both approaches? Because I would compare Puppy’s development to evolution itself - variety is hugely important. Greater central control brings less variety. There is no reason why multiple “Puppy breeds

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#178 Post by sunburnt »

amigo; You`re knowledgeable and so opinionated. A good place to be. This tends to make one brutally honest, but I`m not saying that`s you...
This all of course makes a target of you. My same problem is with politics. Different parts of anything are seen as good or bad, depends on your view.

My continued view is that there`s a lot to like about Squash files. There`s many ways to use them in a O.S., some fix problems, some don`t.
Read-only files should be in a Squash file without a writable union over it. This makes everything in the Squash file "rather" secure compared to Puppy.
Don`t use a union to add execs. and libs., use the paths and added dirs. /root/my-applications/bin:/root/my-applications/sbin ... /root/my-applications/lib
If these paths are at the end of PATH then what`s added can`t overshadow anything. If you want it to act like Puppy does, just put these paths at the start of PATH.
To change what`s in the Squash file a simple remake takes a few minutes. Not much worse than installing many of the .pet apps., all in all...
I hate long dir. path names "my-applications and my-documents". Use /root/apps. and /root/docs instead.
Last edited by sunburnt on Tue 28 Feb 2012, 07:19, edited 2 times in total.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#179 Post by amigo »

I actually (optionally) use a couple of SFS's in my distro. wait, let me say exactly what I mean: I use file-system images which use the squashfs. I use them (optionally, remember) for the kernel-source and for kernel-modules -where read-only is usually no problem. *But*, they are part of a package so that the package-accounting still works. I only use them for these large components (kernel-source 460MB unpacked and modules ~60MB at the moment).

My arguments have not been against the use of squashfs in itself, in the right places. I'm against the idea of composing them from a mongrel-mix of files from all over the place without any way to account for them, or more importantly, easily reproduce the image when needed. I certainly see the logic in combining several *packages* inside an SFS, but including a bunch of maybe-duplicated or mixed-source libraries is not so good. You know, you could always just use a ramdisk or tmpfs and 'install' packages there on the fly, too. I just think that most of the ideas in use for adding 'modularity' to LiveCD's are a little crude.

It's kinda like the arguments about using AppDirs (or other similar concepts), where everything goes in its' own directory -which necessarily are gonna be non-standard locations -vis-a-vis PATH and LIBRARY_PATHs. Libs should nearly always be put in the standard locations -otherwise every program which is gonna use them must be told where they are. This is fine when you need to have multiple versions, for example, but should be avoided when possible. AppDirs, if placed under /opt are in the correct place, and it is 'legal' to create a link in /opt/bin to point to them -that means you don't need wrappers and PATH-setting for every program contained in an AppDir.

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#180 Post by Q5sys »

amigo wrote:It's kinda like the arguments about using AppDirs (or other similar concepts), where everything goes in its' own directory -which necessarily are gonna be non-standard locations -vis-a-vis PATH and LIBRARY_PATHs. Libs should nearly always be put in the standard locations -otherwise every program which is gonna use them must be told where they are. This is fine when you need to have multiple versions, for example, but should be avoided when possible. AppDirs, if placed under /opt are in the correct place, and it is 'legal' to create a link in /opt/bin to point to them -that means you don't need wrappers and PATH-setting for every program contained in an AppDir.
You didnt chime in on the /opt threada little while ago. Could you chime in there with your insight on that thread too? I'm curious about your thoughts on that issue alone.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#181 Post by sunburnt »

amigo; A possible clarification to your post:
Libs should nearly always be put in the standard locations
I think you mean: " shared libs. ". If the lib. is rarely used, then why share it? Statically compile it and it`s conflicting with other libs. problem is solved.

I looked at ROXapps and didn`t see that they were any better than an SFS. I don`t know the structure of a AppDir, just the app.`s files all in a dir. I guess.
Obviously having all of an apps. files isolated has advantages. But shared libs. and a good O.S. API go a long way toward the reusable software model.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#182 Post by amigo »

Q5sys, "curious about your thoughts" -Boy is something going wrong here... I thought I had commented on the /opt thread. I'll maybe waste some time there a little later...

sunburnt, of course I meant shared libs. The static/shared question has several aspects that should be taken inti account. Usually, when a library is very small, devs who want to use it in their application simply add it to their sources and include it statically, unless it is already a commonly-used and commonly-available shared library.

The problem comes with trying to judge how 'rarely' the libs might be used. Sometimes it is hard to tell, but sometimes it is obvious. Linking statically usually takes extra work, so I don't practice it much -I have so much nicer things to be doing with my time :D If binary compatibility with other OS versions is the prime issue, then, of course, linking statically can be a great help. And, if you like compressing your binaries, then linking statically can help you achieve smaller bins. That sounds backwards, but the thing is that you can't really compress *shared* libs, while executable binaries you can. So, often a statically-linked binary can be compressed to a smaller size than having a dynamically-linked binary of a smaller size and then compressed, plus the size of the un-compressable shared library.

I think that usually trying to achieve really tiny binaries is a waste of time and ruins the general outcome when practised often. You either use limited versions of the program, hack out portions of a working program and remove things that you don't use -thinking that nobody else will want them either -often a wrong assumption. Then, you compress everything as much as possible -even when it is tiny to start with. Running compressed binaries is actually wasteful because it requires extra time to uncompress them and extra RAM in which to do that.

About AppDirs. The concept of software 'bundles' is old. Lots of early UNIX software was meant to be used from within a single directory -usually right fom inside the sources. Any shared data or config files would all be located in the same dir. As used by ROX, AppDirs become special by being executable directories -just click on them and they run. At the same time, you can cd into the dir and './AppRun' and get the same thing. Or, you can link the AppRun anywhere you like. All of this fits exactly with what /opt was designed for.

Interestingly, SFS's as used by Puppy and others, share a couple of aspects with AppDirs. They both contain everything under one dir. Even if you union the squashfs, the original mount of it is under one dir. And, they are both extensions of the idea of 'uninstalled' software which can be added or deleted dynamically.

Most of the 'original' AppDirs actually contain their uncompiled sources. 'Installing' them simply means un-packing the archive -basically anywhere you like or have rights to write and execute. Then, clicking on the AppDir compiles the sources and runs the program. Succeeding runs of course do not re-compile the app. Pretty slick actually, but it can get messy just like with uninstalled sfs files -or any other kind of FS-image. There is a project which lets you use a type of AppDir where the AppRun is actually a binary instead of a script and the other AppDir contents are an *.iso image. Running the app mounts the image. The same thing could be done with sfs's or any other FS image -ext2, iso or whatever.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#183 Post by sunburnt »

I think a ROXapp is also essentially what you`re describing. It has a script and a Squash file inside, I didn`t see it as an improvement over an SFS.

I hadn`t considered source code in a AppDir, for small apps. maybe okay, but big ones would be cumbersome and slow to run, but more portable...

Yes, a good statistical analysis of lib. common usage would be difficult to compile, but well worth the effort in organizing Linux package management.

I agree... Trying to compress anything to the Nth is pointless most of the time. It costs too much and storage space just isn`t that critical anymore.

I didn`t know that static compiling libs. was such a pain, maybe it could be improved somewhat? The real pain seems to be just getting the right lib.

gcmartin

#184 Post by gcmartin »

Here, here!
amigo wrote: ... I think that usually trying to achieve really tiny binaries is a waste of time and ruins the ...
Well spoken. Hope many are clear on what you accurately share.

Hope this helps

disciple
Posts: 6984
Joined: Sun 21 May 2006, 01:46
Location: Auckland, New Zealand

#185 Post by disciple »

Ooh - I'm surprised the static lib / tiny binary enthusiasts haven't turned up yet after that post ;)

On Roxapps and squashfiles: if you just chuck an AppRun (and a .DirIcon or whatever else you want) in the top level of a squashfile it would contain an AppDir. Just mount the squashfile and there you have it. If I wanted to use a bunch of non-unioned squashfiles, I think that's the way I'd go.
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER

Post Reply