The State of Package Management

What features/apps/bugfixes needed in a future Puppy

Should Puppy's package format be changed?

Yes, without backwards compatibility.
11
28%
Yes, with backwards compatibility.
10
26%
No, but the PET format should be standardized/stricter.
8
21%
No, the PET format works fine.
10
26%
 
Total votes: 39

Message
Author
noryb009
Posts: 634
Joined: Sat 20 Mar 2010, 22:28

#31 Post by noryb009 »

noryb009; The Save is only needed if the app. has config. files to be written to.
But this can easily be worked around. And there`s no reason a CD shouldn`t work.
I know the save file isn't needed, but Puppy won't currently let you use SFS files while you don't have a save file. As you say, this can be implemented, though.
jemimah; How about a GUI for .deb to SFS, it`d kinda solve all of this.
...
Well, I dunno, but slackware has been good so far
.....and Arch seems to have the best package management/update process I've come across....
...
Has Fedora/Yum been tried, or gentoo/portage?
This is leading back to what we have now: a fragmented package management system taken from different distributions. We have tried it with ubuntu, debian, slackware, etc. With every new version, an entire old system is thrown away.
Puppy needs to stop using other package management systems, and start using it's own, compiled specially for puppy.

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#32 Post by Aitch »

Has Fedora/Yum been tried, or gentoo/portage?
noryb009, that WAS tongue in cheek.... :wink:

English humour, sorry.... :lol:

I'm with Jemimah
The problem isn't the ppm, it's the repos.....The only way any of this works at all is if puppy developers compile and test the packages, and specify the dependencies correctly, and if the users stick to installing stuff from the tested repositories only.
That seems simple enough...? - though I don't compile/develop.....

thanks devs! :D

Aitch :)

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#33 Post by scsijon »

noryb009 wrote:This is leading back to what we have now: a fragmented package management system taken from different distributions. We have tried it with ubuntu, debian, slackware, etc. With every new version, an entire old system is thrown away.
Puppy needs to stop using other package management systems, and start using it's own, compiled specially for puppy.
No, what we need to do is either, do away with the whole built distribution idea and work directly from source as Iguleder is working on, then we would not have the problem; or create a multi-source distribution systems re-packager (and deal with that whole problems that would bring).

and
I don't have any problems with new2dir for building new pets, simple to use, try it with a build.

User avatar
harii4
Posts: 448
Joined: Fri 30 Jan 2009, 04:08
Location: La Porte City, IA , U.S.A.
Contact:

#34 Post by harii4 »

big_bass always said - use standard linux tools, and had great success with txz_pup
I never had a dependency/install/uninstall problem ......
TXZ_pup only used slackware's pkgtools - so simple yet so powerful. :)
Big_Bass wanted to follow linux standards that way - he kept the puppy ppm for us noobs :)
Looking forward to a src2pkg approach that perhaps you and Amigo can suggest.
+1 for that 8)
3.01 Fat Free / Fire Hydrant featherweight/ TXZ_pup / 431JP2012
----------------------------------------------------------------------------------------
Peace and Justice are two sides of the same coin.

noryb009
Posts: 634
Joined: Sat 20 Mar 2010, 22:28

#35 Post by noryb009 »

No, what we need to do is either, do away with the whole built distribution idea and work directly from source as Iguleder is working on, then we would not have the problem; or create a multi-source distribution systems re-packager (and deal with that whole problems that would bring).
The entire point of this thread is to look at the possibility of the first part. We are currently using the second part through woof and the PPM, but it isn't working. As jemimah said:
jemimah wrote:If J. Random User picks some random package from the Ubuntu repo, downloads 300MB worth of deps overwriting system files in the process, what do you think is going to happen?

I don't have any problems with new2dir for building new pets, simple to use, try it with a build.
new2dir and dir2pet are good for quickly creating PET files, but they aren't good for recompiling a PET, as it doesn't save any settings, patches, etc. A script based packager would solve this.

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#36 Post by Aitch »

A script based packager would solve this.
If you build it, we'd no doubt use it...

Even if you made a start, it could be developed...whereas now, we just have real estate on a forum

Aitch :)

stu90

#37 Post by stu90 »

noryb009 wrote: This is leading back to what we have now: a fragmented package management system taken from different distributions. We have tried it with ubuntu, debian, slackware, etc. With every new version, an entire old system is thrown away.
Puppy needs to stop using other package management systems, and start using it's own, compiled specially for puppy.
As it stands now package selection for puppy is sparse at best and that is with source compiled, package conversion and dir2pet.
With the lack on man power and knowledge to do it (i include myself in that category) i would hazard a guess compiling everything specifically for puppy would cut down an already sparse package selection quite drastically.

Really though the state of package management is part of a wider question regarding the state or puppy as a whole - what is puppy, where is it going and along what path.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#38 Post by jemimah »

My position is, if we do things right, and don't try to take shortcuts that decrease quality, pay more attention to look and feel, and have a coherent community structure, we're more likely to attract and keep developers.

We do have one problem that makes it difficult to have a community repo. How do we handle concurrent editing of the Packages file? The only way I can think of is to have one person manage the file and all the other maintainers need to talk to him/her to get packages added. It's not exactly ideal.

User avatar
sickgut
Posts: 1156
Joined: Tue 23 Mar 2010, 19:11
Location: Tasmania, Australia in the mountains.
Contact:

#39 Post by sickgut »

The only reason Puppy cant run other distros packages well is because of the extremely stripped down kernel. As soon as you try to install a package that relies on stuff that should really be in a linux kernel that puppy doesnt have then it wont work.

And the reason there is this sloppy "hey we can run ubuntu packages too!.."
functionality in the PPM that lets you try to install from Ubuntu or other repos (even if they dont actually work) , is because its an attempt to ease the pressure on devs who are overworked trying to compile and maintain a repo of current applications that work with the newest puppy. There is obviously a problem here, as its a huge workload to compile and keep current a puppy repo that is comprehensive enough that newbies dont have to move to other OSes to run their specific programs. The time spent on making packages for Puppy is wasted time as that time would be better spent on the OS itself.

The only clear answer as i have discovered is to start with a more compatible base. There is no point in trying to retro fit another OSes packages on top of Puppy.

it doesnt matter if you have fitted apt-get or synaptic or whatever to puppy, the package manager isnt the issue the issue is that the puppy base itself isnt compatible with the packages you will wish to install.

you can have the same speed/ size and other characteristics of puppy and the compatibility with other major distros repositories by starting with the smallest possible debian base (or ubuntu or slackware or whatever, i chose debian) then proceed to add everything you need, the same rox filer, jwm, xorg desktop and most the same applications as puppy and after manually slimming down you will have something the same size and with similar performance but also the ability to simply apt-get install <packagename> or use synaptic or whatever is the prefered method of the base you chose to install stuff.

If you want to supply slimmed packages? dont. Supply instead a script that apt-get installs what program you need, then removes all the unwanted junk files. Release a script instead of the package.

it is possible to make an OS in this way and also make it look and work in the same way (from the users point of view) as puppy. I have created pussy linux:

http://www.thepussycatforest.info

http://www.murga-linux.com/puppy/viewtopic.php?t=69475

Something like this can be made to seem to be more puppy like if there is a small group of puppy devs interested in doing so. This will solve the package management problems.

noryb009
Posts: 634
Joined: Sat 20 Mar 2010, 22:28

#40 Post by noryb009 »

If you build it, we'd no doubt use it...

Even if you made a start, it could be developed...whereas now, we just have real estate on a forum
Linky. It's a small script based on arch's makepkg. It makes PET packages right now, but it's designed to make another package format easily.
With the lack on man power and knowledge to do it (i include myself in that category) i would hazard a guess compiling everything specifically for puppy would cut down an already sparse package selection quite drastically.
It's not lack of man power and knowledge. We have many puppy devs, but they are all reinventing the wheel. It seems like everyone is working on a different puplet, not working together.
We do have one problem that makes it difficult to have a community repo. How do we handle concurrent editing of the Packages file? The only way I can think of is to have one person manage the file and all the other maintainers need to talk to him/her to get packages added. It's not exactly ideal.
A version control system like GIT would work great for a repository of scripts. It would allow there to be one copy of each script, and people can commit to that. There could be multiple maintainers, and it would record changes to scripts.

sickgut: Using another distribution as a base is one way to solve this problem, but this thread is about the other solution: using our own packages. Your project looks interesting, though. I'll have to try it out some time.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#41 Post by amigo »

"The only reason Puppy cant run other distros packages well is because of the extremely stripped down kernel" Not at all true. the kernl has nothing to do with software compatibility. Software-compatibility is all down to the libs, mate. Things don't always run right because there is library version-mismatch or differing configuration options at compile-time.
As you said, "There is no point in trying to retro fit another OSes packages on top of Puppy."

2byte
Posts: 353
Joined: Mon 09 Oct 2006, 18:10

Regarding package management

#42 Post by 2byte »

Regarding package management

Why oh why hasn’t anyone considered using a real database to oversee the package installation and removal? Puppy comes with one already, SQLite, a self-contained, serverless, zero-configuration, transactional SQL database engine. It can be used from bash, C or almost any other programming language.

Consider this scenario.
Create a SQLite system database with two tables, named Packages and PackageFiles.

1. When the base puppy is created add a ‘system’ name to the Packages table with an ID of 1.
Record the path/file name of all libraries, links, and whatever other files the system is comprised of and put a record of each file in PackageFiles table of the database. Add a value of 1 to the count field in the table. All of these files will belong to the ‘system’ package. This database becomes a permanent part of the distro.

2. Installing a package. (not part of a system upgrade)
When a package is to be installed store the package name in the Packages table’. It will have a unique ID number greater than 1.
Parse each path/file name in the package and run a SELECT query for the exact path/file name on the PackageFiles table. If no record is returned install that file, add the file name to the PackageFiles table and add 1 to the count field of that record. Give this record the same ID as the package name. If the file already exists then increment the count field of the existing record but do not install the file.
Repeat until all new files are copied.

3. Uninstalling a package
Query the database for all files that belong to that package. For each file returned examine the count field. If the count is more than 1, decrement the count and leave the file installed. If the count is 1 and it is not a system file (ID=1), delete the file and delete the file record from the PackageFiles table . If the count is 1 and it is a system file then it is part of the distro and is to be left alone.
Repeat until all valid files are removed.
Delete the package’s record from the Packages table.

There it is in a nut shell. No system files ever get deleted or over written by a software package, plus safe install and removal of software using only existing tools in Puppy. All that’s missing is integration with a PPM.

Additional benefits:
A record of all packages can be retrieved at any time.
A record of all files actually installed for any software package can be retrieved at any time.
A record of the original system files can be retrieved at any time.
The number of applications using a particular library can be retrieved.
Foreign packages could be installed without fear.
The database will be small in KB.
The package format could be almost the same as a pet. A compressed base directory containing all of the package files, links and dirs. A pkg.list file could be required to speed up installation. Do away with pinstall scripts too.
Attachments
pkg-tables.jpg
(14.46 KiB) Downloaded 608 times


User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#43 Post by sunburnt »

amigo; I assume that you mean other OS`s compiled packages.?
So making SFS files from Slackware or other source packages is best.?
The trick then becomes satisfying all the dependencies if not provided.

I`ve always had good luck making Debian binary packages work in Puppy.
But recently that doesn`t seem to be the case, don`t know what changed.
Mainly multimedia apps. don`t work ( lots of libraries ), so maybe the app.?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#44 Post by amigo »

Any sort of software installation -whether 'really installed' or dynamically loaded (sfs tcz, etc) can only be sanely managed by respecting the 'unit'. That means one_source=one_package (except for split sub-packages NLS, DOC, etc). Anything which violates this principle is gonna be a mess -guaranteed.
If you want a system which always works and brings no surprises, then all binary material should be compiled on, by and for that very system -not taken from *anywhere* else. The only excpetion to that should be for non-open-source stuff where the sources are not available -and maybe, with lots of care to supply *all* the dependencies (from your own stock), things like firefox -where you aren't always allowed to re-compile it and keep the name.

The methods used around here are simply a huge mess -there has never been a single release which was free from the problems caused by mixing/matching bins from all over the place. Even if you decide to minimize this by sticking to only bins from a single distro, as soon as you see that you need different options for one of the standard libs/progs from upstream, then you've already begub to break things. Your own toolchain, your own binaries for everything or you are fighting an up-hill battle.

The trick is to supply your own packages where each source supplies one program/lib or group of programs/libs in one package. External dependenices get the same treatment -logically, orderly *packaged*. Every single file, directory and link should be accounted for by at least one package (yes there are duplications which does not always mean conflicts)

Sometimes it's good to accept a bit of old-fashioned methodology from the old guys who have been doing this for decades. Do you really think your ideas are new and have never been tried and possibly discarded for good reasons?

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

Re: Regarding package management

#45 Post by jemimah »

2byte wrote:Regarding package management

Why oh why hasn’t anyone considered using a real database to oversee the package installation and removal? Puppy comes with one already, SQLite, a self-contained, serverless, zero-configuration, transactional SQL database engine. It can be used from bash, C or almost any other programming language.
Puppy already has a flat file database. Using SQL would only make it less transparent.
2byte wrote: If the file already exists then increment the count field of the existing record but do not install the file.
This would cripple the functionality.
2byte wrote:
Additional benefits:
A record of all packages can be retrieved at any time.
A record of all files actually installed for any software package can be retrieved at any time.
A record of the original system files can be retrieved at any time.
We already have this.
2byte wrote: Foreign packages could be installed without fear.
Not really. As long as you allow binaries or scripts, damage can be done easily.
2byte wrote: Do away with pinstall scripts too.
We need this functionality - not for everything, but when it's needed - nothing else will do.

2byte
Posts: 353
Joined: Mon 09 Oct 2006, 18:10

#46 Post by 2byte »

I knew before posting the idea would be rejected out of hand. Amigo is 100% right, although I don’t understand the point of that snide remark in the last sentence.

Jemimah
It might cripple the functionality of the installed software but it would prevent the base system from being crippled by a poorly packaged pet. I am talking about user installed software, not system upgrades. Those things you say we already have, yep. The idea doesn’t do away with those as you pointed out. As for foreign packages, I was referring to installing without fear of overwriting critical system files. Damage from running foreign software is unpreventable.

As long as the PPM blindly installs everything in a pet and blindly executes any pinstall script and then blindly removes every file that was in the pet people are going to continue to fubar their system.

If the Puppy package management isn’t broken, then don’t fix it.

No hard feelings :)


User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#47 Post by jemimah »

If you want to prevent overwriting system files, by far the easiest and most effective way is to mount the save file below the main sfs - as we discovered in a saluki-008 bug. :lol:

Perhaps it is better to have the user confirm system file overwrite.

It is important to understand that good package management (in the style of other distros) and read-only filesystems can be rather at odds with each other.

If you want apt-style package management, the only option is the one Sickgut concluded with - you must design the entire distro around it. You must discard Puppy entirely and start over. Anything less is doomed to failure.

2byte
Posts: 353
Joined: Mon 09 Oct 2006, 18:10

#48 Post by 2byte »

Well I'll just say this, then butt out. I have been using this sort of thing as my own package control system since Lucid was first released and it has performed very well.

Best regards to all


User avatar
darkcity
Posts: 2534
Joined: Sun 23 May 2010, 19:16
Location: near here
Contact:

#49 Post by darkcity »

Hi 2byte

It is possible you could upload your system to the forum so other could test it?

:D

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#50 Post by Lobster »

If Sqlite is too complex a flatfile database that creates a text file
is available as a base
http://puppylinux.info/topic/p-data-alp ... e-database
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

Post Reply