packdude

[Iguleder]

Now it's in good shape - I fixed another memory leak and added support for querying the package database. I did extensive testing in the past two weeks and it works very well.

Now, I want to add a package description field (for display in graphical frontends).

puppyluvr - I want the name to be generic and not associated with a single distro. As a community distro, we're very poor and I don't want to deter other distros from using this code, since that's the best way to lower the maintenance burden. When others use your code, they develop the motivation and man power to fix bugs and add more features.

EDIT: I just submitted committed many efficiency improvements. packdude is way faster now

[slavvo67]

Can I suggest implementing this as a pet, first? The shock of replacing PPM, regardless of how bad it is might be a lot to handle.

If possible, it might be better to allow for pet or sfs install and allow users to provide feedback first.

Best,

Slavvo67

[Iguleder]

Backwards compatibility is easy to do. It's not that hard to unpack a PET package using C code. It's just a .tar.gz archive with the MD5 appended.

However, I see no reason to do this, because this makes the package manager design more complicated. Also, our PET repositories are really small - it's much easier to convert them to another format. For all the PETs hanging around in the forums, we could use a temporary, "pirate" PET installation tool which extracts packages to / and doesn't integrate with the new package manager.

I just added XZ support to packdude, so packages are tiny. I also added some efficiency improvements, so package download and installation is insanely fast now.

I decided to freeze the code and keep the current code base "stable". More features means more bugs and memory leaks, while at the moment, packdude already does everything I want it to. I might do a packdude 2.0, if I change my mind.

[mavrothal]

Backwards compatibility is easy to do. It's not that hard to unpack a PET package using C code. It's just a .tar.gz archive with the MD5 appended.

However, I see no reason to do this, because this makes the package manager design more complicated. Also, our PET repositories are really small - it's much easier to convert them to another format. For all the PETs hanging around in the forums, we could use a temporary, "pirate" PET installation tool which extracts packages to / and doesn't integrate with the new package manager.

The problem with a "pirate" tool is that you also need a tool to record them and uninstall them, that looks like a second package manager.
If the "unofficial" pets are allowed, would be better if the package manager is aware of them and handle them properly. Maybe by including the pet conversion script/program that will be used to convert the pet repos, as a first step in the process.

[stemsee]

When can we get it?
ht
Today i built Arch and trusty, but there is something wrong with my kernel (i hope it is only that!) I will rebuild a new kernel and insert it tonight. I would like to include packdude.

[Iguleder]

Everything (including documentation) is in GitHub - see the first post.

You'll have to set up a repository before you can use packdude.

[stemsee]

How would I do it locally? adding a dvd, or directory such as /var/apt/cache or woof-ce local-repositories, for testing. So that if i wanted to use some packages from wheezy in a precise build i could use packdude to manage local repos as there is only one pet repo but several .deb repos. Or not?

[Iguleder]

Nope.

packdude has its own package format and repositories format. You'll have to convert Debian/Ubuntu repositories to packdude's format first.

[technosaurus]

The new pkg mgr is cool. I love that its expandable.

I just wanted to note that ppm could be 20x faster if the scripts used:
IFS="|"
while read field1 field2 ... fieldN: do

instead of the hacky hodge podge combo of read awk, grep, sed cut
... and others

shell does fine at text processing if its done right, unfortunately most scripters don't have a good understanding of the powerful built-ins.

to do the same in C, strsep or strtok (both are hellacious) are often used or fscanf with a special string that must account for versions with different number of fields. While it is still faster, the code can grow pretty complex ... and complexity has a cost, which is a perfectly good reason to use an actual database like sqlite or lmdb.

[technosaurus]

In addition if you host binary packages, you should also host the sources (according to GPL)

No, it is perfectly legit to make the source available via snail mail on 8 inch floppies at the expense of any requester who cannot trust that they are the same as upstream. There was a whole debate on this with damn small linux ~10 years or so ago.

[mavrothal]

I just wanted to note that ppm could be 20x faster if the scripts used:
IFS="|"
while read field1 field2 ... fieldN: do

instead of the hacky hodge podge combo of read awk, grep, sed cut
... and others

20x? wow!
PPM has about 700 of those. About 50 in tandem!
However, a lot of these are for writing in tmp for latter use...
Maybe you could provide a working example for one/any of the PPM scripts
(if you do, maybe in another thread - not to highjack this one)

[Iguleder]

It's fine - go ahead and hijack this thread. It has only one purpose - improve package management in Puppy.

[technosaurus]

I could probably help put together a script to convert pet .packages databases or pet.specs files directly into PDDB (pack dude database) with ash and sqlite3. Its just a matter of translating the columns into their respective database fields and writing an "INSERT into ...". The biggest thing that would slow that conversion process down would be trying to do each record one at a time. Sqlite can be pretty quick, but not if you open a database write 1 record and close it 1000 times ... better to open it write 1000 records and close it (this is why fastcgi is so much faster than cgi too)

[Iguleder]

Sounds like a good idea, I'll give it a go.

It's even easier - packdude's repository creation tool (repodude) accepts a CSV file very similar to Puppy's package lists. Conversion should be extremely easy.

EDIT: I have a working prototype here it's Puppy's "common" repository, converted to packdude's format. Build the latest packdude and use -u to force the use of this repository.

EDIT 2: it works! I was able to install a52dec. packdude's repositories are limited to one version of each package, so I had to filter only the most recent version of each package.

EDIT 3: Puppy's dependencies are a mess. PPM doesn't really handle dependencies, so some package specify non-existing ones, although they work just fine. For example - "urxvt" as a dependency of a gtkdialog-based package, while the right package name is "rxvt-unicode".

EDIT 4: I'm experimenting with taking things two steps forward. I'm trying to generate a self-hosting toolchain using the last GPLv2 versions of GCC and Binutils, plus the latest musl. I think it would be extremely cool to be able to install a very thin, Aboriginal-style distro to a directory using packdude, then build the entire distro (e.g all packages) inside a chroot environment.

[technosaurus]

're: self hosting

I have been wanting to do the same with llvm, but I forked binutils, gcc, mupdf gpl2 versions @github.com/technosaurus just in case.

[stemsee]

Have you seen this. Might be some useful scripts in there.

http://4mlinux.blogspot.co.uk/2013/12/a ... m-txz.html

cheers

[Iguleder]

I'm aware of alien and I think it's nice, but I want something different - a repository converter. I want to convert a repository myself, so they user doesn't have to go through the slow conversion process locally.

Regarding compiler stuff, my GCC cross-compiler fails to build a native self. I'm trying to cross-compile a static GCC using Aboriginal's toolchain.

[technosaurus]

Regarding compiler stuff, my GCC cross-compiler fails to build a native self. I'm trying to cross-compile a static GCC using Aboriginal's toolchain.

linuxfromscratch had a couple of patches for this problem... the autotools garbage messes up the 2nd pass iirc.

[gcmartin]

Sqlite should probably have been a standard feature in PUPs since smartPhones arrived years ago. 2 of the members here are WOOF-CE contributors. Please add for future PUPs as this PPM utility moves forward into mainstream.

[8-bit]

One thing I never did like about PPM was the update process.
When run, it would attempt to update all repositories.
I would have preferred to be able to choose the ones that got updated.
And if one tries to interrupt the update process, the conversion of the file lists never gets completed and I have crashed PPM in the process.

WIll packdude have a graphical user display that shows available files in a repository?
Or is SQlite responsible for that?