Add support for other compressions than gzip for dotPets

[Karl Godt]

This is study to support other compressions for .pet .

The current compression used by the petget install utility is gzip/gz .

There had been discussion on the forum to add other/newer compressions .

*

This is a first study of the /usr/local/petget/petget script for supporting these :

Code: Select all

# ADDS/CHANGES by Karl Godt :
# 1.) usage message, called with $1 "-h" using bash shell
#		buittin getopts
# 2.) Support for .pet with other compressions as gzip :
#		bzip2,lzop,lzma,xz ;needs altered pet2tgz/tgz2pet,dir2pet
# 3.) though added in later Puppies, have manually adjusted support for .rpm
# 4.) Added support for checking pets in /tmp with extracting them there .
#		Code should be nearly the same as in installpkg.sh
#		On this installation still a TODO

Code: Select all

	$0 [-h] FILE_NAME_TO_INSTALL

	Puppy Linux .pet installer script .

	Needs at least the parameter FILE_NAME_TO_INSTALL.ext

	Currently supported file formats/.ext :

	original : .pet
	new      : .b2pet,.lopet,.lapet,.xzpet
	other    : .tgz,.tar.gz,.deb,.rpm

	[unsupported : .pup]

	Depends on :
	pet2tgz, installpkg.sh, tar, gzip, dpkg-pkg, rpm
	[,bzip2,lzop,lzma,xz]

[Karl Godt]

Today the whole bunch of files ,

working until now :
pet2tgz and similar functions .

Mime handling is handled as a work around by a /root/my-roxapps/mime.sh .

[Karl Godt]

first alpha snapshot based on Puppy-430 : likely not bugfree

was fixing several of mine and
one misleading error message in
downloadpkgs.sh
where
ONEFILE was "pet_packages-4/dotpuphandler-0.0.4-2.pet"
changed to
BN_ONEFILE="${ONEFILE##*/}"
if [ -f /root/"$BN_ONEFILE" ];then

also added a function file with three functions from /sbin/pup_event_frontend_d
if /tmp/pup_event_sizefreem file does not exist ie the daemon was stopped by eventmanager .

[Karl Godt]

Today found that technosaurus already was working on such case :

Petget enhanced and many version updates

from year 2009

[amigo]

Really, the pet package format is so bad that any effort to extend it is only gonna make it worse. The one thing this distro has always needed is a decent, workable package format -then the tools to use and create those packages reliably, repeatably and flexibly.

[01micko]

Really, the pet package format is so bad that any effort to extend it is only gonna make it worse. The one thing this distro has always needed is a decent, workable package format -then the tools to use and create those packages reliably, repeatably and flexibly.

Ok, but now it's in the hands of the community the ethos of 'open source' can be applied liberally

[sc0ttman]

Really, the pet package format is so bad that any effort to extend it is only gonna make it worse. The one thing this distro has always needed is a decent, workable package format -then the tools to use and create those packages reliably, repeatably and flexibly.

Ok, but now it's in the hands of the community the ethos of 'open source' can be applied liberally

Is this is a good time to request that we include dependency, AND compile time (config) options in the pet specs for all pkgs? Then we'll know exactly how a pkg was compiled, making it much easier to re-compile an update, or change some options etc....

[01micko]

Really, the pet package format is so bad that any effort to extend it is only gonna make it worse. The one thing this distro has always needed is a decent, workable package format -then the tools to use and create those packages reliably, repeatably and flexibly.

Ok, but now it's in the hands of the community the ethos of 'open source' can be applied liberally

Is this is a good time to request that we include dependency, AND compile time (config) options in the pet specs for all pkgs? Then we'll know exactly how a pkg was compiled, making it much easier to re-compile an update, or change some options etc....

Join GitHub (you have of course already), create a fork, work it in, test, present a pull request..

Optionally, VVVVVVVVVVVVV .. there's a mailing list...

[mikeb]

Could then join the RPM club for the most confusing plethora of formats for software packaging

seriously does this distro need yet more confusion added to save a few kilobytes? Chasing goalposts does not promote real development.

mike

[01micko]

Could then join the RPM club for the most confusing plethora of formats for software packaging

How does a simple gzip (and now xz) archived file with an md5 appended compare with a cpio archive with a bunch of metadata stuffed in?

seriously does this distro need yet more confusion added to save a few kilobytes?

Where is the confusion? It's only a few lines of code.. only confused ones will be the ones that can't help themselves with that clicky thing..

Chasing goalposts does not promote real development.

mike

No?
.

[mikeb]

You are suggesting adding a compression format... which effectively adds a configuration to to its structure. Ie it may be one of two compression formats. rpm handling can involve one of several approaches mainly due to the use of different compression formats. Just don't want to see pets wander down a similar road of confusion...it may not be wonderful but at least now pets are tried and tested with one familiar system.

sfs could provide one simple package format...use as is or extract for conventional install.... now that's simple and simple is good. Ah hang on its format changes all the time or is that compression or does it matter...as far as anyone is concerned they have to be handled differently and need this weeks fun bunny script/kernel.

Working with certain consistencies saves time having to work around changes. Every time python/gtk/kernel devs change a function, /sys layout and so on, someone has to spend time amending code and scripts.

Well at least you read my posts which is always appreciated,

mike

[01micko]

Nobody likes change Mike

Thing is sfs, went to xz a few years back.. users adapt. They have to. Microsoft forces them to, at their expense.

xz compressed pets is not new, it's just not mainstream, yet.

You must maintain a few scripts ? I do, and every time upstream changes something I have to adapt, kind of like evolution, which can be good or bad, depends on your angle.

[mikeb]

Just not keen on lifespans of a few months ... get a bit tedious.
Tell Bill I still run windows 2000 on a daily basis and have a working install of NT4.

I like change for the better... I don't go around battering spinning jennys and I hated DOS though development can mean simplifying too .. a healthy refinement. Don't make me quote the laff-splash saga

Well hope it goes well and its amazing how passionate we become over these overgrown calculators.

mike

[amigo]

Yeah, nothing like having a moving-target vis-a-vis compression. squashfs is not really a suitable format -it is pottentially subject to changes in both the compression format and the squashfs itself.
Instead of thinking we'll extract the sfs when needed, it would make more sense to 'mount' the normal tarred/compressed archives. Sure, use xz for the compression. It is stable and of course tar is also.

It can be debated about whether to use cpio or tar -no others are featureful enough. But you can still debate whether to use bsdtar/cpio or GNU versions. But the decision can only be made using specific criteria. If you ever wondered why slackware uses a very old version of tar specifically to create and install packages, it is because of 3 certain behaviors which were changed in later versions of tar.

There is much to be said for using someone elses' package format -it is not easy to come up with anything better. But, if you have special software-deployment needs, then creating a format may be the best thing. I'm not suggesting that already-created packages from some other distro should be used! I mean using the rpm, deb or 'arch' package formats -and their package creation tools and methods.

The chief objection to the rpm format is that you really have to use rpm to build them -and that means only being able to create packages with a well-written spec file. The rpm library contains more than 250 macros for carrying out package building and installation/deinstallation steps. It's a pretty steep learning curve.

debian builds are not much better, as there are about 50 macros and at least a couple of extra files have to be supplied -sometimes 15-20 extra files. debian package systems depend lots on perl.

So, why are packages even necessary? So that installations/fixes/changes/upgrades/rollbacks can be made available and carried out in a completely modular fashion. If something is implemented or fixed at the package level, then that unit can be applied to already-installed systems or incorporated into future appliances. And, changes shoudl not be done manually to packages -you have to get the recipe for the package working so that the package can be rebuilt without any intervention, from start to finish by recipe. It simply is impossible to manage hundreds/thousands of packages any other way. And creating the packages yourself is the only way to get control of the process, so software can be configured, compiled and packaged for the needs of the distro. The further your needs are from those of any/all other distros, the more important this is. Instead of posting packages on some forum, post recipes instead.

[Karl Godt]

The .pet format is a little bit crazy because it is a compressed .tar archive - compression by default ever and only had been libz compression - and the crazyness is to add a md5sum to the file.

To show .pet with a specific icon needs adjustments to a few files in /usr/share/mime directory,
that likely would vanish if run some update-mimedatabase program , that ignores or does not know the .pet format .
I remember , that Iwas stunned by my first experiments with the mime database because of fixing .pet mime type .
Therefore I somehow can uderstand the term "horrible" . :LOL:

But changes need time, likely as much time as it was developed and lasting stable.

I am mostly suspicious on radical changes, so I prefer step by step adjustments .

And of course everyone is welcome to post his own petget files in his own thread like amigo did with his src2pkg .

[Ibidem]

Debian format is actually pretty basic:
an ar archive containing two tarballs (control.tar.* & data.tar.*) and "debian-binary", which is a version stamp.
In control.tar.gz is a description (./control) and md5sums for the files installed (./md5sums).

So one could more or less duplicate it by installing to $INSTDIR then going

Code: Select all

cd $INSTDIR; find * -type f -exec md5sum '{}' \; | tac > ../md5sums
tar -c * |gzip -9c > ../data.tar.gz
cd ..
#write a file called "control" containing the description
tar -c ./control ./md5sums | gzip -9c >control.tar.gz
echo "2.0" > debian-binary
ar rc new.deb debian-binary control.tar.gz data.tar.gz

rpm is an underdocumented header prepended to a compressed cpio archive. To create it, you can use rpm, rpm5, or xrpm

Now, is *.pet/pxt/... multiple formats?
Not really. Tar autodetects compression type. So you don't need _any_ new code, apart from mime types.

PS: there is at least one other archiver that could work for packaging: pax.
There's also the SVRx package tools (heirloom packagetools being the relevant port), but _so far_ no one has used those.

[SFR]

Just for the record: there's also pfs format (discovered by me yesterday, anyone knew about this one?), used in PuppyRus.
http://wiki.puppyrus.org/puppyrus/pr218/pfs (google translate helps )
ftp://mirror.yandex.ru/puppyrus/puppyru ... 3.4-pr.pfs
Might be worth to investigate...

Greetings!

[mikeb]

Debian format is actually pretty basic:
an ar archive containing two tarballs (control.tar.* & data.tar.*) and "debian-binary", which is a version stamp.

Perhaps there is an element of re-inventing the wheel here.

Is there a reason why puppy packages cannot be packaged in deb format.?.... they are used all the time with ubuntu based puppies and 95% of my software and libraries comes from deb land. Also that version stamp doo dah might be used in a way to help avoid such as forum packages ending up being installed to incompatible puppy versions. It might be an established format but effectively be a move forward for puppy. Also, as mentioned, higher compression could be used transparently as an option.

mike

[Ibidem]

Debian format is actually pretty basic:
an ar archive containing two tarballs (control.tar.* & data.tar.*) and "debian-binary", which is a version stamp.

Perhaps there is an element of re-inventing the wheel here.

Is there a reason why puppy packages cannot be packaged in deb format.?.... they are used all the time with ubuntu based puppies and 95% of my software and libraries comes from deb land. Also that version stamp doo dah might be used in a way to help avoid such as forum packages ending up being installed to incompatible puppy versions. It might be an established format but effectively be a move forward for puppy. Also, as mentioned, higher compression could be used transparently as an option.

mike

The version stamp is a package format version, not a system version.

By the way, new compression systems are not always completely transparent.
The archiver autodetects compression based on filename extension or on the file magic.
If you use the .pet format, with a constant extension, yes it can be completely transparent...
If you use deb, you will be using (for example) data.tar.xz/control.tar.xz instead of data.tar.gz/control.tar.gz.
And the change in the name of the files has the potential to require a few lines of code.

[mikeb]

Ah ok ...thanks for the clarification

mike