[RELEASE] src2pkg-3.0 - create packages from source

Miscellaneous tools
Message
Author
scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#31 Post by scsijon »

Yes, I agree, I've been having a play and also have found the src2pkg website, which is extremely helpfull.

I think my first problem will be an 'order-of-build' problem as I will need to have dependancies built and installed first before the packages needing them.

So I somehow need to list them, build them and install them before building the parent packages.

The -T I think will tell me if it needs a dependancy, thereby if so put it into build-later-list2, if not into build-first-list1.

I wonder what format it comes back in though, just the packagename or a lot of unwanted info? I think if it's useable like that, that is all I would need added if it's not there already in the switches, to complete an autobuild with dependancies.

This would autoloop with build-first-list1 being installed and build-later-list2 becoming the new source-list fed into the -T until there is nothing in either build* lists.

The -I will of course install the dependancy once built.

I shall play further, but I wonder if in the future it would be possible to build linux totally from source without the need to have interims such as T2, etc.

Seems like i'm back to scripting again, oh well :roll:

I think for now, i'll just build with src2pkg the minimum I need for T290 and work it from there. ie do the other packages from within a T290 base, not give up.

This leaves just which Kernel and it's settings left to choose!

Also, as a fyi, I just found that src2pkg --more-help, listed in the help, just errors out while src2pkg -hh is ok.

regards
scsijon

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#32 Post by scsijon »

amigo, I'm having fun playing with this and do think it will be of much use to me, especially when I start to create all the extra T2 packages.

However, for now, I can't seem to find what output switch sets the final package as a $package.tar.bz2, can you help me please. It's needed for the way BarryK has setup the new T2 packaging system and i'd like to keep that part of it as the overall build seems to have become 'somewhat quicker'. It looks like it requires less post-build scripting work to happen than a pure .pet package does.

thanks
scsijon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#33 Post by amigo »

You mean you want the final packages to have '.tar.bz2' suffix? And, that they use only a name without any ARCH or VERSION info in the name?
src2pkg does offer bzipped packages as an option, using '-TBZ'. However, you may find it quirky since nobody ses such packages. It was intended for use with vector linux IIRC. But, when you ask for a *.pet it will automatically set and use gzip compression with .pet suffix.

The best way to solve the above (either question), would be to write a small extension to src2pkg which would re-format and re-name the end result. You can create extension which are run at any and all steps of the build -either before or after the regular code of the step is run. You can see an explanation of the mechanism starting at line 2492 of the ChangeLog file in the docs and the feature must be enabled in the conf file -see notes at line 289. The extensions feature allows you to radically alter what src2pkg does, so it provides fertile ground for developing new features. Let me know if you still don't understand how to create and use them and I'll help you.

I like the T2 build system -but I don't like their package format as it is way too basic. I have developed a more usable package standard (actually two of them) which can be created by src2pkg -maybe you'd like to use it to build a base on?? The *.tpkg and *.tpm packages provide very rich package information -including package dependency info. The tpkg and tpm package management tools do not (yet) actually resolve depends for you (I am an old slacker, you know), however, the mechansim for doing so is quite simple *once the info is generated or specified and included in the package*. The nice thing about T2 is that it generates even BuildRequires info for you. I've done some work towards integrating this into src2pkg (for *.tpm packages). The run-time depends-generation already works great.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#34 Post by scsijon »

amigo wrote:The best way to solve the above (either question), would be to write a small extension to src2pkg which would re-format and re-name the end result. You can create extension which are run at any and all steps of the build -either before or after the regular code of the step is run. You can see an explanation of the mechanism starting at line 2492 of the ChangeLog file in the docs and the feature must be enabled in the conf file -see notes at line 289. The extensions feature allows you to radically alter what src2pkg does, so it provides fertile ground for developing new features. Let me know if you still don't understand how to create and use them and I'll help you..
ok, i'll have a look and a go before asking for help
amigo wrote:I like the T2 build system -but I don't like their package format as it is way too basic.
/cut/
The nice thing about T2 is that it generates even BuildRequires info for you. I've done some work towards integrating this into src2pkg (for *.tpm packages). The run-time depends-generation already works great.
Mageia does this also with their system, although sometimes there is too much info. given.

I have an idea this is what barryk was working towards using in his version of woof (not woof-ce) and his racy, which is T2. But when he went back to quirky (his favorite plaything) it was overtaken with another idea. He had a few notes about generation and integration some time ago on his blog, but they were only 'roughs'.

I'd really like to one day, just tell a working system that I want to add a new package by name, have it d/l the source, find dependancies from inside the package, check what dependancies I have already and if versions are sufficent, give me an option to go ahead or not, if it's a yes d/l missing or dependancies that need to be updated, build and install dependancies, build and install the new package and let me test it and either accept it or reject it and if rejected remove the package and all the new dependancies or step those that were already in back to the earlier versions. Pie in the sky at present, but it should happen one day.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#35 Post by amigo »

Your last paragraph describes exactly what nearly everyone wants -and what major distros implement to a great degree. Mageia is an rpm distro as is fedora and suse. debian/ubuntu & Co. use dpkg with *.deb packages. Both systems require a recipe for every single package which includes everything from the source url to the last script actions used to install the package. woof is completely different as it just uses pre-built packages.

There is no magic way to resolve dependencies properly. It requires the generation of a lot of accurate information which must be collected and collated into a useful form *at the time the package is created*, and then passed on *within the package* to package *management* tools.

generate, collect, collate/format -that's the hard part. This has to be done by the package *creation* software at package build-time.

The package *format* is the central concept of any package system. It must be capable of containing and transmitting all the info needed to carry out proper installation.

Dependency-resolution and proper up/down-grading, done of course by the package installation/mgmt tools, also depend on the use of known, sane repositories of packages. The repo-side of the equation also involves creating and making available information about the packages. repo creation and management is done by still another class of software.

There is one example extension (for vector linux in fact), here:
http://distro.ibiblio.org/amigolinux/do ... xtensions/

Try just creating a harmless extension with echo statements, so you can see if it is working and how/when.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

64bit?

#36 Post by scsijon »

I'm going to be greedy for mick and ask when there will likely be a 64bit version for his slacko64?

please, pretty please.

:lol:

scsijon

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

extension?

#37 Post by scsijon »

I was wondering if it as possible to build a package from within a jailed (your term i think) system. Something along the lines of your -JAIL setting for an install, but extended to look for any dependancies there (say /var/tree_root) and not in the normal ('/') tree. You would need to set the tree base somewhere. Then as far as src2pkg was concerned, if the wanted prerequisite package or file is not already there it would fail or a version needs updating past what you had already built and installed in the jailed area.
It would mean that you would need a basic filesystem structure (/bin, /lib, /etc, ...) of course. In our case an opened puppy...sfs should suffice, but there is no reason something like aboriginal linux shouldn't be able to provide this structure in it's package.
It would at least then give you the ability to chroot for testing or even (maybe) develop for one system on top of another.

I hope you understand what i mean from this.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#38 Post by amigo »

As for a 64-bit version, I would try installing src2pkg normally and running 'src2pkg --setup', but I'm not sure that will compile correctly on *puppy* 64-but -it does work on Slackware 64-bit. On puppy, it might possibly compile for 64-bit usage but be named as a 32-bit.

As to using jails, that should work if the jail is created and used properly. If you chroot into the 'jailed' directory struvture, then references to the database path would reference that path within the jail. If you need to reference paths outside the 'jail', then you'd need to 'mount --bind ...' the needed items inside the jail.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#39 Post by scsijon »

amigo wrote:As for a 64-bit version, I would try installing src2pkg normally and running 'src2pkg --setup', but I'm not sure that will compile correctly on *puppy* 64-but -it does work on Slackware 64-bit. On puppy, it might possibly compile for 64-bit usage but be named as a 32-bit.

As to using jails, that should work if the jail is created and used properly. If you chroot into the 'jailed' directory struvture, then references to the database path would reference that path within the jail. If you need to reference paths outside the 'jail', then you'd need to 'mount --bind ...' the needed items inside the jail.
I tried with barry's april64 (with devx) and it installed but failed miserably with the --setup so I didn't go further for the moment. Partly due I think to things like april uses a few packages that are later than the current src2pkg so maybe a new version is due.

regards

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#40 Post by amigo »

What was the output from 'src2pkg --setup'?

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#41 Post by scsijon »

amigo wrote:What was the output from 'src2pkg --setup'?
-------------------------------------------------------------------------
[1;36m Notice - [0;39mCreating src2pkg-helpers:
src2pkg uses a shared library and a few programs
when creating packages. For best compatibility,
these binaries will be compiled on your system.
They are then installed in a private directory.
When done, src2pkg is ready for use.

TEMP_DIR=/usr/src/src2pkg/builds/src2pkg-helpers
Starting build in 5 seconds
[1;34mUnpacking sources - [0;39m[1;32mOK[0;39m
[1;34mCreating libsentry - [0;39m[1;32mOK[0;39m
[1;34mCreating tar-1.13 - [0;39m[1;32mOK[0;39m
[1;34mCreating coreutils - [0;39m[1;32mOK[0;39m
[1;34mCreating unionfs-fuse - [0;39m[1;31mFailed! [0;39mError is non-fatal. Continuing...
[1;34mCreating pet package - [0;39m[1;32mDone [0;39m
[1;34mFinished pet package is:[0;39m
/usr/src/src2pkg/src2pkg-helpers/src2pkg-helpers-1.6-i486-1.pet
[1;34mInstalling pet package - [0;39m[1;31mFailed! [0;39m
Failed to install package: src2pkg-helpers-1.6-i486-1.pet
---------------------------------------------------------------------------------
Actually, looking and comparing with my 551, I think it's saying it's 'failing' for the unionfs package, which quirky April
doesn't use, it seems to install all the other files in the right src2pkg places, although the packages coreutils5.21b (8.22)
and tar1.13 (1.28 ) are older than April uses so don't know if that affects it also.

I shall have a go at a build and install with something simple this evening, (it's woodcutting season here at present) days are busy.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#42 Post by amigo »

Yeah, unionfs-fuse is optional. The rest looks alright. Don't worry about the coreutils and tar version -these tools get installed in to a 'proprietary' path away from your system tools where they are available to src2pkg. It uses the old coreutils version because their behavior is known and uses old tar for creating slackware-compatible packages.

It does say that the final src2pkg-helpers package installation is failing, though. If true, you may need to manually install it -it should be under /usr/src/src2pkg/src2pkg-helpers. But you can check to see if it is already installed by running 'which src2pkg'.

User avatar
harii4
Posts: 448
Joined: Fri 30 Jan 2009, 04:08
Location: La Porte City, IA , U.S.A.
Contact:

#43 Post by harii4 »

Would src2pkg work with/on a raspberry pi (raspbian) ?
Or only on a full blown debian or puppy?

Might have to update to a newer puppy to use src2pkg-3.0 due to still using Tgz-pup.
3.01 Fat Free / Fire Hydrant featherweight/ TXZ_pup / 431JP2012
----------------------------------------------------------------------------------------
Peace and Justice are two sides of the same coin.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#44 Post by amigo »

src2pkg will work anywhere you have enough development tools installed.

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#45 Post by live »

Hi,

On tahrpup64 with its devx, what do I need to compile 64bits pet?
By default it builds i486 pets

into "etc/src2pkg/src2pkg.conf", I tried to modify
[[ $ARCH ]] || ARCH="m64"
[[ $MARCH_FLAG ]] || MARCH_FLAG="m64"
And ran

Code: Select all

src2pkg -CWD -PET --arch=m64 <source tar>
But it fails

Thanks

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#46 Post by amigo »

You need this in the conf file:

Code: Select all

[[ $ARCH ]] || ARCH="x86_64"

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#47 Post by s243a »

amigo wrote:You need this in the conf file:

Code: Select all

[[ $ARCH ]] || ARCH="x86_64"
I haven't gone through this thread yet, but I presume that you would have to run the configure scripts before running this pet, in order to make sure that you have all the correct dependencies installed.

A good test for this package would be the ruby package since some environmental variables in ruby are hard coded at compile time.

The difficulty I had with making pets is that for some packages the compilation may depend on where you compile to, and this makes setting the prefix and then using dir2pet not a directly workable approach.

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#48 Post by live »

Thanks

User avatar
Keef
Posts: 987
Joined: Thu 20 Dec 2007, 22:12
Location: Staffordshire

#49 Post by Keef »

src2pkg will run configure. You can use your own options with:

Code: Select all

src2pkg -e='--disable-something --with-summatelse' tarball-name
etc.

You can also create build-scripts with your options, so it is easily repeatable.

The documentation is well worth a read.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#50 Post by scsijon »

sorry amigo but v3.0 is failing with barryk's latest quirky 8.1.6, just junk in output package and things like wrong directories's etc. Wonder if it's because it's working from /file and not /root.

regards

Post Reply