[RELEASE] src2pkg-3.0 - create packages from source

Miscellaneous tools
Message
Author
live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#16 Post by live »

Are you using a '--prefix=??' on the command-line
No, as you see in the preceedeing post my CLI is

Code: Select all

src2pkg -CWD -PET --arch=i686 openssl-1.0.1g.tar.gz 

or in the src2pkg.conf file?
No, I installed your posted pet and unmodified.

Now, it's interesting to look at this conf file.
Does "my" command line generate compiled code working for i486, but when possible optimises it for i686?

Code: Select all

# settings for generic puppy
[[ $ARCH ]] || ARCH="i486"
[[ $MARCH_FLAG ]] || MARCH_FLAG="i486"
[[ $PKG_FORMAT ]] || PKG_FORMAT=pet
PET2=1
[[ $INIT_DIR ]] || INIT_DIR=/etc/rc.d
RE: page 229 of GCC manual http://gcc.gnu.org/onlinedocs/gcc-4.8.2 ... tions.html

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#17 Post by amigo »

Yes, it does. 'march' is the bottom limit and 'mtune' is the upper limit, which gets set to i686 for ARCH=i486. These are the standard settings long used by most mainstream distros. So, you should be fine using the standard ARCH=i486 settings.

Sorry about the prefix stuff, I should have looked here at my build script for openssl to see if there were any abnormalities to the build. Turns out it is rather involved, so I'll attach my materials.

The file openssl.src2pkg.gz is a fake gz file, simply rename to remove the '.gz' suffix. That is then the build recipe for openssl. It will use a couple of patches included in the (real) archive named openssl-patches.tar.bz2. Simply place those two files alongside the openssl sources and run 'src2pkg -X' to use that recipe.
Attachments
openssl.src2pkg.gz
(3.47 KiB) Downloaded 271 times
openssl-patches.tar.bz2
(4.82 KiB) Downloaded 275 times

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#18 Post by live »

>Sorry about the prefix stuff
I'm very glad to learn and exchange. If at the same time it provides enhancement it's an additional benefit for everyone.
Yes, it does. 'march' is the bottom limit and 'mtune' is the upper limit, which gets set to i686 for ARCH=i486. These are the standard settings long used by most mainstream distros.
Unfortunately I do know and I do consider it has a performance and ecological absurdity has i686(=Pentium Pro max 200Mhz) ceased to be produced in 1998 (now 16 years ago) and i486 SX has no floating point unit so it must be emulated by the compiler into fewer integer registers, but that might be considered as personal philosophy and I modified your config file to march=i686.
FYI: Intel history list of CPU, http://www.intel.com/pressroom/kits/quickreffam.htm

For the rest of your last post, would you be so kind to explain in other words, as I think my lack of experience in this matter prevents me of fully seizing the importance of all the concentrate information you give.
It does perfectly work and install this time.
Now interestingly, terminal suggest to it had wishes for SSE2 optimisation http://en.wikipedia.org/wiki/SSE2

Code: Select all

sh-4.1# src2pkg -X
Running build script: openssl.src2pkg
Found source archive: openssl-1.0.1g.tar.gz
Creating working directories:
   PKG_DIR=/root/Downloads/openssl-1.0.1g-i686-1
   SRC_DIR=/root/Downloads/openssl-1.0.1g-src-1
Unpacking source archive - Done
Decompressing openssl-patches.tar.bz2 - Done
Correcting source permissions - Done
Applying patches - from User-Supplied PATCHLIST
Applying: openssl.soname.diff
Applying: openssl.no-extssl.diff
Applying: i486-i686.diff
   Notice - Running 'make clean' in SRC_DIR - Done
Running user-specified CONFIG_COMMAND: ./config --prefix=/usr --openssldir=/etc/ssl no-idea no-rc5 no-sse2 no-engine shared
Compiling sources - Using: 'make depend all'
Compiling has been - Successful!
Checking for Makefile rule: 'install' Okay
Checking support for DESTDIR (or similar) - Found INSTALL_PREFIX
Creating package content using:
   make INSTALL_PREFIX=/root/Downloads/openssl-1.0.1g-i686-1 install
Installation in INSTALL_PREFIX - Successful
Processing package content:
Correcting package permissions - Done
Stripping ELF binaries - Using: strip -p --strip-unneeded Done
Stripping static archives - Using: strip -p --strip-debug Done
Stripping shared libraries - Using: strip -p --strip-unneeded Done
Checking for standard documents - Done
Verified slack-desc.openssl found - Inserting in package as PKG_DIR/install/slack-desc
Adding src2pkg script to package in: usr/doc/openssl-1.0.1g
Rechecking package correctness:
Rechecking package permissions - Done
Checking for dead links in PKG_DIR - Done
Copying content for: openssl-SOLIBS
*** buffer overflow detected ***: /usr/libexec/src2pkg/bin/tar-1.13 terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x45)[0xb76a60e5]
/lib/libc.so.6(+0x102eba)[0xb76a4eba]
/lib/libc.so.6(+0x1021ed)[0xb76a41ed]
/usr/libexec/src2pkg/bin/tar-1.13[0x804ff86]
/usr/libexec/src2pkg/bin/tar-1.13[0x80505b9]
/usr/libexec/src2pkg/bin/tar-1.13[0x8051cd0]
/usr/libexec/src2pkg/bin/tar-1.13[0x804a9b7]
/lib/libc.so.6(__libc_start_main+0xf3)[0xb75bb4d3]
/usr/libexec/src2pkg/bin/tar-1.13[0x804ad01]
======= Memory map: ========
08048000-08067000 r-xp 00000000 00:0b 66774      /initrd/pup_rw/usr/libexec/src2pkg/bin/tar-1.13
08067000-08068000 r-xp 0001e000 00:0b 66774      /initrd/pup_rw/usr/libexec/src2pkg/bin/tar-1.13
08068000-08069000 rwxp 0001f000 00:0b 66774      /initrd/pup_rw/usr/libexec/src2pkg/bin/tar-1.13
08069000-0808a000 rwxp 00000000 00:00 0          [heap]
b7582000-b759e000 r-xp 00000000 07:00 35738      /initrd/pup_ro2/lib/libgcc_s.so.1
b759e000-b759f000 r-xp 0001b000 07:00 35738      /initrd/pup_ro2/lib/libgcc_s.so.1
b759f000-b75a0000 rwxp 0001c000 07:00 35738      /initrd/pup_ro2/lib/libgcc_s.so.1
b75a0000-b75a2000 rwxp 00000000 00:00 0 
b75a2000-b7745000 r-xp 00000000 07:00 35707      /initrd/pup_ro2/lib/libc-2.15.so
b7745000-b7747000 r-xp 001a3000 07:00 35707      /initrd/pup_ro2/lib/libc-2.15.so
b7747000-b7748000 rwxp 001a5000 07:00 35707      /initrd/pup_ro2/lib/libc-2.15.so
b7748000-b774d000 rwxp 00000000 00:00 0 
b774d000-b776d000 r-xp 00000000 07:00 35686      /initrd/pup_ro2/lib/ld-2.15.so
b776d000-b776e000 r-xp 0001f000 07:00 35686      /initrd/pup_ro2/lib/ld-2.15.so
b776e000-b776f000 rwxp 00020000 07:00 35686      /initrd/pup_ro2/lib/ld-2.15.so
bff2e000-bff4f000 rw-p 00000000 00:00 0          [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0          [vdso]
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
      Creating: 'pkg-manifest' Done
Searching for links in: openssl-1.0.1g-i686-1 - Done
Adding links to doinst.sh - Adding links-creation to the doinst.sh
Found '.new' files in PKG_DIR - Adding '.new' file routine to doinst.sh
Deleting symbolic links - Removing links from the package directory
Creating package: 'openssl_SOLIBS-1.0.1g-i686-1.pet' - Done
Creating package: 'openssl-1.0.1g-i686-1.pet' - Done
Package Creation - Successful! Package location:
/root/Downloads/openssl-1.0.1g-i686-1.pet
sh-4.1# openssl version
OpenSSL 1.0.1g 7 Apr 2014
Finally, I do not know if it is a scripting issue, but if I compile geany-1.24.tar.gz
Under "root/Downloads" I have a successful build
However if I compile on my HDD, it fails to compile, as if they were some hard coded path.

Thanks

Code: Select all

sh-4.1# src2pkg -CWD -PET geany-1.24.tar.gz
Found source archive: geany-1.24.tar.gz
Creating working directories:
   PKG_DIR=/mnt/sda7/ITC/SharedWinXp/Linux/Puppy save/pet/dev/src/geany-1.24-i686-1
   SRC_DIR=/mnt/sda7/ITC/SharedWinXp/Linux/Puppy save/pet/dev/src/geany-1.24-src-1
Unpacking source archive - Done
Correcting source permissions - Done
Checking for patches - None found
Found configure script - Done
Configuring sources using:
 LDFLAGS="-Wl,-O1,-L/lib,-L/usr/lib,--relax,--sort-common,--no-keep-memory" CFLAGS="-O2 -m32 -pipe -fomit-frame-pointer -fno-strict-aliasing -Wno-shadow -Wno-unused -march=i686 -mtune=i686" ./configure --prefix=/usr --libdir=/usr/lib
Configuration has been - Successful!
Compiling sources - Using: 'make'
Compiling has been - Successful!
Checking for Makefile rule: 'install' Okay
Checking support for DESTDIR (or similar) - Found DESTDIR
Creating package content using:
   make DESTDIR=/mnt/sda7/ITC/SharedWinXp/Linux/Puppy save/pet/dev/src/geany-1.24-i686-1 install
Notice - Possible error running 'make install'
FATAL! Running make install has failed with error: 1 
Try using INSTALL_LINE 'make -i install'  Exiting...
sh-4.1#

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#19 Post by amigo »

No, there are no hard-coded paths. It may have something to do with the aufs layering. That's a strange tar-1.13 error that I've never seen either. Do you get that with the failed geany build?

About sse, you may want to leave out the no-sse2 and see if that works for you -this is directly related to the march choice. As for generally trying to adjust march/mtune when compiling, this makes no sense unless you build a whole system using 'better' settings -starting with glibc itself. src2pkg itself defaults to using the most conservative and universal settings for obvious reasons. You can adjust any settings you like, as far as compiler options goes. Have a look at and near lines 258-259 for some of them. Actually the defaults there are not completely conservative -the -fomit-frame-pointer option is not good on some arches. And the -pipe option could even be causing the tar-1.13 or build errors.

You can find lots of pre-made recipes here:
http://distro.ibiblio.org/amigolinux/di ... .0/SOURCE/
Again about openssl, its build system is a home-grown thing which works pretty different than most builds -that's why the build script has so many lines. geany, OTOH, is pretty normal. The point is that some sources require extra work to get what you want, whereas something like geany can usually be compiled without any recipe at all.

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#20 Post by live »

>That's a strange tar-1.13 error that I've never seen either. Do you get that with the failed geany build?
It's a reproducable path bug and here is the way:
If my path is:
mnt/sda7/ITC/SharedWinXp/Linux/Puppysave/pet/dev/src
build is OK (thought contrary to a build in /root/Downloads I have way more output lines in Terminal...).
My initial path contained a blank in the folder's name so was 'Puppy save' instead of now Puppysave and there it stops on error 1.
I tested with non US char (namley: ésrc) it's ok.

I have to investigate other things regarding last post, however some others:

1/ On line 32 & 33 of /etc/src2pkg/src2pkg.conf if I modify them as

Code: Select all

[[ $ARCH ]] || ARCH="i686"
[[ $MARCH_FLAG ]] || MARCH_FLAG="i686"
And then execute:

Code: Select all

src2pkg -PET -CWD --arch=pentium4 <tar source>
Then I see

Code: Select all

Configuring sources using:
 LDFLAGS="-Wl,-O1,-L/lib,-L/usr/lib,--relax,--sort-common,--no-keep-memory" CFLAGS="-O2 -m32 -pipe -fomit-frame-pointer -fno-strict-aliasing -Wno-shadow -Wno-unused -march=i686 -mtune=i686" ./configure --prefix=/usr --libdir=/usr/lib
Notice -march=i686 -mtune=i686

If I now modify src2pkg.conf and sets both param as pentium4
and execute the command I see:

Code: Select all

Configuring sources using:
 LDFLAGS="-Wl,-O1,-L/lib,-L/usr/lib,--relax,--sort-common,--no-keep-memory" CFLAGS="-O2 -m32 -pipe -fomit-frame-pointer -fno-strict-aliasing -Wno-shadow -Wno-unused -march=pentium4 -mtune=i686" ./configure --prefix=/usr --libdir=/usr/lib
Notice -march=pentium4 -mtune=i686


2/ Are the files in /usr/include sometimes used for linking/dependicies during build process?

3/
this makes no sense unless you build a whole system using 'better' settings -starting with glibc itself.
OK, good to know, any other?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#21 Post by amigo »

"Notice -march=pentium4 -mtune=i686" I'll work on making it possible to get what you want there. The '--arch=??' option to src2pkg only sets the ARCH portion of the package name and does not directly affect the gcc options.

"Are the files in /usr/include sometimes used for linking/dependicies during build process?" Yes, of course, this is where development headers(include files) are located. They are not used at run-time.

"tar-1.13" So, the error occurs only and always when there is a space in the name? That can be fixed -but seeing this reminds me that for pets a normal version of tar could be used instead.

"build a whole system using 'better' settings" Since nearly everything on the system links to glibc, a small performance improvement can be had by compiling glibc to match your hardware -**but** re-compiling glibc implies re-compiling everything that gets linked to it. Further, very few sources/programs actually use 'enhanced' CPU features -the big exception is audio/video editing software and high-performance number-crunching software.

So, optimizing libraries and applications for better CPU features will usually get you no noticeable improvement. There is some noticeable improvement to be had by re-compiling the kernel to match your hardware.

User avatar
Puppus Dogfellow
Posts: 1667
Joined: Tue 08 Jan 2013, 01:39
Location: nyc

#22 Post by Puppus Dogfellow »

i've decided to try this even though i don't know what i'm doing....

gleaning/misreading that running

Code: Select all

src2pkg -CWD -PET --arch=i686 mixxx-1.11.0-src.tar.gz 
from a terminal in a folder containing mixxx-1.11.0-src.tar.gz would give me the pet i want, i get:

# src2pkg -CWD -PET --arch=i686 mixxx-1.11.0-src.tar.gz
Notice - Creating src2pkg-helpers:
src2pkg uses a shared library and a few programs
when creating packages. For best compatibility,
these binaries will be compiled on your system.
They are then installed in a private directory.
When done, src2pkg is ready for use.

Code: Select all

  TEMP_DIR=/usr/src/src2pkg/builds/src2pkg-helpers
  Starting build in 5 seconds
Unpacking sources - OK
Creating libsentry - OK
Creating tar-1.13 - OK
Creating coreutils - OK
Creating unionfs-fuse - OK
Creating pet package - Done 
Finished pet package is:
/usr/src/src2pkg/src2pkg-helpers/src2pkg-helpers-1.6-i486-1.pet
Installing pet package - 
--it's just hanging, so i figured i'd take this opportunity to ask what i've done wrong...(about a half hour so far on a moderately fast machine running precise 5.5.. lxtask shows no change in memory use and 0% cpu use).

thanks in advance.

(just checked the location the above helper pet is supposed to be in--scripts are there, but no pet.)

will try again...got a little further (helper pet created and installed) on a slower machine running 571, "build failed, must be a poor..."

:oops: forgot to copy the actual error message. (it also said something about there was no file with dependency information or something to that effect...sorry.)

attempt 2 with the 5.5. yielded no output. i hit enter with # src2pkg -CWD -PET --arch=i686 mixxx-1.11.0-src.tar.gz
and just got a cursor. back to the 571...

which gives me:

Code: Select all

# src2pkg -CWD -PET --arch=i686 mixx-1.11.0-src.tar.gz
FAILED! File: mixx-1.11.0-src.tar.gz not found!
Check for errors in the NAME, VERSION or SRC_SUFFIX. 
# src2pkg -CWD -PET --arch=i686 mixxx-1.11.0-src.tar.gz 
Found source archive: mixxx-1.11.0-src.tar.gz
Creating working directories:
   PKG_DIR=/initrd/mnt/dev_save/dwnldsfff/mix111/mixxx-1.11.0-i686-1
   SRC_DIR=/initrd/mnt/dev_save/dwnldsfff/mix111/mixxx-1.11.0-src-1
Unpacking source archive - Done
Correcting source permissions - Done
Checking for patches - None found
Found SConstruct file - Configuring using: 'scons configure prefix=/usr' Done
Compiling sources - Using: 'scons'
ERROR! Compiling source code has failed.
This usually happens because of missing libraries, or
badly written Makefiles or configure scripts.
Sorry! No Dependency or Requirements information found.

# 

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#23 Post by live »

i've decided to try this even though i don't know what i'm doing....
As a math teacher told me, people doing stats without understanding, is like playing with grenades :)

Now seriously,

Mixxx 1.10 is in PPM and needs about 94Mb of Qt4 dependicies.

--it's just hanging, so i figured i'd take this opportunity to ask what i've done wrong...
I experience the same, haven't try with mixxx but:
1/ run src2pkg cmd
2/ on prompting for installing the help pet: cancel.
3/ install the help pet.
4/ re-run src2pk cmd

src2pkg can not readily build everything, for example:
gedit-2.91.11.tar.gz
gcc 8.2
testdisk 7.0
SMPlayer 14
SeaMonkey

But can build others :)

User avatar
Puppus Dogfellow
Posts: 1667
Joined: Tue 08 Jan 2013, 01:39
Location: nyc

#24 Post by Puppus Dogfellow »

live wrote:
i've decided to try this even though i don't know what i'm doing....
As a math teacher told me, people doing stats without understanding, is like playing with grenades :)

Now seriously,

Mixxx 1.10 is in PPM and needs about 94Mb of Qt4 dependicies.


--it's just hanging, so i figured i'd take this opportunity to ask what i've done wrong...
I experience the same, haven't try with mixxx but:
1/ run src2pkg cmd
2/ on prompting for installing the help pet: cancel.
3/ install the help pet.
4/ re-run src2pk cmd

src2pkg can not readily build everything, for example:
gedit-2.91.11.tar.gz
gcc 8.2
testdisk 7.0
SMPlayer 14
SeaMonkey

But can build others :)
yeah, but on one machine i broke it and can't seem to repair it so i figured i'd try it this way...

will mess around some more later. thanks for the insight.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#25 Post by amigo »

The build for mixx is most likely failing because of missing the 'scons' program.
src2pkg can't do magic -there is too much variation in the methods used to configure and build software- as the scons example shows. src2pkg can sometimes create a correct package without any special options being passed at all. But there will always be packages which need special config options or scripted steps to build them.

I've used src2pkg to build more than 10,000 distinct sources and it allows enough flexibility to handle any sort of build -but a recipe or package-specific options are usually needed about 60% of the time. So don't be surprised or discouraged if it fails to do it automatically the first time. Many times, builds fail because of missing prerequisites.

User avatar
Puppus Dogfellow
Posts: 1667
Joined: Tue 08 Jan 2013, 01:39
Location: nyc

#26 Post by Puppus Dogfellow »

amigo wrote:The build for mixx is most likely failing because of missing the 'scons' program.
src2pkg can't do magic -there is too much variation in the methods used to configure and build software- as the scons example shows. src2pkg can sometimes create a correct package without any special options being passed at all. But there will always be packages which need special config options or scripted steps to build them.

I've used src2pkg to build more than 10,000 distinct sources and it allows enough flexibility to handle any sort of build -but a recipe or package-specific options are usually needed about 60% of the time. So don't be surprised or discouraged if it fails to do it automatically the first time. Many times, builds fail because of missing prerequisites.
it seems fairly amazing. i'm not discouraged at all.

:)

live
Posts: 223
Joined: Wed 10 Feb 2010, 21:04

#27 Post by live »

>amigo
All I meant, is what you said "src2pkg can't do magic" sometimes it works as is, some times not and gave some examples.

Sorry, if they were some misinterpretation.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#28 Post by amigo »

You can find lots of build scripts with patches, etc, for a reference implementation here:
http://distro.ibiblio.org/amigolinux/distro/kiss-4.0/

There are also many, many older scripts here:
http://distro.ibiblio.org/amigolinux/download/

You can have src2pkg generate a build script for you with:
src2pkg -N -Q tarball-name
or:
src2pkg -A tarball-name

The build scripts are easily modifiable to do anything at all. And they provide a repeatable, upgradeable recipe for later use. Provided that all needed depends are already present, most sources can be successfully compiled, to your needs, by only adding whatever options the sources configure script needs -although src2pkg handles the routine options like --prefix, --libdir, etc, automatically.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#29 Post by scsijon »

Very nice package amigo, that can save a lot of time for me with T290 building. There's about 5100 pets to create in all of the T2 V9.0 set and I was wondering how I was going to manage it all. I know I could just do those I need, but I would like to have all of them available.

Is there a simple way to :

-just point it at a directory and tell it to create pet's from all sources in that directory;

- output resultant pets to another named directory;

-error files to a third so I can just work on those with problems;

- creates a copy of the onscreen data into a single named file so that you can initially just leave it alone to run it's course, and later check back when working on a problem.

Oh yes, and could it maybe be told to deal with patch scripts if their in the same directory as the source package.

I know a tall order, but anything to make life easier.... :wink:

regards
scsijon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#30 Post by amigo »

src2pkg lends itself well to being driven by other scripts -see the example GroupBuild scripts in the documentation. For best results, each source should be in its own sub-directory so that files generated or used by each build are kept separately. It also makes it possible to run more than one build at a time.

Any patches placed alongside the sources will be automatically applied.

Location of final packages can be controlled by setting PKG_DEST_DIR=/path/to/destination

The '-LOG' option will log the output of all important commands run during the build.

src2pkg is meant to create great packages -but *one at a time*. While it would be possible to make it run through a list of sources as a single run, it would defeat more important purposes. Irt can't do magic, so building 5,000+ sources is gonna take some work no matter what system you use.

While src2pkg can build lots of sources without any help or special options, there will always be some which need patches, special options or that must be done in a certain order. This is the same for all build systems, so at some point human-knowledge must come into play -which means writing build recipes, etc. However, src2pkg can leverage previous knowledge, by using rpm *.spec files, SlackBuild scrips or debian build files. This makes the process of creating a repo of sources for src2pkg much easier.

Take a good look at the example *.src2pkg scripts in the docs as there are examples for building inter-dependent packages and other unusual builds.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#31 Post by scsijon »

Yes, I agree, I've been having a play and also have found the src2pkg website, which is extremely helpfull.

I think my first problem will be an 'order-of-build' problem as I will need to have dependancies built and installed first before the packages needing them.

So I somehow need to list them, build them and install them before building the parent packages.

The -T I think will tell me if it needs a dependancy, thereby if so put it into build-later-list2, if not into build-first-list1.

I wonder what format it comes back in though, just the packagename or a lot of unwanted info? I think if it's useable like that, that is all I would need added if it's not there already in the switches, to complete an autobuild with dependancies.

This would autoloop with build-first-list1 being installed and build-later-list2 becoming the new source-list fed into the -T until there is nothing in either build* lists.

The -I will of course install the dependancy once built.

I shall play further, but I wonder if in the future it would be possible to build linux totally from source without the need to have interims such as T2, etc.

Seems like i'm back to scripting again, oh well :roll:

I think for now, i'll just build with src2pkg the minimum I need for T290 and work it from there. ie do the other packages from within a T290 base, not give up.

This leaves just which Kernel and it's settings left to choose!

Also, as a fyi, I just found that src2pkg --more-help, listed in the help, just errors out while src2pkg -hh is ok.

regards
scsijon

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#32 Post by scsijon »

amigo, I'm having fun playing with this and do think it will be of much use to me, especially when I start to create all the extra T2 packages.

However, for now, I can't seem to find what output switch sets the final package as a $package.tar.bz2, can you help me please. It's needed for the way BarryK has setup the new T2 packaging system and i'd like to keep that part of it as the overall build seems to have become 'somewhat quicker'. It looks like it requires less post-build scripting work to happen than a pure .pet package does.

thanks
scsijon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#33 Post by amigo »

You mean you want the final packages to have '.tar.bz2' suffix? And, that they use only a name without any ARCH or VERSION info in the name?
src2pkg does offer bzipped packages as an option, using '-TBZ'. However, you may find it quirky since nobody ses such packages. It was intended for use with vector linux IIRC. But, when you ask for a *.pet it will automatically set and use gzip compression with .pet suffix.

The best way to solve the above (either question), would be to write a small extension to src2pkg which would re-format and re-name the end result. You can create extension which are run at any and all steps of the build -either before or after the regular code of the step is run. You can see an explanation of the mechanism starting at line 2492 of the ChangeLog file in the docs and the feature must be enabled in the conf file -see notes at line 289. The extensions feature allows you to radically alter what src2pkg does, so it provides fertile ground for developing new features. Let me know if you still don't understand how to create and use them and I'll help you.

I like the T2 build system -but I don't like their package format as it is way too basic. I have developed a more usable package standard (actually two of them) which can be created by src2pkg -maybe you'd like to use it to build a base on?? The *.tpkg and *.tpm packages provide very rich package information -including package dependency info. The tpkg and tpm package management tools do not (yet) actually resolve depends for you (I am an old slacker, you know), however, the mechansim for doing so is quite simple *once the info is generated or specified and included in the package*. The nice thing about T2 is that it generates even BuildRequires info for you. I've done some work towards integrating this into src2pkg (for *.tpm packages). The run-time depends-generation already works great.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#34 Post by scsijon »

amigo wrote:The best way to solve the above (either question), would be to write a small extension to src2pkg which would re-format and re-name the end result. You can create extension which are run at any and all steps of the build -either before or after the regular code of the step is run. You can see an explanation of the mechanism starting at line 2492 of the ChangeLog file in the docs and the feature must be enabled in the conf file -see notes at line 289. The extensions feature allows you to radically alter what src2pkg does, so it provides fertile ground for developing new features. Let me know if you still don't understand how to create and use them and I'll help you..
ok, i'll have a look and a go before asking for help
amigo wrote:I like the T2 build system -but I don't like their package format as it is way too basic.
/cut/
The nice thing about T2 is that it generates even BuildRequires info for you. I've done some work towards integrating this into src2pkg (for *.tpm packages). The run-time depends-generation already works great.
Mageia does this also with their system, although sometimes there is too much info. given.

I have an idea this is what barryk was working towards using in his version of woof (not woof-ce) and his racy, which is T2. But when he went back to quirky (his favorite plaything) it was overtaken with another idea. He had a few notes about generation and integration some time ago on his blog, but they were only 'roughs'.

I'd really like to one day, just tell a working system that I want to add a new package by name, have it d/l the source, find dependancies from inside the package, check what dependancies I have already and if versions are sufficent, give me an option to go ahead or not, if it's a yes d/l missing or dependancies that need to be updated, build and install dependancies, build and install the new package and let me test it and either accept it or reject it and if rejected remove the package and all the new dependancies or step those that were already in back to the earlier versions. Pie in the sky at present, but it should happen one day.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#35 Post by amigo »

Your last paragraph describes exactly what nearly everyone wants -and what major distros implement to a great degree. Mageia is an rpm distro as is fedora and suse. debian/ubuntu & Co. use dpkg with *.deb packages. Both systems require a recipe for every single package which includes everything from the source url to the last script actions used to install the package. woof is completely different as it just uses pre-built packages.

There is no magic way to resolve dependencies properly. It requires the generation of a lot of accurate information which must be collected and collated into a useful form *at the time the package is created*, and then passed on *within the package* to package *management* tools.

generate, collect, collate/format -that's the hard part. This has to be done by the package *creation* software at package build-time.

The package *format* is the central concept of any package system. It must be capable of containing and transmitting all the info needed to carry out proper installation.

Dependency-resolution and proper up/down-grading, done of course by the package installation/mgmt tools, also depend on the use of known, sane repositories of packages. The repo-side of the equation also involves creating and making available information about the packages. repo creation and management is done by still another class of software.

There is one example extension (for vector linux in fact), here:
http://distro.ibiblio.org/amigolinux/do ... xtensions/

Try just creating a harmless extension with echo statements, so you can see if it is working and how/when.

Post Reply