LD Cmake / binutils pet query [closed]

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Post Reply
Message
Author
scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

LD Cmake / binutils pet query [closed]

#1 Post by scsijon »

Is anyone else having / had problems with the LD Dynamic Linker Loader in the binutils pet that we are using?

I eventually built the razorqt package by manually fixing the file with paths/*.so files.

But i've come across the same problem again with another package!

And I now have the same problem result ocurr when building on top of wary522!

I've decided to try building the latest binutils (2.22) with a later kernal and see if that is the problem as the binutils pet we have in the wary5 stream seems to have been built with 2.6.16.

regards
scsijon
Last edited by scsijon on Mon 06 Feb 2012, 01:09, edited 1 time in total.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#2 Post by amigo »

So just what version of binutils is that and where does it *really* come from? This sounds like one of those subtle problems that crop up when you patch together a lil' OS using bits and pieces from all over.... Toolchain integrity is primal.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#3 Post by jemimah »

I did need to update ld to get syslinux4 to compile.

It's was this bug:
http://www.syslinux.org/archives/2010-July/014994.html

Puppy is clearly not the only OS that has this problem - seems to be with specific versions of ld.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#4 Post by scsijon »

amigo wrote:So just what version of binutils is that and where does it *really* come from? This sounds like one of those subtle problems that crop up when you patch together a lil' OS using bits and pieces from all over.... Toolchain integrity is primal.
amigo,
It's the one in the base puppy woof-packages you would normally find in the puppy tree and use if you were creating your own t2 based puppy (wary/racy522) when building with woof, filename is 'binutils-2.20.51.0.2.pet'.
And I agree it's subtle, but did expect the toolchain to be ok as everything I've added has been built on the current or an earlier version with the same package base.
That is why I put the query out there. problem ocurring once can be the program, a second appearance to me means something's not right somewhere!
I did wonder if recompiling it with a 2.8.20~ kernal would fix the problem and will now do that in parallel with trying to build a later release.

Jemimah,
Barry had 2.20.51.0.2, they talk about 2.20.51.0.10 being ok so maybe, just maybe the problem was still in there, just didn't appear. Thank you for the syslinux database referance, I didn't know it existed and will add it to my problem lookup sitelist.
I've had one go at building the latest stable 2.22, which has failed to build properly, I shall persevere today, if it doesn't happen I shall download and try the one they are talking about as good,
I've just downloaded barry's, from his tree, so will have a go at that with a 2.8.xx~ kernal, just in case that is all the problem is, but will persevere with a later one as there are quite a few changes.


Thank you both for answering as it gives me a point to work from, as I have about a dozen packages that build using the LD command before I can release my second qtpuppy alpha, so far everything else has gone well.

regards
scsijon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#5 Post by amigo »

Kernel version has *no* bearing on the problem.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#6 Post by scsijon »

amigo wrote:Kernel version has *no* bearing on the problem.
Could you please expand amigo? I'm trying to learn as I go!

I have found in binutils 2.20.51.0.11 and it's additional patches 1.0 file, patches that fix the LD problem I keep hitting as well as a couple of others, so I plan on building that. If I built 2.22, the latest, there are some 5 additional buildingblock packages that need to be updated also, including gcc, so for now i don't want to go that way, just fix the current problem and get back to qtpuppy!

However I'm still getting fails in trying to build any version of a standard binutils+patches with wary522, so i'm going to have an indepth look at barryk's source of binutils-2.20.51.0.2 on his tree, in case he made some special patches to get around them, in which case I shall see if adding them will fix the build problems for me.

thanks to all

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#7 Post by jemimah »

I think you are overthinking it.

To fix my syslinux problem I just downloaded the latest binutils from the gnu site and compiled it without additional patches. It compiled without trouble on racy.

Note that I only installed the updated ld, not the rest of binutils, and that was enough to get syslinux to compile.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#8 Post by scsijon »

!OH!

I never even thought of that, and ld does have it's own 'bit' in the binutils package too. I thought you wood need to do it all as it's components interworked.

!DAMN!

And just as I have completed, with an awfull lot of help from a binutils guru (my name for her), building a binutils-2.22-i686 for puppy that seems to work on a number of puppy types, we couldn't do a i486 sorry.
I just have to do a final QA before pet packaging. Then the idea was to ask some of the puppy builders to give it a try. It won't have the 'gold' components though, that was just too hard and complicated to add without doubling it's size.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#9 Post by amigo »

Building and using any one of the main components of the toolchain should almost never be approached from the standpoint of what is the latest version -they are just as likely to contain bugs in new code as they are regressions. The thing is that the toolchain must be a sane, working set of tools -with 'circular dependency' between all elements. And most version-combination choices are not usable on every architecture -ARM and PPC are particularly troublesome.

Lat year I started a complete rebuild of my KISS system and chose the toolchain versions mostly copying Slackware current -keying on the ARM branch since I have my sights on that arch. Anyway, after getting past a couple of hundred packages and making it to grub, I found that the 'as' assembler was broken. The armed-slack didn't realize it because ARM uses no grub. The official current branch still doens't know about it because they haven't tried to recompile grub -they never re-compile until it is found out that something doesn't *run* anymore. So I had to abandon that version of binutils and find another (earlier) version (2.21.something) and go with that.

But you have to understand that that meant going back and starting completely over with the circular dependency procedure, and then start re-compiling all the hundreds of programs from scrathc -in their proper order. Assuming that a particular combination of glibc, gcc and binutils *will* work on the targeted arch, these tools must still all by compiled by, linked by and linked *against* each other. glibc must be compiled by the new gcc and linked by the new binutils. gcc must be linked by the new binutils against the new glibc. And binutils must be compiled by the new gcc and linke dagainst the new glibc. Then all other programs must be compiled using this *same set* of tools. That's the way you ensure that everything will run without problems -glibc warnings, library incompatibilities, linker/target issues, etc.

So simply upgrading one component of the toolchain and hoping for the best is no way to go. At the least, the new binutils should built, then rebuild the old gcc version using the new bintuils, then re-compile glibc using the new combination, then re-compile bintuils using the new gcc-glibc combination. But this will still not ensure compatibility for all the other rpograms and libraries of the system. The only sane way to build a system is to build everything in a certain order. Pieces of the toolchain should never be considered indivudually upgradeable.

That's why I ask where did the original non-working version come from. The answer was 'wary' or something which doesn't really tell us anything -is it really part of original deb (or is it ubuntu) combination and is it their binary? Or is something that BK or some contrib has 'whipped up' and thrown in the mix without sanely up/down -grading?

The same incident I described, also made me decide to never trust Slackware's choice of versions, either. I've known for a long time that their upgrade process is not really sane -and seen the errors that come along as a result.

As for kernel-version pertinence, the version of the running kernel has absolutley no direct bearing on library/program compatibility. Running a different kernel version will never help you solve a compiling/assembling/linking problem.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#10 Post by jemimah »

amigo wrote:So simply upgrading one component of the toolchain and hoping for the best is no way to go. At the least, the new binutils should built, then rebuild the old gcc version using the new bintuils, then re-compile glibc using the new combination, then re-compile bintuils using the new gcc-glibc combination. But this will still not ensure compatibility for all the other rpograms and libraries of the system. The only sane way to build a system is to build everything in a certain order. Pieces of the toolchain should never be considered indivudually upgradeable.
Certainly. But if you have one program that won't build, and a quick temporary update of one toolchain element fixes it, that's fine for situations like this as long as the result works. I don't suggest adding the new ld or whatever to the distro. I put everything back the way it was once I have my package built.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#11 Post by scsijon »

thank you amigo for a full explanation of the problem, that does open my eyes to the problems involved with working on the core package-set.

Not having built anything that could be considered a 'core' component of Puppy before, I had no idea of the level of inter-relationships that must be considered. In the past i've left the 'core' alone as it hasn't given problems, however in this case, until the ld problem has been solved my qtpuppy is dead in the water as the qt4.7.x and qt4.8.x, application building processes make serious use of this function in the build structure, and in the specs of the new qt5 it is even proposed to be used as a standard tool.

As far as the version mentioned at the start was concerned, all I knew was that it was Woof's designated version when building from WARY. ?Now I am wondering what to do with the new 2.22 pet, is it safe to release or not, and the same relates to a 2.20.51.0.11-w5c pet I am working on, built on a raw wary522 itself as a simple upgrade to fix the problem for all.

And your last statement did suprise me, I had considered the precept that maybe a component of the later kernal may be an added necessary function for the later core component version to work! If I understand you correctly, this is not so and there are no inter-relationships of this nature to be requirements.

And Jemimahs statement of returning it all back to standard once the problem was surmounted, I hadn't even considered an available option. I had just considered adding the upgraded package instead of the faulty one a requirement for it all to work properly in the future.

So, what I need to work on if I get everyone's ideas right, is to have two individual environments. One with ONLY the standard environment that would become the release environment and used for testing, running and release ONLY. A second self-contained environment would be needed to also exist, however that would be dynamic in nature with components being updated / downgraded and-or added / removed as necessary for the building ONLY. ?Or is this only when a 'core' component is involved, otherwise, both should be the same!


thank you both for expanding my thought patern in this matter.

regards
scsijon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#12 Post by amigo »

If one of the core componenets is not working then you have made a bad choice of distro to use -period.

About interoperability of the kernel and the rest of the system -there is one component that I didn't mention and that is the kernel headers, also called libc headers. When compiling glibc, it uses the kernel headers to establish the ABI-compatibility with the kernel. Changes to ABI-level code are quite rare, but do cause some hickups from time to time. Major 'watershed' versions of the kernel such as 2.6.16, 2.6.27 do contain such changes, although backwared-compatibility is usually maintained. One of the configuration options of glibc is 'kernel compatibility', where you tell the compilation process what the earliest version of the kernel y<ou want to be supported. This allows compiling to build whatever support is needed for kernel versions which are earlier than the kernel headers. The kernel headers version which is chosen is important, but has no relationship to the kernel version which is being run at compilation time. The only limiting factor when booting with a new glibc is that the minimum kernel version chosen during configuration must not be newer than the kernel you are trying to boot with.

For a long time, the headers from 2.6.16 were used by most distros, even when the running kernel versions were much later. This was done because a standardized set of manually 'sanitized' headers was made available. Later kernel versions (don't remember what version this started) have a Makefile rule which produces a set of sanitized headers for you -usually without any manual intervention -though there are still some headers which aren't properly handled.

The kernel version used to supply the headers for glibc is central to library/program compatibility and should never be changed unless glibc is being updated. Kernel headers should never be upgraded when the kernel is updated.

Many people on this forum make the mistake of thinking that the running kernel version has some bearing on compiling and library/program compatibility, but that is simply not the case. No programs communicate directly with the kernel. That is all done through glibc, using the ABI dictated by the choice of kernel header headers when glibc was compiled. The kernel may be the heart of the system, but glibc is the arteries and veins of it. All programs and libraries on the system use glibc to indirectly communicate with the kernel when kernel functions (system calls) are required.

On the other hand, *hardware* compatibility is all about the kernel and its' modules (and sometimes firmware, of course). Keeping these points in mind can save you lots of misery and 'stabs in the dark'. I remember a thread here where a supposedly knowledgable dev/contrib thought that his bash code using 'echo -n' was not working beacuse of the kernel version. So he lost some time looking there -in vain of course.

To be clear, upgrading your running kernel to a newer version should never cause software breakage, but mixing/matching/altering toolchain components will always cause breakage -sometimes the errors are very subtle, or involve little-used tools, so it often goes undetected for a long time. And binary-compatibility(ABI) is often maintained even when compile-time(API) compatibility has been lost -that means that a program may run error free, but if you tried to re-compile it (using the new toolchain), it would fail to compile.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#13 Post by scsijon »

thank you amigo, it is starting to make sense now.

Post Reply