Kernel 2.6.27 and squashfs 3.4 patch problem [solved]

[MU]

ok, uploaded the 4th iso.
ftp://ks301128.kimsufi.com/testing/puppy4.1-k2.6.27/

it is based on version 2, the kernel did not change.
So no lzma.

- I added the firmware from Puppy 4.1, most is compressed.
I could confirm with a DVBT stick, that the extraction works, after modifying /etc/modules/firmware.dep.2.6.27 according to Barrys Blog:
http://www.puppylinux.com/blog/?viewDetailed=00421

- added networkwizard oct-16th.

- used the modified initrd.gz from version 3.

- I went through all the 3rd party modules.
Many did not compile, due to changes in the API.
So for those, new patches must be searched in the web.
But I think, those are used quite seldom, like the exotic modem drivers.

These did compile:
/lib/modules/2.6.27/extra/atl2.ko
/lib/modules/2.6.27/extra/ungrab-winmodem.ko
/lib/modules/2.6.27/slmodem/slamr.ko
/lib/modules/2.6.27/slmodem/slusb.ko
/lib/modules/2.6.27/kernel/drivers/net/r8101.ko

I also patched ndiswrapper and used the current madwifi snapshot.
Then these compiled, too.

The 2.6.27sourcecompiled_410.sfs includes all drivers sources (precompiled) in /usr/src/patches
I also added comments in:
modules-compile.txt
modules-compile-extra.txt
modules-compile-MU.txt

So you see, which ones did not compile (I pasted error-messages).
You could google for the error messages, to find patches for them.
That is, what I did for madwifi and ndiswrapper.

Please let me know, if this iso fixes the issues from the older versions.
If yes, I will start on monday, to upgrade Muppy with the same kernel.

Mark

[big_bass]

I updated this post to reflect new changes in the script #v4.02



#v4.02 firmware in /lib/all-firmware now tarballs. now /lib/modules/all-firmware.
#v4.02 /lib/modules/firmware.dep.<kern>,DOTconfig-K<kern> moved to /etc/modules.
#v406 now can select different kernels in same version. (IDE, SCSI, LZMA, etc.)
#v406 kernels/<sub dir> name can now be named <kern version>-<text>



#-------------------------ok the new relative code ---------------------
****this is cut and paste of the newly modified code for #ver4.02
and the changes pertaining to #v4.02 /lib/modules/firmware.dep.<kern>,DOTconfig-K<kern> moved to /etc/modules.
#-----------------------------------------------------------------------

#v2.12 now have complete set of modules...
echo "Creating modules-complete/..."
rm -rf modules-complete
cp -a kernels/$DIRKERNEL/all-modules modules-complete #v406
#mkdir -p modules-complete/etc/modules
#cp -a kernels/$USEREKNEL/firmware.dep.$USEKERNEL modules-complete/etc/modules/

---------------------------------------------------------------------


#v4.00 v406
cp -a kernels/$DIRKERNEL/DOTconfig-K${USEKERNEL} rootfs-complete/etc/modules/
cp -a kernels/$DIRKERNEL/firmware.dep.${USEKERNEL} rootfs-complete/etc/modules/
--------------------------------------------------------------------




#cp -a modules-complete/lib/modules/firmware.dep.$USEKERNEL boot/initrd-tree/etc/modules/
for ONECHOSENMOD in `cat /tmp/zdrvcutdown.list`
do
CHOSENPATH="`dirname $ONECHOSENMOD`"
mkdir -p boot/initrd-tree/lib/modules/$USEKERNEL/$CHOSENPATH
cp -af modules-complete/lib/modules/$USEKERNEL/$ONECHOSENMOD boot/initrd-tree/lib/modules/$USEKERNEL/$CHOSENPATH/
done
fi
sync
find boot/initrd-tree/lib/modules -type f -name *.ko |
while read ONEKO
do
gzip $ONEKO
sync
done

-------------------------------------------------------------------------------------------------------

P.S Thanks BarryK for this tip also
For 2.6.27, you can just copy all of puppy-unleashed/kernels/2.6.25.16/all-firmware, and copy firmware.dep.2.6.25.16 (and rename to firmware.dep.2.6.27) to puppy-unleashed/kernels/2.6.27/.

big_bass

[jamesbond]

Hi Mark,

Tested. I agree with SHS, your 4th disc (which uses the same kernel as the 2nd) is noticeably faster than 3rd disc.

iwl3945 Wireless flawless works also (this time with network-wizard).

cheers!

[Sit Heel Speak]

The 2nd disc is still the champion (but I do not use wireless).

The 4th disc still shows 4x Xeon processors in Hardinfo, just like the 3rd.

Zlib score (average of several runs)
is 16477 in the 4th (doesn't vary much between runs) and...
16201 in the 3rd (but with occasional, about 1-time-out-of-8, runs over 18000)...and...
in the high 17000's for the 2nd (varies a lot between runs, I saw as low as 16780 and as high as 18660).

The "missing hyperthread" problem, did show up just now, in the 3rd disc.

Seamonkey feels most crisp in the 2nd disc. In fact, it and ttuuxxx's firepup-.02 are very nearly as delightful-to-use in your 2nd disc, as Seamonkey is when running under my personalized Puppy 2.17-1, which uses a Con Kolivas-scheduler-patched 2.6.21.5 SMP kernel compiled in Gentoo.

I have downloaded the 4.1 Unleashed and kernel source packages and will follow your steps this weekend. Maybe I can spot some oddity.

[MU]

Sit Heel Speak
I'm puzzled,as the kernel is the same, it should at least show the same results in the amount of processor cores...

Can you please compare the size of vmlinuz in 2nd and 4th?
Maybe we talk of a different second release, and I used one, that already had changes?
Did you download the source of the second?
If yes, could you attach /usr/src/2.6.27/.config ?
Mark

[MU]

please also tell me the md5sum of the second iso, that works fastest and shows the corrects amount of processor cores.
md5sum Puppy-410-2.6.27.iso
Mark

[Sit Heel Speak]

On-2nd-disc vmlinuz size: 2284K
On-3rd-disc vmlinuz size: 1928K
On-4th-disc vmlinuz size: 2284K

md5sum's:

2nd: 1afd280c17563842e447db391bb4c11b Puppy-410-2.6.27.iso
3rd: 3f19c2e2403b606e24c3f94d19ae7b44 Puppy-410-2.6.27.iso
4th: fc44eb3cdae809c197ab64ba0ddf0027 Puppy-410-2.6.27.iso

All 3 match the md5.txt.

I downloaded the Unleashed + kernel source last night, and some overnight while I slept, so if there was a change in .config or the kernel source+patches then I do not have it.

And now Hardinfo is incorrectly showing 4x Xeon's

...***EDITED 4x CPU's is correct. So says everyone over on the Gentoo forum. Each hyperthread is seen as a separate CPU.***

[MU]

thanks,
md5sum and kernelsize indicate, that I used the "correct" second iso to create the 4th.
So the kernel is identical.
You could try to use vmlinuz from 2nd in the 4th, if you run them frugal, to verify it.

The one thing that changed, is initrd.gz (the delay).
So you could try, too to use the 2nd in the 4th.

The other thing that changed, is that there are some more additional modules and firmware from the 3rd-party drivers (modem, network).
Because of this, also /lib/modules/2.6.27/modules.dep is huger in version 4.
You could try to use the one from version 2 to see, if it makes any difference. But I don't think so.
And delete (move somwhere) the two files in /etc/modules.
So Puppys eventsystem ignores them.

So I have no idea at moment, why now 4 cores are shown

Mark

[Sit Heel Speak]

@Mark: you may wish to wait another day before building your new Muppy. I think I have built a better 4.1 SMP kernel than yours. I have --finally,...it took all day-- figured out amigo's lzma patch--at least, one way of making it work correctly...

It is late here now, and I must hit the sack. I have tomorrow (Monday) free and am strongly tempted to write a How-To.

[MUguest]

ok, I will wait for it, great.

I must build some other packages like openoffice with languagepacks, so will do this today.

Mark

[amigo]

Sit Heel Speak, was there a problem with the patch?

[Sit Heel Speak]

Sit Heel Speak, was there a problem with the patch?

Not exactly a problem with the patch itself, rather with the combination of the patch and the source. The patch has include statements for several routines, e.g. decompress_unlzma.c (or -.h), which are not present in the 2.6.27 source as supplied in Mark's .sfs. These routines are also not present in the latest stable release from kernel.org, 2.6.27.2--it appears that Messrs. Torvalds et al are leaning away from lzma and toward lz1mo. However, the missing routines are components of busybox--but it is not clear to me whether busybox routines are available to the compiler. No matter though, because they are supplied by Puppy user "gray" in the 2.6.25.16 source he has made available. The missing routines are also available here and I am in fact right now double-checking gray's versions against these. Another issue is, I may end up deciding to change the default location of the routines as specified in the patch--not sure yet, I'm still tinkering.

[amigo]

The patch *adds* those routines to the sources -that's what the patch is supposed to do. What the patch does is allow for the kernel and/or initrd to be compressed using bzip2 or lzma instead of the usual gzip. It does this by adding both compressor and decompressor to the the kernel source. You must enable the options for that when you configure the kernel. This does not rely on external ltma or bzip2 programs or libs as they are built into the kernel. This is also *not* the patch which provides lzma compression for squashfs.
I think Mark stated that in the end he wound up leaving out the lzma patch -but I do Ät think that the problems he was having were due to that patch -although they might be.
Anyway, applying the patch to the sources he includes should be straightforward enough once you have them on a read-write drive. I am ineterested to know if you have problems with it working as it is supposed to. I use a similar patch for 2.4 kernels which is what the current 2.6 patch is based on.

[Sit Heel Speak]

The patch *adds* those routines to the sources...

Aha! I see! The patch creates the necessary include files, not modifies existing ones! D'oh!

OK...it'll be a few hours yet...I got distracted by the urge to write a How-to covering sha1sum, gpg and the public key of kernel.org...I'll get back to the business at hand now...

[Sit Heel Speak]

Progress report. This is starting from the official stable 2.6.27.2 source. All discrete patches except amigo's are from MU's 2.6.27-SMP source package or puptrix.org/sources:

1. Edited printk.c as per Barry's instructions in patches-required.txt...check
2. Applied latest unionfs patch for 2.6.27: patch -p1 < ../unionfs-2.5_for_2.6.27-rc6.diff ...OK, 3 offset warnings.
3. Installed lzma-458.pet...installed OK. Rebooted.
4. Applied Barry's lzma-squashfs patch: patch -p1 < ../4300_squashfs-3.3.patch ... OK, 4 offset warnings.
5. Applied Barry's fsync_super patch: patch -p1 < ../fsync_super-2.6.19.patch ...OK, 1 offset warning.
6. Applied Barry's deny_write_access patch: patch -p1 < ../deny_write_access.patch ... OK, 1 offset warning.
7. Backed up the entire /usr/src/linux-2.6.27.2 subdir.
8. Applied amigo's lzma patch:
cd /usr/src
patch -p0 < ./kernel-patch-updated-lzma-2.6.27.diff ...OK, no offset warnings.

Note: name of amigo's patch may not be as supplied, I renamed it and then re-renamed it.

Amigo's patch does indeed go off smoothly, and it does create the three header files.

9. Copied ti_fw_3410h, ti_fw_5052.h, ti_usb_3410_5052.c, and ti_usb_3410_5052h from the 2.6.25.15-patched .sfs. These have already been patched with the four ti_fw multitech patches.

make clean
make mrproper
make oldconfig...
Kernel compression mode 3 (LZMA)
compress initrd using GZIP...yes.
Union file system?..........yes.
Unionfs extended attributes...yes
Debug Unionfs?...yes (hey, it's my kernel...)

Continuing...

[MU]

good work, I'm curious, if we may expect a tar.gz of your /usr/src/ tomorrow?

I have built a multilanguage Openoffice.sfs now, and started to create a Muppy based on 2.6.27.
Must refine some issues tomorrow concerning zmsy_084.sfs, but in general it works.
I can watch DVBT with it and also my wireless stick works, those are the two critical test candidates I have.

So now most of the "infrastructure" is done, and it will be not too difficult, to replace the kernelmodules with newer ones from 2.6.27.2.

But now I need some sleep first.
Mark

[Sit Heel Speak]

good work, I'm curious, if we may expect a tar.gz of your /usr/src/ tomorrow?

Yes. (knock on wood...) Should be available with your morning coffee.

Be aware, I did make one change, to amigo's patch. Changed the "2.6.27" references to "2.6.27.2" to reflect the fact, this is my linux source subdir's name.

Earlier tonight I was having trouble with it, until I realized I was making this simple bonehead mistake.

[MU]

fine.
Ok, but the kernel itself still is called 2.6.27, right?
I would fear confusion with new versions in future, if we would use our own version-numbering.

Your lzma patch is ok, as long as all steps are documented and reproducable. (as you did above).
One reason I started the compilation was, that I wanted to be able, to understand the mechanisms.
So what the patches are good for, and how they are applied.

So if I get requests for Muppy concerning issues lateron, I can solve them on my own

Mark

[Sit Heel Speak]

OK, not quite ready in time for breakfast in Karlsruhe...maybe ready in time for supper.

I spent about an hour in "make gconfig" and then it segfaulted on exit. Ah, the joy of "make menuconfig"...

...and then wasted another 15 minutes --the time it takes for this machine to compile the kernel-- when it goes to lzma-compress the kernel I see an error message:

Code: Select all

lzma: /lib/libstdc++.so.6: no version information available (required by lzma)
lzma: /lib/libstdc++.so.6: no version information available (required by lzma)

Error: Incorrect command

So...let me go get a fresh copy of libstdc++.so.6 and try again...

...Ok, but the kernel itself still is called 2.6.27, right?

Yes, I only used "2.6.27.2" as the foldername because I already had /usr/src/2.6.27, your source.

Your lzma patch is ok, as long as all steps are documented and reproducable. (as you did above).

Maybe. Let me compile a fresh copy of libstdc++.so.6 and see if lzma works then, to compress the kernel.

If not, then I will do the lzma patch to the latest squashfs myself, and try patching from the start again.

[Sit Heel Speak]

Unstripped copies of libstdc++.6.0.9 (size: 1147K) (I compiled it in Puppy 2.17-1, a long time back), placed in /usr/lib and /lib, gets rid of the "no version information available" error lines, but I still get the "Incorrect command" lzma error.

Also I see about a dozen errors of the general type:

fs/unionfs/debug.c warning: passing argument 1 of 'd_deleted' discards qualifiers from pointer target type

or

passing argument 5 of 'kmem_cache_create' from incompatible pointer type

***
Bedtime now, back in 16 hours. I won't upload this until I get a good kernel compile.