LZMA in 2.12 - possible?

Using applications, configuring, problems
Post Reply
Message
Author
shankargopal
Posts: 295
Joined: Sat 03 Dec 2005, 11:30

LZMA in 2.12 - possible?

#1 Post by shankargopal »

Is there anyway to get LZMA squashfs compression as an option after 2.10? I am more than willing to pay the price in speed, but I need better compression.

Here's why. I have a flash Puppy functioning from a 256 MB pen drive. It's my daily working system, on which I do all my routine and other work and is in use typically for ten hours a day, seven days a week. I have such a severe space crunch that I run 2.10 from a Live CD and keep only the pup_save.3fs file on the pen drive.

Unfortuntaely the only other thing I need to keep on the pen drive is OpenOffice 2.0, which I require (AbiWord is too unreliable on Word documents, particularly those with revisions). The LZMA verison of the OO squashfs from grafpup is almost 16 MB smaller than the gzip version. On a 256 MB pen drive, that's a lot, and I simply can't afford the space for the gzip version. Therefore, while I really like the additional features of 2.12 - particularly the zdrv system - it seems I can't use it.

Any ideas?

User avatar
Dougal
Posts: 2502
Joined: Wed 19 Oct 2005, 13:06
Location: Hell more grotesque than any medieval woodcut

#2 Post by Dougal »

It is trivial once you have a kernel that's compiled with lzma support...

Then all you need to do is compile the squashfs utils with lzma support (or steal them from an older Pup) and remaster.
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

User avatar
Gn2
Posts: 943
Joined: Mon 16 Oct 2006, 05:33
Location: virtual - Veni vidi, nihil est adpulerit

#3 Post by Gn2 »

It is trivial once you have a kernel that's compiled with lzma support
http://www.digitalhermit.com/linux/Kern ... HOWTO.html
http://www.kernel.org
>> /usr/src/linux/(dot).config {kernel makefile)
Cd /path_to_Kernel_sources_and_.config
Edit makefile,save > re-compile kernel
cp bzImage to /compressed boot image storage locaation.
If needed, edit grub/lilo.conf
If Lilo loader used - re-run /sbin lilo -v

http://tree.celinuxforum.org/CelfPubWiki/SquashFs[code] ## Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
#
## Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
CONFIG_SQUASHFS=m
# CONFIG_SQUASHFS_EMBEDDED is not set
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
# CONFIG_SQUASHFS_VMALLOC is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set [/code]

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#4 Post by BarryK »

The squashfs part of the kernel source has to be patched.
I don't know where or if it has been done, but you will have to locate
a patch for the 2.6.18.1 kernel.


shankargopal
Posts: 295
Joined: Sat 03 Dec 2005, 11:30

#6 Post by shankargopal »

Thanks a lot for the responses. GIven the seeming complexity and time involved in patching and recompiling the kernel, I think that I'll pay the price in loss of flexibility and just expand the LZMA squashfs on to a hard drive, load up to 2.12, mount OpenOffice and remaster a 2.12 CD with OpenOffice - the simple solution I ought to have thought of a long long time ago :)).

It means I will lose some flexibility, but then I get an extra 100 megs of space on the pen drive :).

User avatar
Gn2
Posts: 943
Joined: Mon 16 Oct 2006, 05:33
Location: virtual - Veni vidi, nihil est adpulerit

#7 Post by Gn2 »

You are welcome - glad to be of any help.
According to Gentoo developers (VERY expert coders)
LZMA is only a bit tighter compresion than zlib -
Decompression carries overhead & time = they are not switching at this time.

However - IF smallest compressed size is ultimate goal -LZMA
produces smallest archives.

User avatar
Dougal
Posts: 2502
Joined: Wed 19 Oct 2005, 13:06
Location: Hell more grotesque than any medieval woodcut

#8 Post by Dougal »

The guy from Tukaani Linux actually created a lzma utility that seems to work about as good as gzip. What he seems to have done is create separate static libraries for the encoder and decoder.

I thought it might be possible to improve the speed of squashfs-lzma decompression by using his code rather than the original 7zip code, but I never found anyone to try implementing this… (I'm not a programmer)
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

User avatar
Gn2
Posts: 943
Joined: Mon 16 Oct 2006, 05:33
Location: virtual - Veni vidi, nihil est adpulerit

#9 Post by Gn2 »

Hi Dougal

Thnx - will have to check that one out ~ Actually, - I think LZMA will become the standard ?
Gentoo is probably very reluctant to change because it has such an impact on all their E-builds.

Hard enough they have now impemented the modularization of Xorg (and KDE builds will soon do same)

Those changes are a Great big Headache - as it blocks up-grades, necessitating a lot of extra work.
I am putting it off as long as possible.
NTIM > Here's a Partial list of VERY long output:
Calculating world dependencies ..... .. ... done!
[blocks B ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/kbproto-1.0.3)
[blocks B ] <=x11-base/xorg-x11-6.9 (is blocking x11-misc/util-macros-1.1.0)
[blocks B ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libX11-1.0.3)
[blocks B ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xextproto-7.0.2)
[blocks B ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86bigfontproto-1.1.2)

can8v
Posts: 586
Joined: Sat 15 Jul 2006, 08:20
Location: Yuba City, CA
Contact:

#10 Post by can8v »

shankargopal,
Very interesting thread. I especially enjoyed the part about the new LZMA utility. Thanks Dougal. My only contribution is that a 1GB pen drive can be had on ebay for less than $30 a 2GB goes for about $45 and for about $60 a 4GB pen drive can be had. Shipping to your location could drive the price a little higher, but I am thinking if you are using Puppy 10 hours a day 7 days a week that an upgrade at least to 512 MB may be worth while. I am a starving student and $50 is a lot of money to me, but that said, you sound really busy and your time might be worth the upgrade.
can8v

marksouth2000
Posts: 622
Joined: Wed 05 Apr 2006, 20:43

#11 Post by marksouth2000 »

To echo what can8v said, here in not-exactly-the-cheapest-place-to-live Switzerland I bought myself a new 1GB key yesterday for about CHF 25.- That's about US $20, although I'm sure http://www.xe.comwill be happy to supply more precision.

See it here: http://prodimex.ch/pinfos.asp?idxART=22 ... o=&SEARCH=

The packaging even said that it works with Linux! (Yes, I know they all do, but I prefer to buy brands that say so....)

Cheers,
Mark 8)

shankargopal
Posts: 295
Joined: Sat 03 Dec 2005, 11:30

#12 Post by shankargopal »

I would love to buy a new 1 GB pen drive, but I live in India and work as a rights activist (hence my love for flash Puppy - I don't have a computer of my own, just use many different ones), and this 256 meg one was itself a gift, so not really an option....

That said, by the way, the remastered 2.12 CD did not work. I got a message at bootup complaining that the pup_212.sfs could not be loaded - the error messages reported something like "tried to read beyond end of file". I suspect this either means that the CD was burned badly or that there wasn't enough memory.

The latter doesn't seem to make sense considering that puppy is supposed to not load the sfs file if there isn't enough memory... correct? So does that mean it must just be a CD error?

Post Reply