Kernel build script

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#21 Post by Iguleder »

It's easy to tweak it to work with .37.x, just change from "2.6.32.40" to "2.6.37.whatever" and put matching patches in the patches directory. The rest is fully automated.

Let me know if you need me to tweak the script, I can do the testing and take care of everything.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#22 Post by 01micko »

I reckon the kernel version could be passed as a commandline param.. that would nullify your little read at the start! :P (no param, no start)

As for patches.. well aufs gets the right version from git anyway, printk.c patch is basic arithmetic.. the stumbling block would be the aufs-allow-sfs.patch.. however the version for 2.6.32 seemed to work perfectly for 2.6.37.6. (I haven't tested outside of the full install as yet, so that's tentative :wink: ). Where does that patch come from anyway??? Is it a BK orig?

The vortex patch is machine specific and not needed for a main puppy kernel and probably (though I didn't test) wont work in newer kernels anyways. Perhaps a simple test there.

Ah.. then the config.. that would be daunting for a kernel noob (such as myself) .. so maybe a pause in the script so a config option could be run? Maybe based on a latish default DOTconfig? That way all the normal puppystuff is there.... and new stuff is tagged as (NEW) with a default recommendation (if using the commandline). The GUI version is super easy.. a dot (like a radiobutton) for module or a check for builtin. Just food for thought :) . Even just stop the script instead of erroring to alert the user that a valid DOTconfig is needed. It can be restarted just fine.
Puppy Linux Blog - contact me for access

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#23 Post by Iguleder »

I hand-crafted the aufs-allow-sfs.patch patch to be generic - the two lines it replaces are common to all Aufs branches, so it should work on any 2.x version, until the two lines it replaces are gone from Aufs.

The BFS patch and the vortex patch are Barry's. However, the bfs_cpufreq_tweaks patch is something I had to backport from later kernels, because Con Kolivas doesn't make one for 2.6.32.x.

Finally, regarding the .config - the defaults are really stupid in many cases, so it's impossible to make a .config generator. And the patches are suitable for a certain kernel version - you'll have to download patches that suit your kernel version manually, no matter what, that's why I don't add a command-line argument for the kernel version. Well, that's how I see it.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
pemasu
Posts: 5474
Joined: Wed 08 Jul 2009, 12:26
Location: Finland

#24 Post by pemasu »

About aufs-allow-sfs-as-branch patch. This may be relevant for it:
http://bkhome.org/blog/?viewDetailed=01470

User avatar
pemasu
Posts: 5474
Joined: Wed 08 Jul 2009, 12:26
Location: Finland

#25 Post by pemasu »

More info about the need of aufs-allow-sfs.patch

http://bkhome.org/blog/?viewDetailed=01731

Iguleders patch patches the loop.c from different row than Barrys patch and is hacked somehow also.

Iguleders patch:

Code: Select all

@@ -28,6 +28,9 @@
  */
 int au_test_loopback_overlap(struct super_block *sb, struct dentry *h_adding)
 {
+	/* allow Squashfs file systems to be used as Aufs branches */
+	return 0;
+	
 	struct super_block *h_sb;
 	struct loop_device *l;
My patch according to the Barrys patches:

Code: Select all

@@ -31,6 +31,7 @@ int au_test_loopback_overlap(struct supe
 	struct super_block *h_sb;
 	struct loop_device *l;
 
+    return 0;
 	h_sb = h_adding->d_sb;
 	if (MAJOR(h_sb->s_dev) != LOOP_MAJOR)
 		return 0;

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#26 Post by Iguleder »

There are two reasons why my patch is better (IMHO):
- If the lines Barry's patch relies on are gone in a future Aufs version, a new patch is needed. Mine relies only on two lines, which persist through the Aufs history.
- Two structs are allocated on the stack and the function returns 0 without using them ... so why allocate them? GCC won't compile the code after the "return 0" and the structs won't be allocated with my patch.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
pemasu
Posts: 5474
Joined: Wed 08 Jul 2009, 12:26
Location: Finland

#27 Post by pemasu »

Anyway. The kernel-kit works like a charm. It is great almost automatic batched kernel builder. And also removes the minor number.

Absolutely wonderful platform.

Keep them coming :)

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#28 Post by Iguleder »

I'm currently doing a test build of a new version of the script. I already mentioned it in Barry's blog and the spup thread.

It's a Python script which generates a Bash build script. It has the ability to deblob the kernel with the Linux-libre scripts and has many improvements.

Also, it's much tidier because I replaced some strings. For example, instead of using "linux_kernel-$kernel_version-$suffix" in many places, I created a variable which stores this string, so many lines of code are shorter and clearer.

Moreover, there's a small bonus ... it patches the kernel with a patch which adds a 224-color Puppy Linux logo. People who use framebuffer consoles with enough colors should see it 8)

I already built a 5.2.5 with a previous version, but it wasn't Linux-libre. The logo appeared and it worked nicely, just like the 2.6.32.40 built by the script.

Here's the output of the latest development version (Linux-libre 2.6.32.41 with BFS):

Code: Select all

#!/bin/sh

##########################################
#	Linux build script for Puppy Linux   #
##########################################

# the Linux version to build
LINUX_VERSION="2.6.32.41"

# the file name of the Linux sources tarball
LINUX_SOURCES_TARBALL="linux-2.6.32.41.tar.bz2"

# the Linux download mirror
LINUX_MIRROR="http://www.il.kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32"

# the Linux image and modules package name
LINUX_PACKAGE_NAME="linux_kernel"

# the Linux API headers package name
LINUX_HEADERS_PACKAGE_NAME="kernel_headers"

# the Linux sources package name
LINUX_SOURCES_PACKAGE_NAME="kernel_sources"

# the Aufs version to use
AUFS_VERSION="2.1"

# get the Linux major version number
LINUX_MAJOR_VERSION=`echo $LINUX_VERSION | cut -f 1-3 -d .`

# get the Linux release number
LINUX_RELEASE_NUMBER=`echo $LINUX_VERSION | cut -f 3 -d .`

# get the Linux minor version number
LINUX_MINOR_VERSION=`echo $LINUX_VERSION | cut -f 4 -d .`

# the package name suffix
PACKAGE_NAME_SUFFIX="${LINUX_MINOR_VERSION}-lts-libre-bfs"

# the Linux-libre scripts to use
LINUX_LIBRE_SCRIPTS="deblob-${LINUX_MAJOR_VERSION} deblob-check"

# the Linux-libre mirror to download the scripts from
LINUX_LIBRE_MIRROR="http://www.fsfla.org/svn/fsfla/software/linux-libre/scripts"

# get the build date
BUILD_DATE=`date +%d%m%Y`

# the absolute log file path									
LOG_PATH=`pwd`/build.log

# delete old logs
[ -f "$LOG_PATH" ] && rm -f "$LOG_PATH"

# create directories for the output
mkdir -p dist/sources/{patches,vanilla}

# download the Linux sources tarball
if [ ! -f dist/sources/vanilla/$LINUX_SOURCES_TARBALL ]; then
	echo "Downloading the Linux sources tarball"
	wget -P dist/sources/vanilla $LINUX_MIRROR/$LINUX_SOURCES_TARBALL > "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Linux sources tarball."
		exit 1
	fi
fi

# check whether the Linux-libre scripts are missing
download=0
for script in $LINUX_LIBRE_SCRIPTS
do
	if [ ! -f dist/sources/vanilla/$script ]
	then
		download=1
		break
	fi
done

# if at least one script is missing, download it
if [ $download -eq 1 ]
then
	echo "Downloading the Linux-libre deblob scripts"
	for script in $LINUX_LIBRE_SCRIPTS
	do
		if [ ! -f dist/sources/vanilla/$script ]
		then
			wget -P dist/sources/vanilla $LINUX_LIBRE_MIRROR/$script >> build.log 2>&1
			if [ $? -ne 0 ]
			then
				[ -f dist/sources/vanilla/$script ] && rm -f dist/sources/vanilla/$script
				echo "Error: failed to download the $script script."
				exit 1
			fi
		fi
	done	
fi

# get the full Aufs package name
AUFS_FULL_PACKAGE_NAME=aufs${AUFS_VERSION}-${LINUX_RELEASE_NUMBER}-git${BUILD_DATE}

# download the Aufs sources through git
if [ ! -f dist/sources/vanilla/${AUFS_FULL_PACKAGE_NAME}.tar.bz2 ]
then
	echo "Downloading the Aufs sources through git"
	git clone http://git.c3sl.ufpr.br/pub/scm/aufs/aufs2-standalone.git $AUFS_FULL_PACKAGE_NAME >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Aufs sources through git."
		exit 1
	fi
	cd $AUFS_FULL_PACKAGE_NAME
	git checkout origin/aufs${AUFS_VERSION}-${LINUX_RELEASE_NUMBER} >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Aufs sources through git."
		exit 1
	fi
	rm -rf .git
	cd ..
	echo "Creating the Aufs sources tarball"
	tar -c $AUFS_FULL_PACKAGE_NAME | bzip2 -9 > dist/sources/vanilla/${AUFS_FULL_PACKAGE_NAME}.tar.bz2
else
	echo "Extracting the Aufs sources tarball"
	tar xf dist/sources/vanilla/${AUFS_FULL_PACKAGE_NAME}.tar.bz2 >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to extract the Aufs sources tarball."
		exit 1
	fi
fi

# patch Aufs
echo "Patching the Aufs sources"
patch -d $AUFS_FULL_PACKAGE_NAME -p1 < aufs-allow-sfs.patch >> "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to patch the Aufs sources."
	exit 1
fi
cp aufs-allow-sfs.patch dist/sources/patches

# extract the Linux sources tarball
echo "Extracting the Linux sources tarball"
tar xf dist/sources/vanilla/$LINUX_SOURCES_TARBALL >> "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to extract the Linux sources tarball."
	exit 1
fi

echo "Deblobing the Linux sources"
echo "  creating a clean copy of the sources"
cp -r linux-${LINUX_VERSION} linux-${LINUX_VERSION}-orig
for script in $LINUX_LIBRE_SCRIPTS
do
	chmod +x dist/sources/vanilla/$script
done

echo "  deblobing"
cd linux-${LINUX_VERSION}
../dist/sources/vanilla/deblob-${LINUX_MAJOR_VERSION} > "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to deblob the Linux sources."
	exit 1
fi

echo "  creating a patch"
cd ..
diff -rupN linux-${LINUX_VERSION}-orig linux-${LINUX_VERSION} > dist/sources/patches/deblob.patch
rm -rf linux-%{LINUX_VERSION}-orig

mkdir dist/packages

cd linux-${LINUX_VERSION}

echo "Cleaning the Linux sources"
make clean >> "$LOG_PATH" 2>&1
make mrproper >> "$LOG_PATH" 2>&1

echo "Creating the Linux API headers package"
echo "  checking the headers"
make headers_check >> "$LOG_PATH" 2>&1
LINUX_HEADERS_FULL_PACKAGE_NAME=${LINUX_HEADERS_PACKAGE_NAME}-${LINUX_MAJOR_VERSION}-${PACKAGE_NAME_SUFFIX}
echo "  installing the headers"
make INSTALL_HDR_PATH=$LINUX_HEADERS_FULL_PACKAGE_NAME headers_install >> "$LOG_PATH" 2>&1
echo "  removing unneeded files"
find $LINUX_HEADERS_FULL_PACKAGE_NAME/include \( -name .install -o -name ..install.cmd \) -delete
mv $LINUX_HEADERS_FULL_PACKAGE_NAME ../dist/packages

# patch the Linux sources with the Aufs patches and add the file system code itself
echo "Adding Aufs to the Linux sources"
for i in kbuild base standalone; do
	patch -p1 < ../$AUFS_FULL_PACKAGE_NAME/aufs2-$i.patch >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to add Aufs to the Linux sources."
		exit 1
	fi
done
cp -r ../$AUFS_FULL_PACKAGE_NAME/{fs,Documentation} .
cp -r ../$AUFS_FULL_PACKAGE_NAME/include/linux/aufs_type.h include/linux
cat ../$AUFS_FULL_PACKAGE_NAME/include/linux/Kbuild >> include/linux/Kbuild

echo "Resetting the minor version number"
cp Makefile Makefile-orig
sed -i s/'^EXTRAVERSION = .*$'/'EXTRAVERSION = '/ Makefile
echo "  creating a patch"
diff -up Makefile-orig Makefile > ../dist/sources/patches/extra-version.patch
rm Makefile-orig

echo "Reducing the number of consoles"
cp kernel/printk.c kernel/printk.c-orig
sed -i s/'#define MAX_CMDLINECONSOLES 8'/'#define MAX_CMDLINECONSOLES 5'/ kernel/printk.c
echo "  creating a patch"
diff -up kernel/printk.c-orig kernel/printk.c > ../dist/sources/patches/less-consoles.patch

echo "Reducing the verbosity level"
cp -f kernel/printk.c kernel/printk.c-orig
sed -i s/'#define DEFAULT_CONSOLE_LOGLEVEL 7 \/\* anything MORE serious than KERN_DEBUG \*\/'/'#define DEFAULT_CONSOLE_LOGLEVEL 3 \/\* anything MORE serious than KERN_ERR \*\/'/ kernel/printk.c
echo "  creating a patch"
diff -up kernel/printk.c-orig kernel/printk.c > ../dist/sources/patches/lower-verbosity.patch

echo "Applying patches"
for patch in ../patches/*
do
	echo "  applying $patch"
	patch -p1 < $patch >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to apply $patch on the Linux sources."
		exit 1
	fi
	cp $patch ../dist/sources/patches
done

echo "  cleaning up patch clutter"
find . -name '*.orig' -delete
find . -name '*.rej' -delete
find . -name '*~' -delete

echo "Building Linux"
cp ../DOTconfig .config
make bzImage modules >> "$LOG_PATH" 2>&1
if [[ ! -f arch/x86/boot/bzImage || ! -f System.map ]]; then
	echo "Error: failed to build Linux."
	exit 1
fi
cp .config ../dist/sources/DOTconfig-${LINUX_VERSION}-${BUILD_DATE}

echo "Creating the Linux image and modules package"
LINUX_FULL_PACKAGE_NAME=${LINUX_PACKAGE_NAME}-${LINUX_RELEASE_NUMBER}-${PACKAGE_NAME_SUFFIX}
echo "  installing the kernel modules"
make INSTALL_MOD_PATH=$LINUX_FULL_PACKAGE_NAME modules_install
echo "  removing firmware"
rm -rf $LINUX_FULL_PACKAGE_NAME/lib/firmware
echo "  removing links to the sources"
rm -f $LINUX_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/{build,source}
ln -s $LINUX_MAJOR_VERSION $LINUX_FULL_PACKAGE_NAME/lib/modules/$LINUX_VERSION
echo "  stripping modules"
for module in `find $LINUX_FULL_PACKAGE_NAME -name *.ko -type f`
do
	strip --strip-unneeded $module > "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to strip the Linux modules."
		exit 1
	fi
done
echo "  adding the kernel image and System.map"
mkdir $LINUX_FULL_PACKAGE_NAME/boot
cp `find arch -name bzImage -type f` $LINUX_FULL_PACKAGE_NAME/boot/vmlinuz
cp System.map $LINUX_FULL_PACKAGE_NAME/boot
mv $LINUX_FULL_PACKAGE_NAME ../dist/packages

echo "Cleaning the Linux sources"
make clean >> "$LOG_PATH" 2>&1
make prepare >> "$LOG_PATH" 2>&1

cd ..

echo "Creating a Linux sources SFS extension"
LINUX_SOURCES_FULL_PACKAGE_NAME=${LINUX_SOURCES_PACKAGE_NAME}-${LINUX_RELEASE_NUMBER}-${PACKAGE_NAME_SUFFIX}
mkdir -p $LINUX_SOURCES_FULL_PACKAGE_NAME/usr/src
mv linux-${LINUX_VERSION} $LINUX_SOURCES_FULL_PACKAGE_NAME/usr/src/linux
mkdir -p $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION
echo "  creating required links"
ln -s /usr/src/linux $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/build
ln -s /usr/src/linux $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/source
mksquashfs $LINUX_SOURCES_FULL_PACKAGE_NAME dist/sources/$LINUX_SOURCES_FULL_PACKAGE_NAME.sfs

echo "Compressing the log"
bzip2 -9 build.log 
cp build.log.bz2 dist/sources

echo "Done!"
aplay /usr/share/sounds/2barks.au
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#29 Post by Iguleder »

Here it is, the new kernel building kit. Much improved!

Features:
- A Python script which generates a kernel build script, more dynamic, more programmable.
- Support for Linux-libre deblobbing for free software lovers
- Support for firmware removal
- Patches the kernel for compatibility with all bug-fix releases of the same stable release (e.g all 2.6.32.x kernels are named "2.6.32")
- Automatically reduces the number of consoles and the kernel verbosity, to save memory and additional gray hair
- Automatically patches the kernel with a Puppy Linux logo
- Builds kernel headers and sources packages
- Strips kernel modules to reduce size
- Better!

How to use it?
- Create a directory named after the kernel branch you use (e.g 2.6.35 for 2.6.35.13) and a configuration file named DOTconfig-$kernel_version. See the 2.6.32 example.
- Run create_kit.sh and put the desired kernel version and download mirror.
- ROX will open in the output directory. Take all files from there to your desired build location and run build.sh.

Simple as A-B-C. For Linux 2.6.32.x you can skip the first step, because the kit already has the patches and the configuration file. I want to add another kernel to the kit - the more kernels I add, the easier things get. This mechanism allows Puppy kernel guys to build multiple kernel versions and track down changes.

The deblobbing needs some testing - the latest deblob-2.6.32 script broke 2.6.32.41 for some reason. It was some syntax error in a driver which got stripped by it. Also, the package creation part needs testing because so far it failed again and again as I added the more advanced features. However, an initial build of this kit produced a really solid kernel.

Here's the build script (the latest one, using it right now - it's a traditional, non-libre build with firmware) produced by it:

Code: Select all

#!/bin/sh

######################################
# Linux build script for Puppy Linux #
######################################

################################
# Linux and Aufs configuration #
################################

# the Linux version to build
LINUX_VERSION="2.6.32.41"

# the file name of the Linux sources tarball
LINUX_SOURCES_TARBALL="linux-2.6.32.41.tar.bz2"

# the Linux download mirror
LINUX_MIRROR="http://www.il.kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32"

# the Aufs version to use
AUFS_VERSION="2.1"

################################
# Linux version number parsing #
################################

# get the Linux major version number
LINUX_MAJOR_VERSION=`echo $LINUX_VERSION | cut -f 1-3 -d .`

# get the Linux release number
LINUX_RELEASE_NUMBER=`echo $LINUX_VERSION | cut -f 3 -d .`

# get the Linux minor version number
LINUX_MINOR_VERSION=`echo $LINUX_VERSION | cut -f 4 -d .`

#########################
# package configuration #
#########################

# the Linux image and modules package name
LINUX_PACKAGE_NAME="linux_kernel"

# the Linux API headers package name
LINUX_HEADERS_PACKAGE_NAME="kernel_headers"

# the Linux sources package name
LINUX_SOURCES_PACKAGE_NAME="kernel_sources"

# the package name suffix
PACKAGE_NAME_SUFFIX="${LINUX_MINOR_VERSION}-lts-bfs"

#############
# boot logo #
#############

# the PNG image used as the boot logo
LOGO_IMAGE="/usr/share/doc/puppylogo96.png"

# the boot logo width
LOGO_WIDTH=80

# the boot logo height
LOGO_HEIGHT=80

##############################
# Output paths configuration #
##############################

# the root build directory
BUILD_ROOT=`pwd`

# the output directory
OUTPUT_DIR="$BUILD_ROOT/dist"

# the directory packages go to
PACKAGES_DIR="$OUTPUT_DIR/packages"

# the directory sources go to
SOURCES_DIR="$OUTPUT_DIR/sources"

# the directory patches go to
PATCHES_DIR="$OUTPUT_DIR/sources/patches"

# the log path								
LOG_PATH="$BUILD_ROOT/build.log"

# get the build date
BUILD_DATE="`date +%d%m%Y`"

####################

# delete old logs
[ -f "$LOG_PATH" ] && rm -f "$LOG_PATH"

# create directories for the output
mkdir -p "$SOURCES_DIR" "$PATCHES_DIR" "$PACKAGES_DIR"

# download the Linux sources tarball
if [ ! -f "$SOURCES_DIR/$LINUX_SOURCES_TARBALL" ]; then
	echo "Downloading the Linux sources tarball"
	wget -P "$SOURCES_DIR" $LINUX_MIRROR/$LINUX_SOURCES_TARBALL > "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Linux sources tarball."
		exit 1
	fi
fi

# get the full Aufs package name
AUFS_FULL_PACKAGE_NAME="aufs${AUFS_VERSION}-${LINUX_RELEASE_NUMBER}-git${BUILD_DATE}"

# download the Aufs sources through git
if [ ! -f "$SOURCES_DIR/${AUFS_FULL_PACKAGE_NAME}.tar.bz2" ]
then
	echo "Downloading the Aufs sources through git"
	git clone http://git.c3sl.ufpr.br/pub/scm/aufs/aufs2-standalone.git $AUFS_FULL_PACKAGE_NAME >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Aufs sources through git."
		exit 1
	fi
	cd $AUFS_FULL_PACKAGE_NAME
	git checkout origin/aufs${AUFS_VERSION}-${LINUX_RELEASE_NUMBER} >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to download the Aufs sources through git."
		exit 1
	fi
	rm -rf .git
	cd ..
	echo "Creating the Aufs sources tarball"
	tar -c $AUFS_FULL_PACKAGE_NAME | bzip2 -9 > "$SOURCES_DIR/${AUFS_FULL_PACKAGE_NAME}.tar.bz2"
else
	echo "Extracting the Aufs sources tarball"
	tar xf "$SOURCES_DIR/${AUFS_FULL_PACKAGE_NAME}.tar.bz2" >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to extract the Aufs sources tarball."
		exit 1
	fi
fi

# patch Aufs
echo "Patching the Aufs sources"
patch -d $AUFS_FULL_PACKAGE_NAME -p1 < aufs-allow-sfs.patch >> "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to patch the Aufs sources."
	exit 1
fi
cp aufs-allow-sfs.patch "$PATCHES_DIR"

# extract the Linux sources tarball
echo "Extracting the Linux sources tarball"
tar xf "$SOURCES_DIR/$LINUX_SOURCES_TARBALL" >> "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to extract the Linux sources tarball."
	exit 1
fi


cd linux-${LINUX_VERSION}

echo "Cleaning the Linux sources"
make clean >> "$LOG_PATH" 2>&1
make mrproper >> "$LOG_PATH" 2>&1

# get the full Linux API headers package name
LINUX_HEADERS_FULL_PACKAGE_NAME="${LINUX_HEADERS_PACKAGE_NAME}-${LINUX_MAJOR_VERSION}-${PACKAGE_NAME_SUFFIX}"

echo "Creating the Linux API headers package"
echo "  checking the headers"
make headers_check >> "$LOG_PATH" 2>&1
echo "  installing the headers"
make INSTALL_HDR_PATH=$LINUX_HEADERS_FULL_PACKAGE_NAME headers_install >> "$LOG_PATH" 2>&1
echo "  removing unneeded files"
find $LINUX_HEADERS_FULL_PACKAGE_NAME/include \( -name .install -o -name ..install.cmd \) -delete
mv $LINUX_HEADERS_FULL_PACKAGE_NAME "$PACKAGES_DIR"

# patch the Linux sources with the Aufs patches and add the file system code itself
echo "Adding Aufs to the Linux sources"
for i in kbuild base standalone; do
	patch -p1 < ../$AUFS_FULL_PACKAGE_NAME/aufs2-$i.patch >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to add Aufs to the Linux sources."
		exit 1
	fi
done
cp -r ../$AUFS_FULL_PACKAGE_NAME/{fs,Documentation} .
cp -r ../$AUFS_FULL_PACKAGE_NAME/include/linux/aufs_type.h include/linux
cat ../$AUFS_FULL_PACKAGE_NAME/include/linux/Kbuild >> include/linux/Kbuild

echo "Resetting the minor version number"
cp Makefile Makefile-orig
sed -i s/'^EXTRAVERSION = .*$'/'EXTRAVERSION = '/ Makefile
echo "  creating a patch"
diff -up Makefile-orig Makefile > "$PATCHES_DIR/extra-version.patch"
rm Makefile-orig

echo "Reducing the number of consoles"
cp kernel/printk.c kernel/printk.c-orig
sed -i s/'#define MAX_CMDLINECONSOLES 8'/'#define MAX_CMDLINECONSOLES 5'/ kernel/printk.c
echo "  creating a patch"
diff -up kernel/printk.c-orig kernel/printk.c > "$PATCHES_DIR/less-consoles.patch"

echo "Reducing the verbosity level"
cp -f kernel/printk.c kernel/printk.c-orig
sed -i s/'#define DEFAULT_CONSOLE_LOGLEVEL 7 \/\* anything MORE serious than KERN_DEBUG \*\/'/'#define DEFAULT_CONSOLE_LOGLEVEL 3 \/\* anything MORE serious than KERN_ERR \*\/'/ kernel/printk.c
echo "  creating a patch"
diff -up kernel/printk.c-orig kernel/printk.c > "$PATCHES_DIR/lower-verbosity.patch"

echo "Applying patches"
for patch in ../patches/*
do
	echo "  applying `basename $patch`"
	patch -p1 < "$patch" >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to apply $patch on the Linux sources."
		exit 1
	fi
	cp "$patch" "$PATCHES_DIR"
done

echo "Adding the Puppy Linux boot logo"
echo "  creating a clean copy of the sources"
cp -r drivers/video/logo drivers/video/logo-orig
echo "  applying the Puppy Linux logo patch"
patch -p1 < ../puppy-logo.patch >> "$LOG_PATH" 2>&1
if [ $? -ne 0 ]
then
	echo "Error: failed to apply the Puppy Linux logo patch on the Linux sources."
	exit 1
fi

# resize the image and make the background black	
resized_image=`mktemp -u`
pngtopnm -mix -background black "$LOGO_IMAGE" | pnmscale -width $LOGO_WIDTH -height $LOGO_HEIGHT > $resized_image
if [ $? -ne 0 ]
then
	echo "Error: failed to resize ${LOGO_IMAGE}."
	exit 1
fi

# create 224-color, 16-color and monochrome variants
echo "  creating 224-color, 16-color and monochrome variants"
cat $resized_image | ppmquant 224 | pnmnoraw > drivers/video/logo/logo_puppy_clut224.ppm
cat $resized_image | ppmquant 16 | pnmnoraw > drivers/video/logo/logo_puppy_vga16.ppm
cat $resized_image | ppmquant 2 | pnmnoraw > drivers/video/logo/logo_puppy_mono.ppm

# delete the resized image
rm -f $resized_image

cd ..
echo "  creating a patch"
diff -rupN linux-${LINUX_VERSION}/drivers/video/logo-orig linux-${LINUX_VERSION}/drivers/video/logo > "$PATCHES_DIR/puppy-logo.patch"
rm -rf linux-${LINUX_VERSION}/drivers/video/logo-orig
cd linux-${LINUX_VERSION}

echo "  cleaning up patch clutter"
find . -name '*.orig' -delete
find . -name '*.rej' -delete
find . -name '*~' -delete

echo "Building Linux"
cp ../DOTconfig .config
make bzImage modules >> "$LOG_PATH" 2>&1
if [[ ! -f arch/x86/boot/bzImage || ! -f System.map ]]; then
	echo "Error: failed to build Linux."
	exit 1
fi
cp .config "$SOURCES_DIR/DOTconfig-${LINUX_VERSION}-${BUILD_DATE}"

# get the full Linux image and modules package name
LINUX_FULL_PACKAGE_NAME="${LINUX_PACKAGE_NAME}-${LINUX_RELEASE_NUMBER}-${PACKAGE_NAME_SUFFIX}"

echo "Creating the Linux image and modules package"
echo "  installing the kernel modules"
make INSTALL_MOD_PATH=$LINUX_FULL_PACKAGE_NAME modules_install
echo "  removing links to the sources"
rm -f $LINUX_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/{build,source}
ln -s $LINUX_MAJOR_VERSION $LINUX_FULL_PACKAGE_NAME/lib/modules/$LINUX_VERSION
echo "  stripping modules"
for module in `find $LINUX_FULL_PACKAGE_NAME -name *.ko -type f`
do
	strip --strip-unneeded $module >> "$LOG_PATH" 2>&1
	if [ $? -ne 0 ]
	then
		echo "Error: failed to strip the Linux modules."
		exit 1
	fi
done
echo "  adding the kernel image and System.map"
mkdir $LINUX_FULL_PACKAGE_NAME/boot
cp `find arch -name bzImage -type f` $LINUX_FULL_PACKAGE_NAME/boot/vmlinuz
cp System.map $LINUX_FULL_PACKAGE_NAME/boot
mv $LINUX_FULL_PACKAGE_NAME "$PACKAGES_DIR"

echo "Cleaning the Linux sources"
make clean >> "$LOG_PATH" 2>&1
make prepare >> "$LOG_PATH" 2>&1

cd ..

# get the full Linux sources package name
LINUX_SOURCES_FULL_PACKAGE_NAME="${LINUX_SOURCES_PACKAGE_NAME}-${LINUX_RELEASE_NUMBER}-${PACKAGE_NAME_SUFFIX}"

echo "Creating a Linux sources SFS extension"
mkdir -p $LINUX_SOURCES_FULL_PACKAGE_NAME/usr/src
mv linux-${LINUX_VERSION} $LINUX_SOURCES_FULL_PACKAGE_NAME/usr/src/linux
mkdir -p $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION
echo "  creating required links"
ln -s /usr/src/linux $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/build
ln -s /usr/src/linux $LINUX_SOURCES_FULL_PACKAGE_NAME/lib/modules/$LINUX_MAJOR_VERSION/source
mv $LINUX_SOURCES_FULL_PACKAGE_NAME "$PACKAGES_DIR"

echo "Compressing the log"
bzip2 -9 "$LOG_PATH" 
cp "${LOG_PATH}.bz2" "$SOURCES_DIR"

echo "Compressing the patches"
for patch in "$PATCHES_DIR"/*
do
	bzip2 -9 "$patch"
done

echo "Done!"
aplay /usr/share/sounds/2barks.au
EDIT: forgot to mention - it won't support .0 kernels (e.g 2.6.32, 2.6.33, 2.6.34), only stable releases. That's a very logical assumption, since Puppy is a "just works" distro.
Attachments
kernel-kit-001.tar.bz2
(115.97 KiB) Downloaded 467 times
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

majorfoo
Posts: 448
Joined: Mon 07 Mar 2011, 22:27
Location: Wish I knew

#30 Post by majorfoo »

James C wrote:
stu90 wrote:Here is my lucid 525 EZ-woof build.

Full Lucid Puppy with updated:
kernel - 2.6.32-40-lts
flash
gnome-mplayer
geko-mediaplayer
mtpaint
lupu-update
mint openbox theme
changed icon set and wallpaper

(screen shot in my post above)
http://smokey01.com/stu90/lupu-540.iso
md5: 45c80dd429d6839875dfea8bf4febd14
Since I can't find a thread for this I'll make a quick post here. :) Did a frugal install on the old P4 test box and everything appears to be working correctly ootb.
Stu - could you create a thread specific to lupu-540?

looks good so far -booted straight to desktop. sound working properly. added seamonkey 2.0.11, roxterm-1.17.1, deadbeef-0.42, abiword-2.90

only concern so far is networking did not autostart as it did in lupu 525
will continue testing. want to do a remaster after i load some more applications and do a full install - will advise later

thanks - majorfoo

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#31 Post by Iguleder »

Just compiled linux_kernel-2.6.32-41-lts. Uploading to my repo.

Haven't tested it yet; this one was built on a Debian Wheezy chroot with GCC 4.5.3 and has BFS 404 and a Puppy boot logo for framebuffer consoles.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
dejan555
Posts: 2798
Joined: Sun 30 Nov 2008, 11:57
Location: Montenegro
Contact:

#32 Post by dejan555 »

Sounds good Iguleder, I will try to build lts for dpup485, so I see script also downloads kernel and does everything auto.
How much space is required, after devx is loaded, to download and compile/package the kernel?
I get about 700+ MB free RAM when I load devx so I'd compile it in ram if possible
puppy.b0x.me stuff mirrored [url=https://drive.google.com/open?id=0B_Mb589v0iCXNnhSZWRwd3R2UWs]HERE[/url] or [url=http://archive.org/details/Puppy_Linux_puppy.b0x.me_mirror]HERE[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#33 Post by Iguleder »

Quite a lot ... I suggest you to have at least 1 GB. The kernel sources are several hundreds of MBs.

I used to do that in RAM on the PC with 2 GB of RAM, but now I do it on the netbook and it has just 1 GB, so I run it on a partition.

By the way - once the upload is over, I'll upload the build scripts. My wireless connection keeps disconnecting so I can't upload the cool stuff in parallel :lol:

btw: the only thing the script doesn't do is the actual package creation. It doesn't run dir2pet and mksquashfs; it just creates the packages in their extracted form.

btw II: you can use a chroot environment to run the script. This one (the .41) was built in a Debian Wheezy chroot environment with GCC 4.5.3, with Debian's goodness atop - the result is a 2 MB smaller package, compared to the .40 I built under 5.2.5. However, this kernel has been around for a long time, so you should use either 4.4.x or 4.5.x with it ... 4.6.0 has regressions and there is a possibility the kernel won't build with it.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
dejan555
Posts: 2798
Joined: Sun 30 Nov 2008, 11:57
Location: Montenegro
Contact:

#34 Post by dejan555 »

Haven't used chroot enviroments yet. Could it be compiled with dpup's devx?
puppy.b0x.me stuff mirrored [url=https://drive.google.com/open?id=0B_Mb589v0iCXNnhSZWRwd3R2UWs]HERE[/url] or [url=http://archive.org/details/Puppy_Linux_puppy.b0x.me_mirror]HERE[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#35 Post by Iguleder »

Yep, of course. Doing it the chroot just makes it ... more geeky :p

Btw - I uploaded the build script with all patches, etc' - here.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
dejan555
Posts: 2798
Joined: Sun 30 Nov 2008, 11:57
Location: Montenegro
Contact:

#36 Post by dejan555 »

Ok, cool I tried your latest script and built kernel - works like a charm.
Booted in ram and loaded devx, then ran build.sh from partition.

Afterwards I replaced vmlinuz, unpack/resquashed initrd.gz and sfs with new modules and it booted OK.
I had to manually modprobe PS/2 mouse to work though :?
What could be the problem there, I tried adding input drivers including psmouse.ko to initrd.gz but it wouldn't work on boot anyway.
Net, sound and video modules work ok as well as usb mouse, just PS/2 mouse remains frozen.
I can add modprobe psmouse line to some of rc scripts maybe?
puppy.b0x.me stuff mirrored [url=https://drive.google.com/open?id=0B_Mb589v0iCXNnhSZWRwd3R2UWs]HERE[/url] or [url=http://archive.org/details/Puppy_Linux_puppy.b0x.me_mirror]HERE[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#37 Post by Iguleder »

I think I saw some line which says that Barry made that module built-in since some kernel.

Do a diff between your rc.sysint or the initramfs' init to see the difference. I'm almost certain you need that line for kernel older than some kernel, I think it was the 4.3.1 one.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#38 Post by Iguleder »

Just built linux_kernel-2.6.32-42-lts; uploading it right now. I haven't tested it yet but it should work ... like the previous ones :D

The directory contains the sources (plus patches, Aufs, etc'), the kernel package, the kernel headers package, the kernel sources SFS and the build script. Have fun :)

There's also a build log, for the skeptic :lol:
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#39 Post by sc0ttman »

Could your kernel build scripts be added to your Pur-tools thing?
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#40 Post by Iguleder »

Theoretically, yes - but it's a very bad idea, for two reasons:
1) You can't install a kernel in Puppy, you'll have to remaster or rebuild your Puppy for that.
2) The kernel is a very sensitive component - it deserves its own build system.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

Post Reply