How to compile Linux kernel

This tutorial is intended to help students to install the tools and the adequate environment to build the Linux kernel. In addition, it provides the steps to compile the Linux kernel.

Tip

Please have a first look on the slides before doing the tutorial.

There exist several ways to compile the Linux kernel. In this tutorial, we will explain 2 different ways:

  1. Compiling the kernel on NON-reference image: The first way requires two Linux machines. One is the bare metal OS and is used for kernel compilation. The other is a VirtualBox VM which contains a reference image and which is used for kernel testing. For this case, see the conventions before starting the tutorial.

  2. Compiling the kernel on the reference image: The other way is to work directly on the reference image within a virtual machine (VM). You will compile and test/install the kernel in the same environment (if you use qemu on macOS, compiling can take a lot of time…).

Danger

Normally, you should have already configured Virtual Box with the reference VM, but here is a small reminder. The new kernel will be tested with VirtualBox. The virtual machine is Ubuntu 32 bits. The reference image can be downloaded here. Only this image, which is a 32 bits Ubuntu, must be used for testing. The user is student and the password is student.

Method1: Compiling the kernel on NON-reference image

In this method, some commands are entered in the host machine, and others in the virtual machine. The conventions are the following:

host$ command # in the terminal of the host machine
virtual$ command # in the terminal of the virtual machine

Download the kernel sources

The first step is to download and extract the kernel 4.15 sources in your host machine by typing the following commands:

host$ mkdir /home/your_username/OS
host$ cd /home/your_username/OS
host$ sudo apt install xz-utils
host$ wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.15.tar.xz
host$ tar -Jxvf linux*
host$ cd linux-4.15

Configure a Git folder

In order to generate/apply patches, you need to save the initial state of this folder using Git. Enter the following commands to do it:

host$ cd linux-4.15
host$ touch .scmversion
host$ git init
host$ git config --global user.name "Your Name" # Configuration
host$ git config --global user.email "you@example.com" # Configuration host$ git init
host$ git add .
host$ git commit -m "first commit" # Saving the initial state
host$ git log

Installing the requirements

In order to compile the kernel, we’ll need to first install a few requirements. This can be done with a single command:

host$ sudo apt install build-essential libncurses5-dev libssl-dev flex libelf-dev bc bison gcc make bc

Configuring the kernel

Before we actually compile the kernel, we must first configure it. The Linux kernel configuration is usually found in the kernel source in the .config file. In that case, we already generated the righ configuration in order to save a lot of time. Enter the following commands in order to configure your kernel:

host$ wget --no-check-certificate https://people.montefiore.uliege.be/gain/courses/info0940/asset/linux-4.15-32bit-config
host$ mv linux-4.15-32bit-config .config
host$ make olddefconfig

Build the kernel

Now it’s time to actually compile the kernel. Use the make command. Specify the number of threads (-j option) and the architecture (32bits). The -j option is only use to speed up application build, it determines how many jobs can spawn for the build. You can either set -j <nb_cores> or use -j $(nproc) in order that the compilation can happen in parallel. Enter the following commands in your kernel folder:

host$ make ARCH=i386 bzImage -j$(nproc)

Then, compile the modules (you’ll need to compile them again only if you modify files chosen to be compiled as modules):

host$ make ARCH=i386 modules -j$(nproc)

Danger

On some recent (host) systems, you may have an error relative to address family and seccmap (#error New address family defined, please update secclass_map). If this is the case, please perform the following procedure below.

host$ cd /home/username/OS/linux-4.15
host$ wget --no-check-certificate http://www.montefiore.ulg.ac.be/~gain/courses/info0940/asset/fix_secclass.sh
host$ chmod +x fix_secclass.sh
host$ ./fix_secclass.sh

And then, run make again to compile your kernel (see steps above).

Installing the kernel

Danger

Do not enter make install && make modules_install command(s) on your host machine!!!

If you type the normal make install && make modules_install, it will install it in your host! Instead, we will install the files in a temporary directory so you can copy them to the virtual machine after that:

host$ mkdir /home/username/OS/boot
host$ export INSTALL_PATH=/home/username/OS/boot
host$ mkdir /home/username/OS/mods
host$ export INSTALL_MOD_PATH=/home/username/OS/mods
host$ make modules_install
host$ make install

Once the kernel and its modules are compiled and installed, you can copy them to the virtual machine. We will use the rsync command to transfer the files from the host to the virtual machine. Enter the following commands in your host machine:

host$ cd /home/username/OS/
host$ rsync -vazr -e 'ssh -p 6543' mods/ student@localhost:/tmp/mods/
host$ rsync -vazr -e 'ssh -p 6543' boot/ student@localhost:/tmp/boot/

These two commands transfer the entire mods/ and boot/ folders, which will be temporarily stored in the /tmp folder on the reference VM. After transferring the kernel image and the modules from your host to the reference VM (/tmp), they need to be copied to their appropriate locations respectively. The kernel image is copied to the /boot directory, and the modules are copied to the root directory. The -rf option is used to copy the entire content of the folders.

virtual$ cd /home/student/OS
virtual$ sudo cp -rf /tmp/boot/* /boot
virtual$ sudo cp -rf /tmp/mods/* /

After copying the files, you need to update the initramfs and the grub configuration. The update-initramfs command is used to update the initial ramdisk, and the update-grub command is used to update the grub configuration. Finally, you need to reboot your virtual machine to load the new kernel.

virtual$ sudo update-initramfs -u -k 4.15.0
virtual$ sudo update-grub
virtual$ sudo reboot

Checking your current version

In order to check if the new version of your kernel is correctly installed, use the uname command

virtual$ uname -a
Linux student 4.15.0 #1 SMP Tue Jan 29 09:41:40 JST 2020 i686i686 i686 GNU/Linux

If you see a “+” at the end of the version line (e.g., 4.15+), make sure you have an empty .scmversion file in the root of the kernel sources. If you do not have this file, enter the following commands:

host$ cd /home/username/OS/linux-4.15
host$ touch .scmversion
host$ make ARCH=i386 bzImage -j$(nproc)
host$ make ARCH=i386 modules -j$(nproc)
host$ make modules_install
host$ make install

If you still see kernel version 4.4, please choose 4.15 from the grub menu which is the black screen at the boot phase.

Method2: Compiling the kernel on the reference image

Important

This method is globally quite similar to the previous one except that you must execute all commands on the virtual machine. Therefore, the rsync/sshfs sections must be skipped. In addition, when installing the kernel, you just need to execute make install on the virtual machine (you must skip the copy with sshfs). We have written a small script which does all the job. You can refer to the slides if you are a bit lost.

The following commands compile and install a new kernel on your virtual machine. After the procedure is finished, please reboot your VM for loading the new kernel. If you are busy, you just need to type the following 4 lines of commands, and do not need to check further :) Execute the following commands on your VM:

$ cd ~
$ wget --no-check-certificate https://people.montefiore.uliege.be/gain/courses/info0940/asset/compile_kernel.sh
$ chmod 755 ./compile_kernel.sh
$ ./compile_kernel.sh

Danger

If you compile and install on the same VM, don’t forget to do an initial snapshot of the reference VM. Please, see the following link. In addition, you need to use git to have a copy of your files. If you DO NOT use Git, you can lose your files in case of kernel panic (trust me, it’s quite frequent).

What’s done by the script?

The script carries out 7 steps (Please also refer to the comments in the script):

  1. Install the required packages

    For kernel compilation, at least we need gcc, make, libssl-dev, flex, bc, and bison. In Ubuntu, we can use the apt command for installing packages. xz-utils is needed for extracting the kernel source. git is used for making patches that to be submitted as your homework.

  2. Download kernel source

    The source code of Linux is found at the Linux Kernel Archives https://www.kernel.org/. On the web page, we can explore varied versions and branches of the Linux kernel. In this time, we download the source from a specific URL https://cdn.kernel.org/pub/linux/kernel/v4.x/${KERNEL}.tar.xz which gives us a specific version of the kernel. On Linux, we can use the wget command for downloading content over HTTP.

  3. Extract kernel source

    The downloaded source is compressed in the xz format. We can extract by using the tar command with the “xvf” option.

  1. Prepare a config file

    For compiling the Linux kernel, we always need to prepare a config file. The config file allows us to choose features to be enabled. When we compile a kernel, we make a config file from old version of config file. In this time, we download the one prepared by the TAs. Alternatively we can use by copying one of config files in /boot. The command make olddefconfig is typed for adjusting the old version of config file (downloaded one) to the kernel that we are about to compile.

  2. Compile kernel

    Now, preparation is done, so let’s start compilation. All compilation is done by make ARCH=i386 commands.

  3. Install kernel

    Finally, we install the built kernel by make modules_install and make install. The command make modules_install installs kernel modules, and make install installs the kernel itself.

Further information

For booting a new kernel, essentially we need the new kernel image, initial ramdisk (initrd), and kernel modules including device drivers.

The command make modules_install makes the following directory that retains kernel modules.

  • /lib/modules/KERNEL_VERSION

In the installation process, make install produces the following files in your system.

  • /boot/vmlinuz-KERNEL_VERSION

  • /boot/initrd.img-KERNEL_VERSION

vmlinuz is the compiled kernel. initrd (initial ramdisk) contains files used for boot-up.

In general, OSes are loaded by boot loaders. On Linux, GRUB is widely used, and for loading the newly built kernel, we need to reconfigure GRUB. make install performs reconfiguration of GRUB. Usually, a new boot entry is added for the new kernel.

In short, make modules_install installs kernel modules at /lib/modules/, and make install locates the kernel image on /boot, and updates configuration of the boot loader.

How to Re-Compile Linux Kernel

While you are working on your assignment, you will recompile your kernel for debugging. If you compile the kernel once, you do not need to repeat the entire process, and you can skip Step 1, 2, 3, and 5. In short, you can use the following script which cuts unnecessary steps.

#!/bin/bash

KERNEL=linux-4.15

# go to user home directory
cd ~

# Step 4: enter kernel source directory
cd ${KERNEL}

# Step 6: kernel compilation
make ARCH=i386 bzImage -j$(nproc) && make ARCH=i386 modules -j$(nproc)

# Step 7: kernel installation
# Step 7.1: install kernel modules
sudo make modules_install
# Step 7.2: install kernel itself
sudo make install