Understanding the Linux Boot Process

Linux Tutorial #2

The Linux Boot Process

To be able to troubleshoot boot issues, one can do hundreds of Internet searches, and try just as many solutions. If you not fully understand what has gone wrong, it can be it difficult to find the right solution.

Overview of the Linux boot process

1. When the computer starts, it loads the BIOS or UEFI, depending what’s installed on the system.

2. BIOS/UEFI looks for some bootloader code. In linux, these are found inside GRUB = Grand Unified Boot loader.

3. GRUB, or most likely GRUB2, loads the Linux Kernel and the initrd = Initial RAM Disk or initramFS = Initial RAM File System. GRUB tells the Linux kernel where the initrd or initramFS filesystems are located. GRUB2 is the latest version of GRUB.

4. The linux kernel is a file called vmlinux. Usually you will see a file called vmlinuz. It is the same file, but compressed. When the Linux kernel loads, it first mounts the initrd or initramFS filesystem.

5. Either initrd or initramFS is mounted into RAM (InitramFS is usually more used in complex boot environments). initrd contains the necessary tools and scripts to mount the needed drivers and file systems.

6. When initrd or initramFS has completed it step, the linux kernel goes ahead and loads all it’s modules and filesystems. That process is called init.

Troubleshoot Boot Issues

Kernel Panics

Kernel Panics will make your computer crash unexpectedly, or hinder your computer from booting. Kernel Panics can happen by either:

• Hardware Error: Faulty RAM brick, Overclocked CPU, faulty disk or any faulty or incorrectly installed expansion card, like a video graphics card.

• After a Software Update.

Note: Not all boot problems are caused by kernel panics.

If the problem is caused by a faulty hardware, update the drivers, or change replace the hardware is the most obvious solutions. If it is software related however, a rollback to a previous version might solve the problem temporarily.

Rolling Back to a previous version of the Linux Kernel

If you are not able to boot after a update, you can do an advanced startup:

1. Turn off the computer.

2. Then hold, or press repeatedly, the left shift key on your keyboard while turning on your PC.

3. You should get a GRUB menu with Boot Ubuntu/Kubuntu or Advanced Startup Alternatives.

4. Inside Advanced Startup Alternatives, you should be able to choose between at least 2 different versions of the Linux kernel. Try choosing the one with the lower version number and see if it boots up.

Note: This is only a temporary workaround until a fix is implemented. To find out what is causing the problem, might be difficult unless your an expert. If you are lucky, the problem that caused the machine not to boot might have been discovered by some nerd in the near future, so you could just wait a few months and try to update again.

Checking the Linux kernel version and OS version

If you need support, it’s a good thing to know how to check what version of the kernel and the OS = Operative System that is installed. The Linux kernel file is stored under the /boot filesystem and can be checked with this command:

user@kubuntu:~$ ls /boot | grep vmlinu
vmlinuz
vmlinuz-5.19.0-32-generic
vmlinuz-5.19.0-35-generic
vmlinuz.old

Note: Unless you have specified anything else, you are most likely running the latest version. Linux usually stores at least one backup version, in case you are experiencing issues with a new version.

To check the OS release, run this command:

user@ubuntu:~$ cat /etc/os-release  
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Appendix

Deep Dive

The boot process for Linux is quite complex, and can be subjected to different “chicken or the egg” scenarios. Some files need to be configured so that other files can load etc. Down below I’m doing my best to comprehend how everything works together.

The Initial RAM Disk

Also referred as the Initial Root Disk in some documentation (I have no idea which one is the “right” definition). It is a filesystem mounted in RAM memory (it mounts to /dev/ram0/ ) by the Linux Kernel.

I guess it is referred as the initial root disk because it loads a temporary root ( the “/”) filesystem in RAM memory that contains just enough information for the kernel to load all required filesystems and drivers to boot the system.

When the kernel is finished loading all required drivers and partitions, the init phase begins. It then switches the root file system to the “real” partition and loads the rest of the file systems (the /home, /etc, and so forth). When the kernel is fully loaded (i.e machine has booted up and you are presented with a login screen), the initrd filesystem is never used used again.

The Initial RAM File System

Just like the initrd, it contains the tools and scripts needed to boot. The biggest benefit of initramFS is that the filesystem is flexible in size (known as a tmpfs-based filesystem), while initrd is static in size. It means that initramFS is more memory efficient than initrd. InitramFS is often used in complex boot environments, including:

• Encrypted File systems

• RAID configurations

All files, tools, libraries, configuration settings (if applicable), etc. related to the initramFS are put into the cpio archive, (a compressed file, like a .tar file) alongside the linux kernel. The boot loader, i.e GRUB will then offer it to the Linux kernel at boot time, so the kernel knows an initramFS is needed.

After the initramFS phase is done, the kernel starts the init phase, (switching to the real root and loads all the filesystems). Once the kernel is fully loaded, initramFS is never used again.

dracut

dracut is a tool created for the sole purpose of managing initramfs files.

Grand Unified Boot loader

GRUB is responsible for telling the Linux Kernel which modules should be loaded by the kernel and where to find the partitions that contains the drivers and modules required to boot the system. Most Linux distributions today are running GRUB2 (version 2), that includes the Debian-based distributions like Ubuntu and Mint, and Redhat-based distributions like Fedora and CentOS.

GRUB settings

GRUB settings can be found inside /boot/grub.

• If you find a file called menu.lst and grub.conf, the system is running GRUB version 1.

• If you find a file called grub.cfg, the system is running GRUB version 2.

To check which version of GRUB you are using, you can use this command:

user@ubuntu:~$ ls /boot/grub/ | grep -E 'grub|menu'
grub.cfg
grubenv

Edit the GRUB settings

The root file system needs to be accessible by the Linux kernel for it to load. If for example, the root filesystem is located on a RAID disk, or an NVME (M.2 SSD) drive, and the kernel doesn’t have the necessary instructions on how to load those drives, the machine won’t boot. That is when you need to edit the GRUB settings.

You don’t edit the boot settings under /boot/grub/grub.cfg directly. On Ubuntu, the file used to edit the bootloader is located under /etc/default/grub

root@ubuntu:~$ nano /etc/default/grub

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""

#   most comments are omitted for brievity

As noted inside the file, you need to run the update-grub command when finished editing:

root@ubuntu:~$ update-grub

Note: Editing and updating GRUB settings varies on different Linux distributions.

Note: There are GUI based alternatives for editing the bootloader. One tool for Ubuntu is called The Grub Customizer.

The Linux Kernel

The Linux Kernel is the core of the operating system and the common denominator for every Linux distribution. It is the interface between the user and the hardware. I’m quoting Red Hat here:

It is responsible for:

1. Memory management: Keep track of how much memory is used to store what, and where

2. Process management: Determine which processes can use the CPU = central processing unit, when, and for how long.

3. Device drivers: Act as mediator/interpreter between the hardware and processes

4. System calls and security: Receive requests for service from the processes

The kernel, if implemented properly, is invisible to the user, working in its own little world known as kernel space, where it allocates memory and keeps track of where everything is stored. What the user sees—like web browsers and files—are known as the user space. These applications interact with the kernel through a System Call Interface (SCI).

https://www.redhat.com/en/topics/linux/what-is-the-linux-kernel

Kernel Modules

A kernel module can be a human interface device, like a keyboard, mouse or a printer. It can also be drivers for the network card or graphic card. Basically all components in a computer has its own kernel module.

When a Linux machine boots, the Linux kernel includes only some basic modules, just enough to be able to boot the system. This is because it would be inefficient to load modules that are not in use. When the init phase starts, the Linux kernel starts to load all it’s file systems and installed hardware modules.

Managing Kernel Modules

Usually the Linux kernel is able to detect the hardware and load the correct modules. In some cases you might want to manually load a kernel module, or not load a kernel module.

To instruct the kernel to load a specific module at boot time, add them to this file:

root@ubuntu:~$ nano /etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

Sometimes a module can cause a conflict for another module, and therefore have to be disabled. To instruct the kernel to not load a specific module at boot time, add them to this file:

root@ubuntu:~$ nano /etc/modprobe.d/blacklist.conf

Following is the default settings inside blacklist.conf on a Ubuntu machine:

# This file lists those modules which we don't want to be loaded by
# alias expansion, usually so some other driver will be loaded for the
# device instead.

# evbug is a debug tool that should be loaded explicitly
blacklist evbug

# these drivers are very simple, the HID drivers are usually preferred
blacklist usbmouse
blacklist usbkbd

# replaced by e100
blacklist eepro100

# replaced by tulip
blacklist de4x5

As you can see, there are some modules that are disabled by default for different reasons.

Note: To load kernel modules after bootup, you can use modprobe.

Sources

• https://www.kernel.org/doc/html/latest/admin-guide/initrd.html

• https://wiki.gentoo.org/wiki/Initramfs/Guide

• https://www.redhat.com/en/topics/linux/what-is-the-linux-kernel

• Linux Server Administration on

https://www.cbtnuggets.com/

Special Mention

Thank you Shawn Powers for your comprehensive video series on Linux Server Administration on CBT Nuggets. I will study every video.