Skip to content

(Yes, the project is still alive πŸ˜ƒ) This procedure shows how to create a bootable and installable Ubuntu Live (along with the automatic hardware detection and configuration) from scratch. A Linux to call your own.

License

Notifications You must be signed in to change notification settings

mvallim/live-custom-ubuntu-from-scratch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

How to create a custom Ubuntu live from scratch

This project guides you through building a fully customized version of Ubuntu Linux from scratch. It covers creating a live ISO image that includes pre-installed packages, configurations, and custom scripts tailored to your needs. The steps walk you through setting up the environment, configuring the chroot, installing software, modifying the kernel, and finally generating the ISO image. This is ideal for those who want complete control over their Linux distribution, whether for personal or professional use.

Requirements

  • Proficiency with Linux shell commands and scripting.
  • Sufficient disk space and memory for building an ISO.

Steps

  1. Prepare the Environment: Install necessary dependencies.
  2. Create a Base System: Use debootstrap to set up a minimal Ubuntu system.
  3. Customize Packages: Add/remove software, configure kernel.
  4. Generate ISO: Package the system into a bootable ISO.

Authors

  • Marcos Vallim - Founder, Author, Development, Test, Documentation - mvallim
  • Ken Gilmer - Commiter, Development, Test, Documentation - kgilmer

See also the list of contributors who participated in this project.

Ways of Using this Tutorial

  • (Recommended) follow the directions step by step below to understand how to build an Ubuntu ISO.
  • Run the build.sh script in the scripts directory after checking this repo out locally.
  • Fork this repo and run the github action build. This will generate an ISO in your github account.

build-bionic build-focal build-jammy build-noble

Terms

  • build system - the computer environment running the build scripts that generate the ISO.
  • live system - the computer environment that runs from the live OS, generated by a build system. This may also be referred to as the chroot environment.
  • target system - the computer environment that runs after installation has completed from a live system.

Prerequisites (GNU/Linux Ubuntu)

Important

It is very important to remember that the version you are generating depends on the version being used on the host machine.

Example: If I am generating a from scratch bionic version, it is necessary for the host machine to have a bionic version or higher installed.

Scratch Host
bionic >= bionic
focal >= focal
jammy >= jammy
noble >= noble

Install packages we need in the build system required by our scripts.

sudo apt-get install \
   debootstrap \
   squashfs-tools \
   xorriso
mkdir $HOME/live-ubuntu-from-scratch

Bootstrap and Configure Ubuntu

debootstrap is a program for generating OS images. We install it into our build system to begin generating our ISO.

  • Checkout bootstrap

    sudo debootstrap \
       --arch=amd64 \
       --variant=minbase \
       noble \
       $HOME/live-ubuntu-from-scratch/chroot \
       http://us.archive.ubuntu.com/ubuntu/

    debootstrap is used to create a Debian base system from scratch, without requiring the availability of dpkg or apt. It does this by downloading .deb files from a mirror site, and carefully unpacking them into a directory which can eventually be chrooted into.

  • Configure external mount points

    sudo mount --bind /dev $HOME/live-ubuntu-from-scratch/chroot/dev
    
    sudo mount --bind /run $HOME/live-ubuntu-from-scratch/chroot/run

    As we will be updating and installing packages (grub among them), these mount points are necessary inside the chroot environment, so we are able to finish the installation without errors.

Define chroot environment

A chroot on Unix operating systems is an operation that changes the apparent root directory for the current running process and its children. A program that is run in such a modified environment cannot name (and therefore normally cannot access) files outside the designated directory tree. The term "chroot" may refer to the chroot system call or the chroot wrapper program. The modified environment is called a chroot jail.

Reference: https://en.wikipedia.org/wiki/Chroot

From this point we will be configuring the live system.

  1. Access chroot environment

    sudo chroot $HOME/live-ubuntu-from-scratch/chroot
  2. Configure mount points, home and locale

    mount none -t proc /proc
    
    mount none -t sysfs /sys
    
    mount none -t devpts /dev/pts
    
    export HOME=/root
    
    export LC_ALL=C

    These mount points are necessary inside the chroot environment, so we are able to finish the installation without errors.

  3. Set a custom hostname

    echo "ubuntu-fs-live" > /etc/hostname
  4. Configure apt sources.list

    cat <<EOF > /etc/apt/sources.list
    deb http://us.archive.ubuntu.com/ubuntu/ noble main restricted universe multiverse
    deb-src http://us.archive.ubuntu.com/ubuntu/ noble main restricted universe multiverse
    
    deb http://us.archive.ubuntu.com/ubuntu/ noble-security main restricted universe multiverse
    deb-src http://us.archive.ubuntu.com/ubuntu/ noble-security main restricted universe multiverse
    
    deb http://us.archive.ubuntu.com/ubuntu/ noble-updates main restricted universe multiverse
    deb-src http://us.archive.ubuntu.com/ubuntu/ noble-updates main restricted universe multiverse
    EOF
  5. Update indexes packages

    apt-get update
  6. Install systemd

    apt-get install -y libterm-readline-gnu-perl systemd-sysv

    systemd is a system and service manager for Linux. It provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic.

  7. Configure machine-id and divert

    dbus-uuidgen > /etc/machine-id
    
    ln -fs /etc/machine-id /var/lib/dbus/machine-id

    The /etc/machine-id file contains the unique machine ID of the local system that is set during installation or boot. The machine ID is a single newline-terminated, hexadecimal, 32-character, lowercase ID. When decoded from hexadecimal, this corresponds to a 16-byte/128-bit value. This ID may not be all zeros.

    dpkg-divert --local --rename --add /sbin/initctl
    
    ln -s /bin/true /sbin/initctl

    dpkg-divert is the utility used to set up and update the list of diversions.

  8. Upgrade packages

    apt-get -y upgrade
  9. Install packages needed for Live System

    apt-get install -y \
       sudo \
       ubuntu-standard \
       casper \
       discover \
       laptop-detect \
       os-prober \
       network-manager \
       net-tools \
       wireless-tools \
       wpagui \
       locales \
       grub-common \
       grub-gfxpayload-lists \
       grub-pc \
       grub-pc-bin \
       grub2-common \
       grub-efi-amd64-signed \
       shim-signed \
       mtools \
       binutils
    apt-get install -y --no-install-recommends linux-generic
  10. Graphical installer

    apt-get install -y \
       ubiquity \
       ubiquity-casper \
       ubiquity-frontend-gtk \
       ubiquity-slideshow-ubuntu \
       ubiquity-ubuntu-artwork

    The next steps will appear, as a result of the packages that will be installed from the previous step, this will happen without anything having to be informed or executed.

    1. Configure keyboard

    2. Console setup

  11. Install window manager

    apt-get install -y \
       plymouth-themes \
       ubuntu-gnome-desktop \
       ubuntu-gnome-wallpapers
  12. Install useful applications

    apt-get install -y \
       clamav-daemon \
       terminator \
       apt-transport-https \
       curl \
       vim \
       nano \
       less
  13. Install Visual Studio Code (optional)

    1. Download and install the key

      curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
      
      install -o root -g root -m 644 microsoft.gpg /etc/apt/trusted.gpg.d/
      
      echo "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list
      
      rm microsoft.gpg
    2. Then update the package cache and install the package using

      apt-get update
      
      apt-get install -y code
  14. Install Google Chrome (optional)

    1. Download and install the key

      wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
      
      echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
    2. Then update the package cache and install the package using

      apt-get update
      
      apt-get install google-chrome-stable
  15. Install Java JDK 8 (optional)

    apt-get install -y \
        openjdk-8-jdk \
        openjdk-8-jre
  16. Remove unused applications (optional)

    apt-get purge -y \
       transmission-gtk \
       transmission-common \
       gnome-mahjongg \
       gnome-mines \
       gnome-sudoku \
       aisleriot \
       hitori
  17. Remove unused packages

    apt-get autoremove -y
  18. Reconfigure packages

    1. Generate locales

      dpkg-reconfigure locales
      1. Select locales

      2. Select default locale

    2. Configure network-manager

      1. Create config file

        cat <<EOF > /etc/NetworkManager/NetworkManager.conf
        [main]
        rc-manager=none
        plugins=ifupdown,keyfile
        dns=systemd-resolved
        
        [ifupdown]
        managed=false
        EOF
      2. Reconfigure network-manager

        dpkg-reconfigure network-manager

Create the image directory and populate it

We are now back in our build environment after setting up our live system and will continue creating files necessary to generate the ISO.

  1. Create directories

    mkdir -p /image/{casper,isolinux,install}
  2. Copy kernel images

    cp /boot/vmlinuz-**-**-generic /image/casper/vmlinuz
    
    cp /boot/initrd.img-**-**-generic /image/casper/initrd
  3. Copy memtest86+ binary (BIOS and UEFI)

     wget --progress=dot https://memtest.org/download/v7.00/mt86plus_7.00.binaries.zip -O /image/install/memtest86.zip
     unzip -p /image/install/memtest86.zip memtest64.bin > /image/install/memtest86+.bin
     unzip -p /image/install/memtest86.zip memtest64.efi > /image/install/memtest86+.efi
     rm -f /image/install/memtest86.zip

GRUB menu configuration

  1. Create base point access file for grub

    touch /image/ubuntu
  2. Create image/isolinux/grub.cfg

    cat <<EOF > /image/isolinux/grub.cfg
    
    search --set=root --file /ubuntu
    
    insmod all_video
    
    set default="0"
    set timeout=30
    
    menuentry "Try Ubuntu FS without installing" {
       linux /casper/vmlinuz boot=casper nopersistent toram quiet splash ---
       initrd /casper/initrd
    }
    
    menuentry "Install Ubuntu FS" {
       linux /casper/vmlinuz boot=casper only-ubiquity quiet splash ---
       initrd /casper/initrd
    }
    
    menuentry "Check disc for defects" {
       linux /casper/vmlinuz boot=casper integrity-check quiet splash ---
       initrd /casper/initrd
    }
    
    grub_platform
    if [ "\$grub_platform" = "efi" ]; then
    menuentry 'UEFI Firmware Settings' {
       fwsetup
    }
    
    menuentry "Test memory Memtest86+ (UEFI)" {
       linux /install/memtest86+.efi
    }
    else
    menuentry "Test memory Memtest86+ (BIOS)" {
       linux16 /install/memtest86+.bin
    }
    fi
    EOF

Create manifest

Next we create a file filesystem.manifest to specify each package and it's version that is installed on the live system. We create another file filesystem.manifest-desktop which specifies which files will be installed on the target system. Once the Ubiquity installer completes, it will remove packages specified in filesystem.manifest that are not listed in filesystem.manifest-desktop.

  1. Generate manifest

    dpkg-query -W --showformat='${Package} ${Version}\n' | sudo tee /image/casper/filesystem.manifest
    
    cp -v /image/casper/filesystem.manifest image/casper/filesystem.manifest-desktop
    
    sed -i '/ubiquity/d' /image/casper/filesystem.manifest-desktop
    
    sed -i '/casper/d' /image/casper/filesystem.manifest-desktop
    
    sed -i '/discover/d' /image/casper/filesystem.manifest-desktop
    
    sed -i '/laptop-detect/d' /image/casper/filesystem.manifest-desktop
    
    sed -i '/os-prober/d' /image/casper/filesystem.manifest-desktop

Create diskdefines

README file often found on Linux LiveCD installer discs, such as an Ubuntu Linux installation CD; typically named β€œREADME.diskdefines” and may be referenced during installation.

  1. Create file /image/README.diskdefines

    cat <<EOF > /image/README.diskdefines
    #define DISKNAME  Ubuntu from scratch
    #define TYPE  binary
    #define TYPEbinary  1
    #define ARCH  amd64
    #define ARCHamd64  1
    #define DISKNUM  1
    #define DISKNUM1  1
    #define TOTALNUM  0
    #define TOTALNUM0  1
    EOF

Creating image

  1. Access image directory

    cd /image
  2. Copy EFI loaders

    cp /usr/lib/shim/shimx64.efi.signed.previous isolinux/bootx64.efi
    cp /usr/lib/shim/mmx64.efi isolinux/mmx64.efi
    cp /usr/lib/grub/x86_64-efi-signed/grubx64.efi.signed isolinux/grubx64.efi
  3. Create a FAT16 UEFI boot disk image containing the EFI bootloaders

    (
       cd isolinux && \
       dd if=/dev/zero of=efiboot.img bs=1M count=10 && \
       mkfs.vfat -F 16 efiboot.img && \
       LC_CTYPE=C mmd -i efiboot.img efi efi/ubuntu efi/boot && \
       LC_CTYPE=C mcopy -i efiboot.img ./bootx64.efi ::efi/boot/bootx64.efi && \
       LC_CTYPE=C mcopy -i efiboot.img ./mmx64.efi ::efi/boot/mmx64.efi && \
       LC_CTYPE=C mcopy -i efiboot.img ./grubx64.efi ::efi/boot/grubx64.efi && \
       LC_CTYPE=C mcopy -i efiboot.img ./grub.cfg ::efi/ubuntu/grub.cfg
    )
  4. Create a grub BIOS image

    grub-mkstandalone \
       --format=i386-pc \
       --output=isolinux/core.img \
       --install-modules="linux16 linux normal iso9660 biosdisk memdisk search tar ls" \
       --modules="linux16 linux normal iso9660 biosdisk search" \
       --locales="" \
       --fonts="" \
       "boot/grub/grub.cfg=isolinux/grub.cfg"
  5. Combine a bootable Grub cdboot.img

    cat /usr/lib/grub/i386-pc/cdboot.img isolinux/core.img > isolinux/bios.img
  6. Generate md5sum.txt

    /bin/bash -c "(find . -type f -print0 | xargs -0 md5sum | grep -v -e 'isolinux' > md5sum.txt)"

Cleanup the chroot environment

  1. If you installed software, be sure to run

    truncate -s 0 /etc/machine-id
  2. Remove the diversion

    rm /sbin/initctl
    
    dpkg-divert --rename --remove /sbin/initctl
  3. Clean up

    apt-get clean
    
    rm -rf /tmp/* ~/.bash_history
    
    umount /proc
    
    umount /sys
    
    umount /dev/pts
    
    export HISTSIZE=0
    
    exit

Unbind mount points

sudo umount $HOME/live-ubuntu-from-scratch/chroot/dev

sudo umount $HOME/live-ubuntu-from-scratch/chroot/run

Compress the chroot

After everything has been installed and preconfigured in the chrooted environment, we need to generate an image of everything that was done by following the next steps in the build environment.

  1. Access build directory

    cd $HOME/live-ubuntu-from-scratch
  2. Move image artifacts

    sudo mv chroot/image .
  3. Create squashfs

    sudo mksquashfs chroot image/casper/filesystem.squashfs \
       -noappend -no-duplicates -no-recovery \
       -wildcards \
       -comp xz -b 1M -Xdict-size 100% \
       -e "var/cache/apt/archives/*" \
       -e "root/*" \
       -e "root/.*" \
       -e "tmp/*" \
       -e "tmp/.*" \
       -e "swapfile"

    Squashfs is a highly compressed read-only filesystem for Linux. It uses zlib compression to compress both files, inodes and directories. Inodes in the system are very small and all blocks are packed to minimize data overhead. Block sizes greater than 4K are supported up to a maximum of 64K. Squashfs is intended for general read-only filesystem use, for archival use (i.e. in cases where a .tar.gz file may be used), and in constrained block device/memory systems (e.g. embedded systems) where low overhead is needed.

  4. Write the filesystem.size

    printf $(sudo du -sx --block-size=1 chroot | cut -f1) | sudo tee image/casper/filesystem.size

Create ISO Image for a LiveCD (BIOS + UEFI + Secure Boot)

  1. Access build directory

    cd $HOME/live-ubuntu-from-scratch/image
  2. Create iso from the image directory using the command-line

    sudo xorriso \
       -as mkisofs \
       -iso-level 3 \
       -full-iso9660-filenames \
       -J -J -joliet-long \
       -volid "Ubuntu from scratch" \
       -output "../ubuntu-from-scratch.iso" \
       -eltorito-boot isolinux/bios.img \
         -no-emul-boot \
         -boot-load-size 4 \
         -boot-info-table \
         --eltorito-catalog boot.catalog \
         --grub2-boot-info \
         --grub2-mbr ../chroot/usr/lib/grub/i386-pc/boot_hybrid.img \
         -partition_offset 16 \
         --mbr-force-bootable \
       -eltorito-alt-boot \
         -no-emul-boot \
         -e isolinux/efiboot.img \
         -append_partition 2 28732ac11ff8d211ba4b00a0c93ec93b isolinux/efiboot.img \
         -appended_part_as_gpt \
         -iso_mbr_part_type a2a0d0ebe5b9334487c068b6b72699c7 \
         -m "isolinux/efiboot.img" \
         -m "isolinux/bios.img" \
         -e '--interval:appended_partition_2:::' \
       -exclude isolinux \
       -graft-points \
          "/EFI/boot/bootx64.efi=isolinux/bootx64.efi" \
          "/EFI/boot/mmx64.efi=isolinux/mmx64.efi" \
          "/EFI/boot/grubx64.efi=isolinux/grubx64.efi" \
          "/EFI/ubuntu/grub.cfg=isolinux/grub.cfg" \
          "/isolinux/bios.img=isolinux/bios.img" \
          "/isolinux/efiboot.img=isolinux/efiboot.img" \
          "."

Alternative way, if previous one fails, create an Hybrid ISO

  1. Create a ISOLINUX (syslinux) boot menu

    cat <<EOF> isolinux/isolinux.cfg
    UI vesamenu.c32
    
    MENU TITLE Boot Menu
    DEFAULT linux
    TIMEOUT 600
    MENU RESOLUTION 640 480
    MENU COLOR border       30;44   #40ffffff #a0000000 std
    MENU COLOR title        1;36;44 #9033ccff #a0000000 std
    MENU COLOR sel          7;37;40 #e0ffffff #20ffffff all
    MENU COLOR unsel        37;44   #50ffffff #a0000000 std
    MENU COLOR help         37;40   #c0ffffff #a0000000 std
    MENU COLOR timeout_msg  37;40   #80ffffff #00000000 std
    MENU COLOR timeout      1;37;40 #c0ffffff #00000000 std
    MENU COLOR msg07        37;40   #90ffffff #a0000000 std
    MENU COLOR tabmsg       31;40   #30ffffff #00000000 std
    
    LABEL linux
     MENU LABEL Try Ubuntu FS
     MENU DEFAULT
     KERNEL /casper/vmlinuz
     APPEND initrd=/casper/initrd boot=casper
    
    LABEL linux
     MENU LABEL Try Ubuntu FS (nomodeset)
     MENU DEFAULT
     KERNEL /casper/vmlinuz
     APPEND initrd=/casper/initrd boot=casper nomodeset
    EOF
  2. Include syslinux bios modules

    apt install -y syslinux-common && \
    cp /usr/lib/ISOLINUX/isolinux.bin image/isolinux/ && \
    cp /usr/lib/syslinux/modules/bios/* image/isolinux/
  3. Access build directory

    cd $HOME/live-ubuntu-from-scratch/image
  4. Create iso from the image directory

    sudo xorriso \
       -as mkisofs \
       -iso-level 3 \
       -full-iso9660-filenames \
       -J -J -joliet-long \
       -volid "Ubuntu from scratch" \
       -output "../ubuntu-from-scratch.iso" \
     -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin \
     -eltorito-boot \
         isolinux/isolinux.bin \
         -no-emul-boot \
         -boot-load-size 4 \
         -boot-info-table \
         --eltorito-catalog isolinux/isolinux.cat \
     -eltorito-alt-boot \
         -e /EFI/boot/efiboot.img \
         -no-emul-boot \
         -isohybrid-gpt-basdat \
     -append_partition 2 0xef EFI/boot/efiboot.img \
       "$HOME/live-ubuntu-from-scratch/image"

Make a bootable USB image

It is simple and easy, using "dd"

sudo dd if=ubuntu-from-scratch.iso of=<device> status=progress oflag=sync

Summary

This completes the process of creating a live Ubuntu installer from scratch. The generated ISO may be tested in a virtual machine such as VirtualBox or written to media and booted from a standard PC.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use GitHub for versioning. For the versions available, see the tags on this repository.

License

This project is licensed under the GNU GENERAL PUBLIC LICENSE - see the LICENSE file for details

About

(Yes, the project is still alive πŸ˜ƒ) This procedure shows how to create a bootable and installable Ubuntu Live (along with the automatic hardware detection and configuration) from scratch. A Linux to call your own.

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages