Automated Installation of Ubuntu 18.04

When you try new things, you break old things. In order to avoid the fear of losing something important I figured out how to automate the reinstallation of my laptop.

Target Audience

This shall be a detailed guide to preseeding Ubuntu 18.04 Desktop and automatically configuring some software. You will heavily use the terminal while following in my footprints. Apart from executing bash commands, not programming knowledge is required.

If you don't have already a device running linux, I recommend doing as I did: Set up a normal installation USB stick with the official Ubuntu 18.04 LTS Desktop ISO. Because we will be modifying the bootable ISO, and I did not manage to do that correctly from Windows using balenaEtcher or Cygwin.

I have previously used linux on my raspberry pi and on compute clusters, and cygwin on windows, but am kinda new to ubuntu.

Goal

I'd like to be able to plug the USB Stick into my laptop, walk away, and when I return 2 hours later my ubuntu installation should be restored to a clean but perfectly usable state. My dual-boot windows partitions should remain unaffected. All programs already configured so that I don't need to reconfigure anything manually.

You could just make a disk image. But that requires way more storage space and is a bit harder to keep clean yet up-to-date regarding which programs it configures.

Since almost all of the documentation is either plainly wrong, outdated, or inexistent, here is a detailed guide.

Structure

Every Section will outline how you do something and possibly how you can automate that. I will provide relevant links often, but you should hopefully rarely need to consult them.

Installing Linux

If you have a windows partition, it's worth checking whether bitlocker is on. If it is, I recommend having a look at Chris Xiao's Guide (Last Updated April 2019). He also recommends to disable Secure Boot from the UEFI BIOS settings.

Even if you choose a distro that supports Secure Boot, it can potentially cause issues, especially when you’re installing extra drivers or manually upgrading kernel. Therefore, you may want to disable Secure Boot to make your life easier.

I won't cover flashing the ISO to a usb stick in detail. Just download the ISO, flash it on the stick using e.g. balena etcher (this will of course wipe any preexisting data on the stick) and boot your laptop from the stick.

Something Went Terribly Wrong (Optional)

If you only get a black screen telling you that something has gone seriously wrong, it may be because you aborted a previous installation after checking the box about using proprietary drivers. In that case, the installer remembers that it wants to start off a different file on the stick, which has not yet been created, and is hence slightly broken now. I followed this answer to resolve that problem:

Now when I try to run the installer from my bootable USB I get the following error


Failed to open \EFI\BOOT\mmx64.efi - Not Found
Failed to load image \EFI\BOOT\mmx64.efi: Not Found
Failed to start MokManager: Not Fond
Something has gone seriously wrong: import_mok_state() failed

I booted an Ubuntu Live USB stick, went intomy hard disk drive's /boot/efi folder and renamed the file grubx64.efi to mmx64.efi

Reboot the machine and it should work.

Partitioning Manually

Choose something else as the installation type. That gives you more control and is safer than choosing to install alongside windows.

There are different reasonable choices that you can make for splitting your linux installation into partitions. One possibility I did not explore was using a separate home partition to make upgrading easier or to use at the same time in different systems. What I am using is this:

/boot partition (ext2), 300MB
/tmp partition (ext2), 7GB
swap space: 20GB because I have 16GB RAM Hibernating with less swap space than RAM is not possible, and since I want to be able to hibernate... Recommended is 2 times your RAM size as swap space, but that sounds overkill to me.
/ partition (ext4), remaining space

Preseeding

Partitioning Automatically

Preseeding means adding commands onto the stick so that the installer will have to ask less questions. However, this is very limited in regards to partitioning since the underlying tool partman only allows you to either wipe the whole hard disk (including the windows partition) or to only use the free space (so you would have to manually free the previously used partitions for the reinstall).

This 2017 thread claims to have a solution, but I have not tested it. Similarly, this blog post outlines how to customize the partitioning within those constraints.

I hope my directions will be enough, but if not here are some links. Skip this paragraph until you need it. You can consult the official ubuntu documentation on preseeding and on customizing the LiveCD and a self-proclaimed beginner outlines hist process here. There is also a Step-By-Step guide for ubuntu 16.04 and the Ubuntu Help example-preseed.txt (which is specifically linked to as "for bionic" even though the file itself says it's for debian stretch.) and a third-pary preseed file that uses ubiquity. This QA contains valuable information on creating a customized ISO, as does this article and this guide. You could also consider running Ubuntu in qemu for testing purposes. This answer explains how you should specify the file path

Modifying the ISO

The Ubuntu Help claims

B.2.1. Loading the preconfiguration file
If you are using initrd preseeding, you only have to make sure a file named preseed.cfg is included in the root directory of the initrd. The installer will automatically check if this file is present and load it.
For the other preseeding methods you need to tell the installer what file to use when you boot it. This is normally done by editing the bootloader configuration file (e.g. parmfile.ubuntu) and adding the parameter to the end (however, the file is empty by default).

That documentation is pretty useless. There is no parmfile.ubuntu and while placing a preseed.cfg in the initrd (more on what "initrd" even is later) may work but is harder than necessary.

You have the option to get the preseed config file from the network, but that will bring its own issues: You'd have to specify some settings manually because they happen to be asked before the network is loaded (Or perhaps you can specify them in the boot arguments).

So here's what we do instead:

We write our preseed commands into a file that you name however you want. I named mine my.seed. We will unpack the official ubuntu iso, modify the contents, and repack it. I will explain this later in more detail, but the modified iso shall contain our preseed file, and information on where to find it.

The following commands are to be executed on any linux system. It must not necessarily be the machine you will in the end automatically install ubuntu on.

If something is unclear to you, perhaps I phrased it better in my answer on how to preseed wirieless authentication.


xxxxxxxxxx
# mount iso readonly because it is readonly anyhow
sudo mount -o loop -t iso9660 ~/ubuntu.iso /mnt/myiso -o ro
# copy data to a writeable directory
mkdir ~/Downloads/iso.new
cp -ra /mnt/myiso/* ~/Downloads/iso.new

Place your preseed file somewhere within ~/Downloads/iso.new. You could choose any place. I created the directory iso.new/preseed and placed it there as iso.new/preseed/my.seed. Keep it as an empty file for now. We will iteratively work on it, so that you have an easier time debugging in the case of something going wrong.

Boot Arguments

There is a shitload of documentation around on how to tell the ubuntu installer where to check for your preseed file. Most of those don't work. Some suggest to use iso.new/isolinux/txt.cfg, which does indeed work. It is in the end included in iso.new/isolinux/isolinux.cfg, so I specified my boot arguments directly there.

Modify isolinux.cfg so that it looks like this:


x
# D-I config version 2.0
# search path for the c32 support libraries (libcom32, libutil etc.)
path 
include menu.cfg
#default vesamenu.c32
default install
prompt 1
timeout 50
ui gfxboot bootlogo
label install
  menu label ^LucidBrot 18.04 v12
  kernel /casper/vmlinuz
  append auto file=/cdrom/preseed/my.seed boot=casper debug-ubiquity automatic-ubiquity initrd=/casper/initrd DEBCONF_DEBUG=5 ---

The line label install declares a new block which is by default selected due to the earlier line default install. Anything that comes after menu label ^ is what will be displayed as the name. I use a version counter here so that I can see early whether I'm using the version of my iso that I thought I am using. But of course you will need to manually update that number.

The line append specifies the arguments that actually interest us. First of all, you need to specify auto. That enables the usage of a preseed file, which you specify with a leading file=/cdrom because your usb stick will actually be mounted as /cdrom.

There are several features of the Ubuntu Installer that combine to allow fairly simple command lines at the boot prompt to result in arbitrarily complex customized automatic installs.
This is enabled by using the Automated install boot choice, also called auto for some architectures or boot methods. In this section, auto is thus not a parameter, it means selecting that boot choice, and appending the following boot parameters on the boot prompt.
Source

boot=casper initrd=/casper/initrd is always there.

debug-ubiquity DEBCONF_DEBUG=5 increases the verbosity of the installer. That is useful for finding out why it's not working as expected. The log file can be found during the installation at /var/log/installer/debug.

And automatic-ubiquity is again a must.

automatic-ubiquity: Pass --automatic to ubiquity which enables automatic mode. Questions that have been marked seen will be skipped and pages of the interface that ask no questions (because they are all skipped) will not be shown. Implies only-ubiquity (see below).
Source

Installer Background Image

To see whether our modifications have any effect, let us modify the background image that is shown during the installation (but not on the installed system). The idea comes from here. I use the normal background turned blue - you can grab it here.

Store it in iso.new/preseed/warty-final-ubuntu-blue.png (or wherever you like).

To get it to display, it needs to be placed where it will be loaded from, before the installer actually is fully ready. To that end, we add this command to iso.new/preseed/my.seed:


xxxxxxxxxx
d-i preseed/early_command string cp /root/cdrom/preseed/warty-final-ubuntu-blue.png /root/usr/share/backgrounds/warty-final-ubuntu.png

Note that I prefixed both paths with /root. That is because the preseed/early_command is actually executed by casper (Source). In order to understand what that means, here's a short overview of filesystems involved in the installation process.

There are multiple filesystems.
The filesystem you are running all these commands in
The iso we mounted on /mnt/myiso and copied the contents of over to ~/Downloads/iso.new. It contains some information for the bootloader.
A ~/Downloads/iso.new/casper/initrd file which contains an initial ram filesystem that will be booted into once you select "Install Ubuntu" in grub.
A ~/Downloads/iso.new/casper/filesystem.squashfs file which contains a compressed filesystem. This filesystem will be chrooted into after some initial setup done in the initramfs. Once that is up, it's possible to log into a tty by pressing CtrlAltF2 (or F3) and enter ubuntu as the username. The squashfs contains scripts for ubiquity - the installer.
Source: My Answer

I advise you to keep the whole command on one line. It's technically allowed to have newlines in there, but not everywhere.

Write The Modified ISO To The Stick

You might have to install isohybrid and mkisofs. I'm not sure anymore but I think only the latter was preinstalled.


xxxxxxxxxx
# create a new iso
sudo mkisofs -J -l -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table -z -iso-level 4 -c isolinux/isolinux.cat -o ~/Downloads/outputubuntu.iso -joliet-long iso.new

I do not understand what all of these flags do. Feel free to check out the mkisofs man page to learn more about them before reaching the conclusion that you don't care as long as it works™.


xxxxxxxxxx
# make the iso bootable from an usb stick (it's only bootable from cdrom currently)
sudo isohybrid ~/Downloads/outputubuntu.iso

If you want to, you can test whether the iso is bootable in qemu. However, I did not really use this as I tested it on another laptop instead.


xxxxxxxxxx
# test it in qemu
# create some file I don't fully understand
qemu-img create -f qcow2 /tmp/ubuntu.qcow2  10G
# run it in qemu
sudo qemu-system-x86_64 -m 1G -cdrom ~/Downloads/ubuntu18.04.new.iso -hda /tmp/ubuntu.qcow2
# or don't do above two commands and just do 
qemu-system-x86_64 -cdrom ubuntu-9.04.1-desktop-i386-custom.iso -boot d -m 512

To test it on a real laptop, starting from the stick, you will need to write it to that stick first of course. dd is not forgiving when you do mistakes, so make sure you're writing to the correct device and that it is not mounted and that it is actually plugged in.


xxxxxxxxxx
# write it to usb stick by unmounting the usb stick first, and then using dd
sudo umount /dev/sda1
sudo dd bs=4M if=path/to/input.iso of=/dev/sda conv=fdatasync  status=progress

If you accidentally run this command before plugging the stick in, the unmount part will fail. If you run the dd command without the stick plugged in, it will create a file /dev/sda that you'll have to delete before being able to mount the stick again.

conv=fdatasync makes sure that the command only returns once it is done. This command will take a few minutes!

Boot From Stick

There are enough guides in the web for this. Plug the stick into your device, select it as boot device in bios, and then it should show you several options. One of them being titled the way you named it - and that should be autoselected anyway.

If you'd like to experiment with boot parameters, you can do that here as well.

Once your installer has started, the background image should be blue and it will ask you some questions which we will get rid of at a later point.

Isolinux.cfg Additions

UbiquityAutomation implies that you need to specify languages differently than with debconf d-i. That site was updated in 2019:

Furthermore, there are a few other components necessary for an automated installation with ubiquity:
languagechooser/language-name: choose among the available languages, eg English
countrychooser/shortlist: choose a country, territory or area, eg US
localechooser/supported-locales: choose other locales to be supported, eg en_US.UTF-8

This gist implies that even with ubiquity, it's still possible to set up the locale using the d-i commands... but it also specifies everything as boot arguments. I read in some official docs that you cannot have more than 8 arguments there, but I guess that no longer applies.

I set the installer to use a swiss german keyboard layout but the English language. Somehow, with this configuration, I ended up with some german logging messages, but I didn't mind because I change those settings in the final install anyway.


xxxxxxxxxx
label install
  menu label ^LucidBrot 18.04 v55
  kernel /casper/vmlinuz
  append auto=true file=/cdrom/preseed/my.seed boot=casper debug-ubiquity automatic-ubiquity initrd=/casper/initrd DEBCONF_DEBUG=5 debian-installer/locale=de_CH.UTF-8 keyboard-configuration/layoutcode=ch languagechooser/language-name=English countrychooser/shortlist=CH localechooser/supported-locales=en_US.UTF-8 ---

You can modify your iso.new/isolinux/isolinux.cfg to match your preferences. I advise you to recreate the stick and test whether it works as you're expecting it to every time you perform a few modifications.

Preseed Commands

Again, I advise you to iteratively add a few commands to test if they work instead of just dumping everything in at once - at least if you're doing something differently than me.

Background

We've been over this already.


xxxxxxxxxx
# machine-dependent wireless interface
d-i preseed/early_command string cp /root/cdrom/preseed/warty-final-ubuntu-blue.png /root/usr/share/backgrounds/warty-final-ubuntu.png

Localization


xxxxxxxxxx
### Localization
# The values can also be preseeded individually for greater flexibility.
# See locale and locale -a in a running system
d-i debian-installer/language string en
d-i console-setup/ask_detect boolean false
d-i debian-installer/country string CH
d-i debian-installer/locale string en_US.UTF-8
# Optionally specify additional locales to be generated.
d-i localechooser/supported-locales multiselect de_CH.utf8
# Keyboard selection.
# Disable automatic (interactive) keymap detection.
d-i console-setup/ask_detect boolean false
# specify the layout (and variant) as found in a running system with setxkbmap -query
d-i keyboard-configuration/xkb-keymap select ch
# To select a variant of the selected layout:
d-i keyboard-configuration/xkb-keymap select de_nodeadkeys
# I think gnome handles toggling with Win+Space and Win+Shift+Space anyway
d-i keyboard-configuration/toggle select No toggling

This results in a system with the following output of locale && localectl. Except that I later modified the compose key.


xxxxxxxxxx
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=de_CH.UTF-8
LC_TIME=de_CH.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=de_CH.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=de_CH.UTF-8
LC_NAME=de_CH.UTF-8
LC_ADDRESS=de_CH.UTF-8
LC_TELEPHONE=de_CH.UTF-8
LC_MEASUREMENT=de_CH.UTF-8
LC_IDENTIFICATION=de_CH.UTF-8
LC_ALL=
   System Locale: LANG=en_US.UTF-8
                  LANGUAGE=en_US:en
                  LC_NUMERIC=de_CH.UTF-8
                  LC_TIME=de_CH.UTF-8
                  LC_MONETARY=de_CH.UTF-8
                  LC_PAPER=de_CH.UTF-8
                  LC_NAME=de_CH.UTF-8
                  LC_ADDRESS=de_CH.UTF-8
                  LC_TELEPHONE=de_CH.UTF-8
                  LC_MEASUREMENT=de_CH.UTF-8
                  LC_IDENTIFICATION=de_CH.UTF-8
       VC Keymap: sg
      X11 Layout: ch,us
       X11 Model: pc105
     X11 Variant: de_nodeadkeys,
     X11 Options: compose:lwin-altgr

Wireless Networking

We want our ubuntu installer to automatically connect to the Wifi Network. In older versions of ubuntu, this could have been set up using d-i netcfg. The latest version does no longer support that, since it uses ubiquity as the installer instead of the good old debian-installer.

Preseeding keys for the following installer components will not be used in Ubiquity, usually because they do not fit with Ubiquity's mode of operation:
netcfg
LVM and RAID partitioning
base-installer
pkgsel/tasksel
finish-install
Source: UbiquityAutomation

I have tried replacing the d-i commands with ubiquity as follows, but I'm pretty sure having that in my preseed file did little if anything at all. Still, it currently is in my working preseed file, so it doesn't hurt to have that there at least.


xxxxxxxxxx
# wlp3s0b1 would be the device-specific wifi interface
ubiquity netcfg/choose_interface select wlp3s0b1
ubiquity netcfg/wireless_show_essids select manual
ubiquity netcfg/wireless_essid string myWifiName
ubiquity netcfg/wireless_essid_again string myWifiName
ubiquity netcfg/wireless_security_type select wpa
# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string
# The wacky dhcp hostname that some ISPs use as a password of sorts.
#d-i netcfg/dhcp_hostname string radish
# If non-free firmware is needed for the network or other hardware, you can
# configure the installer to always try to load it, without prompting. Or
# change to false to disable asking.
#d-i hw-detect/load_firmware boolean true

I have also tried using preseed/early_command, which does not work because it is executed too early.

Evan needs to add a "a bit later than early command" preseed variable so that testing scripts can run once the desktop is available.

Evan didn't. But we can. I'll give you a quick run-through on how to modify ubiquity itself so that it allows us to run our custom not-too-early commands. If you need more explanation, I highly recommend you to check out my answer on askubuntu - I have taken the following excerpts from there.

Open The Squashfs


xxxxxxxxxx
# isohybrid and unmksquashfs
sudo apt-get install syslinux-utils squashfs-tools
# unpack the squashfs to a new dir
mkdir ~/Downloads/squashy && cd ~/Downloads/squashy
sudo unsquashfs ~/Downloads/iso.new/casper/filesystem.squashfs
# now we have a new directory "squashfs-root"

Modify Ubiquity

Open ~/Downloads/squashy/squashfs-root/usr/bin/lib/ubiquity/ubiquity/plugin_manager.py (I'd say as root, but not sure if that's necessary) and add the import statement import subprocess after all the other imports at the start of the file. Then navigate to line 42 where the function def load_plugins(): should be defined. Here what I inserted at the start of it, though the print statements are only for the logs and can be skipped.


xxxxxxxxxx
def load_plugins():
    print("flappy plugin_manager: load_plugins()")
    mywifires = subprocess.run("/cdrom/preseed/inb4.sh", stderr=subprocess.PIPE, stdout=subprocess.PIPE)
    print("flappy stdout wifi: {}".format(mywifires))
    print("flappy usr/lib/bin/ubiquity/ubiquity/plugin_manager: post-wifi", file=sys.stderr)
    modules = []

Set Up inb4.sh

It is now possible for us to run arbitrary bash commands at the start of the installation by entering them into ~/Downloads/iso.new/preseed/inb4.sh. Since I simply want to connect to a normal WPA2 network, here's how mine looks (make sure you use your own interface as ip link reports it instead of my wlp3s0b1):


xxxxxxxxxx
# file iso.new/preseed/inb4.sh
#!/bin/sh
set -ux
sudo ip link set dev wlp3s0b1 up
sudo nmcli d wifi connect myWiFiHotspot password MyPassw0rd

Make inb4.sh executable

Just to be safe.


xxxxxxxxxx
sudo chmod a+rx ~/Downloads/iso.new/preseed/inb4.sh

Pack Squashfs

Make sure you currently don't have a file called filesystem.squashfs in your squashy directory, because that will cause the next commands to create /dev_1 instead of /dev and such things...


xxxxxxxxxx
cd ~/Downloads/squashy
sudo rm filesystem.squashfs
sudo mksquashfs squashfs-root/ filesystem.squashfs -b 1024k -comp xz -Xbcj x86 -e sudo cp filesystem.squashfs ../iso.new/casper/filesystem.squashfs

This command will take about 10 minutes on my laptop - be patient.

Update Filesystem Size Metadata


xxxxxxxxxx
printf $(sudo du -sx --block-size=1 chroot | cut -f1) >  ~/Downloads/iso.new/casper/filesystem.size

Test

Now is a good time to test your work again. Here's a reminder in case you can't be bothered to scroll back up to the section Write The Modified ISO To The Stick


xxxxxxxxxx
# copy the modified squashfs back to the iso filesystem data
sudo cp ~/Downloads/squashy/filesystem.squashfs ~/Downloads/iso.new/casper/filesystem.squashfs
# build iso. Again, the flags are beyond be, but they work
sudo mkisofs -J -l -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table -z -iso-level 4 -c isolinux/isolinux.cat -o ~/Downloads/outputubuntu.iso -joliet-long iso.new
# make the iso bootable from an usb stick (it's only bootable from cdrom currently)
sudo isohybrid ~/Downloads/outputubuntu.iso
# plug usb stick in. Unmount it and flash the iso
cd ~/Downloads
sudo umount /dev/sda1 && sudo dd bs=4M if=ubuntuoutput.iso of=/dev/sda conv=fdatasync status=progress

Hostname

Remember how the documentation states that netcfg preseed commands are no longer used? Well, that was a lie. To set up your device's hostname:


xxxxxxxxxx
# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string motorbrot
d-i netcfg/get_domain string unassigned-domain
# If you want to force a hostname, regardless of what either the DHCP
# server returns or what the reverse DNS entry for the IP is, uncomment
# and adjust the following line.
#d-i netcfg/hostname string somehost

Mirrors


xxxxxxxxxx
### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string archive.ubuntu.com
d-i mirror/http/directory string /ubuntu
d-i mirror/http/proxy string
# Alternatively: by default, the installer uses CC.archive.ubuntu.com where
# CC is the ISO-3166-2 code for the selected country. You can preseed this
# so that it does so without asking.
#d-i mirror/http/mirror select CC.archive.ubuntu.com
# Suite to install.
#d-i mirror/suite string bionic
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string bionic
# Components to use for loading installer components (optional).
#d-i mirror/udeb/components multiselect main, restricted
# I could just set this up to use everything but that might be dumb:
#d-i mirror/udeb/components multiselect main, restricted, universe, multiverse

Account Setup


xxxxxxxxxx
# Skip creation of a root account (normal user account will be able to
# use sudo). The default is false; preseed this to true if you want to set
# a root password.
#d-i passwd/root-login boolean false
# Alternatively, to skip creation of a normal user account.
#d-i passwd/make-user boolean false
# Root password, either in clear text
#d-i passwd/root-password password r00tme
#d-i passwd/root-password-again password r00tme
# or encrypted using a crypt(3)  hash.
#d-i passwd/root-password-crypted password [crypt(3) hash]
# To create a normal user account.
d-i passwd/user-fullname string generic
d-i passwd/username string generic
# Normal user's password, either in clear text
#d-i passwd/user-password password insecure
#d-i passwd/user-password-again password insecure
# or encrypted using a crypt(3) hash.
# Create the first user with the specified UID instead of the default.
#d-i passwd/user-uid string 1010
# The installer will warn about weak passwords. If you are sure you know
# what you're doing and want to override it, uncomment this.
d-i user-setup/allow-password-weak boolean true
# The user account will be added to some standard initial groups. To
# override that, use this.
#d-i passwd/user-default-groups string audio cdrom video
# Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false

Time


xxxxxxxxxx
### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true
# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string Europe/Zurich
# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true
# NTP server to use. The default is almost always fine here.
#d-i clock-setup/ntp-server string ntp.example.com

Package Selection

Again, I find the documentation to be confusing at best and wrong at worst. But despite what they say, this does choose a minimal install for me:


xxxxxxxxxx
### Package selection
tasksel tasksel/first multiselect 
ubiquity ubiquity/minimal_install boolean true
# Policy for applying updates. May be "none" (no automatic updates),
# "unattended-upgrades" (install security updates automatically), or
# "landscape" (manage system with Landscape).
d-i pkgsel/update-policy select unattended-upgrades

Grub Alongside Windows


xxxxxxxxxx
# Alternatively, if you want to install to a location other than the mbr,
# uncomment and edit these lines:
# since my /boot is on nvme0n1p5, I hope that's the correct partition
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true

It is possible to specify a disk, but I let it choose automatically since I only have one hard drive anyway.


xxxxxxxxxx
# d-i grub-installer/bootdev string /dev/nvme0n1p5
#d-i grub-installer/bootdev string /dev/sda
# To install grub to multiple disks:
#d-i grub-installer/bootdev  string (hd0,1) (hd1,1) (hd2,1)

And if you have seen the official example with all their options... I have not made use of them and it works fine:


xxxxxxxxxx
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
#id-i grub-installer/only_debian boolean true
# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
#d-i grub-installer/with_other_os boolean true
# Due notably to potential USB sticks, the location of the MBR can not be
# determined safely in general, so this needs to be specified:
#d-i grub-installer/bootdev  string /dev/sda
# To install to the first device (assuming it is not a USB stick):
#d-i grub-installer/bootdev  string default

Further options which I did not use yet include the possibility to set a password for grub:


xxxxxxxxxx
# Optional password for grub, either in clear text
#d-i grub-installer/password password r00tme
#d-i grub-installer/password-again password r00tme
# or encrypted using an MD5 hash, see grub-md5-crypt(8).
#d-i grub-installer/password-crypted password [MD5 hash]
# Use the following option to add additional boot parameters for the
# installed system (if supported by the bootloader installer).
# Note: options passed to the installer will be added automatically.
#d-i debian-installer/add-kernel-opts string nousb

Finishing

There are some hints in the official example file. This first one does not automatically reboot for me.


xxxxxxxxxx
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

I have not tried this second tip, but it could work.


xxxxxxxxxx
# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.
#d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
#d-i debian-installer/exit/poweroff boolean true

Late Command

In ubiquity, that's now called success_command and you can in addition also specify a failure_command.

You can only specify one of each, and each can only be on one single line unless you use line continuation characters. Because that gets annoying, I have moved all my late commands to bash files and only have this at the end of my preseed file:


xxxxxxxxxx
ubiquity ubiquity/success_command \
    string bash -c 'sudo /cdrom/preseed/l8r.bootstrap.sh |& sudo tee /target/var/log/lucidbrot.log'

I will show you more about this in the next section.

Configuring Software

I have added two files to iso.new/preseed that contain my commands for installing and configuring applications. One is called l8r.sh (but you could choose your own name) and is responsible for the actual installation process. The other one is called l8r.bootstrap.sh and is responsible for setting up the environment and logging. Of course, both are executable and have a shebang.

l8r.bootstrap.sh

This file is run from my ubiquity/success_command instruction and hence from the installer's perspective - not in the target filesystem.

In the installer, /target represents the target filesystem and /cdrom represents the USB Stick.


xxxxxxxxxx
#!/bin/bash
# expects to be run from anywhere where both /target and /cdrom are mounted
# This echo goes to stdout - which my preseed file pipes on to /target/var/log/lucidbrot.log
echo "starting late commands"
# Enable debugging traces
set -x
# Create a directory for us to use
mkdir /target/install
# Test and Log whether internet and DNS resolving works
echo "--- networking diagnostics in initramfs ---"
ping -c 1 8.8.8.8
ping -c 1 www.google.com
nmcli d wifi list
echo "--- preparing dns resolve in target ---"
sudo cp /etc/resolv.conf /target/etc/resolv.conf
sudo cat /target/etc/resolv.conf
# Prepare the environment for our setup script
echo "--- preparing execution of l8r.sh ---"
sudo cp '/cdrom/preseed/l8r.sh' '/target/install/l8r.sh'
sudo mount -t proc /proc /target/proc/
sudo mount --rbind /sys /target/sys/
sudo mount --rbind /dev /target/dev/
sudo mkdir /target/cdrom
sudo mount --rbind /cdrom /target/cdrom
sudo chroot /target bash -c "chmod +x /install/l8r.sh"
# Execute our setup script in the target
echo "--- executing l8r.sh ---"
sudo chroot /target bash -c 'sudo /install/l8r.sh'
# Undo any environment changes. If that fails, it's okay - we will reboot afterwards anyway.
echo "--- cleaning up ---"
sudo umount /target/dev
sudo umount /target/sys
sudo umount /target/proc
sudo umount /target/cdrom
echo "finished late commands."

l8r.sh

Anything that works in a root bash shell in the target system should also work from this file.

Consider this an example file. I am not showing you everything I'm doing, because I'm still trying things out. There might follow another post on this.


xxxxxxxxxx
#!/bin/bash
# To be executed as superuser by ubiquity on success_command
# Stop execution on error, Enable debugging output
set -ex
# wipe and recreate the apt-get lists because why not. The internet recommended this. Never doubt the internet.
rm -rf /var/lib/apt/lists
apt-get -y update
# This was recommended by the internet, but it does not run in chroot... or rather:
# This simply required the mounting of /proc /dev and /sys (see my.seed success_command l8r.bootstrap.sh)
apt-get -y dist-upgrade
# install some basic tools
apt-get -y install wget curl vim-gnome
# prepare install typora
wget -qO - https://typora.io/linux/public-key.asc | apt-key add -
add-apt-repository 'deb https://typora.io/linux ./'
# prepare install syncthing
curl -s https://syncthing.net/release-key.txt | apt-key add -
echo "deb https://apt.syncthing.net/ syncthing stable" | tee /etc/apt/sources.list.d/syncthing.list
# install typora and syncthing
apt-get -y update
apt-get -y install typora syncthing
apt-get -y upgrade
# install some tweaks for gnome
apt-get -y install gnome-tweak-tool
# and configure them
# Nightlight
gsettings set org.gnome.settings-daemon.plugins.color night-light-enabled true
# Enable Mousepad while typing
dconf write /org/gnome/desktop/peripherals/touchpad/disable-while-typing false
# Increase Trackpad speed by skipping some pixels
# TODO: machine-dependent
# xinput --set-prop 'SYNA8004:00 06CB:CD8B Touchpad' 'Coordinate Transformation Matrix' 2.100000 0.000000 0.000000 0.000000 2.100000 0.000000 0.000000 0.000000 1.000000
# Show battery percentage at the top right
dconf write /org/gnome/desktop/interface/show-battery-percentage true
# Show the Month and Day in addition to the time in the top middle
dconf write /org/gnome/desktop/interface/clock-show-date true
# install ifconfig and filezilla
apt-get -y install net-tools filezilla
# install telegram
snap install telegram-desktop
# do some apt cleanup and hope nothing breaks
apt-get -y autoremove
apt-get -y autoclean
apt-get clean
# set up correct keyboard layouts and variants
# Apparently, in my "correct" setup, the localectl is being ignored anyways...
#
# temporary for this X11 session - but it doesn't work
# setxkbmap -layout ch,us -variant de_nodeadkeys -options 'compose:lwin-altgr'
# permanent for future sessions - but it also doesn't work
# localectl set-x11-keymap ch,us pc105 de_nodeadkeys 'compose:lwin-altgr'
# 
# I guess gnome is to blame
# Alternative way to activate the compose key and set the keyboard.
# Win-AltGr does rarely work with this setting - just use AltGr-Win instead.
dconf write /org/gnome/desktop/input-sources/xkb-options "['compose:lwin-altgr']"
# Order matters - it's priority
dconf write /org/gnome/desktop/input-sources/sources "[('xkb', 'ch+de_nodeadkeys'), ('xkb', 'us')]"
# mru-sources stands for most-recently-used sources and hence should not be important.
# dconf write /org/gnome/desktop/input-sources/mru-sources "[('xkb', 'ch+de_nodeadkeys'), ('xkb', 'us')]"
# log success
echo "Lucidbrot l8r.sh finished."

Anything else that you'd like to run automatically after the OS is installed successfully, you can add in here.

Note that I wrote this file so that it instantly stops on an error. If you don't like this, remove set -e.

You find the logs of the latest install in /var/log/lucidbrot.log when you're normalled logged into the targed system.

Comments

Any reactions, replies or comments are welcome at comments@mink.li.

If you have any addition, I'll be happy to edit this article.

TODO

These are things that I am still working on automating fully and that I would like to include in a future post.

Hibernate (See here)
Automatically configuring Syncthing without confirming on other devices
Automatically configuring Telegram without confirming 2FA
GNOME Workspaces Shortcuts (See here)
Copying over dotfiles
OwnCloud Mountpoint (See here)
Block Caribou
Install Anaconda and further useful programs
Using qemu to boot a dual-boot windows partition within linux

Licence

You are free to quote me and to improve on my work and redistribute it. You need to attribute me. You may not use my work for commercial purposes.

In more formal terms: This article is distributed under the CC BY-NC-SA 4.0 Licence which you can find here