Ubuntu

Open source tools like Ollama and Open WebUI are convenient for building local LLM inference stacks that let you create a ChatGPT-like experience on your own infrastructure. Whether you are a hobbyist, someone concerned about privacy, or a business looking to deploy LLMs on-premises, these tools can help you achieve that. Prerequisites We assume here that you are running an LTS version of Ubuntu (NVIDIA and AMD tooling is best supported on LTS releases) and that you have a GPU installed on your machine (either NVIDIA or AMD). If you don’t have a GPU, you can still follow this guide, but inference will be much slower as it will run on CPU. ...

Intro A few years ago I built a tool called genesis to build images of Ubuntu. The tool has a very basic CLI interface and is written in Python. I was asked recently if I had ever built desktop images of Ubuntu from scratch and had to admit that I had never tried. I decided to give it a go and found the process rather straight forward. Steps First, install genesis in a venv: ...

For a while, I wanted to set up an Ubuntu archive mirror using Cloudflare. It felt like a natural idea: the archive is a set of static files that could be easily cached, and Cloudflare is very good at caching files close to users around the world. What is an archive mirror? If you have ever run apt update on Ubuntu, you have used the archive. It is a big collection of files: packages (.deb files) and index files (Packages.gz, Release, etc.) that tell apt what is available. ...

Make sure the system is up-to-date This section is important to avoid pulling DKMS NVIDIA drivers during the installation. First make sure your server is up-to-date: 1 2 sudo apt update sudo apt full-upgrade -y If your system needs reboot, reboot it before running: 1 sudo apt autoremove -y Note: You can check if your system needs to be rebooted by checking if this file exists: /var/run/reboot-required. ...

Basic Config [Distribution] Distribution=ubuntu [Output] Format=disk [Content] Packages= apt cloud-init dbus grub-efi-amd64-signed iproute2 linux-virtual netplan.io openssh-server openssl shim-signed ssh-import-id sudo systemd systemd-resolved udev vim Bootloader=grub ShimBootloader=signed BiosBootloader=none Bootable=true RootPassword=ubuntu KernelCommandLine=console=ttyS0 Hostname=ubuntu Then simply run mkosi. Boot the image Use this script (use –no-snapshot to make the changes persist): 1 ./start-vm.sh ./image.raw To go further At the moment, mkosi only supports producing raw disk images. To convert the image to qcow2: 1 qemu-img convert -f raw -O qcow2 /tmp/image.raw /tmp/ubuntu.img And to make it (virtually) bigger: ...

Boot process - context In general, the boot process looks like this: ROM | DISK Pre-EFI initialization -> EFI firwmare -|> shim -> grub -> Linux The pre-efi initialization is about initializing CPU and devices. Especially, it is responsible for initializing the DRAM controller on the CPU. Before this step the system is in a very precurious state and can only use its cache as memory (aka Cache as RAM). ...

systemd-tmpfiles /tmp and other temporary directories and files are now managed by systemd and are not tmpfs. sytemd-tmpfiles gives to the user the ability to choose what they want to do with temporary directories/files. There are a ton of options that the user can choose from and everything is managed though config files (see man tmpfiles.d). Create a temporary directory in your HOME folder Using the global /tmp directory can be security issue as anyone can read this directory. If the user is not very carefull with the permissions they set on their files, confidential information might leak. Also, programs packaged with snap cannot access the global temporary directory /tmp by default. ...

Genesis a CLI project written in Python. It can build Ubuntu images from scratch. The tool is named genesis (because you start from nothing). And is available as a python package: https://github.com/gjolly/genesis (it’s also packaged as a deb in a PPA. A basic example We are going to create a very minimal image of Ubuntu 23.04 (Lunar Lobster) and try to boot from it using qemu. Creating a base image First you want to start by bootstrapping a basic filesystem: ...

The flow In order to decrypt the root filesystem, the kernel uses a initial ram disk (initramfs). The initramfs provides an temporary filesystem from which extra kernel modules can be loaded, it also contains a set of scripts used to boot the system including scripts to decrypt the user’s root filesystem. This initramfs image is a file stored un-encrypted next to the kernel image. However, unlike the kernel image, it is not signed by the kernel publisher as the iniramfs is generated locally and can be modified by the user. Thus, anyone with physical access to the user’s drive can inject a malicious initramfs that would log the user’s passphrase and thus make FDE useless. ...

I recently enabled the Firewall on my desktop on Ubuntu. I probably did a quick lookup online to find out that sudo ufw enable was enough to enable it. I entered the command and forgot about it. 1 2 $ sudo ufw enable Firewall is active and enabled on system startup Obviously, (and to be honest I was waiting for it), it didn’t take long for things to go bad. A few weeks later, while I was not at home and wanted to SSH on my machine via tailscale, I realized that I couldn’t and quickly remember about the Firewall. ...