Why Nix and NixOS are the future of software deployment
Nix
The traditional Linux package management workflow is great in theory: you are able to install a package from a centralized source without annoying installers and with automatic dependency resolution.
That should be enough, right?
No, it’s not.
What happens if two packages need two different versions of the same library?
[root@arch test]# pacman -Syu
:: Starting full system upgrade...
:: Replace dbus-python with extra/python-dbus? [Y/n]
resolving dependencies...
looking for conflicting packages...
:: ffmpeg-2:7.0.1-1 and ffmpeg-obs-6.1.1-10 are in conflict. Remove ffmpeg-obs? [y/N]
error: unresolvable package conflicts detected
error: failed to prepare transaction (conflicting dependencies)
:: ffmpeg-2:7.0.1-1 and ffmpeg-obs-6.1.1-10 are in conflicts
These problems with the traditional approach aren’t new and here are a few that I consider very important:
- No atomic package upgrades
There is a time window in which we have some of the files of the old version, and some of the new version – Eelco Dolstra
- No rollbacks
When we upgrade components, it is important to be able to undo, or roll back the effects of the upgrade, if the upgrade turns out to break important functionality. This requires both that we remember what the old configuration was, and that we have some way to reproduce the old configuration – Eelco Dolstra
- Upgrading dynamic libraries often breaks applications
This is the well-known DLL hell, where upgrading or installing one application can cause a failure in another application due to shared dynamic libraries. It has been observed that software systems often suffer from the seemingly inexplicable phenomenon of “bit rot,” i.e., that applications that worked initially stop working over time due to changes in the environment. – Eelco Dolstra
That’s where Nix comes in: it stores packages in isolation using cryptographic hashes in a central store (e.g., /nix/store/m9s6nrg172nc055vsyr27avlzxb7gd7h-ffmpeg.drv), preventing undeclared dependencies and enabling multiple versions to coexist and also multiple versions of dynamic libraries. It supports lock files like, for instance, the cargo package manager and can be used in almost any Linux distribution and macOS.
Nix packages are defined using the Nix programming language, a purely functional and declarative language designed for reproducible builds.
Nix is not a container, and so it doesn’t suffer from the same common Docker problems:
- Build once, Run anywhere but actually not.
FROM foo:latestis a terrible practice because it’s inherently not reproducible and Docker doesn’t have the concept of lock files. - Docker-based development IDE support is tricky, especially for LSPs and debugging tools. Since Nix integrates directly with the system environment, you get native IDE support without the complexity of containerized workarounds.
I am not saying that Docker is not useful, it is, but not for every use case (you can even build Docker images with Nix!)
With Nix you start a shell with Ansible 2.16 and another with 2.17 in the same system just by running nix-shell -p ansible_2_16 and nix-shell -p ansible_2_17.
Nix drawbacks
Nix breaks many standard Linux conventions and therefore has some drawbacks:
- Downloading random application binaries from the internet is unlikely to work as they assume system libraries are in standard locations.
nix-ldandenvfsprovide a mostly pain-free compatibility layer, but they break Nix’s core principles of purity and reproducibility. - It has quite a big learning curve largely due to being a functional programming language.
What about NixOS?
NixOS is a full operating system built around the principles of Nix.
Instead of manually configuring system dependencies, NixOS lets you declare your entire system state in a config file (configuration.nix), making rollbacks and reproducibility trivial.
The following snippet declares a minimal system with the sshd systemd service that only accepts key logins, the systemd-boot bootloader and a few packages available at the shell:
{
lib,
config,
pkgs,
nixpkgs,
...
}:
{
imports = [
./hardware-configuration.nix
# Minimal stuff
(nixpkgs.outPath + "/nixos/modules/profiles/minimal.nix")
];
networking.hostName = "bar";
users.users = {
alice = {
initialPassword = "bob";
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAQQCo9+BpMRYQ/dL3DS2CyJxRF+j6ctbT3/Qp84+KeFhnii7NT7fELilKUSnxS30WAvQCCo2yU1orfgqr41mM70MB foobar"
];
# sudoer
extraGroups = [ "wheel" ];
};
};
# sshd systemd service
services.openssh = {
enable = true;
settings = {
PermitRootLogin = "no";
PasswordAuthentication = false;
};
};
# Bootloader
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
boot.loader.systemd-boot.configurationLimit = 10;
environment.systemPackages = [ pkgs.ansible pkgs.htop ];
# This value determines the NixOS release from which the default
# settings for stateful data, like file locations and database versions
# on your system were taken. It‘s perfectly fine and recommended to leave
# this value at the release version of the first install of this system.
# Before changing this value read the documentation for this option
# (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
system.stateVersion = "24.11"; # Did you read the comment?
}
NixOS VM tests
NixOS has built-in support for VM-based integration testing. It provides a declarative way to define and run system tests using QEMU-based virtual machines. It includes a very good python framework to write the tests.
With NixOS VM tests, you can:
- Define a full system environment with services, users, and network configurations.
- Spin up multiple VMs to simulate real-world distributed systems like testing VPN or DNS servers connectivity with their clients.
- Test declaratively in a hermetic (fully isolated with no Internet access) environment.
- Have a CI for your IaC.
For example, this is the upstream NixOS headscale integration test:
import ./make-test-python.nix (
{ pkgs, lib, ... }:
let
tls-cert = pkgs.runCommand "selfSignedCerts" { buildInputs = [ pkgs.openssl ]; } ''
openssl req \
-x509 -newkey rsa:4096 -sha256 -days 365 \
-nodes -out cert.pem -keyout key.pem \
-subj '/CN=headscale' -addext "subjectAltName=DNS:headscale"
mkdir -p $out
cp key.pem cert.pem $out
'';
in
{
name = "headscale";
meta.maintainers = with lib.maintainers; [
kradalby
misterio77
];
nodes =
let
headscalePort = 8080;
stunPort = 3478;
peer = {
services.tailscale.enable = true;
security.pki.certificateFiles = [ "${tls-cert}/cert.pem" ];
};
in
{
peer1 = peer;
peer2 = peer;
headscale = {
services = {
headscale = {
enable = true;
port = headscalePort;
settings = {
server_url = "https://headscale";
ip_prefixes = [ "100.64.0.0/10" ];
derp.server = {
enabled = true;
region_id = 999;
stun_listen_addr = "0.0.0.0:${toString stunPort}";
};
dns.base_domain = "tailnet";
};
};
nginx = {
enable = true;
virtualHosts.headscale = {
addSSL = true;
sslCertificate = "${tls-cert}/cert.pem";
sslCertificateKey = "${tls-cert}/key.pem";
locations."/" = {
proxyPass = "http://127.0.0.1:${toString headscalePort}";
proxyWebsockets = true;
};
};
};
};
networking.firewall = {
allowedTCPPorts = [
80
443
];
allowedUDPPorts = [ stunPort ];
};
environment.systemPackages = [ pkgs.headscale ];
};
};
testScript = ''
start_all()
headscale.wait_for_unit("headscale")
headscale.wait_for_open_port(443)
# Create headscale user and preauth-key
headscale.succeed("headscale users create test")
authkey = headscale.succeed("headscale preauthkeys -u test create --reusable")
# Connect peers
up_cmd = f"tailscale up --login-server 'https://headscale' --auth-key {authkey}"
peer1.execute(up_cmd)
peer2.execute(up_cmd)
# Check that they are reachable from the tailnet
peer1.wait_until_succeeds("tailscale ping peer2")
peer2.wait_until_succeeds("tailscale ping peer1.tailnet")
'';
}
)
It starts the VMs headscale, peer1, peer2 and checks connectivity between the peers through the VPN.