Browse Source

Fledged out NixOS LXC

Victor Roest 2 years ago
Signed by: 0x76 GPG Key ID: A3923C699D1A3BDA
  1. 1
  2. BIN
  3. 10
  4. 142
  5. 2


@ -0,0 +1 @@
*.png filter=lfs diff=lfs merge=lfs -text

assets/images/nixos-buildproduct.png (Stored with Git LFS)

Binary file not shown.


@ -2,9 +2,17 @@ baseURL:
languageCode: en-gb
title: Blog
theme: PaperMod
enableRobotsTXT: true
copyright: "Source available on [Gitea]("
enableRobotsTXT: true
buildDrafts: false
buildFuture: false
buildExpired: false
disableXML: true
minifyOutput: true
env: production
description: "A blog about things"


@ -1,5 +1,5 @@
title: "Proxmox NixOS LXC"
title: "NixOS inside LXC"
date: 2021-02-21T18:56:48+01:00
draft: true
@ -9,8 +9,138 @@ categories:
- tutorial
* set networking to dhcp inside of proxmox
* (apparmor)
* + unpriv
* (proc)
This tutorial will go through on how to install NixOS as an LXC container inside of proxmox.
## Getting the container tarball
Go to the pipeline for the [NixOS 20.09 Container Tarball][nixos-tar]
Then click on the latest successful build and download the corresponding `.tar.xz`.
{{< img path="images/nixos-buildproduct.png" >}}
After it's downloaded we should rename the file to follow proxmox conventions (recommended but optional):
mv nixos-system-x86_64-linux.tar.xz nixos-$RELEASE-default_$BUILDID_amd64.tar.xz
### Uploading to proxmox
Uploading it to proxmox is quite easy just go to your storage,
most likely called "local" then in "CT Templates" and click on upload and upload the tarball.
## Creating the container
To create the container on proxmox we need to either `ssh` into it or use the web shell.
After in a shell on the proxmox host execute the following command. But, make sure you understand what the options
do before executing it. You can see the [proxmox docs][proxmox-lxc] if you are unsure.
pct create $(nextid) \
--arch amd64 \
--description nixos-template \
local:vztmpl/nixos-$RELEASE-default_$BUILDID_amd64.tar.xz \
--ostype unmanaged \
--net0 name=eth0 \
--storage local-lvm \
--unprivileged 1
after running this the container should show up in the Proxmox Web UI.
### Fix LXC config
After creating the container we need to make a simple edit to the lxc config file located in `/etc/pve/lxc/$ID.conf`,
`$ID` being the ID you passed in the previous step, if unsure you can check the web UI.
After opening the file add the following line to the bottom:
lxc.init.cmd: /init
This will point proxmox to the correct init binary.
### Fix Network settings
If you didn't specify a full network configuration during container creation you must now
do so in the web UI or else the container won't start. The easiest being to just set ipv4 and ipv6 to dhcp.
### In-Container tweaks
Now you can finally start up the container! But we are not done yet, we need to set some minor settings
to make NixOS play nice with the fact that it is running inside of an lxc container.
#### /proc fix
For some reason `/proc` is mounted with (to NixOS) unexpected permissions to fix this we need to
run the following:
mkdir -p /mnt/proc
mount -t proc proc /mnt/proc
#### Populate Nixpkgs
`nixpkgs` isn't properly initialized when booting a fresh container, which would result in errors when running other
nix commands, to fix this simply run:
nix-channel --update
#### configuration.nix tweaks
Finally we will add some tweaks inside of `/etc/nixos/configuration.nix`. The first one being simply making
the previously done `/proc` hack permanent, and the second one suppresses some annoying systemd service warnings.
# Make /proc hack permanent
fileSystems."/mnt/proc" = {
fsType = "proc";
device = "/proc";
# Supresses a bunch of systemd errors caused by running inside of LXC
systemd.suppressedSystemUnits = [
## Expected Errors
As this setup is fairly unconvential there are some errors that will occur when running nix,
especially `nixos-rebuild switch`. However as far as I can tell these don't seem to pose any real problem.
For example, these are the errors that I get
> nixos-rebuild switch
building Nix...
building the system configuration...
activating the configuration...
setting up /etc...
mount: /dev: cannot remount devtmpfs read-write, is write-protected.
mount: /dev/pts: cannot remount devpts read-write, is write-protected.
mount: /dev/shm: cannot remount tmpfs read-write, is write-protected.
mount: /proc: cannot remount proc read-write, is write-protected.
mount: /run: cannot remount tmpfs read-write, is write-protected.
mount: /run/keys: cannot remount ramfs read-write, is write-protected.
mount: /run/wrappers: cannot remount tmpfs read-write, is write-protected.
Activation script snippet 'specialfs' failed (32)
reloading user units for root...
setting up tmpfiles
fchownat() of /run/keys failed: Read-only file system
fchownat() of /run/keys failed: Read-only file system
warning: error(s) occurred while switching to the new configuration
nixos-rebuild switch 14.80s user 12.45s system 17% cpu 2:38.52 total
Some (or all) of these errors can be fixed by running a privileged and apparmor unconstrained container,
but it doesn't seem to have any real effect I wouldn't recommend doing this. There are some open issues
on the nix repos about these problems (linked below), but no real progress has been made it seems.
# References
This guide is based heavily on the resources below
* [Proxmox LXC Docs][proxmox-lxc]
* [NixOS LXC Docs][nixos-lxc]
* [NixOS issue 1 on LXC][nixos-issue-1]
* [NixOS issue 2 on LXC][nixos-issue-2]
* [Nix Generate issue on lxc][nix-generate-issue]


@ -0,0 +1,2 @@
{{ $image := resources.Get (.Get "path") }}
<img src="{{ $image.Permalink }}" alt="">