8 changed files with 132 additions and 147 deletions
@ -1 +1,2 @@ |
|||
*.png filter=lfs diff=lfs merge=lfs -text |
|||
*.webm filter=lfs diff=lfs merge=lfs -text |
|||
|
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,117 @@ |
|||
--- |
|||
title: "NixOS inside LXC" |
|||
date: 2021-02-21T18:56:48+01:00 |
|||
draft: false |
|||
tags: |
|||
- Proxmox |
|||
- NixOS |
|||
categories: |
|||
- tutorial |
|||
--- |
|||
|
|||
This tutorial will explain how to install NixOS as an LXC container inside of proxmox. |
|||
Do be warned that this setup isn't extensively tested for stability or officially supported. |
|||
|
|||
# Getting the container tarball |
|||
The first step is to download the NixOS container tarball. To do so |
|||
simply go to its [pipeline][nixos-tar]. And click on the latest successful build, and download the corresponding `.tar.xz` file. |
|||
{{< img path="images/nixos-buildproduct.png" alt="A screenshot of the NixOS container tarball download page" >}} |
|||
|
|||
After it's downloaded we should rename the file to follow proxmox conventions (recommended but optional): |
|||
```sh |
|||
mv nixos-system-x86_64-linux.tar.xz nixos-$RELEASE-default_$BUILDID_amd64.tar.xz |
|||
``` |
|||
|
|||
## Uploading to proxmox |
|||
Uploading it to proxmox is quite easy just go to your storage, |
|||
most likely called "local" then to "CT Templates" and click on upload and upload the tarball. |
|||
|
|||
{{< webm path="webm/upload-nixos-container.webm" >}} |
|||
|
|||
# Creating the container |
|||
To create the container on proxmox we need to either `ssh` into it or use the web shell. |
|||
After in a shell on the proxmox host execute the following command. But, make sure you understand what the options |
|||
do before executing it. You can see the [proxmox docs][proxmox-lxc] if you are unsure. |
|||
|
|||
```sh |
|||
pct create $(nextid) \ |
|||
--arch amd64 \ |
|||
--description nixos-template \ |
|||
local:vztmpl/nixos-$RELEASE-default_$BUILDID_amd64.tar.xz \ |
|||
--ostype unmanaged \ |
|||
--net0 name=eth0 \ |
|||
--storage local-lvm \ |
|||
--unprivileged 1 |
|||
``` |
|||
|
|||
after running this the container should show up in the Proxmox Web UI. |
|||
|
|||
## Fix LXC config |
|||
After creating the container we need to make a simple edit to the lxc config file located in `/etc/pve/lxc/$ID.conf`, |
|||
`$ID` being the ID you passed in the previous step, if unsure you can check the web UI. |
|||
|
|||
After opening the file add the following line to the bottom: |
|||
```conf |
|||
lxc.init.cmd: /init |
|||
``` |
|||
This will point proxmox to the correct init binary of NixOS. |
|||
|
|||
## Enable nesting |
|||
We are now done with all the cli configuration and need to enable a feature flag in the web interface. |
|||
Under container options, go to "Features" and enable "Nesting". This is needed as nix makes heavy use of sandboxing. |
|||
|
|||
{{< img path="images/nixos-proxmox-lxc-features.png" alt="Picture showing the proxmox featues dialog with nesting enabled" >}} |
|||
|
|||
_Do note the following (from the [proxmox wiki][proxmox-lxc])_ |
|||
> Nesting is best used with unprivileged containers with additional id mapping. Note that this will expose procfs and sysfs contents of the host to the guest. |
|||
|
|||
## Fix Network settings |
|||
If you didn't specify a full network configuration during container creation you must now |
|||
do so in the web UI or else the container won't start. The easiest being to just to set both ipv4 and ipv6 to dhcp. |
|||
|
|||
Do mind that these network settings seem mostly to just be defaults that are able to be overwritten inside of the container, |
|||
except things like MAC address presumably. |
|||
|
|||
## In-Container tweaks |
|||
Now you can finally start up the container! But we are not done yet, we need to set some minor settings |
|||
to make NixOS play nice with the fact that it is running inside of an lxc container. |
|||
|
|||
### Populate Nixpkgs |
|||
`nixpkgs` isn't properly initialized when booting a fresh container, which would result in errors when running other |
|||
nix commands, to fix this simply run: |
|||
```bash |
|||
nix-channel --update |
|||
``` |
|||
|
|||
### configuration.nix tweak |
|||
Finally we need to add a small tweak inside of `/etc/nixos/configuration.nix`. We will |
|||
supress the `sys-kernel-debug.mount` systemd unit as it will otherwise error upon |
|||
every invocation of `nixos-rebuild switch` which can be annoying. |
|||
```nix |
|||
# Supress sys-kernel-debug |
|||
systemd.suppressedSystemUnits = [ |
|||
"sys-kernel-debug.mount" |
|||
]; |
|||
``` |
|||
|
|||
## Done! |
|||
After doing the final tweaks you should be done and able to run \ |
|||
`nixos-rebuild switch` without any errors. |
|||
|
|||
{{< img path="images/nixos-rebuild-switch.png" alt="A screenshot of a succesful `nixos-rebuild switch` invocation" >}} |
|||
|
|||
# References |
|||
This guide is based heavily on the resources below |
|||
|
|||
* [Proxmox LXC Docs][proxmox-lxc] |
|||
* [NixOS LXC Docs][nixos-lxc] |
|||
* [NixOS issue 1 on LXC][nixos-issue-1] |
|||
* [NixOS issue 2 on LXC][nixos-issue-2] |
|||
* [Nix Generate issue on LXC][nix-generate-issue] |
|||
|
|||
[nixos-tar]: https://hydra.nixos.org/job/nixos/release-20.09/nixos.containerTarball.x86_64-linux |
|||
[nixos-lxc]: https://nixos.wiki/wiki/Proxmox_Linux_Container |
|||
[proxmox-lxc]: https://pve.proxmox.com/wiki/Linux_Container |
|||
[nixos-issue-1]: https://github.com/NixOS/nixpkgs/issues/9735 |
|||
[nixos-issue-2]: https://github.com/NixOS/nixpkgs/issues/43781 |
|||
[nix-generate-issue]: https://github.com/nix-community/nixos-generators/issues/41 |
@ -1,146 +0,0 @@ |
|||
--- |
|||
title: "NixOS inside LXC" |
|||
date: 2021-02-21T18:56:48+01:00 |
|||
draft: true |
|||
tags: |
|||
- Proxmox |
|||
- NixOS |
|||
categories: |
|||
- tutorial |
|||
--- |
|||
|
|||
This tutorial will explain on how to install NixOS as an LXC container inside of proxmox. |
|||
Do be warned that this setup isn't entirely stable or supported. |
|||
|
|||
# Getting the container tarball |
|||
Go to the pipeline for the [NixOS 20.09 Container Tarball][nixos-tar] |
|||
Then click on the latest successful build and download the corresponding `.tar.xz`. |
|||
{{< img path="images/nixos-buildproduct.png" >}} |
|||
|
|||
After it's downloaded we should rename the file to follow proxmox conventions (recommended but optional): |
|||
```sh |
|||
mv nixos-system-x86_64-linux.tar.xz nixos-$RELEASE-default_$BUILDID_amd64.tar.xz |
|||
``` |
|||
|
|||
## Uploading to proxmox |
|||
Uploading it to proxmox is quite easy just go to your storage, |
|||
most likely called "local" then in "CT Templates" and click on upload and upload the tarball. |
|||
|
|||
# Creating the container |
|||
To create the container on proxmox we need to either `ssh` into it or use the web shell. |
|||
After in a shell on the proxmox host execute the following command. But, make sure you understand what the options |
|||
do before executing it. You can see the [proxmox docs][proxmox-lxc] if you are unsure. |
|||
|
|||
```sh |
|||
pct create $(nextid) \ |
|||
--arch amd64 \ |
|||
--description nixos-template \ |
|||
local:vztmpl/nixos-$RELEASE-default_$BUILDID_amd64.tar.xz \ |
|||
--ostype unmanaged \ |
|||
--net0 name=eth0 \ |
|||
--storage local-lvm \ |
|||
--unprivileged 1 |
|||
``` |
|||
|
|||
after running this the container should show up in the Proxmox Web UI. |
|||
|
|||
## Fix LXC config |
|||
After creating the container we need to make a simple edit to the lxc config file located in `/etc/pve/lxc/$ID.conf`, |
|||
`$ID` being the ID you passed in the previous step, if unsure you can check the web UI. |
|||
|
|||
After opening the file add the following line to the bottom: |
|||
```conf |
|||
lxc.init.cmd: /init |
|||
``` |
|||
This will point proxmox to the correct init binary. |
|||
|
|||
## Fix Network settings |
|||
If you didn't specify a full network configuration during container creation you must now |
|||
do so in the web UI or else the container won't start. The easiest being to just set ipv4 and ipv6 to dhcp. |
|||
|
|||
## In-Container tweaks |
|||
Now you can finally start up the container! But we are not done yet, we need to set some minor settings |
|||
to make NixOS play nice with the fact that it is running inside of an lxc container. |
|||
|
|||
### /proc fix |
|||
For some reason `/proc` is mounted with (to NixOS) unexpected permissions to fix this we need to |
|||
run the following: |
|||
```bash |
|||
mkdir -p /mnt/proc |
|||
mount -t proc proc /mnt/proc |
|||
``` |
|||
|
|||
### Populate Nixpkgs |
|||
`nixpkgs` isn't properly initialized when booting a fresh container, which would result in errors when running other |
|||
nix commands, to fix this simply run: |
|||
```bash |
|||
nix-channel --update |
|||
``` |
|||
|
|||
### configuration.nix tweaks |
|||
Finally we will add some tweaks inside of `/etc/nixos/configuration.nix`. The first one being simply making |
|||
the previously done `/proc` hack permanent, and the second one suppresses some annoying systemd service warnings. |
|||
```nix |
|||
# Make /proc hack permanent |
|||
fileSystems."/mnt/proc" = { |
|||
fsType = "proc"; |
|||
device = "/proc"; |
|||
}; |
|||
|
|||
# Supresses a bunch of systemd errors caused by running inside of LXC |
|||
systemd.suppressedSystemUnits = [ |
|||
"console-getty.service" |
|||
"getty@.service" |
|||
"systemd-udev-trigger.service" |
|||
"systemd-udevd.service" |
|||
"sys-fs-fuse-connections.mount" |
|||
"sys-kernel-debug.mount" |
|||
"dev-mqueue.mount" |
|||
]; |
|||
``` |
|||
|
|||
# Expected Errors |
|||
As this setup is fairly unconvential there are some errors that will occur when running nix, |
|||
especially `nixos-rebuild switch`. However as far as I can tell these don't seem to pose any real problem. |
|||
|
|||
For example, these are the errors that I get |
|||
```shell |
|||
> nixos-rebuild switch |
|||
building Nix... |
|||
building the system configuration... |
|||
activating the configuration... |
|||
setting up /etc... |
|||
mount: /dev: cannot remount devtmpfs read-write, is write-protected. |
|||
mount: /dev/pts: cannot remount devpts read-write, is write-protected. |
|||
mount: /dev/shm: cannot remount tmpfs read-write, is write-protected. |
|||
mount: /proc: cannot remount proc read-write, is write-protected. |
|||
mount: /run: cannot remount tmpfs read-write, is write-protected. |
|||
mount: /run/keys: cannot remount ramfs read-write, is write-protected. |
|||
mount: /run/wrappers: cannot remount tmpfs read-write, is write-protected. |
|||
Activation script snippet 'specialfs' failed (32) |
|||
reloading user units for root... |
|||
setting up tmpfiles |
|||
fchownat() of /run/keys failed: Read-only file system |
|||
fchownat() of /run/keys failed: Read-only file system |
|||
warning: error(s) occurred while switching to the new configuration |
|||
``` |
|||
|
|||
Some (or all) of these errors can be fixed by running a privileged and apparmor unconstrained container, |
|||
but it doesn't seem to have any real effect I wouldn't recommend doing this. There are some open issues |
|||
on the nix repos about these problems (linked below), but no real progress has been made it seems. |
|||
|
|||
# References |
|||
This guide is based heavily on the resources below |
|||
|
|||
* [Proxmox LXC Docs][proxmox-lxc] |
|||
* [NixOS LXC Docs][nixos-lxc] |
|||
* [NixOS issue 1 on LXC][nixos-issue-1] |
|||
* [NixOS issue 2 on LXC][nixos-issue-2] |
|||
* [Nix Generate issue on lxc][nix-generate-issue] |
|||
|
|||
[nixos-tar]: https://hydra.nixos.org/job/nixos/release-20.09/nixos.containerTarball.x86_64-linux |
|||
[nixos-lxc]: https://nixos.wiki/wiki/Proxmox_Linux_Container |
|||
[proxmox-lxc]: https://pve.proxmox.com/wiki/Linux_Container |
|||
[nixos-issue-1]: https://github.com/NixOS/nixpkgs/issues/9735 |
|||
[nixos-issue-2]: https://github.com/NixOS/nixpkgs/issues/43781 |
|||
[nix-generate-issue]: https://github.com/nix-community/nixos-generators/issues/41 |
@ -1,2 +1,2 @@ |
|||
{{ $image := resources.Get (.Get "path") }} |
|||
<img src="{{ $image.Permalink }}" alt=""> |
|||
<img src="{{ $image.Permalink }}" alt="{{ (.Get "alt") }}"> |
|||
|
@ -0,0 +1,4 @@ |
|||
{{ $webm := resources.Get (.Get "path") }} |
|||
<video controls autoplay loop width="100%"> |
|||
<source src="{{ $webm.Permalink }}" type="video/webm"> |
|||
</video> |
Loading…
Reference in new issue