Archived
beatzaplentyandClaude Sonnet 5
2aa625d566
Both existed only so the installer could boot as an LXC container and nixos-install some other host from within it, but lxc-* targets are already excluded from the install menu (nixos-install can't touch its own running root filesystem), and now have their own direct tarball path anyway. That left the installer's own LXC form with no real use case, and packages.all with only two members worth bundling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
203 lines
9.0 KiB
Markdown
203 lines
9.0 KiB
Markdown
# Auto-installer
|
|
|
|
This flake builds a self-contained NixOS installer environment that can
|
|
install any host exposed by its own `nixosConfigurations`. It was migrated
|
|
from a formerly-separate `nix-auto-installer` repo — everything it did now
|
|
lives here.
|
|
|
|
The installer provides a small NixOS install environment (ISO, or the same
|
|
image netbooted via PXE) with SSH access, Git support, and an interactive
|
|
installation script.
|
|
Logging in as any user (root or `nixos`) runs `/etc/auto-install.sh`,
|
|
discovers available hosts from this same flake, lets the operator choose a
|
|
target, applies that host's Disko storage configuration, installs NixOS, and
|
|
reboots.
|
|
|
|
**This applies to every `nixosConfigurations` target except `lxc-*` hosts —
|
|
see "LXC hosts" immediately below for why those are different.**
|
|
|
|
## LXC hosts
|
|
|
|
`lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
|
|
`lxc-gui`, `lxc-pxe-boot`) are **not** installed via `auto-install.sh` — the
|
|
interactive menu deliberately excludes them. Don't try to select one there;
|
|
`nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
|
|
disk to partition) and then refuse to touch the filesystem it's currently
|
|
running on — it's designed to protect exactly this case, so it just fails.
|
|
|
|
`modules/platforms/lxc.nix` imports nixpkgs' own
|
|
`virtualisation/proxmox-lxc.nix` module, which gives every `lxc-*` host a
|
|
`config.system.build.tarball` output — a complete, directly Proxmox-importable
|
|
container image, no install step at all:
|
|
|
|
```sh
|
|
nix build .#nixosConfigurations.lxc-minimal.config.system.build.tarball
|
|
```
|
|
|
|
Then, on the Proxmox host, `pct restore` (or the GUI's "Create CT" → upload
|
|
template flow) that tarball directly as a new container. First boot runs
|
|
`boot.postBootCommands` (registers the Nix store DB and system profile) —
|
|
there's no separate activation step to run yourself.
|
|
|
|
Host keys still need pre-seeding the same way as any other host (see "Host
|
|
keys" below) — the sops-nix activation-vs-first-boot race is identical
|
|
regardless of how the image reaches the machine. `NIXOS_HOST_KEYS_DIR=...
|
|
nix build ... --impure` bakes the matching key into the tarball the same way
|
|
it does for the ISO/PXE installer images.
|
|
|
|
## Layout
|
|
|
|
- `modules/installer/common.nix` — shared by every installer target: SSH
|
|
access, users, the generated `/etc/auto-install.sh` script, and the
|
|
`programs.bash.loginShellInit` hook that runs it on login.
|
|
- `modules/installer/iso.nix` — ISO/netboot-specific: imports the stock
|
|
`installation-cd-minimal.nix` module plus `common.nix`. Also used, paired
|
|
with `netboot-minimal.nix`, to build the PXE netboot variant (see
|
|
`docs/pxe-boot.md`).
|
|
- `modules/installer/host-keys.nix` — optionally bakes pre-generated SSH
|
|
host keys into the image; see "Host keys" below.
|
|
- `scripts/prepare-host-key.sh` — admin-workstation pre-flight tool, see
|
|
"Creating a New Machine" below.
|
|
|
|
Flake outputs:
|
|
|
|
```nix
|
|
nixosConfigurations.installer # ISO/netboot installer image
|
|
|
|
packages.x86_64-linux.iso # installer ISO/netboot image
|
|
packages.x86_64-linux.pxe # netboot-ipxe + netboot-initrd + netboot-kernel, bundled
|
|
```
|
|
|
|
```sh
|
|
nix build .#iso
|
|
nix build .#pxe
|
|
```
|
|
|
|
There's no `nixosConfigurations.proxmox-lxc` (installer-boots-as-an-LXC-
|
|
container) or `packages.x86_64-linux.lxc`/`.all` anymore. Both existed only
|
|
to let the installer itself run as an LXC container so you could
|
|
`nixos-install` some *other* host from within it — but LXC targets are
|
|
excluded from the install menu (same bind-mount problem as any LXC
|
|
`nixos-install`), and now have their own direct tarball path anyway (see
|
|
"LXC hosts" above), which left the installer's own LXC form with no real
|
|
use case.
|
|
|
|
The `pxe` variant is also built automatically as part of the `pxe-boot` host
|
|
itself (`modules/pxe-boot/stage-installer-artifacts.nix`) and served over
|
|
iPXE — see `docs/pxe-boot.md`.
|
|
|
|
## Host keys
|
|
|
|
`sops-nix` derives each host's decryption key from its own
|
|
`/etc/ssh/ssh_host_ed25519_key`, generated at **activation** time — before
|
|
systemd would otherwise generate one on first boot. Without pre-seeding this
|
|
key, secrets (including the root/nixos login password) fail to decrypt on a
|
|
genuinely fresh install.
|
|
|
|
Generated host keys live in `host-keys/` at the repo root (`ssh_host_ed25519_key`
|
|
+ `.pub` pairs per hostname). This directory is **gitignored on purpose** —
|
|
private key material must never be committed — which also means flakes can't
|
|
see it through a normal relative path. `modules/installer/host-keys.nix`
|
|
reads it through `builtins.getEnv`, which Nix silently returns as an empty
|
|
string under normal (non-`--impure`) evaluation, so the module is a no-op —
|
|
safe by default, including in CI — unless explicitly opted into:
|
|
|
|
```sh
|
|
NIXOS_HOST_KEYS_DIR="$(pwd)/host-keys" nix build .#iso --impure
|
|
```
|
|
|
|
When built this way, every key currently in `host-keys/` is baked into the
|
|
image at `/etc/host-keys/<hostname>_ssh_host_ed25519_key(.pub)`, and
|
|
`auto-install.sh` automatically installs whichever one matches the flake
|
|
target selected at install time — no manual per-host scp step needed.
|
|
|
|
**Trade-off, accepted deliberately for this LAN-only setup:** baking keys in
|
|
means every key present in `host-keys/` at build time becomes readable by
|
|
anyone who can reach the built image — including, for the PXE variant, anyone
|
|
who can reach the `pxe-boot` host's unauthenticated HTTP server. This is
|
|
considered acceptable here because `pxe-boot` sits behind LAN-only network
|
|
infrastructure, not the open internet. If that ever changes, reconsider this
|
|
default.
|
|
|
|
`auto-install.sh` still supports the older manual path as a fallback: if a
|
|
host's key isn't baked in (`/etc/host-keys`), it checks `/root/host-keys`
|
|
next, where you can `scp` a key in after boot, same as before this migration.
|
|
|
|
## Storage
|
|
|
|
Disk partitioning is handled by Disko — the installer has no hardcoded
|
|
`parted`/`mkfs`/`mkswap`/`mount` commands. Each host defines its own layout
|
|
via `disko.devices`; a host without one is installed in place via bind-mount
|
|
(this is how every LXC container target works, since they have no raw block
|
|
device to partition).
|
|
|
|
## Installer process
|
|
|
|
`/etc/auto-install.sh`:
|
|
|
|
1. Queries `nixosConfigurations` from this flake over the network (`git+https://<lanDomain>/beatzaplenty/nixos.git`) — this happens at *install* time, not build time, so a generic installer image always sees whatever hosts are currently committed, without needing a rebuild.
|
|
2. Presents them as a menu; confirms the choice.
|
|
3. Skips the `nix-cache` substituter when installing a `nix-cache` host itself (consistent with that host's own runtime config).
|
|
4. Runs `disko --mode destroy,format,mount` if the target has a Disko config, otherwise bind-mounts `/` onto `/mnt`.
|
|
5. Installs the target's SSH host key from `/etc/host-keys` or `/root/host-keys` (see "Host keys" above).
|
|
6. Runs `nixos-install --flake <url>#<choice> --no-root-password`.
|
|
7. Cleans up and reboots.
|
|
|
|
## Creating a new machine
|
|
|
|
Do this instead of jumping straight to a plain install whenever the target
|
|
host consumes any sops-nix secret — as of this writing, that's every host
|
|
(`modules/common/configuration.nix` puts the root/nixos password hash and the
|
|
GitHub token behind sops-nix for all of them).
|
|
|
|
1. **On your admin workstation**, decide the new machine's flake target name
|
|
(`<platform>-<buildtype>`, e.g. `proxmox-server`) and generate + register
|
|
its host key:
|
|
|
|
```sh
|
|
./scripts/prepare-host-key.sh <hostname>
|
|
```
|
|
|
|
This generates `host-keys/<hostname>_ssh_host_ed25519_key(.pub)` and
|
|
prints the `.sops.yaml` snippet to add, which secrets files to
|
|
re-encrypt it into, and the next steps below.
|
|
|
|
2. **Edit `.sops.yaml`** with the printed snippet, then re-encrypt every
|
|
secrets file you added the new host to:
|
|
|
|
```sh
|
|
nix-shell -p sops --run 'sops updatekeys secrets/common.yaml'
|
|
```
|
|
|
|
3. **Commit and push.** The flake build the installer uses has to see the
|
|
new recipient before you install, or decryption fails on first boot
|
|
regardless of the next step.
|
|
|
|
4. **Build the installer image with keys baked in** (or reuse an already-serving `pxe-boot` host, which does this automatically once redeployed):
|
|
|
|
```sh
|
|
NIXOS_HOST_KEYS_DIR="$(pwd)/host-keys" nix build .#iso --impure
|
|
```
|
|
|
|
5. **Boot it on the target machine**, log in, select the new host's flake
|
|
target from the menu, confirm. `auto-install.sh` finds the baked-in key,
|
|
runs Disko + `nixos-install`, and reboots.
|
|
|
|
6. **Verify after reboot:**
|
|
|
|
```sh
|
|
ssh <new-host> ls /run/secrets/
|
|
```
|
|
|
|
If that's empty or login fails, the host's age key most likely wasn't in
|
|
`.sops.yaml` (or wasn't re-encrypted into the secrets file it needs) when
|
|
`nixos-install` ran — fix `.sops.yaml`/`secrets/*.yaml`, push, then re-run
|
|
`nixos-install --flake .#<hostname> --no-root-password` from a rescue
|
|
environment against the existing `/mnt`, or just redo the install.
|
|
|
|
## Safety
|
|
|
|
This installer is destructive: `disko --mode destroy,format,mount` erases
|
|
any disk defined by the selected host's Disko configuration. Always verify
|
|
the selected host profile and target machine before confirming.
|