NixOS is awesome for personal servers! (1/?)

I want to share my awe and excitement. I've started configuring my personal server using NixOS and realized how eloquent and satisfying the process is in comparison to AWS or almost any other "serious" hosting platform.

Personal servers usually have different requirements than infrastructure for business apps. For me, a personal server:

  • is not about 99.999% uptime (nice to have, but not necessary)
  • is not about autoscaling (if something needs it - better to move into proper infra like fly.io).
  • should have low maintenance cost (so I do not waste time solving boring problems during weekends)
  • should be easy to experiment with (unlike AWS where complexity makes simple things unbearable sometimes)
  • should be fun to work with!

And my initial goals are:

  • host services for personal needs, such as:
    • FoundryVTT instance (to play Pathfinder with friends)
    • Forgejo instance (to store my private repos, mirror repos I want to preserve)
  • host this site and other simple apps (to avoid paying ~5 EUR for each separate app on fly.io)

In this article I will highlight the first steps: the things I did before installing actual services etc.

Home for the Server

vpsfree is my choice:

  • 1st class NixOS support! They also use Nix internally to manage their VPS cloud!
  • 12 EUR/month = up to 8 cores (3+ GHz), 4 GB RAM, 120 GB disk
  • non-profit organization - with my money I support community of enthusiasts, not merely one more hosting platform

After applying I had the VPS in less than 2 business days. Not in seconds like with business-oriented VPS providers, but in my perception it's ok because the whole process is about healthy long-term relationships with the provider, not "I pay - you serve" attitude.

On the server, configuration was placed in a pretty convenient way: there was an /etc/nixos/ folder with 2 files:

flake.nix. So I already had a nix flake to work with.

{
  description = "vpsAdminOS container";

  inputs = {
    vpsadminos.url = "github:vpsfreecz/vpsadminos";

    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs =
    inputs@{
      nixpkgs,
      vpsadminos,
      ...
    }:
    let
      system = "x86_64-linux";
    in
    {
      nixosConfigurations.vps = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          vpsadminos.nixosModules.containerUnstable
          ./configuration.nix
        ];
        specialArgs = {
          inherit inputs;
        };
      };
    };
}

Pay attention to vpsadminos.nixosModules.containerUnstable NixOS module. It applies all default settings for this VPS provider. As an outcome, my configuration only defines deviations from an "empty" VPS.

configuration.nix - NixOS module with some basic configurations.

{ inputs, lib, pkgs, ... }:
{
  nix.settings.experimental-features = [
    "nix-command"
    "flakes"
  ];

  environment.systemPackages = with pkgs; [
    vim
  ];

  services.openssh = {
    enable = true;

    # Allow root login with password, needed for passwords set through vpsAdmin
    settings.PermitRootLogin = "yes";

    # Needed for public keys deployed through vpsAdmin, can be disabled if you
    # authorize your keys in configuration
    authorizedKeysInHomedir = true;
  };

  # Add your public keys
  #users.users.root.openssh.authorizedKeys.keys =
  #  [ "..." ];

  systemd.settings.Manager = {
    DefaultTimeoutStartSec = "900s";
  };

  time.timeZone = "Europe/Amsterdam";

  system.stateVersion = "26.05";
}

And to apply the configuration, you merely need to run in the server's SSH session

nixos-rebuild --flake /etc/nixos#vps switch

Simplicity of Control

We have a VPS running with an SSH server. The VPS admin panel allows me to add my SSH key to root's .ssh/authorized_keys. I can also add the server IP to /etc/hosts. This means I can have a passwordless root session on the server by merely running

ssh root@server

But opening an SSH session and editing files on the server is not convenient: on the server I have Vim without plugins, while on my local machine I have my own IDE/PDE (also Vim, but with many adjustments). After some brainstorming I found an eloquent approach for my needs: to mount server's /etc/nixos/ folder (rclone+sftp) and edit its files locally!

Considering that I have nix installed on my local machine, the approach has impressive benefits!

To begin with, I do not need any additional server config! SFTP is provided by already installed OpenSSH. rclone needs to be installed only on the local machine.

I can edit files like they're local using my already configured IDE/PDE and tools. Nix-related formatters, linters, language servers, etc - all work in the same way as with any other repo.

I made it a git repo that I do not push anywhere at the moment. Commits are signed using my local SSH and GPG keys (some free consistency that is not critical, but nice to have). Effectively, I have configuration stored both on server and local machine: rclone actually copies files to your machine, so even if you're offline you can see and edit them. It may be inconvenient when your personal server is not that personal and multiple amigos can edit files simultaneously, but for just one person it is an incredibly straightforward setup!

I can validate that Nix code can be successfully evaluated by running the following command locally in the configuration folder:

# add --all-systems if your local machine has a different architecture from the VPS,
# for example when you're on macOS
nix flake check

I can even build the configuration locally if I have build problems, by running

nix build ".#nixosConfigurations.vps.config.system.build.toplevel" --print-out-paths --no-link

This requires the local system to have the same architecture or to be able to build for that architecture. It will print a local path of the build result where you can explore outputs. For example, in the case of successful local build you can read each systemd service definition or contents of /etc folder.

In which other setup can you build your server locally?

And to apply new configuration I do not even need to push or commit anything. Just

ssh root@server 'cd /etc/nixos && nixos-rebuild --flake .#vps switch'

To conclude:

  • with no additional server setup
  • and with only rclone configured locally
  • you have local backup of a server config
  • and can validate many of the server config traits locally
  • and edit your server config using familiar local tools
  • and apply config with just one command

And I haven't even highlighted all the benefits of NixOS itself like reverting to previous configuration with just one command, etc. Captivating, isn't it?

Local Setup with NixOS

I use NixOS on my local machine and it makes achieving the setup described above even simpler than it sounds. I use home-manager so my configuration is split between NixOS module and home-manager module parts.

Let's say that 10.20.30.40 is the VPS's public IP.

NixOS part is only about adjusting /etc/hosts:

{
  networking.hosts = {
    "10.20.30.40" = ["server"];
  };
}

home-manager part is the main player here. Also, it can be reused on any Linux with home-manager support, not only NixOS. (Replace ${username} with your username.)

{
  programs.zsh.shellAliases = {
    server-ssh     = "ssh root@server";
    server-update  = "ssh root@server 'cd /etc/nixos && nixos-rebuild --flake .#vps switch'";
  };

  # for port-forwarding
  programs.zsh.initContent = ''
    server-port() {
      ssh -N -L ''${1}:localhost:''${2} root@server
    }
  '';

  programs.rclone = {
    enable = true;

    remotes.server = {
      config = {
        type = "sftp";
        host = "server";
        user = "root";
        key_file = "~/.ssh/id_ed25519";
        known_hosts_file = "~/.ssh/known_hosts";
        shell_type = "unix";
      };

      # creates systemd service that is responsible for mounting
      mounts = {
        "/etc/nixos" = {
          enable = true;
          mountPoint = "/home/${username}/Projects/server";
        };
      };
    };
  };
};

The main benefit is that home-manager provides a built-in mechanism for rclone mounts: when I define a mount, a corresponding systemd service unit will be created to handle it. On non-NixOS systems, you would most probably need to do it manually: either call rclone mount when needed or write your own systemd user-level service unit.

For reference here is generated .config/rclone/rclone.conf

[server]
host=server
key_file=~/.ssh/id_ed25519
known_hosts_file=~/.ssh/known_hosts
shell_type=unix
type=sftp
user=root

Is It Perfect?

Of course not.

As you may have noticed, I haven't tuned rclone remote & mount options. As a result, git and any write operations are a bit laggy and can take a couple of seconds sometimes.

Also, sometimes git had problems with it and I had to delete the .git/index.lock file manually (the error message explicitly mentions this file, so it was an instant workaround).

Sometimes applying a new config would fail during the evaluation phase because not every file was synced by rclone. Waiting for several seconds and trying again was enough to solve it.

I believe all these problems can be solved by proper rclone configuration. I will do it later, my goal here is different - to highlight that even with default config the setup is pretty functional, rewarding and convenient to use.

Next Steps

In the upcoming articles I'll highlight rclone fine-tuning, deploying FoundryVTT and Forgejo, deploying SvelteKit apps and, most probably, Elixir apps.

So What?

I remember solving something similar with AWS several years ago. It was a convoluted experience. I felt relief when I finally got things configured.

Here, after hours spent with my new server I feel fulfillment and aesthetic pleasure.

To feel the same I recommend doing the process manually, without LLMs, or use them only as productivity helpers (not implementors). If you merely feed an LLM/agent this article and ask it to do the same, you risk losing all the joy of the process.

Comments

To leave a comment, visit the post on Mastodon. Comments to the post will be displayed here.