Gotchas¶

Quirks and footguns worth knowing before touching the stack. Each is also noted on its own service's page; this page is the consolidated view.

Environment¶

• sudo.ws, not sudo. Ubuntu 26.04 ships Rust sudo (sudo-rs) as the default sudo, and Ansible become hangs against it. ansible.cfg points become_exe at sudo.ws instead — this applies to every play. The pve host itself is Debian-based and keeps plain sudo in its host_vars override.
• proxmox_vm_validate_certs: false (group_vars/proxmox_vms.yml) is a deliberate but temporary relaxation: the test Proxmox host presents PVE's default self-signed cluster CA. Re-enable once the host has a trusted certificate.
• Linters (ansible-lint, yamllint) run from pipx on ~/.local/bin, which must be on PATH for make test/make lint to find them — they have gone missing between sessions before.

Known inconsistencies to verify¶

• vault_sysadmin_password is plaintext, not !vault-encrypted, in the live inventory/group_vars/all/vault.yml, unlike every other populated key in that file. Confirm this is intentional for the disposable test host before relying on it being vault-protected, and encrypt it before this stack touches anything that matters.
• timezone is never defined. Pi-hole and Nextcloud key their container timezone off a plain {{ timezone | default('Etc/UTC') }}, but no timezone variable exists anywhere in this repository — only common_timezone (Europe/London, consumed only by the common role) is set. Both appliances silently run on Etc/UTC. Worth fixing before relying on their logs/timestamps being in local time.
• Nagios is pinned below its originally intended tag. 4.5.9 does not exist on Docker Hub for jasonrivers/nagios; 4.5.7 (2024) is the newest tag actually published, and lags upstream Nagios Core. A move to a maintained image is tracked as separate follow-up work.

Authentication model exceptions¶

Forward-auth (authentik@swarm) is the fallback, used only where a service has no native OIDC/SAML — see each service's Auth column on the Services page. Three further exceptions to that rule, each deliberate:

• Emby, Navidrome: kept off Authentik entirely — their TV/mobile/Subsonic clients authenticate directly against the server, and a forward-auth gate on the router would break them.
• Lychee: kept off Authentik because it serves public shared galleries that anonymous visitors must reach without logging in.
• Matomo: uses native OIDC for admin login only; the tracking endpoints (/matomo.php, /matomo.js) must stay public for instrumented sites to post analytics, so there is no blanket forward-auth on the domain.
• Firefly-III: the opposite exception — forward-auth is not just a fallback here but load-bearing. AUTHENTICATION_GUARD=remote_user_guard disables Firefly's own login entirely and trusts the X-authentik-email header completely; removing the middleware without re-enabling native login would let a client spoof that header.
• Ontime: forward-auth currently gates its public output/overlay views too, which arguably should stay open (e.g. for a stream overlay). Flagged as known future work in the compose template, not yet split onto its own unauthenticated router.

Each multi-container stack has a private network¶

Every stack with more than one container defines its own private internal overlay network (Swarm namespaces it as <stackname>_internal, so it never collides across stacks even though every stack names it internal). Backend-only containers (databases, caches, workers) join only internal; the container(s) a stack actually exposes through Traefik join both internal and the shared traefik network. This closes what was previously a real service-name collision risk on Swarm's embedded DNS: several stacks reuse the same generic service name (db, postgresql, redis, postgres), which was only safe once each stack got its own network. Fixed in issue #109. Single-container stacks (beszel, duplicati, emby, linkding, lychee, mealie, nagios, navidrome, ontime, openproject, whatsupdocker, zigbee2mqtt, traefik itself) have nothing to isolate and stay on the shared network alone. The arr stack is the one exception with more than one container where every service still joins the shared network directly: each of its five apps is individually routed through Traefik, and internal is additional, carrying their inter-app RPC (Prowlarr sync, arr-to-Transmission) instead.

A changed Swarm secret fails the deploy until forced¶

docker_secret detects a changed value itself (a stored label hash) and tries to recreate the secret to match, whether or not force is set; Docker secrets are immutable, so recreating means removing the current one first. Docker refuses that removal while any service still references it, so a plain deploy-stacks.yml run fails outright ("... is in use by the following service ...") the moment a vault.yml value changes for a stack that is already deployed — it does not silently keep the old value. That failure is the signal to set stack_force_secret_rotation: true (-e stack_force_secret_rotation=true), which tears the whole stack down first (freeing every one of its secrets), recreates them, and redeploys. Defaults to false so a routine run with no intended secret change never takes a stack offline; expect the stack's full downtime for the run's duration when it is set, not just the affected service restarting. Authentik surfaced the underlying immutability gap as Secret key missing after an earlier run left authentik_secret_key empty; the same docker_secret task creates every stack's Swarm secrets, so any of them can hit this. Fixed in issue #115.

Authentik blueprints must be .yaml and world-readable¶

Authentik's custom-blueprint auto-discovery under /blueprints/custom only scans for .yaml; a blueprint rendered as .yml is silently never applied, no error either. The mounted directory also needs to stay world-readable: the shared stack role's extra-templates directory defaults to 0750, root-only, which blocks the non-root authentik worker from reading it at all. Every entry in the authentik stack's extra_templates (playbooks/deploy-stacks.yml) renders to a .yaml destination and sets dir_mode: "0755" to cover both. Fixed in issue #117.

No explicit startup ordering¶

No compose file in the repository uses depends_on: — Swarm mode doesn't use it. Services either tolerate their database not being ready yet, or (for Gitea's OIDC bootstrap) poll until the dependency is ready. Don't assume a deploy brings services up in dependency order.

Manual post-deploy steps¶

Not automated by Ansible; each is real friction on every fresh deploy of that service, see Deployment for the full table:

• Gitea's OIDC login source (oidc-bootstrap.sh, idempotent and self-polling).
• Fleet's SAML org settings (fleetctl gitops).
• Snipe-IT's SAML config (admin UI only, no config-file surface).
• Matomo's LoginOIDC plugin install and endpoint/client config (admin UI only).
• Nextcloud AIO's and OpenProject's first-run setup wizards.

Not yet live¶

• Zigbee2MQTT is non-functional: the USB adaptor is unchosen (zigbee_usb_hostspec, zigbee_serial_device both empty) and the Home Assistant Mosquitto broker (mqtt_host) is unconfigured.
• Emby runs CPU-only transcoding: no GPU is physically fitted in swarm-media yet (docker_install_nvidia_toolkit: false override).
• Home Assistant's homeassistant_vm_ip (10.36.0.14) is declared for Traefik's file provider, but not applied by Ansible: HAOS has no cloud-init, so a DHCP reservation keyed to home_assistant_mac_address is what makes the address actually match. It also has no host_vars/homeassistant.yml of its own.
• No router DHCP reservation exists yet for any of the five VMs' pinned MACs (proxmox_vm_mac_address/home_assistant_mac_address); see Network.
• Duplicati has nothing to back up until nfs_client_mounts is configured so the shared NFS export is actually mounted on the Swarm managers.
• Media and experimental tier vault_* variables are not populated in any real vault yet (they live in the [DEFERRED] block of vault.yml.example) — don't deploy those tiers against a populated Swarm until they are, or set deploy_stack_tiers to exclude media/ experimental (see Deployment).

Test-environment values, not production facts¶

vault.yml.test's generated secrets, and the Proxmox host address/API token in the live vault (pve.lan.mw.jasonrivers.co.uk, 192.168.0.76), belong to the disposable test deployment. Rotate the Proxmox credentials before reusing this repository against a different or permanent Proxmox host.