Fediversity

Feature Specification: Component windmill

Created: 2026-10-07

Status: Implemented

Input: Reconstructed from the code of core/components/hosting-provider/windmill/ and its OTel wiring at 093e68b6f, pull requests #1622 and #1691.

User Scenarios & Testing (mandatory)

User Story 1 - Run operator deploys as jobs (Priority: P1)

The operator front end deploys and destroys operator deployments through jobs on a Windmill server and its workers, on a node of the hosting-provider group. A caller names an uploaded script and gives a payload, but no code. The script runs the deploy program of 004 through the dev shell. Builds go through the Nix daemon of the host, so they land in the store of the hypervisor.

Why this priority: without the jobs, the front end cannot deploy or destroy.

Independent Test: a unit test imports the workspace scripts and checks that the bootstrap uploads each of them. The integrations-windmill-api-token check runs the token steps of the bootstrap against a Windmill and checks that one token with the label of the bootstrap stays.

Acceptance Scenarios:

  1. Given a fresh node, When the bootstrap runs, Then it logs in as the superadmin, gets an API token, makes the workspace and its folder, and uploads the scripts. It uploads a script again only when its content changed, as a new version that names its parent.
  2. Given a published API token that Windmill accepts, When the bootstrap runs again, Then it uses that token again and mints none.
  3. Given a published API token that Windmill refuses with HTTP 401, or no published token, When the bootstrap runs, Then it mints a new token, writes it to the UI forward-auth and the KV store, and only then deletes each other token with the label of the bootstrap.
  4. Given a script in the folder of the bootstrap that the module does not declare, When the bootstrap runs, Then it deletes that script with each version, which then does not run. It keeps each script in another folder.
  5. Given a job, When the script makes its environment, Then the deploy environment of the node has priority over the payload, and the payload over the process environment.

User Story 2 - Keep the deploy credentials on the node (Priority: P1)

The deploy needs credentials for OpenBao, NetBox, DNS, the registrar and the Incus store. The node gets them itself, never from the front end.

Why this priority: the front end does not hold the credentials of the hosting provider, and a credential in a payload is visible to each user of the job.

Independent Test: a unit test checks that the job environment agrees with the identity of the Incus store.

Acceptance Scenarios:

  1. Given the node keeps its vars in OpenBao, When it renders, Then each deploy secret is one file that only root and the Windmill group can read, and the OpenBao agent renders the Incus credentials.

User Story 3 - Read job logs in the browser (Priority: P2)

Windmill CE has no single sign-on. An Authelia forward-auth check on the UI name presents the API token for an administrator.

Why this priority: the logs show why a deploy failed. The jobs work without the UI.

Independent Test: no test opens the UI.

Acceptance Scenarios:

  1. Given a user who passes one factor, When the user is in the administrator group, Then the request reaches Windmill with the API token. Else Authelia refuses it.

Edge Cases

  • The CE API to set a password fails, so the bootstrap writes the password hash into the database as the database admin role.
  • A token that the bootstrap mints does not expire, thus the bootstrap deletes each token with its label other than the current token. It does not delete a token with a different label. A failed delete fails the bootstrap after the other steps.
  • The folder of the bootstrap belongs to the bootstrap: it also deletes a script that a person adds to the folder. Windmill moves each version to its trash before the delete. A bootstrap that declares the path again creates the script again, and a bootstrap that declares an archived path makes the path active again, also with the same content. A failed list or delete fails the bootstrap, which runs again on the next deploy.
  • When Windmill gives an answer other than HTTP 200 or 401 to the published token, the bootstrap stops and tries again: it does not mint a token because Windmill was not ready.
  • The bootstrap retries without a limit, and retries its login a limited number of times. A new superadmin password restarts it.
  • The database uses rotating credentials, because dynamic credentials restart the units at each SVID rotation.
  • The server and the workers start after the OpenBao agent and restart quickly after a failure.
  • An evaluation without the OpenBao vars backend has no deploy secrets.
  • The API name serves only the API path.

Requirements (mandatory)

Functional Requirements

  • FR-001: The component MUST build through the Nix daemon of the host, MUST use the bootstrap certificate audience, and MUST serve its API on its own subdomain.
  • FR-002: The component MUST take the operator settings for DNS, the NetBox range, the registrar and the UI name under the apex.
  • FR-003: The component MUST request a SPIFFE identity, a certificate, and the netbox, dns and otel resources. The IPAM request MUST write the operator range into the deploy environment.
  • FR-004: The node MUST get the deploy secrets only when it keeps its vars in OpenBao.
  • FR-005: The database request MUST contain only data, with rotating credentials, a user role with all privileges, and an admin role that bypasses row security and is a member of the user role. The node MUST configure its client from the result alone and MUST NOT create a local database (see postgresql).
  • FR-006: The UI forward-auth vhost MUST use one factor, admit only the administrator group, pass websockets, and add the API token (see oidc).
  • FR-007: The node MUST generate the superadmin password, and MUST keep one API token with the label of the bootstrap: the published token while Windmill accepts it, else a new one that it writes to the KV store.
  • FR-008: The server MUST send mail through smtp, with the password as a service secret.
  • FR-009: The base URL MUST be the UI name. The deploy environment MUST carry the git-tracked tree and nixpkgs.
  • FR-010: The package MUST hold only the Python and Bash toolchains.
  • FR-011: The Windmill user MUST be a trusted Nix user, with nixpkgs pinned in the registry, no global registry, and flakes on.
  • FR-012: The workers MUST have Nix, SSH, git and a shell, and MUST stop so that a job can finish its cleanup.
  • FR-013: The deploy script MUST read its coordinates from the job environment: the tree, the attribute, the system, nixpkgs, the SSH key, the hosts, the environments and the state backend.
  • FR-014: The server and the workers MUST get the OTel settings of the group.
  • FR-015: The component MUST stream a backup of its database as root to a bucket of garage.

Key Entities

  • Workspace: the runner, the job environment, and the deploy and destroy scripts. core gives the runner to api as a library.
  • API token: minted when Windmill refuses the published token or there is none, held in the KV store and the UI forward-auth. It does not expire.
  • Deploy environment: the coordinates of the node that the jobs read.
  • Deploy secrets: the deploy credentials, one file each.

Success Criteria (mandatory)

Measurable Outcomes

  • SC-001: The unit tests of the workspace import, the configuration and the Incus store identity pass.
  • SC-002: The OTel wiring VM test passes.
  • SC-003: On a live node, the bootstrap is active and the KV store holds the current API token.

Assumptions

  • The postgresql, smtp, netbox and dns resources resolve on the node, and OpenBao and its agent run on it.
  • The render entry point sets the operator settings.

Known Deviations

  • The node sets the OTLP settings, but the OSS exporters of Windmill are stubs. No signal reaches the collector.