Fediversity

Feature Specification: Component openbao

Created: 2026-10-07

Status: Implemented

Input: Reconstructed from the code of core/components/hosting-provider/openbao/ at 093e68b6f, pull requests #1820, #1822, #1853 and #1956, and issues #1759, #1760 and #1817.

User Scenarios & Testing (mandatory)

User Story 1 - Run an unattended secrets backend (Priority: P1)

A hosting provider deploys the openbao component on one node of the group. The node initializes OpenBao, seals the unseal key to its TPM, and unseals itself at every boot. A KV store then holds the vars of the deployment. The component provides varsBackend and pki: the node is the secrets store of the group and the CA of the openbao-pki provider of ssl.

No root token stays on the node. Each unit that uses OpenBao has a scoped token with only its own grants. An administrator who needs root makes a short-lived root token with a command on the node. OpenBao writes an audit entry for each request and each response.

Why this priority: each other node reads its secrets from this backend.

Independent Test: a VM test that boots spire and openbao together, and a VM test of a node that seals its key to a TPM and audits to the journal.

Acceptance Scenarios:

  1. Given a fresh node with a TPM, When the initialization runs, Then OpenBao is initialized with one key share sealed to the TPM, and unsealed.
  2. Given an initialized node that reboots, When the initialization runs, Then OpenBao is unsealed with no human step.
  3. Given an initialized OpenBao whose sealed key is missing, When the initialization runs, Then it fails and does not initialize again.
  4. Given a node after the initialization, When an administrator examines the disk, Then no root token is there.
  5. Given a client writes and reads a secret, Then the journal holds a request entry and a response entry for each operation, with the secret only as a keyed hash.

User Story 2 - Let each node log in by its SPIFFE identity (Priority: P1)

A node presents its SPIFFE X.509 SVID to OpenBao. Each roster host has one policy and one certificate login role that binds to its SPIFFE ID. The policy gives the host only the vars instances that it owns or that the setup names for it to read. A node of a tenant deployment gets only the objects of its own deployment.

Why this priority: this login is how each consumer reads its secrets without a shared token.

Independent Test: the VM test of spire with openbao, and unit tests of the policy and login role of each kind of roster entry and of the LDAP grants per group.

Acceptance Scenarios:

  1. Given a host on the roster, When it logs in with its SVID, Then it gets a token that lives an hour and renews for at most a day.
  2. Given a host that leaves the roster, When the reconcilers run, Then OpenBao has no policy and no login role for it.
  3. Given a node of the tenant deployment tenant42, When it writes a secret of its own deployment, Then OpenBao accepts the write. It refuses the same write for tenant43, for the shared vars, and for a path under the name of the host.
  4. Given a host that owns no vars instance, Then its policy gives no write on KV.
  5. Given the SPIRE server has a new CA that the local agent does not attest to yet, When the bundle refresh runs, Then it waits a bounded time for the agent, then installs the bundle and writes the login roles again. When the agent does not attest in time, the refresh keeps the current bundle and tries again at its next run.

User Story 3 - Issue internal TLS certificates (Priority: P2)

A node needs a certificate for node-to-node TLS. When the deployment wires the openbao-pki provider of ssl, OpenBao keeps an internal root CA and a role that issues only for the domains of the deployment. OpenBao then also serves a mutual-TLS listener with a leaf from its own CA. A fresh node gets the CA certificate without TLS, because it has no trust anchor yet.

Why this priority: internal TLS is optional. Public application domains use ACME.

Independent Test: the VM test of the TPM-sealed node.

Acceptance Scenarios:

  1. Given the PKI is on, When the PKI reconciler runs, Then OpenBao has the PKI mount, the issuing role and one root CA, which it makes only once.
  2. Given a configuration without the PKI mount, When the PKI reconciler runs, Then the apply fails, and the mount and its root CA stay.

User Story 4 - Mint deployer and rotated credentials (Priority: P2)

CD, a workstation deploy and the operator deploys need tokens. Applications need database, cache and directory passwords that rotate. OpenBao mints these through a token exchange for forge Actions tokens, an OIDC login for persons, and its database and ldap secrets engines.

Why this priority: deploys and rotated credentials depend on it, but a node can boot and serve without them.

Independent Test: a VM test of the CD login, a test of the token exchange against a stub OpenBao, and VM tests of the rotation of the directory bind account and of the cache user.

Acceptance Scenarios:

  1. Given a CD run on the main branch of the configured repository, When it logs in, Then it gets a deployer token that lives 30 minutes, and at most 2 hours with renewal. A pull request run, another branch, another repository or another audience is refused.
  2. Given a person in the administrators group, When that person logs in on the deployer OIDC login, Then the token carries only the deployer policy.
  3. Given a dynamic database role that the configuration does not state, When the database reconciler runs, Then OpenBao revokes its leases and deletes it.
  4. Given the cache engine applied once, When it applies again, Then nothing changes.

Edge Cases

  • The TPM is not ready at boot. The initialization waits for it, and retries a transient read error. It seals a new key before it initializes OpenBao, so a failed seal never loses the key of an initialized OpenBao.
  • The storage is in an older format. OpenBao moves it to raft before it starts.
  • A node of an earlier release has no audit device. A switch does not restart OpenBao, so OpenBao adds the device at its next start.
  • A switch does not restart OpenBao. In the recorded release series, openbao-stale-server restarts OpenBao within 15 minutes, and openbao-init unseals it. After a change of the release series, openbao-stale-server fails until the operator restarts OpenBao.
  • The configured options of the audit device differ from the stored device. OpenBao refuses to unseal. A change of an option thus gives the device a new name, and OpenBao replaces the old device.
  • journald does not accept an audit entry. OpenBao refuses the request, or waits.
  • A reconciler cannot do its work yet: the initialization failed, the OIDC issuer is not up, or the database administrator password is not available. The reconciler fails and tries again, with no limit.
  • The OIDC client secret is empty. The OIDC reconciler refuses it.
  • The OIDC issuer chains to the private PKI CA. The OIDC discovery trusts the system anchors plus that CA.
  • A shell unit of an earlier release made an object that no reconciler state holds. The reconciler adopts it when its configuration names it (FR-015). Otherwise the object stays.
  • The cache node of a required namespace has not published its address. The cache engine configures the other namespaces, then fails, so that it starts again.
  • An operator deployment is gone, but its KV records stay. The cache engine treats it as an optional namespace that does not answer (FR-036).
  • A restore puts back an older snapshot. Reconciler objects come back when the reconcilers restart. The forced rotation of each static role makes OpenBao and the database agree on a password that rotated after the snapshot. A consumer gets the new password at its next login, in at most about an hour. An object that the configuration removed after the snapshot stays, because no state holds it.
  • A restore puts back a snapshot from before a change of the database administrator password or of the LDAP bind password. OpenBao keeps the old password, because the reconcilers write it as a write-only value and see no drift. The forced rotations of that mount then fail, and the restore reports a failure.
  • The node or its TPM is lost. An administrator decrypts the unseal key from the backup and adds it to the archive. The restore then seals the key to the TPM of the new node. Without the key, a restore of a snapshot from another node fails, and OpenBao stays as it was.
  • All holders of the backup identities lose them. The snapshots then restore only on a node that has the original unseal key. When that node is also lost, the secrets that only KV holds are lost: the app encryption keys of mastodon and pixelfed, the OAuth signing keys of pixelfed, the DNSSEC and TSIG keys, and the PKI CA.

Requirements (mandatory)

Functional Requirements

Component and API

  • FR-001: The component MUST register as openbao, with the default subdomain openbao and a second UI domain secrets under the apex. Its certificate audience MUST be bootstrap, because nodes use it before a public certificate can exist.
  • FR-002: When enabled, the component MUST request a SPIFFE identity (spiffe), a TLS certificate for both domains (ssl), an OIDC client for its UI (oidc) and its NixOS module (nixos-module). The OIDC client MUST allow the UI callback on both domains and the local callback of the CLI login, and MUST ask for the group claim.
  • FR-003: The component MUST provide varsBackend and pki, so that the setups route the secrets store and the PKI CA of the group to this node.
  • FR-004: OpenBao MUST serve its admin API on the loopback interface only, and each unit on the node MUST use that API. A reverse proxy MUST serve both domains from it. The unauthenticated root-token generation MUST be off on the proxied listener, and only on a second loopback listener.
  • FR-005: OpenBao MUST use raft storage that is local to the node.

Initialization and tokens

  • FR-006: The initialization MUST initialize OpenBao with one key share, seal the key to the TPM, unseal, and enable the KV store. It MUST keep the unseal key and its root token in memory only. On an initialized node, it MUST make the root token from the unseal key in the TPM. It MUST revoke the root token when it exits, also after a failure.
  • FR-007: The initialization MUST fail, and MUST NOT initialize again, when OpenBao is initialized but the sealed key is missing.
  • FR-008: The initialization MUST encrypt the unseal key to the backup recipients of the hosting provider, again when the recipients change. A failure of this step MUST NOT stop the unseal.
  • FR-009: When the TPM seal of a new key fails, the initialization MUST keep the key in volatile storage for adoption, and fail. When OpenBao is initialized and a key for adoption is present, the initialization MUST seal that key to the TPM in place of the sealed key, unseal with it, encrypt it for the backup, and remove the adoption copy.
  • FR-010: Each unit that uses OpenBao MUST have a scoped token: a policy and a token role of its own, and a token that only that unit reads. The token MUST be orphan and periodic with a period of about a month, and MUST have only its own policy and the default policy. The initialization MUST keep each valid token, make the missing ones, and revoke and delete the scoped tokens that the configuration does not declare. No unit MUST read a root token from a file.
  • FR-011: Each scoped token MUST renew itself daily. The initialization, the renewal and a new token MUST NOT stop or restart a reconciler, an agent or the TLS listener.
  • FR-012: The root-token command MUST make a root token as in FR-006, only for root. It MUST print the token, or run a given command with it and then revoke it and exit with the status of that command. When OpenBao cannot revoke the token, for example while sealed, the command MUST print its accessor for a later revoke. The command and the initialization MUST NOT make a root token at the same time, because a new attempt cancels the other.

Reconcilers

  • FR-013: Each set of declared OpenBao objects (login methods and policies, OIDC, PKI, database, cache engine, LDAP engine) MUST have one reconciler, which applies them with OpenTofu against the admin API. The state MUST be local to the node, because the HTTP state backend uses a certificate from this PKI and is deployed after this node. A reconciler MUST write each secret of its configuration as a write-only value, so that the state holds only a digest of it, and MUST NOT keep a backup of an earlier state.
  • FR-014: A reconciler MUST delete from OpenBao each object that leaves its configuration, except a protected object. The PKI mount and the database mount MUST be protected.
  • FR-015: Before each apply, a reconciler MUST adopt each mount, auth mount, identity group and group alias of its configuration that OpenBao holds and its state does not.
  • FR-016: A reconciler MUST start after the initialization, but MUST NOT depend on its success or restart with it. It MUST try again with no limit until it succeeds.

SPIFFE login and host policies

  • FR-017: OpenBao MUST have one host policy and one certificate login role per roster host. The role MUST admit only the SPIFFE ID of that host and attach its host policy. A host that leaves the roster MUST lose both.
  • FR-018: Each host that reads an object store bucket MUST be on the roster, also when it owns no instance. The evaluation MUST fail and name a host that is not.
  • FR-019: While the SPIRE trust bundle is missing, the certificate login reconciler MUST NOT start and MUST NOT fail, and a timer MUST try it again. The policy reconciler MUST NOT wait for the bundle.
  • FR-020: The bundle refresh MUST read the trust bundle from the local SPIRE agent, and from the bundle file of the agent only when the current bundle has no CA that is valid now. It MUST NOT install a bundle that does not parse or has no valid CA, and MUST keep the current bundle when the agent fails. When the bundle file has a newer CA that the agent does not attest to yet, the refresh MUST wait a bounded time for the agent. It MUST NOT put the bundle file in place of a current bundle that has a valid CA. It MUST write the login roles again only when the bundle changes.
  • FR-021: A host policy MUST read the submission passwords of an SMTP relay only when the setup names that relay for the host. The setup names it for each node of the relay's own group that sends mail: a node whose component requests smtp, and the providers of oidc and ldap. Only the relay hosts MUST read the upstream password of the smarthost.
  • FR-022: A host policy MUST read a vars instance that another host writes only when a read roster of the setup names that instance for the host, or a dedicated roster for that kind of consumer names the host. An instance MUST NOT be readable by each host. This includes the LDAP bind password of a group: without the LDAP engine, only a host that a read roster names reads it. Where an instance holds keys with different readers, a read roster MUST name the single keys, and the policy MUST read only the data of those keys. The deploy runners read the deploy credentials key by key, and MUST NOT read the zones-manager token, which only the front-end node reads.
  • FR-023: For a node of a tenant deployment, the host policy MUST grant KV, cache credentials and LDAP credentials only on the objects of its own deployment. The policy MUST take the deployment from the namespace in the SVID of the login. It MUST NOT grant a path under the name of the host, the shared vars, or the objects of another deployment, except the smarthost read of FR-021.
  • FR-024: A host policy MUST NOT grant a path under the name of the host. It MUST grant a write on KV only on the vars instances that the host owns. A host that owns no instance MUST get no write on KV.

Internal PKI

  • FR-025: When the deployment wires the openbao-pki provider, OpenBao MUST hold the PKI mount, a root CA that it makes once, and a role that issues only for the domains of the deployment. It MUST serve the CA certificate over plain HTTP.
  • FR-026: When the PKI is on, OpenBao MUST serve a mutual-TLS listener on the network that trusts the SPIRE CA for client certificates.

Backup, restore and audit

  • FR-027: The component MUST request a garage bucket, and MUST stream an archive of a raft snapshot and of the encrypted unseal key to it with restic, with its own scoped token. The restore MUST:

    1. restore the snapshot. When the archive holds the plaintext unseal key, it MUST force the restore and hand the key to the initialization for adoption (FR-009). Otherwise, a failed restore MUST tell the administrator to decrypt the key and add it to the archive;
    2. wait a bounded time until this node is active;
    3. run the initialization again, because the restored storage holds the scoped tokens of the time of the snapshot, and read its own token again;
    4. restart each reconciler in dependency order, and the operator deployer token task;
    5. rotate each static role of each database and ldap mount. A missing mount or an empty list MUST NOT be a failure.

    The restore MUST try each step, and MUST report a failure at the end when one step failed.

  • FR-028: OpenBao MUST audit each request and each response to the journal. The audit MUST keep the default keyed hash of secret values, and MUST write a count in place of the keys of a list response. journald MUST NOT drop audit entries by rate limit.

Deployer tokens

  • FR-029: The deployer policy and the operator deployer policy MUST come from one definition, which the test of the operator deployment method also uses.
  • FR-030: The deployer policies MUST NOT grant sudo, a system path, or an auth path other than token creation. The deployer policy MUST NOT write the operator deployer token. A deployer MUST NOT keep a root token: CD, a workstation and the front-end node get a deployer token (FR-031, FR-032, FR-033), and an administrator uses the root-token command for a manual step (FR-012).
  • FR-031: OpenBao MUST keep a periodic operator deployer token in KV, with a period of about a month, and MUST renew it daily. It MUST make the token from an orphan token role that allows only the operator deployer policy, and only when the current token does not renew.
  • FR-032: When the forge issuer is set, OpenBao MUST admit a forge Actions token as a deployer only for the configured repository on its main branch.
  • FR-033: The UI OIDC login MUST map the administrators group of the identity provider to the administrators policy, which MUST grant a write only on the domain claim records. A separate OIDC login MUST give members of that group only the deployer policy.
  • FR-034: The operator deployer policy MUST deny the KV stems of retained backups under the deployment prefixes, and MUST only create retention records. Each backup sweeper host MUST list, read and delete the metadata of these stems and records, and MUST NOT read their data.

Secrets engines

  • FR-035: The database reconciler MUST connect to postgres with TLS required. It MUST keep a static role for each rotating role and a dynamic role for each dynamic role, with no role of the other kind under the same name. It MUST revoke the leases of each dynamic role that the configuration does not state, and delete it.
  • FR-036: The cache engine MUST configure each valkey instance whose password and address are in KV, and MUST remove the configuration of each other instance. A required namespace belongs to the hosting provider. An optional namespace has one instance per operator deployment with a password for it. For an optional namespace, the engine MUST first check that the address answers. When the address is missing or does not answer, the engine MUST keep what its state holds, MUST NOT fail, and MUST run again on a timer.
  • FR-037: The LDAP engine MUST mount the directory of the central group (the group that provides varsBackend) and one directory per operator deployment, and MUST rotate the bind account of each daily. It MUST delete the mount of a deployment whose KV records are gone. While the inputs of another directory are missing, it MUST keep the live mount that its state holds. The management bind password MUST go to OpenBao as a write-only value, so that the state holds only a digest. A mount without configuration MUST be made again, not adopted.
  • FR-038: The setups MUST give the directories of the operator groups to the LDAP engine only on the node of the central group.

Key Entities

  • Initialization: the task that initializes, seals to the TPM, unseals, and makes the scoped tokens with a root token that it revokes at its end.
  • Root-token command: the administrator command for a short-lived root token (FR-012).
  • Scoped token: the policy, token role and token of one unit (FR-010).
  • Reconciler: an OpenTofu apply of one set of declared OpenBao objects, with local state (FR-013).
  • Roster: the hosts that get a login role and a host policy, and the instances that each one owns or reads. The setup derives it from the components on each node. It adds the readers that no component request states: the deploy runners read the deploy credentials, key by key, and the hypervisor deploy tokens, the database engine node reads the database administrator password, and the object store node reads the key pairs of its group that other nodes make.
  • SMTP relays: the roster hosts whose component provides smtp, in each group that the setup collects.
  • Deployer policy and operator deployer policy: the policies of a CD or workstation deploy token, and of the periodic token for operator deploys.
  • Administrators group: the identity group and policy of a person who logs in to the UI.
  • Engines: pki (internal CA), database (postgres and valkey), ldap (bind account rotation).

Success Criteria (mandatory)

Measurable Outcomes

  • SC-001: The VM tests of spire with openbao, of the TPM-sealed node, of the CD login (with its refusals), of the directory bind rotation, and the token exchange test pass.
  • SC-002: The unit tests of the host policies, of the LDAP grants and of the initialization pass.
  • SC-003: After a reboot, OpenBao is unsealed with no human step.
  • SC-004: The VM test of postgres rotation passes, including the restore of an older snapshot: a role that a reconciler made after the snapshot is there again, and a role that rotated by hand after the snapshot has a new password that connects.
  • SC-005: The VM test of the bootstrap backup passes, including a restore on a node with a different TPM and a reboot of that node with no human step.

Assumptions

  • The node has a TPM 2.0 with the storage hierarchy enabled.
  • The setup supplies the rosters from the requests of each component (see setups and wiring).
  • The spire component gives this node its SVID. The deploy writes the bundle file of the local agent.
  • The postgresql component, valkey and lldap serve the backends of the engines.
  • The authelia component is the OIDC issuer of the UI login.
  • The curation of deploy credentials and of operator deployments is not part of this component. The windmill component imports it, so its spec belongs with windmill.

Known Deviations

  • The state of the cache engine holds the valkey passwords that it writes (FR-013). The provider has no write-only value for the generic endpoint that the cache engine writes them with. The cache engine also reads them back from its state for an optional namespace whose KV record is missing.
  • The audit log is not tamper-evident. A user with root on the node can change the journal before the log shipper sends it to Loki, and Loki is not write-once storage. The audit log has the retention of the journal and of Loki, not a retention of its own (#2270). The audit property of a component, the audit path and the audit tenant of telemetry can take it, with a prefix that only its audit entries have.
  • The backup uses a fixed and public restic password, because a restore of this backup comes before the secrets store can give a password. Thus the encryption of the repository gives no protection. The barrier of OpenBao encrypts the data in the snapshot, and age encrypts the unseal key.