Credentials register

This part lists the credentials of the system: where each one is, what makes it, how long it lives, how it changes, and what a change interrupts. The page gives names and locations only. The components that can read each credential are in the access model.

The word "Open" in a cell means: the code and the documentation give no procedure. The page gives a period or a procedure only when the code or a document holds it.

How the register is made

The tables that have the label "generated" come from the code at each build of the documentation (core/docs/md/technical-design/credentials.nix). Each generated table has two inputs:

  • The names, which the build reads from the code: the realised configuration of the nodes with the groups of core/testers/groups.nix, the file core/resources/secrets/secrets.nix, and the workflows in .forgejo/workflows/.
  • The facts that the code does not hold (the procedure and what it interrupts), which a person writes in credentials.nix.

The build compares the two inputs. The check core-docs fails when a name has no facts, or when facts have no name. The other tables are written by hand in this file.

The specification of the secret rotation gives the requirements for rotation. Its table in "Key Entities" adds, for each secret, the backups that depend on the secret and how the consumers get a new value. This page is the source for the list of the credentials and for the state of their rotation.

Credentials that OpenBao rotates

OpenBao makes these credentials and changes them on a schedule. No person and no deploy writes them. A consumer reads the current value through its OpenBao agent. The lifetime of a value is the rotation period. The code does not state what a rotation interrupts: the column "Consumer refresh" of the specification gives how each consumer gets the new value.

Generated:

Credential Path that the consumer reads in OpenBao Rotation period
PostgreSQL role of the database hosting-provider-authelia postgresql/static-creds/hosting-provider-authelia (rotating) 24h
PostgreSQL role of the database hosting-provider-lldap postgresql/static-creds/hosting-provider-lldap (rotating) 24h
PostgreSQL role of the database hosting-provider-netbox postgresql/static-creds/hosting-provider-netbox (rotating) 24h
PostgreSQL role of the database hosting-provider-windmill postgresql/static-creds/hosting-provider-windmill (rotating) 24h
PostgreSQL role of the database operator-authelia postgresql/static-creds/operator-authelia (rotating) 24h
PostgreSQL role of the database operator-lldap postgresql/static-creds/operator-lldap (rotating) 24h
PostgreSQL role of the database operator-mastodon postgresql/static-creds/operator-mastodon (rotating) 24h
PostgreSQL role of the database operator-peertube postgresql/static-creds/operator-peertube (rotating) 24h
PostgreSQL role of the database operator-pixelfed postgresql/static-creds/operator-pixelfed (rotating) 24h
LDAP account authelia-bind, read by the node authelia ldap/static-cred/authelia-bind 24h
LDAP account authelia-bind, read by the node operator-authelia ldap-operator/_shared/static-cred/authelia-bind 24h
LDAP account authelia-bind of each deployment ldap-operator/<namespace>/static-cred/authelia-bind 24h
Valkey user app of the namespace hosting-provider-authelia, read by the node authelia database/static-creds/hosting-provider-authelia See the text below
Valkey user app of the namespace hosting-provider-valkey, read by the node valkey database/static-creds/hosting-provider-valkey See the text below
Valkey user app of the namespace operator-authelia, read by the node operator-authelia database/static-creds/operator-authelia See the text below
Valkey user app of the namespace operator-mastodon, read by the node operator-mastodon database/static-creds/operator-mastodon See the text below
Valkey user app of the namespace operator-peertube, read by the node operator-peertube database/static-creds/operator-peertube See the text below
Valkey user app of the namespace operator-pixelfed, read by the node operator-pixelfed database/static-creds/operator-pixelfed See the text below
Valkey user app of the namespace operator-valkey, read by the node operator-valkey database/static-creds/operator-valkey See the text below

The rotation period of a Valkey namespace is not an option. It is 24 hours in core/components/hosting-provider/openbao/valkey-engine.nix. The engine rotates the user app of a namespace. It connects as the user default, and it does not rotate that user: the password of default is the generated secret valkey-<namespace> in the next section. The LDAP engine rotates only the accounts in the table. It connects with the management account of lldap, and it does not rotate that account (the instances ldap-bind-* in the next section).

Generated secrets in the KV of OpenBao

A node declares each secret that it needs as a generateFiles request. These facts apply to each row of the table:

  • Where: the KV path is <prefix>/secret/<instance>/<file> in the mount secret. The prefix is vars for the hosting provider and vars/operator-deployments/<namespace> for a deployment.
  • Maker: the script of the request makes the value at the first deploy of the node (nix/contracts/nixos/modules/services/security/vars-openbao.nix).
  • Lifetime: the code sets no end. A deploy keeps a value that is in KV.
  • Change: a deploy with VARS_FORCE_UPLOAD=<instance> or VARS_FORCE_UPLOAD=<instance>/<file>, or the command upload-vars-openbao-force on the node, makes a new value and writes it. This is a write, not a rotation procedure: it does not tell what the consumers do with the new value. Thus a row without a procedure for its consumers has "Open".
  • For some keys OpenBao keeps all versions, because a backup from before a change needs the old key. The column "KV keeps all versions" shows them.

The table shows the nodes of the hosting provider group and of one operator group of the testers. A file that is not secret (for example a public key) is not in the table.

Generated:

Credential Instances Files Nodes that request it KV keeps all versions Rotation What a rotation interrupts
Password of the restic repositories of an application authelia-backups-restic, bind-backups-restic, grafana-backups-restic, hosting-provider-authelia-cache-backups-restic, hosting-provider-valkey-cache-backups-restic, lldap-backups-restic, mastodon-backups-restic, netbox-backups-restic, operator-authelia-cache-backups-restic, operator-mastodon-cache-backups-restic, operator-peertube-cache-backups-restic, operator-pixelfed-cache-backups-restic, operator-valkey-cache-backups-restic, peertube-backups-restic, pixelfed-backups-restic, postgresql-backups-restic, windmill-backups-restic password hosting-provider/authelia, hosting-provider/bind, hosting-provider/lldap, hosting-provider/netbox, hosting-provider/postgresql, hosting-provider/telemetry, hosting-provider/valkey, hosting-provider/windmill, operator/authelia, operator/lldap, operator/mastodon, operator/peertube, operator/pixelfed, operator/valkey no By hand: Password of the restic repositories The procedure restarts openbao-agent-default. The repositories accept the old and the new password during the change
Password and salt of the encrypted copy of the media buckets mastodon-media-backup, peertube-media-backup, pixelfed-media-backup password, salt operator/mastodon, operator/peertube, operator/pixelfed no Open Open
Secret key of an S3 key of garage authelia-backups-s3, bind-backups-s3, grafana-backups-s3, hosting-provider-authelia-cache-backups-s3, hosting-provider-valkey-cache-backups-s3, lldap-backups-s3, netbox-backups-s3, openbao-backups-s3, postgresql-backups-s3, telemetry-s3, terraform-backend-backups-s3, windmill-backups-s3 secretKey hosting-provider/garage, hosting-provider/telemetry no Open Open
Admin token and RPC secret of garage garage-admin adminToken, rpcSecret hosting-provider/garage no Open Open
Storage encryption key of authelia authelia-oidc, operator-authelia-oidc storageEncryptionKey hosting-provider/authelia, operator/authelia yes By hand: Authelia storage encryption key Authelia is stopped during the change
OIDC and JWT secrets of authelia authelia-oidc, operator-authelia-oidc jwtSecret, oidcClientSecret, oidcClientSecretDigest, oidcHmacSecret, oidcIssuerPrivateKey hosting-provider/authelia, operator/authelia no Open Open
Session secret of authelia authelia-session sessionSecret hosting-provider/authelia, operator/authelia no Open Open
Password of the user operator of the state backend terraform-backend-operator password hosting-provider/terraform-backend no Open Open
TSIG keys of the name server acme-dns, octodns, operator-dns tsigSecret hosting-provider/bind no Open Open
DNSSEC key of the name server dnssec key, private, state hosting-provider/bind no Open Open
Password of the management account of lldap ldap-bind-hosting-provider, ldap-bind-operator bindPassword hosting-provider/lldap, operator/lldap no Open Open
JWT secret of lldap ldap-jwt-hosting-provider, ldap-jwt-operator jwtSecret hosting-provider/lldap, operator/lldap no Open Open
Key seed of lldap ldap-key-hosting-provider, ldap-key-operator keySeedEnv hosting-provider/lldap, operator/lldap yes Never without loss: Keys that cannot change without data loss lldap does not start with a new seed. With LLDAP_FORCE_UPDATE_PRIVATE_KEY=true it starts, and all users must set a new password
Secret key and token peppers of NetBox netbox apiTokenPeppersFile, secretKeyFile hosting-provider/netbox no Open Open
Password of the PostgreSQL administrator postgresql-admin password hosting-provider/postgresql no Open Open
Submission passwords of the mail server operator-smtp, smtp autheliaPassword, lldapPassword, mastodonPassword, netboxPassword, peertubePassword, pixelfedPassword, telemetryPassword, windmillPassword hosting-provider/smtp, operator/smtp no Open Open
DKIM key of the mail server operator-smtp-dkim, smtp-dkim fediversity.test_fediversity.key hosting-provider/smtp, operator/smtp no Open Open
Secret key of the Grafana of the hosting provider telemetry-grafana secretKey hosting-provider/telemetry yes By hand: Grafana secretKey Grafana restarts
Old secret keys of the Grafana of the hosting provider telemetry-grafana-previous secretKeys hosting-provider/telemetry no Written and emptied in the procedure of the row above See the row above
Admin password of the Grafana of the hosting provider telemetry-grafana adminPassword hosting-provider/telemetry no Open Open
Credentials of the telemetry and audit endpoints audit-ingest-api, metrics-query, node-telemetry, telemetry-ingest authHeader, htpasswd, password hosting-provider/telemetry, operator/grafana no Open Open
Secret key of the Grafana of a deployment grafana-secret secretKey operator/grafana no Open Open
Encryption key of the state backend terraform-backend kmsKey hosting-provider/terraform-backend no Open Open
Password of the Valkey user default of a namespace valkey-hosting-provider-authelia, valkey-hosting-provider-valkey, valkey-operator-authelia, valkey-operator-mastodon, valkey-operator-peertube, valkey-operator-pixelfed, valkey-operator-valkey password hosting-provider/valkey, operator/valkey no Open Open
Password of the superadmin of windmill windmill-superadmin adminPassword hosting-provider/windmill no Open Open
Active Record encryption keys of Mastodon mastodon-keys activeRecordEncryptionDeterministicKey, activeRecordEncryptionKeyDerivationSalt, activeRecordEncryptionPrimaryKey operator/mastodon yes Never without loss: Keys that cannot change without data loss The private keys of the accounts and the two-factor secrets do not decrypt
SECRET_KEY_BASE and VAPID key of Mastodon mastodon-keys secretKeyBase, vapidPrivateKey operator/mastodon yes By hand, with the effects that Keys that cannot change without data loss gives No data is lost. All users must log in again, the links in password reset and confirmation emails stop, and the push subscriptions stop until the clients subscribe again
Secret of PeerTube peertube-secret secret operator/peertube yes Never without loss: Keys that cannot change without data loss The two-factor authentication of all users that use it fails
APP_KEY of Pixelfed pixelfed-app appKey operator/pixelfed yes By hand: Pixelfed APP_KEY Pixelfed restarts two times. A user that did not use pixelfed during the change must log in again
Old APP_KEY values of Pixelfed pixelfed-app-previous appKeys operator/pixelfed no Written and emptied in the procedure of the row above See the row above
Passport key of Pixelfed pixelfed-passport privateKey operator/pixelfed no Open Open

The change of an application encryption key has its own work item (fediversity/fediversity issue 2433).

Other secrets in the KV of OpenBao

No generateFiles request of a node of core makes these entries. The api node and the panel node are not in the setups that the build of this page reads, thus their rows are written by hand.

Credential Access Where Maker Lifetime Rotation What a rotation interrupts
Key of the target of the external copy of the backups The garage node vars/secret/backup-external-copy/<name> A person writes it with bao kv put (hosting provider guide) The code sets no end Open Open
Submission password for the upstream mail relay The node that relays the mail vars/secret/smarthost-upstream/<name> (core/resources/smtp/maddy-provider.nix) Not made by a node The code sets no end Open Open
Token of the api for windmill (WINDMILL_TOKEN) The api node Instance windmill, file apiToken The windmill node, at run time (core/components/hosting-provider/windmill/windmill.nix) The code sets no end Open Open
Service token between the panel and the api (API_SERVICE_TOKEN) The panel node and the api node Instance panel-service, file token (panel/nix/component/panel.nix) The panel node, at its first deploy The code sets no end Open Open
Callback token of the authorization check (AUTHZ_CALLBACK_TOKEN) The panel node and the api node Instance panel-authz, file token (panel/nix/component/panel.nix, api/nix/component/authz-token.nix) The panel node, at its first deploy The code sets no end Open Open
Secret key of the panel The panel node Instance panel, file secret_key The panel node, at its first deploy The code sets no end Open Open
API key of centrifugo The api node Instance api-centrifugo, file apiKey (api/nix/component/default.nix) The api node, at its first deploy The code sets no end Open Open
Credentials that the api deploys with (state backend password, zones token) The api node Instance deployer (core/components/hosting-provider/deploy-credentials.nix) The CD publishes them from the secrets of the forge The code sets no end Open Open
Tokens that a node writes at run time The nodes that the access model gives Instances deploy-tokens (incus), netbox (NetBox) and metrics (Mastodon, PeerTube, Pixelfed) The node that owns the instance, at run time The code sets no end Open Open

Credentials of OpenBao

Credential Access Where Maker Lifetime Rotation What a rotation interrupts
Scoped tokens (list below) The units of the openbao node /var/lib/openbao-init/tokens/<name> on the openbao node openbao-init Periodic token with a period of 768h Automatic: openbao-tokens-renew renews each token daily. openbao-init makes a token again when it is missing or not valid Open
Root token An administrator on the openbao node Not stored: openbao-init revokes its root token when it exits openbao-init, or the command openbao-root-token Until openbao-init exits Does not apply Does not apply
Unseal key The openbao node, and the backup recipients Sealed to the TPM of the node. A copy is encrypted to fediversity.bootstrapBackupRecipients openbao-init The code sets no end Open. When the recipients change, openbao-init encrypts the same key again (core/components/hosting-provider/openbao/escrow.nix) Open
CA of the PKI engine OpenBao The PKI mount of OpenBao (core/components/hosting-provider/openbao/pki-init.nix) The openbao node (pki-init.nix) maxLeaseTtl, 10 years by default. A certificate that it signs lives 30 days at most (roleMaxTtl) Open Open
CA of the admin SSH login OpenBao signs with it. Each node trusts its public key The engine ssh-admin of OpenBao. The forge variable ADMIN_SSH_CA_KEYS holds the public keys The openbao node (ssh-certificates.nix) The code sets no end. A certificate that it signs lives 8 hours (certificateTtl) By hand: Rotate the CA The procedure trusts the two keys until the old certificates end

Generated, the names of the scoped tokens:

database-init, deployer-kv-prune, kv-key-metadata, ldap-engine, oidc-auth, operator-deployer-token, pki-init, pki-leaf, snapshot, spire-auth, spire-cert-auth, ssh-certificates, valkey-engine

Credentials of SPIRE

Each node logs in to OpenBao with an identity that SPIRE gives. Without an upstream root, the SPIRE server signs its own CAs, and each CA lives for about one day. With the upstream root, the root lives 10 years and each CA lives 24 hours. The private key of the root is the forge secret SPIRE_ROOT_KEY in the section about the forge. The specification of spire gives the custody and the rotation of the root in "The upstream root".

Keys of persons

Credential Access Where Maker Lifetime Rotation What a rotation interrupts
SSH certificate of an administrator The administrator With the administrator The engine ssh-admin 8 hours Automatic: the administrator gets a new certificate (Log in with a certificate) Does not apply
SSH key of a contributor The contributor, who has root on the machines of the development infrastructure. The key is also a recipient of the agenix secrets The public half is in core/keys/contributors/ The contributor The code sets no end By hand, no schedule: core/keys/README.md, "Adding a contributor" and "Removing a contributor". A removal needs a rekey, a deploy, and a rotation of the secrets Open
Keys and password hash in ADMIN_EMERGENCY The holder of the offline emergency key, as the account emergency (wheel) on each node. Only for when no certificate is available The forge variable ADMIN_EMERGENCY (core/setups/nixos/users.nix); the private key offline A person The code sets no end Open Open
SSH key of the CD (cd) The CD jobs. The key is also a recipient of the agenix secrets The public half is core/keys/cd-ssh-key.pub A person The code sets no end Open Open
Host key of a system The system. The key is a recipient of the agenix secrets that it reads The public half is in core/keys/systems/ Open The code sets no end Open Open

Secrets of the development infrastructure

The files core/resources/secrets/<name>.age hold the secrets of the forge and its runners. Each file is encrypted to the recipients that core/resources/secrets/secrets.nix gives. A person makes each value, and the code sets no end for it. A change of the recipients is a rekey (core/resources/secrets/README.md, "Rekeying"): it needs a person who is a recipient now, and it does not change the value.

Generated:

File Credential Rotation What a rotation interrupts
anubis-forgejo-key.age Key of the anubis in front of the forge By hand: core/resources/secrets/README.md, "anubis-forgejo-key" Each visitor of the forge solves one challenge again
forgejo-database-password.age Password of the database user of the forge By hand: core/resources/secrets/README.md, "forgejo-database-password" New database connections of the forge fail between the change in the database and the restart of forgejo
forgejo-email-password.age Password of the mail account of the forge By hand: core/resources/secrets/README.md, "forgejo-email-password" Open
forgejo-runner-tokens-dev2.age Registration tokens of the runners of one machine By hand: core/resources/secrets/README.md, "forgejo-runner-tokens-<machine>" Open
forgejo-runner-tokens-dev3.age Registration tokens of the runners of one machine By hand: core/resources/secrets/README.md, "forgejo-runner-tokens-<machine>" Open
forgejo-runner-tokens-dev4.age Registration tokens of the runners of one machine By hand: core/resources/secrets/README.md, "forgejo-runner-tokens-<machine>" Open
forgejo-runner-tokens-fediversity-ci.age Registration tokens of the runners of one machine By hand: core/resources/secrets/README.md, "forgejo-runner-tokens-<machine>" Open
security-pgp-key.age OpenPGP key that reporters of vulnerabilities encrypt to By hand: core/resources/secrets/README.md, "security-pgp-key and security-pgp-passphrase" Each reader of the reports imports the key again
security-pgp-passphrase.age Passphrase of the OpenPGP key By hand: core/resources/secrets/README.md, "security-pgp-key and security-pgp-passphrase" See the row above

Secrets of the forge

The CD jobs and the other workflows read these secrets from the Forgejo Actions secrets of the repository. A person with the rights to set secrets on the forge writes each one. The code sets no end for a value, apart from the incus client certificate: the command in notes/cd-credentials.md makes it for 365 days. .forgejo/workflows/cd-env.example.sh gives the same names for a deploy from a workstation. The forge variable TF_BACKEND_HTPASSWD holds one line for each user of the state backend (cd, operator, and workstation-<person>), with the hash of the password only. The rotation of the password of a workstation-<person> user is Open.

Generated:

Secrets Credential Rotation What a rotation interrupts
SSH_PRIVATE_KEY SSH key that the CD jobs use to reach the nodes Open Open
INCUS_CLIENT_CRT, INCUS_CLIENT_KEY Client certificate of the CD for the incus API By hand: notes/cd-credentials.md, "Rotating it" The procedure adds the new certificate before it removes the old one
TF_HTTP_PASSWORD Password of the user cd of the state backend. The variable TF_BACKEND_HTPASSWD holds its hash. The state backend node makes the password of the user operator Open Open
SMARTHOST_UPSTREAM_PASSWORD Password of the account of the upstream smarthost. The deploy of the hosting provider writes it to the KV entry that the relays read Open Open
DNS_API_TOKEN, OPERATOR_DNS_API_TOKEN, OPERATOR_ZONES_MANAGER_TOKEN Tokens for the API of the DNS provider Open Open
DNS_TSIG_SECRET, DNS_SECONDARIES TSIG keys: the key of a zone that the name server holds, and the transfer keys of the secondaries Open Open
NETBOX_API_TOKEN Token for the API of NetBox Open Open
INWX_USERNAME, INWX_PASSWORD, INWX_SHARED_SECRET, OPENPROVIDER_USERNAME, OPENPROVIDER_PASSWORD, PORKBUN_API_KEY, PORKBUN_SECRET_KEY Credentials of the registrars Open Open
FORGEJO_BACKUP_RESTIC_PASSWORD, FORGEJO_BACKUP_S3_ACCESS_KEY, FORGEJO_BACKUP_S3_SECRET_KEY Password and S3 key of the offsite backup of the forge By hand: set the value, then start the workflow cd-dev-forgejo (notes/cd-credentials.md) Open
RELEASE_SIGNING_KEY SSH key that signs the release tags By hand: Set up the signing key The file .forgejo/workflows/release-allowed-signers keeps the old key for the old tags
SPIRE_ROOT_KEY Private key of the upstream root of SPIRE. The variable SPIRE_ROOT_CERT holds the certificate By hand: the specification of spire, "The upstream root" Open
SBOM_PACKAGE_TOKEN Token that publishes the SBOM package (SBOM) Open Open
BOT_PAT, DEPLOY_KEY Token and SSH key that the automation workflows use on the forge (update.yaml, registry-terms.yaml, vulnerability-scan.yaml) Open Open

The workflows also read these names as secrets. They give an address, a name or a selector, not a credential. Generated:

ADMIN_SOURCES, BIND2_IPV4, BIND2_IPV6, BIND_IPV4, BIND_IPV6, CLUSTER_SOURCES, DNS_APEX_NAMESERVERS, DNS_DOMAIN, DNS_PROVIDER, DNS_SECONDARIES_BY_ADDRESS, DNS_SERVER_URL, NETBOX_CLUSTER_NAME, NETBOX_CUSTOM_FIELDS, NETBOX_PREFIX_SELECTOR, NETBOX_RANGE_SELECTOR, NETBOX_SERVER_URL, NETBOX_TAGS, NETBOX_TENANT, OPERATOR_DNS_PROVIDER, OPERATOR_DNS_SERVER_URL, OPERATOR_NETBOX_RANGE_SELECTOR

Signing key of the bridge cache

The incus host signs the paths that its binary cache serves to its containers.

  • Where: /var/lib/incus-bridge-cache/ on the host. The key does not go out of the host.
  • Maker: the host.
  • Lifetime: fediversity.incus.bridgeCacheKey.rotationDays, 90 days by default.
  • Rotation: automatic. The timer incus-bridge-cache-key-rotate runs one time each day. The containers trust the previous key for fediversity.incus.bridgeCacheKey.overlapDays, 30 days by default.
  • What a rotation interrupts: nix-serve restarts.

The procedure after a compromise is in Signing key of the bridge cache.