State register

This part lists each store of state of the software: where it is, which component owns it, whose personal data it holds, how long it keeps the data, how it is protected, and how to erase data from it. It gives the facts for the record of processing that the data protection page assigns to the operator and to the hosting provider. It is not legal advice, and it gives no legal classification.

How the register is made

The tables on this page are generated at the documentation build from two inputs:

  • The declaration core/docs/md/technical-design/state-register-stores.nix. It gives, for each store, the purpose, the personal data, the retention, the encryption, the backup and the procedure to erase, with the files that these statements come from.
  • The code. The build reads the databases, the buckets, the valkey namespaces and the persistentPaths that the components request, the tables that the migrations of api/ create (api/src/fediversity_api/state_migrations/versions/), and the models of the panel (panel/src/panel/models.py). The components are those of an example deployment with each application enabled.

The build compares the two inputs. The check core-docs fails when:

  • an item in the code is in no store of the declaration,
  • the declaration names an item or a source file that does not exist,
  • a component owns no store and is not in the list of components without a store.

The check sees only the items that the code declares as data. It does not see a state directory that a module sets on a node, for example /var/lib/maddy. The declaration gives these paths by hand.

"Open" in a table means that no source in the tree gives the fact. The hosting provider finds it in its setup or in the upstream documentation, see the unknowns of the data protection page.

Which store holds which

Some stores are inside other stores. A statement about the outer store, for example its encryption, also applies to the stores inside it.

flowchart TD
  pools["Incus storage pools on the hypervisor"]
  nodes["Paths on the nodes"]
  postgresql["PostgreSQL server"]
  garage["Garage"]
  databases["Databases of the applications and the services"]
  buckets["Media and telemetry buckets"]
  backups["Backup buckets"]
  pools --> nodes
  nodes --> postgresql
  nodes --> garage
  postgresql --> databases
  garage --> buckets
  garage --> backups
  databases -- "backup" --> backups
  nodes -- "backup" --> backups

Location and owner

The column "Declared in the code" gives the items that the build found in the code for the store. The databases of the panel and of api/ and their backup buckets are not in the example deployment, thus the build does not check them.

Store Owner Node and path Purpose Declared in the code
api/ database api The api database on the PostgreSQL node, or a SQLite file. The deployments, their builds and operations, their domains, the SBOMs of their nodes, and the objects that a destroy-time step did not remove. table of api/: deployment, build, operation_run, zone_follow, domain_transfer, domain_event, domain_contact, sbom_component, node_sbom, node_sbom_component, left_behind
Application databases mastodon, peertube, pixelfed One database for each application of a deployment, on the PostgreSQL node of the hosting provider. The records of the application. bucket: mastodon-backups, peertube-backups, pixelfed-backups; database: mastodon, peertube, pixelfed
Application files on the node peertube, pixelfed /var/lib/peertube on the PeerTube node and /var/lib/pixelfed/storage/app on the Pixelfed node. The files that the application does not keep in a bucket. --
Application media buckets mastodon, peertube, pixelfed Buckets on the Garage node of the hosting provider, with a prefix for each operator. The media that the application serves. bucket: mastodon, peertube-videos, peertube-playlists, pixelfed
Audit records api The audit_event and audit_chain_anchor tables of the api database. api/ also writes each event to the journal, and the tenant audit of Loki keeps that copy. The record of who did which action on a deployment. table of api/: audit_event, audit_chain_anchor
Authelia authelia The authelia database on the PostgreSQL node, and the authelia namespace of valkey for the sessions. Each group has its own Authelia node. The single sign-on of a group of nodes. bucket: authelia-backups; database: authelia; valkey namespace: authelia
Backups each node The *-backups buckets on the Garage of the hosting provider. A group can also copy the backups to a target outside the site. The restore of a service. --
DNS zones bind /var/lib/named on the BIND node. The DNS zones that the hosting provider serves. bucket: bind-backups
Garage garage /var/lib/garage on the Garage node of the hosting provider, with the metadata snapshots in /var/lib/garage/snapshots. It holds each bucket of this register. The object storage, and the target of the backups. --
Grafana of the hosting provider telemetry A SQLite file on the telemetry node. The accounts of the OIDC logins and their preferences, the service accounts, the silences, the annotations and the dashboards that are not provisioned. bucket: grafana-backups
Grafana of a deployment grafana The file grafana.db in a runtime directory below /run on the Grafana node of the deployment. The local records of the Grafana that shows the metrics of a deployment to its operator. --
Incus storage pools incus The storage pools of fediversity.incus.storagePools on the hypervisor. The root file systems of the nodes. Each store of this register that is a path on a node is in a pool. --
lldap lldap The lldap database on the PostgreSQL node. Each group has its own lldap node. The directory of the users and the groups of a group of nodes. bucket: lldap-backups; database: lldap
Journal and access logs each node The journal of each node, and the access log of nginx on each node that runs nginx. The debugging of a node. --
Mail smtp /var/lib/maddy on the SMTP node of the group. The queue of the mail that the applications and the shared services send. --
NetBox netbox The netbox database on the PostgreSQL node. The address management (IPAM) of the hosting provider. bucket: netbox-backups; database: netbox
OpenBao openbao The raft storage in /var/lib/openbao/raft on the OpenBao node. The secrets of an instance are below the KV path secret/data/vars/secret/<instance>/. The secrets that the components generate and the secrets that the operator brings. bucket: openbao-backups
Panel database panel The panel database on the PostgreSQL node. A panel without a DATABASE_URL uses a SQLite file below /var/lib. The accounts of the operators, their settings and their deployments. model of the panel: Deployment, DomainContact, OperatorSettings
PostgreSQL server postgresql The PostgreSQL node of the hosting provider. It holds each database of this register that is not SQLite. The database server that the components share. bucket: postgresql-backups
SPIRE spire /var/lib/spire-server on the SPIRE node. The identities of the workloads. --
Telemetry (Loki, Tempo, Mimir) telemetry The loki, tempo and mimir buckets on Garage. The logs, the traces and the metrics of the nodes. bucket: loki, tempo, mimir
OpenTofu state backend terraform-backend /var/lib/private/terraform-backend on the node of the state backend. The OpenTofu states of the deployments. bucket: terraform-backend-backups; path: /var/lib/private/terraform-backend
Valkey namespaces valkey /var/lib/redis-<namespace> on the valkey node of the group. An application of a group that has no valkey node uses a local redis on its own node. The cache and the job queue of an application. valkey namespace: mastodon, peertube, pixelfed, valkey
Windmill windmill The windmill database on the PostgreSQL node. The jobs that deploy. bucket: windmill-backups; database: windmill

Personal data and retention

The personal data of each log, trace and audit channel, with its readers and its controls, is in Logs, traces and audit records.

The retention settings that an operator can set for an application are in Personal data per application.

Store Personal data, and of whom Retention of the live store
api/ database Of the operators: the configuration of each deployment, the build logs, the output and the actor of each operation, and the identifier and the handle of each domain contact. services.fediversity-api.logRetentionDays for the build logs and the runs of the operations: 30 days by default, 0 keeps them. The operation_run table keeps at most the last 100 runs of each deployment. No job removes old rows of the build table. A row of the left_behind table stays after its deployment is gone, until a retry removes the object or an operator forgets the row. For the other tables the code sets no period.
Application databases Of the users of the application: the accounts and their email addresses, the posts, the direct messages, the follows and the sessions. For PeerTube also the videos and their metadata, the comments and the watch history. The applications also keep content that they get from other instances. Until the user or a moderator deletes the data. The code sets none of the retention settings of the applications.
Application files on the node Of the users of the application: for example avatars, thumbnails, captions, profile headers and custom emoji. /var/lib/peertube also holds the logs of PeerTube. Until the application deletes the file. The code sets no period.
Application media buckets Of the users of the application: the uploads, for example images, videos and avatars. Mastodon also keeps cached media of other instances. Until the application deletes the object. The code sets none of the media retention settings of the applications.
Audit records Of the operators and the administrators: the actor of each action. services.fediversity-api.auditRetentionDays for the table and services.fediversity-telemetry.auditRetentionDays for the tenant audit, each 365 days by default.
Authelia Of the users of the group: the consents, the second factors, the identity verification tokens, the OIDC state and the sessions. The code sets no period. The code also does not set the session lifetime, the inactivity limit and the regulation. The operator can set them.
Backups That of the store that each backup is from. See its row. See the backup of each row. After a destroy, a *-backups bucket stays for the grace period of Garage.
DNS zones None from the software: the records hold the names and the addresses of the services. Open: no source in the tree gives it.
Garage That of the buckets on it. See their rows. That of the buckets on it. Garage keeps one copy of each object (replication_factor = 1).
Grafana of the hosting provider Of the administrators: the accounts that the first OIDC login makes. The code sets no period.
Grafana of a deployment Of the operators: the accounts that an OIDC login makes. Until the next restart of the service.
Incus storage pools That of the stores on the nodes. See their rows. Until a destroy removes the node.
lldap Of the users of the group: the user names, the email addresses, the names, the avatars and the group memberships. The first administrator gets the email of the configuration. Until a member of lldap_admin deletes the user.
Journal and access logs Of the users, the operators and the administrators: client addresses and user names (sshd, nginx errors, Authelia), email addresses (maddy), the audit lines of OpenBao, and what each application writes to its log. The journal: 30 days and at most 512 MB (MaxRetentionSec, SystemMaxUse). The access log of nginx: logrotate keeps 26 weeks.
Mail Of the senders and the recipients: the addresses, the headers and the content of the mail that passes, for example password resets and notifications. The code sets no retention on the queue.
NetBox Of the administrators: their accounts. If the IPAM records hold personal data is open. Open: no source in the tree gives it.
OpenBao Of the administrators: the names that OIDC logins give them. If a secret is personal data is open. Until deleteData or a destroy removes the KV entries. The code sets no period.
Panel database Of the operators: the accounts, with the email address and the user name from the OIDC login, the sessions, the domain contacts of the operator with their labels and handles, and the deployments with their configuration and the domain contact of each role. The panel keeps the deployments that the operator deleted, for the billing. Until a panel administrator removes the account. The session lifetime is the default of Django, and no job removes expired sessions.
PostgreSQL server That of the databases on it. See their rows. That of the databases on it.
SPIRE Open: no source in the tree gives it. Open: no source in the tree gives it.
Telemetry (Loki, Tempo, Mimir) A copy of the journal of each node of the hosting provider, and the journal lines of priority warning and higher of each node of an operator deployment. See the row of the journal for the personal data. services.fediversity-telemetry.retentionDays, 90 days by default.
OpenTofu state backend Of the domain contacts: the state with the key <base>-<contact>-domain-contact holds the contact that an operator saves through api/: the name, the address, the telephone number and the email address, not hidden. If the other states hold personal data is open. The code sets no period. The state of a contact stays until DELETE /domain_contacts/{contact_id} of api/ erases it.
Valkey namespaces Of the users of the application: what the application caches, for example queued jobs and timelines. The code sets no retention on the cache.
Windmill Of the administrators: their accounts. Of the operators: the arguments, the result and the log of each job, which are the configuration of a deployment, its deploy log and the SBOMs of its nodes. Windmill removes a completed job after services.fediversity-windmill.jobRetentionDays, 30 days by default.

Encryption, backup and erasure

The table gives the encryption that the software adds to a store. The encryption of the disks is not in the code. The contract templates state that the disks of the hypervisors are not encrypted.

The procedures of the hosting provider are in the hosting provider guide: Know how long the backups keep personal data, Erase personal data and Know what a destroy keeps.

Store Encryption at rest Backup Procedure to erase
api/ database None in the code. A restic repository in the api-backups bucket. The backup keeps 30 days (--keep-within 30d). DELETE /deployments/{id} destroys a deployment. DELETE /domain_contacts/{contact_id} removes the handle of a contact. DELETE /deployments/left-behind/{entry_id}/{deployment_id} forgets one left-behind object.
Application databases None in the code. A restic repository in the <app>-backups bucket. The backup keeps 30 days (--keep-within 30d). The account deletion of the application. To erase all data of an application, set enable to false and deleteData to true, or destroy the deployment.
Application files on the node None in the code. A restic repository in the peertube-backups or the pixelfed-backups bucket. The backup keeps 30 days (--keep-within 30d). Open: no source in the tree gives it.
Application media buckets None in the code. The media backup, in the prefix media/ of the <app>-backups bucket. It keeps 30 days of history by default (fediversity.mediaBackup.instances.<app>.retentionDays). The application deletes the object. media-backup-<app> erase <bucket> <key> also removes the object from the media backup and from its history.
Audit records None in the code. With the api database, in the api-backups bucket. The backup keeps 30 days (--keep-within 30d). Open: no source in the tree gives it.
Authelia Authelia encrypts its database with a storage key (storageEncryptionKeyFile). A restic repository in the authelia-backups bucket. The backup keeps 30 days (--keep-within 30d). Open: no source in the tree gives it.
Backups The media backups and the restic backups are encrypted. None in the code. Garage has no backup of its objects. The restic backups have no erase command. An erased record stays in them until they expire. The media backup has media-backup-<app> erase.
DNS zones None in the code. The records of the apex zone on the primary, in the bind-backups bucket. The backup does not hold the zones of the operators, the zones of the challenge lanes or the DNSSEC keys (core/components/hosting-provider/bind/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. Open: no source in the tree gives it.
Garage None in the code. None in the code for the objects. Garage makes snapshots of its metadata on the same node. A group can copy the backups to a target outside the site. See the rows of the buckets. A destroy keeps each *-backups bucket for a grace period (fediversity.garage.server.retainedBackups.gracePeriodDays, 7 days by default), then a timer deletes it.
Grafana of the hosting provider Grafana encrypts the secrets in its database with a secret key. The SQLite database, in the grafana-backups bucket. The secret key of the database is in OpenBao KV, not in the backup (core/components/hosting-provider/telemetry/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. Open: no source in the tree gives it.
Grafana of a deployment None in the code. None. A restart of the service.
Incus storage pools None in the code. None for a pool. See the rows of the stores on the nodes. A destroy of a deployment removes its nodes.
lldap None in the code. A restic repository in the lldap-backups bucket. The backup keeps 30 days (--keep-within 30d). A member of lldap_admin deletes the user at https://users.<domain>/.
Journal and access logs None in the code. None. Loki keeps a copy, see the row of the telemetry. Open: no source in the tree gives it.
Mail None in the code. None. Open: no source in the tree gives it.
NetBox None in the code. Two restic repositories in the netbox-backups bucket: one for the database (core/components/hosting-provider/netbox/default.nix) and one for the files that users give to NetBox (core/components/hosting-provider/netbox/uploads-backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. Open: no source in the tree gives it.
OpenBao The unseal key is sealed to the TPM, and its copy on the disk is encrypted to age recipients. A raft snapshot and the unseal key, in the openbao-backups bucket. The unseal key is encrypted to the admin recipients (core/components/hosting-provider/openbao/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. deleteData of an application deletes its generated secrets and the secrets that the operator brought. A destroy removes the KV entries of the deployment.
Panel database None in the code. A restic repository in the panel-backups bucket. The backup keeps 30 days (--keep-within 30d). A panel administrator removes the account. The panel first erases the domain contacts of the account, and queues the destroy of each deployment of the account.
PostgreSQL server None in the code. Each database has its own backup. Before an upgrade of the major of PostgreSQL, the server also makes a snapshot of its data directory in the postgresql-backups bucket. See the rows of the databases.
SPIRE None in the code. None. Open: no source in the tree gives it.
Telemetry (Loki, Tempo, Mimir) None in the code. None. Open: no source in the tree gives it.
OpenTofu state backend None in the code. The states and the kmsKey that encrypts them, in the terraform-backend-backups bucket. The key is encrypted to the admin recipients (core/components/hosting-provider/terraform-backend/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. For a contact: DELETE /domain_contacts/{contact_id} of api/ deletes the customer in the registrar account and then the state. The call refuses while a domain of the account names the contact. A change of a name deletes the old customer, unless a domain names it.
Valkey namespaces None in the code. A restic repository for each namespace, in the cache backups bucket of the group. A local redis goes to the <app>-backups bucket. The backup keeps 30 days (--keep-within 30d). deleteData of the application deletes its cache.
Windmill None in the code. A restic repository in the windmill-backups bucket. The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. Open: no source in the tree gives it.

Components without a store

Component Why it has no store
otel-collector It sends each signal that it gets to the store of that signal on the telemetry node.
overview It serves a static page that lists the web UIs of the deployment.

Sources

Store Sources
api/ database api/src/fediversity_api/state.py, api/src/fediversity_api/left_behind.py, api/src/fediversity_api/state_migrations/versions/0001_baseline.py, api/src/fediversity_api/state_migrations/versions/0010_left_behind.py, api/nix/component/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Application databases core/components/operator/app-resources.nix, core/components/sharedOptions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Application files on the node core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Application media buckets core/components/operator/app-resources.nix, core/resources/garage/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Audit records api/src/fediversity_api/state.py, api/src/fediversity_api/settings.py, api/nix/configuration.nix, core/components/hosting-provider/telemetry.nix, core/docs/md/guides/hosting-provider/logs.md, core/docs/md/guides/hosting-provider/backups.md
Authelia core/components/shared/authelia/default.nix, core/components/shared/authelia/database.nix, core/components/shared/authelia/sessions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Backups core/components/operator/app-resources.nix, core/components/shared/postgres-backup.nix, core/docs/md/compliance/contract-templates.md, core/docs/md/guides/hosting-provider/backups.md
DNS zones core/components/hosting-provider/bind.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Garage core/resources/garage/server.nix, core/resources/garage/server-options.nix, core/resources/garage/metadata-snapshots.nix, core/resources/garage/migrations.nix, core/docs/md/guides/hosting-provider/backups.md
Grafana of the hosting provider core/components/hosting-provider/telemetry.nix, core/components/hosting-provider/telemetry/backup.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Grafana of a deployment core/components/operator/grafana/node.nix, core/docs/md/compliance/data-protection.md
Incus storage pools core/components/hosting-provider/incus.nix, core/docs/md/guides/hosting-provider/access.md
lldap core/components/shared/lldap/default.nix, core/components/shared/lldap/database.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Journal and access logs core/setups/nixos/common.nix, core/setups/nixos/nginx.nix, core/components/hosting-provider/node-logs.nix, core/docs/md/guides/hosting-provider/logs.md
Mail core/components/shared/smtp/default.nix, core/resources/smtp/maddy-migrations.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
NetBox core/components/hosting-provider/netbox.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
OpenBao core/components/hosting-provider/openbao/openbao.nix, core/components/hosting-provider/openbao/backup.nix, core/components/hosting-provider/openbao/deployer-policies.nix, core/components/sharedOptions.nix, core/docs/md/compliance/threat-model.md, core/docs/md/guides/hosting-provider/backups.md
Panel database panel/src/panel/models.py, panel/nix/component/default.nix, panel/nix/configuration.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
PostgreSQL server core/setups/wiring/postgresql-roster.nix, core/components/hosting-provider/postgresql/default.nix, core/docs/md/guides/hosting-provider/backups.md
SPIRE core/components/hosting-provider/spire/spire.nix, core/docs/md/guides/hosting-provider/backups.md
Telemetry (Loki, Tempo, Mimir) core/components/hosting-provider/telemetry.nix, core/components/hosting-provider/telemetry/object-store.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/logs.md, core/docs/md/guides/hosting-provider/monitoring.md, core/docs/md/guides/hosting-provider/backups.md
OpenTofu state backend core/components/hosting-provider/terraform-backend/state-dir.nix, core/components/hosting-provider/terraform-backend/backup.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md, core/docs/md/guides/hosting-provider/domain-transfers.md
Valkey namespaces core/components/shared/valkey/valkey.nix, core/components/shared/valkey/default.nix, core/components/sharedOptions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md
Windmill core/components/hosting-provider/windmill/windmill.nix, core/components/hosting-provider/windmill/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md