State register
This part lists each store of state of the software: where it is, which component owns it, whose personal data it holds, how long it keeps the data, how it is protected, and how to erase data from it. It gives the facts for the record of processing that the data protection page assigns to the operator and to the hosting provider. It is not legal advice, and it gives no legal classification.
How the register is made
The tables on this page are generated at the documentation build from two inputs:
- The declaration
core/docs/md/technical-design/state-register-stores.nix. It gives, for each store, the purpose, the personal data, the retention, the encryption, the backup and the procedure to erase, with the files that these statements come from. - The code. The build reads the databases, the buckets, the valkey namespaces and the
persistentPathsthat the components request, the tables that the migrations ofapi/create (api/src/fediversity_api/state_migrations/versions/), and the models of the panel (panel/src/panel/models.py). The components are those of an example deployment with each application enabled.
The build compares the two inputs. The check core-docs fails when:
- an item in the code is in no store of the declaration,
- the declaration names an item or a source file that does not exist,
- a component owns no store and is not in the list of components without a store.
The check sees only the items that the code declares as data. It does not see a state directory that a module sets on a node, for example /var/lib/maddy. The declaration gives these paths by hand.
"Open" in a table means that no source in the tree gives the fact. The hosting provider finds it in its setup or in the upstream documentation, see the unknowns of the data protection page.
Which store holds which
Some stores are inside other stores. A statement about the outer store, for example its encryption, also applies to the stores inside it.
flowchart TD pools["Incus storage pools on the hypervisor"] nodes["Paths on the nodes"] postgresql["PostgreSQL server"] garage["Garage"] databases["Databases of the applications and the services"] buckets["Media and telemetry buckets"] backups["Backup buckets"] pools --> nodes nodes --> postgresql nodes --> garage postgresql --> databases garage --> buckets garage --> backups databases -- "backup" --> backups nodes -- "backup" --> backups
Location and owner
The column "Declared in the code" gives the items that the build found in the code for the store. The databases of the panel and of api/ and their backup buckets are not in the example deployment, thus the build does not check them.
| Store | Owner | Node and path | Purpose | Declared in the code |
|---|---|---|---|---|
api/ database |
api |
The api database on the PostgreSQL node, or a SQLite file. |
The deployments, their builds and operations, their domains, the SBOMs of their nodes, and the objects that a destroy-time step did not remove. | table of api/: deployment, build, operation_run, zone_follow, domain_transfer, domain_event, domain_contact, sbom_component, node_sbom, node_sbom_component, left_behind |
| Application databases | mastodon, peertube, pixelfed |
One database for each application of a deployment, on the PostgreSQL node of the hosting provider. | The records of the application. | bucket: mastodon-backups, peertube-backups, pixelfed-backups; database: mastodon, peertube, pixelfed |
| Application files on the node | peertube, pixelfed |
/var/lib/peertube on the PeerTube node and /var/lib/pixelfed/storage/app on the Pixelfed node. |
The files that the application does not keep in a bucket. | -- |
| Application media buckets | mastodon, peertube, pixelfed |
Buckets on the Garage node of the hosting provider, with a prefix for each operator. | The media that the application serves. | bucket: mastodon, peertube-videos, peertube-playlists, pixelfed |
| Audit records | api |
The audit_event and audit_chain_anchor tables of the api database. api/ also writes each event to the journal, and the tenant audit of Loki keeps that copy. |
The record of who did which action on a deployment. | table of api/: audit_event, audit_chain_anchor |
| Authelia | authelia |
The authelia database on the PostgreSQL node, and the authelia namespace of valkey for the sessions. Each group has its own Authelia node. |
The single sign-on of a group of nodes. | bucket: authelia-backups; database: authelia; valkey namespace: authelia |
| Backups | each node | The *-backups buckets on the Garage of the hosting provider. A group can also copy the backups to a target outside the site. |
The restore of a service. | -- |
| DNS zones | bind |
/var/lib/named on the BIND node. |
The DNS zones that the hosting provider serves. | bucket: bind-backups |
| Garage | garage |
/var/lib/garage on the Garage node of the hosting provider, with the metadata snapshots in /var/lib/garage/snapshots. It holds each bucket of this register. |
The object storage, and the target of the backups. | -- |
| Grafana of the hosting provider | telemetry |
A SQLite file on the telemetry node. | The accounts of the OIDC logins and their preferences, the service accounts, the silences, the annotations and the dashboards that are not provisioned. | bucket: grafana-backups |
| Grafana of a deployment | grafana |
The file grafana.db in a runtime directory below /run on the Grafana node of the deployment. |
The local records of the Grafana that shows the metrics of a deployment to its operator. | -- |
| Incus storage pools | incus |
The storage pools of fediversity.incus.storagePools on the hypervisor. |
The root file systems of the nodes. Each store of this register that is a path on a node is in a pool. | -- |
lldap |
lldap |
The lldap database on the PostgreSQL node. Each group has its own lldap node. |
The directory of the users and the groups of a group of nodes. | bucket: lldap-backups; database: lldap |
| Journal and access logs | each node | The journal of each node, and the access log of nginx on each node that runs nginx. | The debugging of a node. | -- |
smtp |
/var/lib/maddy on the SMTP node of the group. |
The queue of the mail that the applications and the shared services send. | -- | |
| NetBox | netbox |
The netbox database on the PostgreSQL node. |
The address management (IPAM) of the hosting provider. | bucket: netbox-backups; database: netbox |
| OpenBao | openbao |
The raft storage in /var/lib/openbao/raft on the OpenBao node. The secrets of an instance are below the KV path secret/data/vars/secret/<instance>/. |
The secrets that the components generate and the secrets that the operator brings. | bucket: openbao-backups |
| Panel database | panel |
The panel database on the PostgreSQL node. A panel without a DATABASE_URL uses a SQLite file below /var/lib. |
The accounts of the operators, their settings and their deployments. | model of the panel: Deployment, DomainContact, OperatorSettings |
| PostgreSQL server | postgresql |
The PostgreSQL node of the hosting provider. It holds each database of this register that is not SQLite. | The database server that the components share. | bucket: postgresql-backups |
| SPIRE | spire |
/var/lib/spire-server on the SPIRE node. |
The identities of the workloads. | -- |
| Telemetry (Loki, Tempo, Mimir) | telemetry |
The loki, tempo and mimir buckets on Garage. |
The logs, the traces and the metrics of the nodes. | bucket: loki, tempo, mimir |
| OpenTofu state backend | terraform-backend |
/var/lib/private/terraform-backend on the node of the state backend. |
The OpenTofu states of the deployments. | bucket: terraform-backend-backups; path: /var/lib/private/terraform-backend |
| Valkey namespaces | valkey |
/var/lib/redis-<namespace> on the valkey node of the group. An application of a group that has no valkey node uses a local redis on its own node. |
The cache and the job queue of an application. | valkey namespace: mastodon, peertube, pixelfed, valkey |
| Windmill | windmill |
The windmill database on the PostgreSQL node. |
The jobs that deploy. | bucket: windmill-backups; database: windmill |
Personal data and retention
The personal data of each log, trace and audit channel, with its readers and its controls, is in Logs, traces and audit records.
The retention settings that an operator can set for an application are in Personal data per application.
| Store | Personal data, and of whom | Retention of the live store |
|---|---|---|
api/ database |
Of the operators: the configuration of each deployment, the build logs, the output and the actor of each operation, and the identifier and the handle of each domain contact. | services.fediversity-api.logRetentionDays for the build logs and the runs of the operations: 30 days by default, 0 keeps them. The operation_run table keeps at most the last 100 runs of each deployment. No job removes old rows of the build table. A row of the left_behind table stays after its deployment is gone, until a retry removes the object or an operator forgets the row. For the other tables the code sets no period. |
| Application databases | Of the users of the application: the accounts and their email addresses, the posts, the direct messages, the follows and the sessions. For PeerTube also the videos and their metadata, the comments and the watch history. The applications also keep content that they get from other instances. | Until the user or a moderator deletes the data. The code sets none of the retention settings of the applications. |
| Application files on the node | Of the users of the application: for example avatars, thumbnails, captions, profile headers and custom emoji. /var/lib/peertube also holds the logs of PeerTube. |
Until the application deletes the file. The code sets no period. |
| Application media buckets | Of the users of the application: the uploads, for example images, videos and avatars. Mastodon also keeps cached media of other instances. | Until the application deletes the object. The code sets none of the media retention settings of the applications. |
| Audit records | Of the operators and the administrators: the actor of each action. | services.fediversity-api.auditRetentionDays for the table and services.fediversity-telemetry.auditRetentionDays for the tenant audit, each 365 days by default. |
| Authelia | Of the users of the group: the consents, the second factors, the identity verification tokens, the OIDC state and the sessions. | The code sets no period. The code also does not set the session lifetime, the inactivity limit and the regulation. The operator can set them. |
| Backups | That of the store that each backup is from. See its row. | See the backup of each row. After a destroy, a *-backups bucket stays for the grace period of Garage. |
| DNS zones | None from the software: the records hold the names and the addresses of the services. | Open: no source in the tree gives it. |
| Garage | That of the buckets on it. See their rows. | That of the buckets on it. Garage keeps one copy of each object (replication_factor = 1). |
| Grafana of the hosting provider | Of the administrators: the accounts that the first OIDC login makes. | The code sets no period. |
| Grafana of a deployment | Of the operators: the accounts that an OIDC login makes. | Until the next restart of the service. |
| Incus storage pools | That of the stores on the nodes. See their rows. | Until a destroy removes the node. |
lldap |
Of the users of the group: the user names, the email addresses, the names, the avatars and the group memberships. The first administrator gets the email of the configuration. |
Until a member of lldap_admin deletes the user. |
| Journal and access logs | Of the users, the operators and the administrators: client addresses and user names (sshd, nginx errors, Authelia), email addresses (maddy), the audit lines of OpenBao, and what each application writes to its log. | The journal: 30 days and at most 512 MB (MaxRetentionSec, SystemMaxUse). The access log of nginx: logrotate keeps 26 weeks. |
| Of the senders and the recipients: the addresses, the headers and the content of the mail that passes, for example password resets and notifications. | The code sets no retention on the queue. | |
| NetBox | Of the administrators: their accounts. If the IPAM records hold personal data is open. | Open: no source in the tree gives it. |
| OpenBao | Of the administrators: the names that OIDC logins give them. If a secret is personal data is open. | Until deleteData or a destroy removes the KV entries. The code sets no period. |
| Panel database | Of the operators: the accounts, with the email address and the user name from the OIDC login, the sessions, the domain contacts of the operator with their labels and handles, and the deployments with their configuration and the domain contact of each role. The panel keeps the deployments that the operator deleted, for the billing. | Until a panel administrator removes the account. The session lifetime is the default of Django, and no job removes expired sessions. |
| PostgreSQL server | That of the databases on it. See their rows. | That of the databases on it. |
| SPIRE | Open: no source in the tree gives it. | Open: no source in the tree gives it. |
| Telemetry (Loki, Tempo, Mimir) | A copy of the journal of each node of the hosting provider, and the journal lines of priority warning and higher of each node of an operator deployment. See the row of the journal for the personal data. |
services.fediversity-telemetry.retentionDays, 90 days by default. |
| OpenTofu state backend | Of the domain contacts: the state with the key <base>-<contact>-domain-contact holds the contact that an operator saves through api/: the name, the address, the telephone number and the email address, not hidden. If the other states hold personal data is open. |
The code sets no period. The state of a contact stays until DELETE /domain_contacts/{contact_id} of api/ erases it. |
| Valkey namespaces | Of the users of the application: what the application caches, for example queued jobs and timelines. | The code sets no retention on the cache. |
| Windmill | Of the administrators: their accounts. Of the operators: the arguments, the result and the log of each job, which are the configuration of a deployment, its deploy log and the SBOMs of its nodes. | Windmill removes a completed job after services.fediversity-windmill.jobRetentionDays, 30 days by default. |
Encryption, backup and erasure
The table gives the encryption that the software adds to a store. The encryption of the disks is not in the code. The contract templates state that the disks of the hypervisors are not encrypted.
The procedures of the hosting provider are in the hosting provider guide: Know how long the backups keep personal data, Erase personal data and Know what a destroy keeps.
| Store | Encryption at rest | Backup | Procedure to erase |
|---|---|---|---|
api/ database |
None in the code. | A restic repository in the api-backups bucket. The backup keeps 30 days (--keep-within 30d). |
DELETE /deployments/{id} destroys a deployment. DELETE /domain_contacts/{contact_id} removes the handle of a contact. DELETE /deployments/left-behind/{entry_id}/{deployment_id} forgets one left-behind object. |
| Application databases | None in the code. | A restic repository in the <app>-backups bucket. The backup keeps 30 days (--keep-within 30d). |
The account deletion of the application. To erase all data of an application, set enable to false and deleteData to true, or destroy the deployment. |
| Application files on the node | None in the code. | A restic repository in the peertube-backups or the pixelfed-backups bucket. The backup keeps 30 days (--keep-within 30d). |
Open: no source in the tree gives it. |
| Application media buckets | None in the code. | The media backup, in the prefix media/ of the <app>-backups bucket. It keeps 30 days of history by default (fediversity.mediaBackup.instances.<app>.retentionDays). |
The application deletes the object. media-backup-<app> erase <bucket> <key> also removes the object from the media backup and from its history. |
| Audit records | None in the code. | With the api database, in the api-backups bucket. The backup keeps 30 days (--keep-within 30d). |
Open: no source in the tree gives it. |
| Authelia | Authelia encrypts its database with a storage key (storageEncryptionKeyFile). |
A restic repository in the authelia-backups bucket. The backup keeps 30 days (--keep-within 30d). |
Open: no source in the tree gives it. |
| Backups | The media backups and the restic backups are encrypted. | None in the code. Garage has no backup of its objects. | The restic backups have no erase command. An erased record stays in them until they expire. The media backup has media-backup-<app> erase. |
| DNS zones | None in the code. | The records of the apex zone on the primary, in the bind-backups bucket. The backup does not hold the zones of the operators, the zones of the challenge lanes or the DNSSEC keys (core/components/hosting-provider/bind/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
Open: no source in the tree gives it. |
| Garage | None in the code. | None in the code for the objects. Garage makes snapshots of its metadata on the same node. A group can copy the backups to a target outside the site. | See the rows of the buckets. A destroy keeps each *-backups bucket for a grace period (fediversity.garage.server.retainedBackups.gracePeriodDays, 7 days by default), then a timer deletes it. |
| Grafana of the hosting provider | Grafana encrypts the secrets in its database with a secret key. | The SQLite database, in the grafana-backups bucket. The secret key of the database is in OpenBao KV, not in the backup (core/components/hosting-provider/telemetry/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
Open: no source in the tree gives it. |
| Grafana of a deployment | None in the code. | None. | A restart of the service. |
| Incus storage pools | None in the code. | None for a pool. See the rows of the stores on the nodes. | A destroy of a deployment removes its nodes. |
lldap |
None in the code. | A restic repository in the lldap-backups bucket. The backup keeps 30 days (--keep-within 30d). |
A member of lldap_admin deletes the user at https://users.<domain>/. |
| Journal and access logs | None in the code. | None. Loki keeps a copy, see the row of the telemetry. | Open: no source in the tree gives it. |
| None in the code. | None. | Open: no source in the tree gives it. | |
| NetBox | None in the code. | Two restic repositories in the netbox-backups bucket: one for the database (core/components/hosting-provider/netbox/default.nix) and one for the files that users give to NetBox (core/components/hosting-provider/netbox/uploads-backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
Open: no source in the tree gives it. |
| OpenBao | The unseal key is sealed to the TPM, and its copy on the disk is encrypted to age recipients. | A raft snapshot and the unseal key, in the openbao-backups bucket. The unseal key is encrypted to the admin recipients (core/components/hosting-provider/openbao/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
deleteData of an application deletes its generated secrets and the secrets that the operator brought. A destroy removes the KV entries of the deployment. |
| Panel database | None in the code. | A restic repository in the panel-backups bucket. The backup keeps 30 days (--keep-within 30d). |
A panel administrator removes the account. The panel first erases the domain contacts of the account, and queues the destroy of each deployment of the account. |
| PostgreSQL server | None in the code. | Each database has its own backup. Before an upgrade of the major of PostgreSQL, the server also makes a snapshot of its data directory in the postgresql-backups bucket. |
See the rows of the databases. |
| SPIRE | None in the code. | None. | Open: no source in the tree gives it. |
| Telemetry (Loki, Tempo, Mimir) | None in the code. | None. | Open: no source in the tree gives it. |
| OpenTofu state backend | None in the code. | The states and the kmsKey that encrypts them, in the terraform-backend-backups bucket. The key is encrypted to the admin recipients (core/components/hosting-provider/terraform-backend/backup.nix). The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
For a contact: DELETE /domain_contacts/{contact_id} of api/ deletes the customer in the registrar account and then the state. The call refuses while a domain of the account names the contact. A change of a name deletes the old customer, unless a domain names it. |
| Valkey namespaces | None in the code. | A restic repository for each namespace, in the cache backups bucket of the group. A local redis goes to the <app>-backups bucket. The backup keeps 30 days (--keep-within 30d). |
deleteData of the application deletes its cache. |
| Windmill | None in the code. | A restic repository in the windmill-backups bucket. The backup keeps the snapshots of --keep-daily 7 --keep-weekly 4 --keep-monthly 6. |
Open: no source in the tree gives it. |
Components without a store
| Component | Why it has no store |
|---|---|
otel-collector |
It sends each signal that it gets to the store of that signal on the telemetry node. |
overview |
It serves a static page that lists the web UIs of the deployment. |
Sources
| Store | Sources |
|---|---|
api/ database |
api/src/fediversity_api/state.py, api/src/fediversity_api/left_behind.py, api/src/fediversity_api/state_migrations/versions/0001_baseline.py, api/src/fediversity_api/state_migrations/versions/0010_left_behind.py, api/nix/component/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Application databases | core/components/operator/app-resources.nix, core/components/sharedOptions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Application files on the node | core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Application media buckets | core/components/operator/app-resources.nix, core/resources/garage/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Audit records | api/src/fediversity_api/state.py, api/src/fediversity_api/settings.py, api/nix/configuration.nix, core/components/hosting-provider/telemetry.nix, core/docs/md/guides/hosting-provider/logs.md, core/docs/md/guides/hosting-provider/backups.md |
| Authelia | core/components/shared/authelia/default.nix, core/components/shared/authelia/database.nix, core/components/shared/authelia/sessions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Backups | core/components/operator/app-resources.nix, core/components/shared/postgres-backup.nix, core/docs/md/compliance/contract-templates.md, core/docs/md/guides/hosting-provider/backups.md |
| DNS zones | core/components/hosting-provider/bind.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Garage | core/resources/garage/server.nix, core/resources/garage/server-options.nix, core/resources/garage/metadata-snapshots.nix, core/resources/garage/migrations.nix, core/docs/md/guides/hosting-provider/backups.md |
| Grafana of the hosting provider | core/components/hosting-provider/telemetry.nix, core/components/hosting-provider/telemetry/backup.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Grafana of a deployment | core/components/operator/grafana/node.nix, core/docs/md/compliance/data-protection.md |
| Incus storage pools | core/components/hosting-provider/incus.nix, core/docs/md/guides/hosting-provider/access.md |
lldap |
core/components/shared/lldap/default.nix, core/components/shared/lldap/database.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Journal and access logs | core/setups/nixos/common.nix, core/setups/nixos/nginx.nix, core/components/hosting-provider/node-logs.nix, core/docs/md/guides/hosting-provider/logs.md |
core/components/shared/smtp/default.nix, core/resources/smtp/maddy-migrations.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
|
| NetBox | core/components/hosting-provider/netbox.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| OpenBao | core/components/hosting-provider/openbao/openbao.nix, core/components/hosting-provider/openbao/backup.nix, core/components/hosting-provider/openbao/deployer-policies.nix, core/components/sharedOptions.nix, core/docs/md/compliance/threat-model.md, core/docs/md/guides/hosting-provider/backups.md |
| Panel database | panel/src/panel/models.py, panel/nix/component/default.nix, panel/nix/configuration.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| PostgreSQL server | core/setups/wiring/postgresql-roster.nix, core/components/hosting-provider/postgresql/default.nix, core/docs/md/guides/hosting-provider/backups.md |
| SPIRE | core/components/hosting-provider/spire/spire.nix, core/docs/md/guides/hosting-provider/backups.md |
| Telemetry (Loki, Tempo, Mimir) | core/components/hosting-provider/telemetry.nix, core/components/hosting-provider/telemetry/object-store.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/logs.md, core/docs/md/guides/hosting-provider/monitoring.md, core/docs/md/guides/hosting-provider/backups.md |
| OpenTofu state backend | core/components/hosting-provider/terraform-backend/state-dir.nix, core/components/hosting-provider/terraform-backend/backup.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md, core/docs/md/guides/hosting-provider/domain-transfers.md |
| Valkey namespaces | core/components/shared/valkey/valkey.nix, core/components/shared/valkey/default.nix, core/components/sharedOptions.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |
| Windmill | core/components/hosting-provider/windmill/windmill.nix, core/components/hosting-provider/windmill/default.nix, core/docs/md/compliance/data-protection.md, core/docs/md/guides/hosting-provider/backups.md |