Fediversity

Deploy

This page tells where the alerting notifications go, where the credentials of the state backend are, and how to deploy from CI and from a workstation.

Receiving alerting notifications

Deployments use the ADMIN_EMAIL environment variable to set email addresses (;-delimited) to send alerting emails, in core/setups/ from Grafana. A deployment with the telemetry node needs it: without it, the evaluation stops. An evaluation in the tree with no ADMIN_EMAIL (core/eval.nix and the CI builds of the nodes) gives the alertEmails argument of core/default.nix in its place.

The credentials of the state backend

Deployments use the TF_BACKEND_HTPASSWD environment variable to set the bcrypt hashes of the deployers at the front of the state backend: one <user>:<hash> line for each of cd, operator and, for each person who deploys from a workstation, workstation-<person>, where <person> has only the characters [a-z0-9] (htpasswd -nB <user>). The access log of the front records the user of each request, so it shows which person sent it. A deployment with the state backend node needs it: without it, the evaluation stops. An evaluation in the tree with no TF_BACKEND_HTPASSWD (core/eval.nix and the CI builds of the nodes) gives the test hashes of core/testers/tf-backend-htpasswd.nix in its place. notes/cd-credentials.md tells how to set the passwords and in which order.

Deploy from CI

The CD workflows deploy the machines. A workflow that runs on main starts on each push to main that changes the files of its machine. You can start each workflow by hand in "Actions":

Workflow Name in "Actions" What it deploys Run on main
cd-dev-dev2.yaml deploy-dev2 The development host dev2, over SSH Each push that changes its files
cd-dev-dev3.yaml deploy-dev3 The development host dev3, over SSH Each push that changes its files
cd-dev-dev4.yaml deploy-dev4 The development host dev4, over SSH Each push that changes its files
cd-dev-fediversity-ci.yaml deploy-fediversity-ci The CI host fediversity-ci, over SSH Each push that changes its files
cd-dev-forgejo.yaml deploy-forgejo The host of the forge, over SSH Each push that changes its files
cd-dns-hosting-provider.yaml publish-dns-hosting-provider The DNS zone of the hosting provider Each push that changes its files
cd-hosting-provider.yaml deploy-hosting-provider All containers of the hosting provider, in one OpenTofu graph Each push that changes its files
cd-hypervisor-incus.yaml deploy-incus The hypervisor that runs these containers, over SSH Each push that changes its files
cd-registrar-hosting-provider.yaml registrar-hosting-provider The registrar resources of the domain of the hosting provider By hand only

To run a different verb, start deploy-hosting-provider by hand in "Actions": select the verb (for example plan) and, if the verb needs them, the args. For example, plan with -destroy shows what a destroy would remove. Dispatching a verb other than apply lists the verbs.

The coordinates of the deploy are repository secrets and variables. notes/cd-credentials.md tells what each one is, and how to make it.

A CD run logs in to the OpenBao of the hosting provider with the OIDC token of the forge. OpenBao accepts only a run on main of this repository. A run on a different branch, or the run of a pull request, cannot get the deployer token, and the deploy refuses to start. A run that you start by hand on main gets the token, also when its ref input selects a different commit.

Deploy from a workstation

A deploy from a workstation uses your checkout, so it can deploy a branch that is not merged. It uses the same scripts as CI.

  1. Enter the core dev shell, which has the tools:

    nix develop ./core

  2. Copy .forgejo/workflows/cd-env.example.sh to a location outside the worktree, and fill in the values. The template groups the variables by purpose, and has the same names as the repository secrets and variables. Get the values from the person who set up the hosting provider.

  3. Load the values, and prepare the deploy:

    . <your copy of cd-env.example.sh>
    . ./.forgejo/workflows/cd-env.sh tf-incus

    cd-env.sh refuses to continue if a required value is missing. With OPENBAO_ADDR set, it opens the SSO login of the hosting provider in your browser, and gets a deployer token that is valid for 30 minutes. The browser must be on the same machine, and port 8250 must be free.

  4. Make a plan, and read it:

    nix run --impure .#hosting-provider-tf-incus-all-plan

  5. Apply:

    nix run --impure .#hosting-provider-tf-incus-all-apply

Note --impure lets the deploy read the coordinates, so a plan proposes to create all containers again.

A full plan builds the whole fleet. To run it on a build host, forward the SSO callback from your workstation:

ssh -N -o ExitOnForwardFailure=yes -L 8250:127.0.0.1:8250 <build-host>

Then do steps 1 to 5 on the build host, and open the URL that bao shows in your browser. A deploy from a workstation tells more.

Be careful:

  • A deploy from a workstation deploys your checkout to the hosting provider that the coordinates name. If they name the production hosting provider, your branch replaces what CI deployed, until the next CD run.
  • Without the OPENBAO_* values, the deploy uses a different lane than CI, and changes every node. The next CD run changes them back.
  • Without CD_OIDC_ISSUER and CD_OIDC_REPOSITORY, the deploy removes the forge login from OpenBao, and CD fails after it. cd-env.sh shows a warning.
  • The filled-in file holds the repository secrets. Do not commit it, and do not load it automatically in each shell.