Backup and Restore¶
Backups are your responsibility. Sovereign Home Cloud does not automatically back up your data and in Early Access every component runs as a single replica. If a volume or the cluster is lost and you have no backup, your organizations, users, services, and their configurations are gone. Set up backups before you rely on the installation, and test a restore.
What to Back Up¶
| What | Holds | Back up | What happens if lost without a backup |
|---|---|---|---|
| Encryption key and secrets | VAULT_LOCAL_KEY and the other values in the release's Secret |
Once, then on every change | Every value encrypted under the key is unreadable, even with a database backup |
| PostgreSQL | The organizations, users, services, datacenters, Event Portal data: almost all of Sovereign Home Cloud's state | Daily, and before every upgrade | Everything is lost |
| Values files | Your installation's configuration | In version control | You cannot reinstall the same way |
| OpenSearch | The audit log (its only copy), and search indexes | Daily | The audit history is lost; search results are empty until the data is written again |
| Event broker services | Each event broker service's messages and configuration, on its own volumes | Per your messaging requirements | The event broker service's queued messages and configuration |
| Object store | Short-lived data passed between services | Optional | Little; in-flight transfers fail and are retried |
| Cache (Valkey) | Sessions and cached data | Not needed | Users sign in again; restart the control plane to reload cached data |
Warning Keep the key with the database. Store them together, outside the cluster, with the same access control as the database backup. A database backup is only usable with the VAULT_LOCAL_KEY it was written under.
The Encryption Key and Secrets¶
Copy the release's Secret, which holds VAULT_LOCAL_KEY, the generated passwords, the session signing key and the API token signing key, and keep it outside the cluster:
kubectl get secret home-cloud-control-plane-secrets -n home-cloud-control-plane -o yaml > home-cloud-control-plane-secrets.yaml
The file contains secrets in Base64-encoded clear text. Store it in your secrets manager or encrypted. Do not store it in plain version control. If you manage the Secret yourself (secrets.create: false), your secrets manager already holds these values.
Also keep the Secrets your values reference: the image pull secret, the ingress TLS Secret, and any CA or certificate Secrets.
PostgreSQL¶
Your own PostgreSQL¶
Use the backups services provided by your database, preferably with point-in-time recovery. Sovereign Home Cloud uses these databases on the server: maas_conductor, maas_core, maas_event_mesh, maas_gateway, maas_billing_v2, maas_monitoring, maas_apim_dev_portal and maas_ep_core.
Bundled PostgreSQL¶
Dump each database from the PostgreSQL pod to your workstation, then move the files to your backup storage:
NS=home-cloud-control-plane
PG=home-cloud-control-plane-postgresql-0
for db in $(kubectl exec -n $NS $PG -- psql -U postgres -Atc \
"select datname from pg_database where datname like 'maas_%'"); do
kubectl exec -n $NS $PG -- pg_dump -U postgres -Fc "$db" > "$db.dump"
done
Each dump is consistent on its own. For a backup with no changes between databases, scale the control plane to zero first and back up again afterwards; otherwise take the dumps at a quiet time.
To run this on a schedule, put the same commands in a Kubernetes CronJob in the namespace, or take snapshots of the data-home-cloud-control-plane-postgresql-0 volume if your storage supports volume snapshots.
OpenSearch¶
- Your own OpenSearch: use its snapshot feature, or your provider's automated snapshots.
- Bundled OpenSearch: take a volume snapshot of
data-home-cloud-control-plane-opensearch-0, if your storage supports volume snapshots.
The audit log exists only in OpenSearch. If you must keep audit history, back it up.
Event Broker Services¶
Each event broker service keeps its messages and configuration on its own volumes in the datacenter's namespace. Their names start with data- and spool-, followed by the event broker service's name. Protect them with volume snapshots, or use your storage's own replication. The control plane's in-cluster event broker has its own volume too, data-home-cloud-control-plane-broker-0.
Restore¶
Restore into a fresh installation of the same Sovereign Home Cloud version the backup was taken from.
- Prepare the values and Secrets. Create the namespace and the Secrets your values reference: the pull secret, the TLS Secret and any CA or certificate Secrets. Check that your values set
secrets.values.VAULT_LOCAL_KEYto the original key, from your saved copy of the release's Secret. Your values file already contains it if you installed with the key in it. If you manage the Secret yourself, restore it from your secrets manager instead. - Install with your saved values files, as you originally did.
-
Stop the control plane, so nothing writes while you restore:
kubectl scale deploy/home-cloud-control-plane-app -n home-cloud-control-plane --replicas=0 -
Restore each database. For bundled PostgreSQL:
for f in maas_*.dump; do db=${f%.dump} kubectl exec -i -n home-cloud-control-plane home-cloud-control-plane-postgresql-0 -- \ pg_restore -U postgres --clean --if-exists -d "$db" < "$f" doneFor your own PostgreSQL, restore with your provider's procedure.
-
Restore OpenSearch from its snapshot, if you have one.
-
Start the control plane:
kubectl scale deploy/home-cloud-control-plane-app -n home-cloud-control-plane --replicas=1 -
Verify: sign in to the Cloud Console and the Ops UI, and check that your services and datacenters are listed.
Event broker services are separate workloads. Restoring the control plane does not restore an event broker service's messages; restore event broker service volumes from their own snapshots.
Tip Test your restore. A backup you have never restored is not yet a backup. Restore into a separate namespace or cluster after you set up backups, and again after major upgrades.
Next Steps¶
- Upgrading: back up before every upgrade.
- Monitoring Your Deployment