Skip to content

Installing for Production

A production installation connects to an event broker you run, uses your own hostnames and TLS certificates, keeps its secrets under your control, and stores its data in data stores you already operate and back up. Read Before You Begin first.

Prerequisites

What Details
Kubernetes cluster As in Before You Begin. It must already run an ingress controller: the chart creates Ingress resources, and your controller serves them.
Chart and image access The chart reference, registry and pull credentials (Before You Begin)
Event broker An event broker for the control plane's own messaging, with a message VPN, a client username and a SEMP administrator, serving TLS with a valid certificate for its hostname. The event broker is required before you install; to run one in Kubernetes, use the Solace Event Broker Operator Quick Start. Or use the chart's embedded broker, at your own risk.
CA certificate and key A CA, with its certificate and its private key, that signs your event broker services' certificates and that the control plane trusts. Every datacenter uses the same CA; see The CA Certificate.
Hostnames One each for the Cloud Console and the Ops UI, and optionally one for the API, with DNS records pointing at your ingress controller's address
TLS certificate A certificate covering those hostnames: in a Kubernetes TLS Secret, or, on Amazon EKS with the AWS Load Balancer Controller, in AWS Certificate Manager
Data stores Recommended: your own PostgreSQL, OpenSearch, Redis-compatible cache, and S3-compatible object store, meeting the requirements in Before You Begin. OpenSearch and the cache take only a hostname and port: they are used without credentials.

Write Your Values File

Create one values file, for example values.local.yaml, from the blocks on this page. It holds secrets, so keep it out of version control or encrypt it.

global:
  imagePullSecrets:
    - <pull secret name>
  persistence:
    storageClassName: ""       # blank: the cluster's default StorageClass
    namespaceId: <prefix>      # always the same value as maasPrefix

maasPrefix: <prefix>           # names your stored data; fixed at installation

organization:
  operator:
    username: <Ops UI administrator email>
    password: <Ops UI administrator password>

secrets:
  values:
    VAULT_LOCAL_KEY: <64 hexadecimal characters>   # openssl rand -hex 32

Warning! Change the default passwords. The Ops User Interface administrator defaults to admin@company.com with the password admin. Set your own before the installation is reachable by anyone else.

Add the blocks below for the event broker, the CA, the ingress, and your data stores.

Event broker for the control plane

Use your own event broker, or one the chart deploys for you.

Give the connection details of the event broker you run. broker.enabled stays false, its default.

ivmr:
  smfHost: tcps://broker.example.com:55443
  mgmtHost: https://broker.example.com:943
  msgVpn: <message VPN>
  username: <client username>
  mgmtUsername: <SEMP admin username>
secrets:
  values:
    EXTERNAL_IVMR_PASSWORD: <client password>
    EXTERNAL_IVMR_MGMT_PASSWORD: <SEMP admin password>

Every datacenter connects to this event broker at the smfHost address, so it must be reachable from the control plane and from every datacenter.

The event broker must present a valid TLS certificate: unexpired, with the smfHost and mgmtHost hostname in its subject alternative names, and signed by a CA the control plane trusts. Sign it with the CA from The CA Certificate, or add the CA that signed it under Other private CAs. Every other client of the event broker, such as the Event Management Agents, must trust that CA too.

The chart deploys a single-node event broker in the release's namespace and sets up the control plane's connection to it. Leave every ivmr.* value unset: the chart refuses them alongside the embedded event broker. It generates the event broker's passwords and keeps them across upgrades.

broker:
  enabled: true
  networkPolicy:
    extraIngress:              # admits the datacenter namespaces you label
      - from:
          - namespaceSelector:
              matchLabels:
                solace.com/home-cloud-mca: "true"
        ports:
          - { protocol: TCP, port: 55555 }

Warning! Use at your own risk. The embedded event broker has no redundancy, so it is down while its pod restarts, and monitoring it is up to you. It speaks plain text, without TLS, and only clients inside the cluster can reach it.

Only datacenters in the same cluster can use it. When you onboard each datacenter, label its namespace so the event broker's NetworkPolicy admits it, and give its agent the event broker's full in-cluster address; see step 5.

This is an installation-time choice. Moving the control plane to another event broker later, including from the embedded event broker to your own, is a reinstall, not an upgrade.

No datacenter or organization at installation

A production installation creates no datacenter and no Cloud Console organization: the chart's datacenter is off by default (mca.enabled: false). After you install, the Ops User Interface (UI) administrator creates the organizations and their users (Administering with the Ops UI), then onboards each datacenter (Onboarding a Datacenter).

Scheduling

global:
  nodeSelector: {}                       # applies to every pod the release schedules
  tolerations: []

The CA Certificate

One CA serves the whole installation:

  • each datacenter's agent signs every event broker service's certificate with it, so it needs the CA's certificate and private key;
  • the control plane trusts it, so it can reach those event broker services over TLS. It needs only the certificate.

Use an intermediate CA dedicated to event broker services, not your root: anyone who can read its Secret can issue certificates your clients trust. It must use an RSA key, be marked CA:TRUE, and have at least seven days of validity left.

  1. Get the CA's certificate as ca.crt and its private key as ca.key, in PKCS#8 format.

    To use an existing CA, export its certificate, followed by its chain, and its private key.

    To create a new CA instead:

    openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes \
      -subj "/CN=Broker Issuing CA" \
      -addext "basicConstraints=critical,CA:TRUE" \
      -addext "keyUsage=critical,keyCertSign,cRLSign" \
      -keyout ca.key -out ca.crt
    

    This writes the key in PKCS#8 format and a certificate valid for 10 years (-days 3650). Event broker service certificates never outlive the CA, so plan to replace it before then.

  2. Encrypt the key. This is optional and recommended: anyone who has ca.key can issue certificates your clients trust.

    openssl pkcs8 -topk8 -v2 aes-256-cbc -in ca.key -out ca.enc.key -passout pass:"$PASSPHRASE"
    
  3. Create the Secret in the release's namespace before you install:

    kubectl create secret generic broker-issuing-ca -n home-cloud-control-plane \
      --from-file=ca.crt=ca.crt \
      --from-file=ca.key=ca.enc.key \
      --from-file=passphrase=<(printf '%s' "$PASSPHRASE")
    

    For an unencrypted key, use --from-file=ca.key=ca.key and leave out passphrase.

  4. Have the control plane trust it. keys mounts only the certificate, never the key:

    composite:
      trustedCA:
        existingSecret: broker-issuing-ca
        keys: [ca.crt]
    

Each datacenter is then pointed at the same CA when you onboard it; see Broker Certificates.

Your Own Data Stores

Turn off the bundled component and give the endpoint. The chart refuses to install with a conflicting or malformed endpoint, rather than silently using the wrong store.

PostgreSQL

persistence-layer:
  postgresql:
    enabled: false
dataLayer:
  external:
    postgresHost: pg.example.internal
    postgresPort: 5432
    postgresUsername: hosted_fleet
secrets:
  values:
    HOSTED_FLEET_POSTGRES_PASSWORD: <password>

When the account has CREATEDB, the control plane creates its databases on first start. Otherwise, create them beforehand and set dataLayer.postgresql.databases: [].

OpenSearch

dataLayer:
  opensearch:
    enabled: false
  external:
    opensearchHost: search.example.internal
    opensearchPort: 443
    opensearchProtocol: https

The control plane connects without credentials, so restrict access to the endpoint by network.

Cache

dataLayer:
  redis:
    enabled: false
  external:
    redisHost: cache.example.internal
    redisPort: 6379

The control plane connects without a username or password, so restrict access by network. The one exception is Amazon ElastiCache with IAM authentication; add the IAM block, and TLS is then turned on automatically:

dataLayer:
  external:
    redisIam:
      enabled: true
      userId: <IAM user ID, identical to the user name>
      replicationGroupId: <replication group>
      region: <region>

The control plane's service account then needs elasticache:Connect on the replication group and the user; see Cloud identity.

Object store

objectStorage:
  enabled: false
  bucket: <bucket>
  region: <region>
  external:
    endpoint: ""            # blank for Amazon S3; the URL of any other S3-compatible store
    forcePathStyle: false   # true for most S3-compatible stores other than Amazon S3
persistence-layer:
  seaweedfs:
    enabled: false

Create the bucket first. Give the control plane access through cloud identity, or set HOSTED_FLEET_S3_ACCESS_KEY and HOSTED_FLEET_S3_SECRET_KEY in the Secret.

Cloud identity

On Amazon EKS, give the control plane an IAM role instead of static keys, through IRSA or EKS Pod Identity:

serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account>:role/<role>

The role needs the S3 permissions in Before You Begin, and elasticache:Connect if you use ElastiCache with IAM.

Other private CAs

If your data stores or your event broker present certificates from another private CA, add it next to the event broker service CA. Every entry under certificates is trusted in addition to existingSecret:

composite:
  trustedCA:
    existingSecret: broker-issuing-ca
    keys: [ca.crt]
    certificates:
      corporate-root.pem: |
        -----BEGIN CERTIFICATE-----
        ...
        -----END CERTIFICATE-----

Without it, those connections fail with PKIX path building failed. A change restarts the control plane. If you update the Secret without running helm upgrade, restart it yourself:

kubectl rollout restart deploy/home-cloud-control-plane-app -n home-cloud-control-plane

Provide Secrets

By default the chart creates one Secret, <release>-secrets, from secrets.values. It generates and keeps the values you leave blank, such as the session signing key. Three values matter beyond the installation:

  • VAULT_LOCAL_KEY encrypts sensitive values in the database. Keep a copy outside the cluster. Losing it loses every value encrypted under it, even if the database survives; see Backup and Restore.
  • JWT_SECRET signs user sessions. Changing it signs every user out.
  • JWT_PRIVATE_KEY signs API tokens. Changing it invalidates every API token; see Using the REST API. To use your own key, supply an RSA private key in PEM format, at least 2048 bits, for example from openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048.

To manage the Secret yourself, for example from an external secrets operator, set:

secrets:
  create: false
  existingSecret: <secret name>

The Secret must then contain every one of these keys:

Key What it is
VAULT_LOCAL_KEY The encryption key: 64 hexadecimal characters
JWT_SECRET Session signing key
JWT_PRIVATE_KEY API token signing key: an RSA private key in PEM format
MAAS_OPERATOR_PASSWORD Ops UI administrator password
HOSTED_FLEET_CONTROL_TOKEN Internal control token
EXTERNAL_IVMR_PASSWORD, EXTERNAL_IVMR_MGMT_PASSWORD The event broker's client and SEMP admin passwords
HOSTED_FLEET_POSTGRES_PASSWORD PostgreSQL password
HOSTED_FLEET_S3_ACCESS_KEY, HOSTED_FLEET_S3_SECRET_KEY Only with static object store keys

Warning! A missing key is not caught when you install. It shows up later as a failed sign-in, or for JWT_PRIVATE_KEY as an API gateway that fails at startup, so check the Secret first if the installation fails.

Hostnames, Ingress and TLS

The chart creates Ingress resources for your hostnames, and the ingress controller already running in your cluster serves them. Point each hostname's DNS record at that controller's address, such as its load balancer.

The Cloud Console and the Ops UI each need their own hostname: both are served from /, so they cannot share one. API requests on either hostname under /api go to the API. The optional API hostname sends every path to the API, for tools that address it directly.

TLS is mandatory once ingress is on. The chart refuses to install an ingress without TLS, because the UIs would load and then fail.

ingress:
  enabled: true
  className: nginx                 # blank: the cluster's default IngressClass
  consoleHost: console.example.com
  opsHost: ops.example.com
  apiHost: api.example.com         # optional
  tls:
    - secretName: home-cloud-control-plane-tls
      hosts: [console.example.com, ops.example.com, api.example.com]
kubectl create secret tls home-cloud-control-plane-tls -n home-cloud-control-plane \
  --cert=<chain.pem> --key=<key.pem>

The certificate stays in AWS Certificate Manager, and the load balancer terminates TLS:

ingress:
  enabled: true
  className: alb
  consoleHost: console.example.com
  opsHost: ops.example.com
  annotations:
    alb.ingress.kubernetes.io/scheme: internal         # or internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: <ACM certificate ARN>
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/ssl-redirect: "443"
  tls: []

Every hostname must be covered by the certificate. On OpenShift, secretName is required: without it, no Route is created and the hostname leads nowhere.

Certificate renewal is your responsibility. Replace the TLS Secret, or the load balancer's certificate, before it expires. A tool such as cert-manager can do this for you.

Deploy

  1. Create the namespace, the pull secret and the Secrets your values reference: the CA Secret and, if you use one, the TLS Secret.

    kubectl create namespace home-cloud-control-plane
    kubectl create secret docker-registry <pull secret name> -n home-cloud-control-plane \
      --docker-server=registry.solace.com --docker-username=<username> --docker-password=<password>
    
  2. Check the values against the cluster without installing anything:

    helm install home-cloud-control-plane oci://registry.solace.com/solace/home-cloud-control-plane/charts/home-cloud-control-plane --version <chart version> -n home-cloud-control-plane \
      -f values.local.yaml --dry-run=server
    
  3. Install:

    helm install home-cloud-control-plane oci://registry.solace.com/solace/home-cloud-control-plane/charts/home-cloud-control-plane --version <chart version> -n home-cloud-control-plane \
      -f values.local.yaml --wait --timeout 30m
    

    The installation waits for the control plane's first start, which can take several minutes.

  4. Watch it in another terminal:

    kubectl get pods -n home-cloud-control-plane -w
    

Verify the Deployment

  1. Every pod is Running.
  2. Open the Ops UI at its hostname over HTTPS, and sign in as the Ops UI administrator from your values file.
  3. Open the Cloud Console at its hostname. It shows its sign-in page; there is no account to sign in with until you create an organization.

Next Steps