Cluster Configuration

Your Anvil cluster can be configured by specifying appropriate options in the Cluster Custom Resource spec. Here is a sample Cluster resource:

apiVersion: anvil.works/v1
kind: Cluster
metadata:
  namespace: anvil
  name: maple
spec:
  versionTag: 2025.01.01
  anvilOrigin: https://anvil.mycorp.com
  appOrigin: https://{{id-or-alias}}.apps.anvil.mycorp.com
  platformServerCount: 3
  loadBalancer:
    internalTls:
      certificateSecretName: anvil-certs

Upgrading Anvil

Update your Anvil cluster by replacing the spec.versionTag property in your Cluster Resource.

Rolling Updates

If a field marked with No Downtime is updated, the Anvil Operator will take care of bringing up servers with the new configuration and draining the old ones, avoiding downtime where possible.

This feature is enabled with the updatePolicy="rolling" setting, and requires a multi-node cluster licence.

Configuration Reference

versionTag

The Anvil Enterprise version to install into the cluster.

anvilOrigin

The base URL used by the Anvil platform server.

appOrigin

The base URL used for individual apps. {{id-or-alias}} will be replaced automatically for each app.

If not provided, apps will be served from <anvilOrigin>/apps/{{id-or-alias}}. This is not recommended for production clusters, as it is more secure to serve unrelated apps from a different origin.

Instead, set appOrigin to ensure each app has a different origin. For example:

anvilOrigin: "https://anvil.example.com"
appOrigin: "https://{{id-or-alias}}.apps.anvil.example.com"

licenceKeySecretName

The name of an Opaque Secret containing the Anvil licence key.

For example, create the following Secret and use it with licenceKeySecretName: anvil-licence-key:

kubectl create -n anvil secret generic anvil-licence-key --from-literal="value=<LICENCE_KEY>"

imagePullSecretName

The name of a kubernetes.io/dockerconfigjson Secret to be used when pulling images from the Anvil container registry.

The default name is overridden if the imagePullSecret Helm chart value is set.

imagePrefix

If set, this overrides the default container image prefix for imagePrefixPublic and imagePrefixPrivate.

imagePrefixPublic

The prefix to use for public Anvil container images. If not specified, imagePrefix is used. If that isn’t set either, "anvil.works/public/" is used.

imagePrefixPrivate

The prefix to use for private Anvil container images. If not specified, imagePrefix is used. If that isn’t set either, "anvil.works/on-site/" is used.

busyboxImage

The Busybox image to use for init containers.

haproxyImage

The HAProxy image to use for load balancers.

sshServerImage

The OpenSSH server image to use for SSH servers.

postgresDbVersion

The PostgreSQL version to use for the standard databases.

timescaleDbVersion

The PostgreSQL version to use for the timescale databases.

platformDbImageOverride

If set, this overrides the platform database container image.

splitDb

If set, the app data tables are stored in a separate database (data-tables-db) instead of the main platform database (platform-db).

platformServerCount

The number of platform servers in the cluster’s main pool.

updatePolicy

The policy to use for updating the Anvil platform servers:

• "rolling": When the configuration changes, create new platform servers with the updated configuration, wait for them to become ready, then drain and delete the old ones.
• "immediate": When the configuration changes, immediately shut down the existing platform servers, and create new ones with the updated configuration.

updateApproval

If set, only platform servers with the specified version tag will be created. This can either be a single version tag, or a dictionary of Deployment Pool IDs to version tags. Any platform servers that already exist are not affected.

Platform servers in the main pool are unaffected by this setting.

extraCertsSecret

The name of an Opaque Secret containing additional certificate files to add to the platform server’s keystore.

For example, you could create the following Secret and use it with extraCertsSecret: my-extra-certs:

kubectl create -n anvil secret generic my-extra-certs --from-file="<PATH_TO_CERTIFICATE_FILE>"

Platform Servers

platformServers.jvmOptions[]

A list of JVM options to use for the platform servers.

platformServers.config

Override the default configuration of the Anvil platform server by setting the fields below inside a platformServers.config block. For example, to disable email verification:

platformServers:
  config:
    noVerifyEmail: true

Some fields below have comments indicating that they can be provided by a Kubernetes Secret. These can be configured as follows:

platformServers:
  config:
    anvilGoogleClientSecret:
      valueFrom:
        secretKeyRef:
        name: anvil-google-client-secret
        key: value

platformServers.crashDumps

platformServers.crashDumps.enabled

If set to true, PVCs will be created for the crash dumps from each of the platform servers.

platformServers.crashDumps.pvc

platformServers.crashDumps.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

platformServers.crashDumps.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

Downlinks

downlinks.mode

Either "podman" to provide container-based Python server environments, or "disabled" to disable this functionality.

downlinks.versionTag

The Dynamic Downlink host version tag to use. If not set, the Anvil cluster versionTag will be used instead.

Dynamic Downlink hosts require downtime when upgraded. You can avoid this when upgrading the main Anvil version by setting downlinks.versionTag to a specific version, and only updating it when a downlink host upgrade is necessary.

downlinks.includeBaseImages

Include all base images in the downlink registry. If not set, base images will be downloaded when the cluster boots.

downlinks.baseImages[]

A list of Python base images to make available for use. If not set, the default list of base images will be used.

For example, to include only the Minimal 3.10 base image:

downlinks:
  baseImages:
    - name: python310-minimal
      title: "Minimal 3.10"
      repo: anvil.works/public/anvil-downlink-base-python310-minimal

downlinks.baseImages[].name

A unique identifier for the base image.

downlinks.baseImages[].repo

The repository URL for the base image container.

downlinks.baseImages[].title

Human-readable title of the base image, shown in the UI. If not set, the name field will be used.

downlinks.registryCerts[]

Custom CA (or even client) certificates for connecting to registries from the downlink host. This is useful when pulling custom base images from private registries in podman mode.

See the containers-certs.d documentation for more details.

downlinks.registryCerts[].registry

The "server:port" of the registry.

downlinks.registryCerts[].secret

The name of an Opaque Secret containing one or more .crt, .cert or .key files.

downlinks.extraVolumes[]

Provide an additional volume to mount in the downlink server container.

downlinks.extraVolumes[].volume

The Kubernetes Volume definition to be added to the downlink server. It must not include the name field, as this will be set automatically.

For example:

downlinks:
  extraVolumes:
    - volume:
        persistentVolumeClaim:
          claimName: my-pvc

downlinks.extraVolumes[].mode

The mount access mode for the volume: ro for read-only, or rw for read-write.

Legacy Downlinks

Provide the legacy ‘Full Python 3` server environment.

legacyDownlinks.count

Enable the legacy (“Full Python 3”) server environments by setting this to greater than zero

legacyDownlinks.image

The container image to use for the Full Python 3 downlink server. The default is the ‘-minimal’ image; for the full (multi-gigabyte) set of packages use the ‘-full’ image.

legacyDownlinks.timeout

The timeout in seconds for the Full Python 3 downlink server. Any server calls that take longer than this will be stopped.

Load Balancer

Configuration for the load balancer used as an entry point to the cluster.

By default, the load balancer is enabled and configured as internalTls. You can override this with one of the following options:

• disabled: Disable the load balancer
• internalTls: Use internal TLS termination (default)
• externalTls: Use external TLS termination
• awsNlb: Use an AWS network load balancer
• insecure: No TLS

loadBalancer.disabled

Disable the load balancer.

loadBalancer.externalDns

Set the domain name for the load balancer service using the external-dns.alpha.kubernetes.io/hostname annotation.

loadBalancer.image

The container image to use for the load balancer. If not set, the cluster’s haproxyImage setting is used.

loadBalancer.ip

The static IP address for the load balancer using the loadBalancerIP service field.

This field is deprecated (see the note in the Kubernetes LoadBalancer docs). Use a provider-specific annotation instead.

loadBalancer.smtpPort

The external port to use for incoming SMTP traffic.

loadBalancer.sshPort

The external port to use for incoming SSH traffic.

loadBalancer.internalTls

Configure the load balancer to use internal TLS termination. This is the default mode of operation.

loadBalancer.internalTls.loadBalancerClass

The value for the loadBalancerClass service field.

loadBalancer.internalTls.annotations

Additional annotations for the load balancer service.

loadBalancer.internalTls.certificateSecretName

The name of a kubernetes.io/tls Secret containing a TLS certificate and key for the cluster load balancer.

If not specified, a self-signed certificate will be generated.

loadBalancer.externalTls

Configure the load balancer to use external TLS termination.

loadBalancer.externalTls.loadBalancerClass

The value for the loadBalancerClass service field.

loadBalancer.externalTls.annotations

Additional annotations for the load balancer service.

loadBalancer.awsNlb

Use an AWS network load balancer. TLS can be terminated either:

• In the AWS NLB by setting certificate_arn.
• In the cluster load balancer by setting certificate_secret_name.
• In the cluster load balancer with a self-signed certificate by not setting either certificate field.

loadBalancer.awsNlb.annotations

Additional annotations for the load balancer service.

loadBalancer.awsNlb.certificateArn

The ARN of a certificate in AWS to use for TLS. If set, this creates a service.beta.kubernetes.io/aws-load-balancer-ssl-cert annotation, and the AWS NLB will terminate TLS.

loadBalancer.awsNlb.certificateSecretName

The name of a kubernetes.io/tls Secret containing a TLS certificate and key for the cluster load balancer.

If neither this nor certificateArn are specified, a self-signed TLS certificate will be generated.

loadBalancer.insecure

Configure the load balancer for http only (no TLS).

loadBalancer.insecure.loadBalancerClass

The value for the loadBalancerClass service field.

loadBalancer.insecure.annotations

Additional annotations for the load balancer service.

Backup

If backups are enabled, the operator will periodically back up all database and source code to the configured backup target.

By default, backups are enabled and will be written to a PVC in the cluster.

backup.disabled

Disable all backup functionality.

backup.encryptionKeySecretName

The name of an Opaque Secret containing an encryption key for the backups. If the secret does not already exist, it will be created with a random key.

backup.maxDelta

The number of delta backups to perform in between full backups. This is the WALG_DELTA_MAX_STEPS setting for WAL-G.

backup.frequency

How often to backup the databases and source code, in pytimeparse2 syntax. The database WAL is continuously archived between backups using PostgreSQL Continuous Archiving.

backup.target

Configure where the backups will be written. This is one of:

• s3: Write to an S3 bucket. Currently this is only possible when running in an AWS EKS cluster.
• ssh: Write to a remote server via SSH.
• nfs: Write to an NFS volume in the cluster.
• pvc: Write to a PVC in the cluster (default).

backup.target.s3

backup.target.s3.bucketName

The name of the S3 bucket to write backups to.

backup.target.s3.pathPrefix

The directory in the S3 bucket used to store the backup files.

backup.target.ssh

backup.target.ssh.hostname

The hostname of the SSH server to write backups to.

backup.target.ssh.port

The port of the SSH server.

backup.target.ssh.directory

The directory on the remote server used to store the backup files.

backup.target.ssh.username

The username to use for the SSH connection.

backup.target.ssh.authSecretName

The name of a kubernetes.io/ssh-auth Secret containing an SSH private key to use for the backup.

backup.target.ssh.knownHostsSecretName

The name of an Opaque Secret containing a known hosts file entry for the SSH server.

For example, you could create the following Secret and use it with knownHostsSecretName: my-known-hosts:

kubectl create -n anvil secret generic my-known-hosts --from-file="known_hosts"

backup.target.nfs

backup.target.nfs.server

The hostname of the NFS server.

backup.target.nfs.export

The exported directory on the NFS server.

backup.target.nfs.directory

The subdirectory inside the NFS export used to store the backup files.

backup.target.pvc

backup.target.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

backup.target.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

backup.target.pvc.storageClass

The storage class to use if creating a new PVC. If not set, the cluster’s storage.defaultRetainedStorageClass will be used.

This field is ignored if the PVC already exists.

backup.target.pvc.directory

The directory in the PVC used to store the backup files.

Database Restore Points

dbRestorePoints.enabled

Enable the ability to perform point-in-time restores of the app databases.

PDF Renderer

The PDF renderer service converts HTML to PDF documents. This is required for the anvil.pdf.render_form() server function.

pdfRenderer.enabled

Enable the PDF renderer.

pdfRenderer.image

The image to use for the PDF renderer.

pdfRenderer.disableDevShm

Disable the use of /dev/shm for the PDF renderer. This reduces memory usage at the expense of performance.

pdfRenderer.certificateSecretName

The name of an Opaque Secret containing a TLS certificate to add to the PDF Renderer’s certificate store. This can be used to render PDFs from a server that uses a self-signed certificate.

For example, you could create the following Secret and use it with certificateSecretName: my-certificate:

kubectl create -n anvil secret generic my-certificate --from-file="tls.crt"

pdfRenderer.disableCertificateCheck

Disable the certificate check for the PDF renderer.

Metrics

Configure metrics collection and monitoring services.

metrics.enabled

Enable metrics services.

metrics.domainName

The domain name for the metrics services. If set, the metrics services will be exposed externally on this domain name.

metrics.grafana

metrics.grafana.image

The container image to use for the Grafana service.

metrics.grafana.adminPasswordSecretName

The name of an Opaque Secret containing the Grafana admin password. If the secret doesn’t exist, it will be created with a random password.

For example, you could create the following Secret and use it with adminPasswordSecretName: my-grafana-admin-password:

kubectl create -n anvil secret generic my-grafana-admin-password --from-literal="value=<ADMIN_PASSWORD>"

metrics.grafana.config

Additional environment variables for the Grafana service.

metrics.grafana.pvc

metrics.grafana.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

metrics.grafana.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

metrics.grafana.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

metrics.prometheus

metrics.prometheus.image

The container image to use for the Prometheus service.

metrics.prometheus.scrapeNodes

If set, a Prometheus Node Exporter will be created to scrape node metrics from the cluster.

This requires the operator to have permission to manage resources at the cluster scope, which is granted when the enableNodeMetrics Helm chart value is set.

The default value is set by the enableNodeMetrics Helm chart value, which is true by default.

metrics.prometheus.pvc

metrics.prometheus.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

metrics.prometheus.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

metrics.prometheus.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

metrics.loki

metrics.loki.image

The container image to use for the Loki service.

metrics.loki.retentionHours

The length of time to retain logs in the Loki service.

metrics.loki.pvc

metrics.loki.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

metrics.loki.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

metrics.loki.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

metrics.alloy

metrics.alloy.image

The container image to use for the Alloy service.

Tempo

Configure the Tempo tracing service.

tempo.enabled

Enable the Tempo service.

tempo.image

The container image to use for the Tempo service.

tempo.config

Additional environment variables for the Tempo service.

tempo.pvc

tempo.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

tempo.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

tempo.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

Pods

Configure the Kubernetes pods used in the cluster.

Each pod definition has a resources and affinity field. If neither of these are specified for a pod, the top-level pods.resources and pods.affinity will be used.

For example, the following specifies a 2GiB memory request for the containers in the downlinkRunner pod, but uses a 1GiB request for the containers in all other pods:

pods:
  resources:
    requests:
      memory: 1Gi
  downlinkRunner:
    resources:
      requests:
        memory: 2Gi

pods.resources

Configure the resources requests and limits for the containers in the pod. See the Kubernetes Resource Management documentation for more information.

pods.resources.requests

pods.resources.limits

pods.affinity

pods.affinity.node

Configure the node affinity for the pod by specifying a dictionary of keys and values to be used as requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms.

For example, the following specifies that the pod must be scheduled on anvil-node-0:

pods:
  affinity:
    node:
      kubernetes.io/hostname: anvil-node-0

See the Kubernetes Node Affinity documentation for more information.

pods.affinity.pod

Configure the pod affinity for the pod by specifying a dictionary of keys and values to be used as requiredDuringSchedulingIgnoredDuringExecution.labelSelectors.

For example, the following specifies that the pod must be scheduled on the same node as any pods with the label foo=bar:

pods:
  affinity:
    pod:
      foo: bar

See the Kubernetes Pod Affinity documentation for more information.

pods.platformServers

Pod configuration (resources and affinity) for the platform servers.

pods.platformServers.gitServer

Pod configuration (resources and affinity) for the git server. If not specified, the parent pods.platformServers configuration will be used.

pods.databases

Pod configuration (resources and affinity) for the databases.

pods.databases.platform

Pod configuration (resources and affinity) for the platform database. If not specified, the parent pods.databases configuration will be used.

pods.databases.dataTables

Pod configuration (resources and affinity) for the data tables database. If not specified, the parent pods.databases configuration will be used.

pods.databases.appLogs

Pod configuration (resources and affinity) for the app logs database. If not specified, the parent pods.databases configuration will be used.

pods.migrator

Pod configuration (resources and affinity) for the database migrator.

pods.loadBalancer

Pod configuration (resources and affinity) for the load balancer.

pods.pdfRenderer

Pod configuration (resources and affinity) for the PDF renderer.

pods.legacyDownlink

Pod configuration (resources and affinity) for the legacy downlink.

pods.downlinkRunner

Pod configuration (resources and affinity) for the downlink runner.

pods.downlinkRegistry

Pod configuration (resources and affinity) for the downlink registry.

pods.backupServer

Pod configuration (resources and affinity) for the backup SSH server.

pods.grafana

Pod configuration (resources and affinity) for the Grafana server.

pods.prometheus

Pod configuration (resources and affinity) for the Prometheus server.

pods.tempo

Pod configuration (resources and affinity) for the Tempo server.

pods.nodeExporter

Pod configuration (resources and affinity) for the Prometheus node exporter.

pods.loki

Pod configuration (resources and affinity) for the Loki server.

pods.alloy

Pod configuration (resources and affinity) for the Grafana Alloy server.

Storage

storage.defaultStorageClass

Override the default storage class for non-retained PVCs. When these PVCs are deleted, their underlying volumes should also be deleted.

storage.defaultRetainedStorageClass

Override the default storage class for retained PVCs. When these PVCs are deleted, their underlying volumes should not be deleted.

storage.appSourceCode

storage.appSourceCode.dataPath

The path inside the PVC to use for storage.

storage.appSourceCode.pvc

storage.appSourceCode.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.appSourceCode.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.appSourceCode.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

storage.platformDb

storage.platformDb.dataPath

The path inside the PVC to use for storage.

storage.platformDb.pvc

storage.platformDb.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.platformDb.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.platformDb.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

storage.appLogsDb

storage.appLogsDb.dataPath

The path inside the PVC to use for storage.

storage.appLogsDb.pvc

storage.appLogsDb.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.appLogsDb.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.appLogsDb.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

storage.dataTablesDb

storage.dataTablesDb.dataPath

The path inside the PVC to use for storage.

storage.dataTablesDb.pvc

storage.dataTablesDb.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.dataTablesDb.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.dataTablesDb.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

storage.downlinkContainerData

Storage configuration for the downlink runners.

storage.downlinkContainerData.pvc

storage.downlinkContainerData.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.downlinkContainerData.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.downlinkContainerData.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

storage.downlinkRegistry

Storage configuration for the downlink registry.

storage.downlinkRegistry.pvc

storage.downlinkRegistry.pvc.existingClaimName

The name of an existing PVC to use. If the PVC doesn’t exist, a new one will be created.

storage.downlinkRegistry.pvc.size

The size of the PVC to create, specified as a Kubernetes Quantity.

This field is ignored if the PVC already exists.

storage.downlinkRegistry.pvc.storageClass

The storage class to use for the PVC. If not set, the cluster’s storage.defaultStorageClass will be used.

This field is ignored if the PVC already exists.

Secret Names

Configure the names of the secrets used by the cluster. These secrets are created automatically if they don’t already exist.

secretNames.platformServerKeys

secretNames.platformDbCreds

secretNames.appLogsDbCreds

secretNames.dataTablesDbCreds

Do you still have questions?

Our Community Forum is full of helpful information and Anvil experts.