SOPS¶
SOPS (Secrets OPerationS) encrypts sensitive values in YAML files so they can be safely committed to Git. The cluster uses SOPS with age as the encryption backend, ensuring that secrets like 1Password credentials, Infisical auth tokens, and Talos machine secrets are protected at rest in the repository.
How It Works¶
flowchart LR
PLAIN[Plaintext YAML\n*.sops.yaml] -->|sops --encrypt| ENC[Encrypted YAML\nValues encrypted\nKeys in cleartext]
ENC -->|git commit + push| GIT[(Git Repository)]
GIT -->|git pull| LOCAL[Local Clone]
LOCAL -->|sops --decrypt| PLAIN2[Plaintext YAML]
AGE[age private key\n~/.config/sops/age/keys.txt] -.->|Used by| ENC
AGE -.->|Used by| PLAIN2 SOPS encrypts only the values in YAML files (controlled by encrypted_regex), leaving keys and structure visible. This allows diffs to show which fields changed without revealing the actual secret data.
Configuration¶
The .sops.yaml file at the repository root defines encryption rules for different file paths:
---
creation_rules:
- path_regex: kubernetes/pitower/.*\.sops\.ya?ml
encrypted_regex: "^(data|stringData)$"
key_groups:
- age:
- "age1tkaddc3hgjx0eagjl6mqpxvzzkerd44e34rua6gzucv6emr5f5fs4mlu67"
- path_regex: pitower/talos/.*\.sops\.ya?ml
encrypted_regex: "^(crt|id|token|key|secret|stringData|secretboxencryptionsecret|bootstraptoken)$"
key_groups:
- age:
- "age1tkaddc3hgjx0eagjl6mqpxvzzkerd44e34rua6gzucv6emr5f5fs4mlu67"
- path_regex: /dev/stdin
key_groups:
- age:
- "age1tkaddc3hgjx0eagjl6mqpxvzzkerd44e34rua6gzucv6emr5f5fs4mlu67"
- path_regex: .*\.sops\.ya?ml$
key_groups:
- age:
- "age1tkaddc3hgjx0eagjl6mqpxvzzkerd44e34rua6gzucv6emr5f5fs4mlu67"
Rule Breakdown¶
| Path Pattern | encrypted_regex | Purpose |
|---|---|---|
kubernetes/pitower/.*\.sops\.ya?ml | ^(data\|stringData)$ | Kubernetes Secret manifests -- encrypts only data and stringData fields |
pitower/talos/.*\.sops\.ya?ml | ^(crt\|id\|token\|key\|secret\|...)$ | Talos machine secrets -- encrypts certificates, tokens, keys, and bootstrap data |
/dev/stdin | (all values) | Piped input for ad-hoc encryption |
.*\.sops\.ya?ml$ | (all values) | Catch-all for any other .sops.yaml files |
File naming convention
SOPS-encrypted files must use the .sops.yaml or .sops.yml extension. This convention makes it immediately clear which files contain encrypted data and ensures the correct .sops.yaml rules are matched.
age Encryption¶
The cluster uses a single age public key for encryption:
The corresponding private key must be available locally for decryption:
Protect the private key
The age private key is the master key for all encrypted secrets in the repository. It must never be committed to Git. Store it securely and ensure it is backed up.
Encrypt / Decrypt Workflow¶
Encrypting a New Secret¶
-
Create the plaintext file with the
.sops.yamlextension: -
Encrypt the file in place:
-
Verify -- the file now has encrypted values but readable keys:
apiVersion: v1 kind: Secret metadata: name: my-secret namespace: my-app stringData: api-key: ENC[AES256_GCM,data:...,type:str] password: ENC[AES256_GCM,data:...,type:str] sops: age: - recipient: age1tkaddc3hgjx0eagjl6mqpxvzzkerd44e34rua6gzucv6emr5f5fs4mlu67 enc: | -----BEGIN AGE ENCRYPTED FILE----- ... -----END AGE ENCRYPTED FILE----- lastmodified: "2025-01-01T00:00:00Z" mac: ENC[AES256_GCM,data:...,type:str] version: 3.9.0 -
Commit the encrypted file to Git.
Decrypting a Secret¶
Editing an Encrypted Secret¶
SOPS can open encrypted files in your editor, decrypting on open and re-encrypting on save:
This opens the file in $EDITOR with plaintext values. When you save and close, SOPS re-encrypts the changed values automatically.
Talos Secrets¶
Talos machine secrets (certificates, tokens, encryption keys) are stored as SOPS-encrypted files under pitower/talos/. The encrypted_regex for this path is broader than for Kubernetes secrets, covering fields specific to Talos:
crt-- TLS certificatesid-- Machine identifierstoken-- Bootstrap and join tokenskey-- Private keyssecret-- General secretssecretboxencryptionsecret-- Kubernetes secret encryption keybootstraptoken-- Cluster bootstrap token
# Decrypt Talos secrets for inspection
sops --decrypt pitower/talos/secrets.sops.yaml
# Edit Talos secrets
sops pitower/talos/secrets.sops.yaml
Integration with ArgoCD¶
ArgoCD does not natively decrypt SOPS files. Instead, bootstrap secrets (like 1Password Connect credentials and Infisical auth) are:
- Encrypted with SOPS and committed to Git
- Decrypted locally and applied directly with
kubectlduring cluster bootstrap - Referenced by External Secrets Operator, which then handles syncing all other secrets
This means SOPS is used primarily for bootstrap secrets -- the minimal set of credentials needed to start External Secrets Operator, which then takes over secret management for all applications.
Best Practices¶
SOPS best practices
- Always use
encrypted_regex-- encrypt only the fields that contain sensitive data, not the entire file. This makes diffs meaningful and review easier. - Use the
.sops.yamlextension -- this ensures SOPS rules are applied correctly and makes encrypted files easy to identify. - Never commit the age private key -- store it in a password manager or secure vault, separate from the repository.
- Rotate secrets periodically -- when rotating the age key, re-encrypt all SOPS files with the new key and update
.sops.yaml. - Verify before committing -- run
sops --decrypton the file to ensure it was encrypted correctly before pushing.