Skip to content
Hands-on IT IT Drills

Day 4: .sops.yaml and creation_rules

Goal: A .sops.yaml in the repo defines which file is encrypted for whom and how – nobody types --age or --encrypted-regex any more, and dev and prod have separate keys. · Duration: approx. 60 min · Prerequisite: Day 3

Briefing

  • .sops.yaml contains creation_rules: a list of rules, each with a path_regex and the keys (age, pgp, hc_vault_transit_uri, …) plus options such as encrypted_regex.
  • sops searches for the file from the current directory upwards. path_regex is matched against the path relative to the .sops.yaml. The first matching rule wins.
  • The rules apply when a file is created (and on updatekeys, Day 5). An already encrypted file carries its recipients itself in the block sops:.
  • Whoever decrypts needs no .sops.yaml – only a matching identity.
 ~/sops/repo/.sops.yaml
   ^prod/.*\.yaml$   → age1r025… (prod pipeline key)   only data/stringData
   ^dev/.*\.yaml$    → age126j… (me)                   only data/stringData
   ^notes/.*         → age126j… (me)

Drill 1: A second key and the first rules file

prod gets its own key – later only the cluster or the pipeline has it.

# on argo
mkdir -p ~/sops/tag04 && cd ~/sops/tag04
age-keygen -o prod.key
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
PROD=$(age-keygen -y prod.key)
cd ~/sops/repo
cat > .sops.yaml <<EOF
creation_rules:
  - path_regex: ^prod/.*\.yaml$
    encrypted_regex: ^(data|stringData)$
    age: $PROD
  - path_regex: ^dev/.*\.yaml$
    encrypted_regex: ^(data|stringData)$
    age: $ICH
  - path_regex: ^notes/.*
    age: $ICH
EOF
cat .sops.yaml

Expected:

Public key: age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
creation_rules:
  - path_regex: ^prod/.*\.yaml$
    encrypted_regex: ^(data|stringData)$
    age: age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
  - path_regex: ^dev/.*\.yaml$
    encrypted_regex: ^(data|stringData)$
    age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
  - path_regex: ^notes/.*
    age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2

The heredoc is deliberately unquoted (<<EOF): the shell substitutes $PROD and $ICH, \. stays as it is. You separate multiple recipients in age: with commas (Day 5).

Save your progress? Sign up for free

Drill 2: Encrypt without options

# on argo
cd ~/sops/repo
mkdir -p dev prod
for e in dev prod; do
  kubectl create secret generic notes-db -n notes-$e \
    --from-literal=password=lab-db-$e-123 --dry-run=client -o yaml > $e/notes-db.yaml
done
sops encrypt -i dev/notes-db.yaml
sops encrypt -i prod/notes-db.yaml
yq '.sops.age[].recipient' dev/notes-db.yaml prod/notes-db.yaml
yq '.kind, .data.password' prod/notes-db.yaml | cut -c1-40

Expected:

age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
---
age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
Secret
ENC[AES256_GCM,data:qJhs0Ct2YVuBXOKbIKY4

-i (in place) replaces the plaintext in the file with the encrypted version. Recipients and encrypted_regex come from the rule that matches the path.

Warning: -i only overwrites the file itself. Copies, editor backups and whatever was already staged with git add stay plaintext – so never stage plaintext files (Day 9).

Save your progress? Sign up for free

Drill 3: The missing key and where sops looks

Symptom: prod cannot be opened with your key.

# on argo
cd ~/sops/repo
sops decrypt prod/notes-db.yaml 2>&1 | head -9

Expected:

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks: FAILED
    - | failed to create reader for decrypting sops data key with
      | age: no identity matched any of the recipients. Did not find
      | keys in locations 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
      | 'SOPS_AGE_SSH_PRIVATE_KEY_CMD', '/home/ubuntu/.ssh/id_rsa',
      | 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', and 'SOPS_AGE_KEY_CMD'.

The message names the recipient for which a key is missing and all the places where sops looked. It found only ~/.config/sops/age/keys.txt – and that one does not match. Three ways to give sops a different key:

# on argo
cd ~/sops/repo
SOPS_AGE_KEY_FILE=~/sops/tag04/prod.key sops decrypt prod/notes-db.yaml \
  | grep password
SOPS_AGE_KEY_CMD="cat $HOME/sops/tag04/prod.key" sops decrypt prod/notes-db.yaml \
  | grep -c password
SOPS_AGE_KEY=$(grep AGE-SECRET ~/sops/tag04/prod.key) \
  sops decrypt prod/notes-db.yaml | grep -c password

Expected:

    password: bGFiLWRiLXByb2QtMTIz
1
1
Variable Content Typical use
SOPS_AGE_KEY_FILE path to an identity file laptop, pipeline with a file
SOPS_AGE_KEY the identity itself CI variable
SOPS_AGE_KEY_CMD command that prints the identity password manager

sops reads the default file keys.txt in addition – even when one of the variables is set. And an SSH key at ~/.ssh/id_ed25519 or id_rsa also counts as an identity (Day 1, Drill 5).

Save your progress? Sign up for free

Drill 4: The first matching rule wins

Break it: Someone adds a rule for all YAML files “just to be safe” – at the top.

# on argo
cd ~/sops/repo
cp .sops.yaml ~/sops/tag04/sops.yaml.gut
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
yq -i '.creation_rules = [{"path_regex": "\\.yaml$", "age": "'$ICH'"}]
  + .creation_rules' .sops.yaml
head -4 .sops.yaml
kubectl create secret generic z --from-literal=k=lab-z-123 --dry-run=client \
  -o yaml > prod/z.yaml
sops encrypt -i prod/z.yaml

Symptom:

# on argo
cd ~/sops/repo
yq '.sops.age[].recipient, .apiVersion' prod/z.yaml | cut -c1-40

Expected:

creation_rules:
  - path_regex: \.yaml$
    age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
  - path_regex: ^prod/.*\.yaml$
age126j96yd6te036uya83l76w0afmlaj9vyylux
ENC[AES256_GCM,data:Wbg=,iv:xyfhbqSe4rID

Cause: The new rule matches every YAML file and comes first. The prod file is therefore encrypted for the wrong key (the pipeline cannot read it) and fully, even apiVersion.

Fix and verify: Specific rules at the top, general ones at the bottom – or leave out the general one entirely.

# on argo
cd ~/sops/repo
rm prod/z.yaml
cp ~/sops/tag04/sops.yaml.gut .sops.yaml
yq '.creation_rules[].path_regex' .sops.yaml

Expected:

^prod/.*\.yaml$
^dev/.*\.yaml$
^notes/.*

Save your progress? Sign up for free

Drill 5: No rule, no configuration

# on argo
cd ~/sops/repo
printf 'a: b\n' > lose.yaml
sops encrypt -i lose.yaml; echo "rc=$?"
rm lose.yaml
cd /tmp
kubectl create secret generic y --from-literal=k=lab-y-123 --dry-run=client \
  -o yaml > ~/sops/repo/dev/y.yaml
sops encrypt -i ~/sops/repo/dev/y.yaml; echo "rc=$?"

Expected:

error loading config: no matching creation rules found
rc=1
config file not found, or has no creation rules, and no keys provided through command
line options
rc=1

The first message is a safeguard: a file without a rule is not encrypted with just any key. The second is a trap: from /tmp, sops does not find the .sops.yaml, because it searches from the current directory, not from the file path. The remedy: work inside the repo or name the configuration – --config is a global option and comes before the subcommand.

# on argo
cd /tmp
sops --config ~/sops/repo/.sops.yaml encrypt -i ~/sops/repo/dev/y.yaml
yq '.sops.age[].recipient' ~/sops/repo/dev/y.yaml
rm ~/sops/repo/dev/y.yaml

Expected:

age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2

From a subfolder of the repo (cd prod), on the other hand, sops finds the file, and the path_regex is still matched against prod/….

Save your progress? Sign up for free

Drill 6: Limit the MAC to encrypted values

Day 3 showed that even a change to the namespace breaks the MAC. In repos where tools such as Renovate or kustomize edit adjust plaintext fields (labels, annotations), that can get in the way. The option mac_only_encrypted exists only in .sops.yaml:

# on argo
cd ~/sops/repo
yq -i '.creation_rules[1].mac_only_encrypted = true' .sops.yaml
kubectl create secret generic m --from-literal=k=lab-m-123 --dry-run=client \
  -o yaml > dev/m.yaml
sops encrypt -i dev/m.yaml
yq '.sops.mac_only_encrypted' dev/m.yaml
sed -i 's/name: m$/name: umbenannt/' dev/m.yaml
sops decrypt dev/m.yaml | grep -E 'name|k:'

Expected:

true
    k: bGFiLW0tMTIz
    name: umbenannt

No MAC mismatch: the name is changed, the encrypted value unchanged. The price: nobody notices any more when an encrypted value moves into another namespace or under another name. That is why the book keeps the default.

# on argo
cd ~/sops/repo
rm dev/m.yaml
cp ~/sops/tag04/sops.yaml.gut .sops.yaml

Save your progress? Sign up for free

Drill 7: Rules and files into the repo

# on argo
cd ~/sops/repo
for f in dev/notes-db.yaml prod/notes-db.yaml notes/app.enc.yaml; do
  printf '%s ' $f; sops filestatus $f; done
git add .sops.yaml dev prod
git commit -q -m "tag04: creation_rules, notes-db für dev und prod"
git push -q
git log --oneline | head -1
git ls-files

Expected:

dev/notes-db.yaml {"encrypted":true}
prod/notes-db.yaml {"encrypted":true}
notes/app.enc.yaml {"encrypted":true}
6fb50ed tag04: creation_rules, notes-db für dev und prod
.sops.yaml
README.md
dev/notes-db.yaml
notes/app.enc.yaml
prod/notes-db.yaml

.sops.yaml belongs in the repo: it contains only public keys and documents who may read what. prod.key does not belong in it.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Show all path_regex entries of the rules file. Target time: 10 s
  2. Show the prod password in plaintext (base64-decoded). Target time: 40 s
  3. Show which rule would apply to dev/x.yaml, without encrypting anything. Target time: 30 s

Solution

# on argo
cd ~/sops/repo
yq '.creation_rules[].path_regex' .sops.yaml
SOPS_AGE_KEY_FILE=~/sops/tag04/prod.key sops decrypt \
  --extract '["data"]["password"]' prod/notes-db.yaml | base64 -d; echo
for r in $(yq '.creation_rules[].path_regex' .sops.yaml); do
  echo dev/x.yaml | grep -qE "$r" && { echo "$r"; break; }; done

Expected:

^prod/.*\.yaml$
^dev/.*\.yaml$
^notes/.*
lab-db-prod-123
^dev/.*\.yaml$

The loop mimics “first matching rule wins” with grep -E – sops uses Go regex; for simple patterns that is equivalent.

Save your progress? Sign up for free

Checkpoint

# on argo
cd ~/sops/repo && git status -sb
sops decrypt dev/notes-db.yaml | yq '.data.password' | base64 -d; echo
yq '.creation_rules | length' .sops.yaml
ls ~/sops/tag04/prod.key

Expected:

## main...origin/main
lab-db-dev-123
3
/home/ubuntu/sops/tag04/prod.key

Challenge

Add a rule that encrypts a dotenv file dev/app.env with your key (all values), create the file with API_KEY=lab-key-123, encrypt it in place and commit both.

Hint: encrypted_regex: ^(data|stringData)$ would encrypt nothing in dotenv – the new rule needs no regex and does not have to come before the YAML rule, because the patterns do not overlap.

Solution

# on argo
cd ~/sops/repo
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
yq -i '.creation_rules += [{"path_regex": "^dev/.*\\.env$", "age": "'$ICH'"}]' \
  .sops.yaml
printf 'API_KEY=lab-key-123\n' > dev/app.env
sops encrypt -i dev/app.env
cut -c1-40 dev/app.env | head -1
git add .sops.yaml dev/app.env
git commit -q -m "tag04: dotenv-Regel für dev" && git push -q
sops decrypt dev/app.env

Expected:

API_KEY=ENC[AES256_GCM,data:N1skRIiKsUbbn
API_KEY=lab-key-123

Save your progress? Sign up for free

Interview Note

  • What is in .sops.yaml and who reads it? creation_rules with path_regex, keys and options. sops reads it when encrypting and on updatekeys, not when decrypting. The first matching rule wins.
  • How do you separate environments? One rule per environment with its own recipients – the prod pipeline has only the prod key, developers only dev.

Enterprise Note

  • Protect .sops.yaml with CODEOWNERS: whoever changes the rules changes who can read secrets in the future.
  • Instead of individual personal keys in prod: KMS or OpenBao Transit (Day 7) plus an emergency key in the vault.
  • No catch-all rule \.yaml$ at the end without a reason – better a file fails with “no matching creation rules” than gets encrypted for the wrong people.
  • SOPS_AGE_KEY_CMD connects sops to the company password manager without the key lying on the laptop as a file.