Day 4: .sops.yaml and
creation_rules
Goal: A
.sops.yamlin the repo defines which file is encrypted for whom and how – nobody types--ageor--encrypted-regexany more, and dev and prod have separate keys. · Duration: approx. 60 min · Prerequisite: Day 3
Briefing
.sops.yamlcontainscreation_rules: a list of rules, each with apath_regexand the keys (age,pgp,hc_vault_transit_uri, …) plus options such asencrypted_regex.- sops searches for the file from the current
directory upwards.
path_regexis matched against the path relative to the.sops.yaml. The first matching rule wins. - The rules apply when a file is created (and on
updatekeys, Day 5). An already encrypted file carries its recipients itself in the blocksops:. - Whoever decrypts needs no
.sops.yaml– only a matching identity.
~/sops/repo/.sops.yaml
^prod/.*\.yaml$ → age1r025… (prod pipeline key) only data/stringData
^dev/.*\.yaml$ → age126j… (me) only data/stringData
^notes/.* → age126j… (me)
Drill 1: A second key and the first rules file
prod gets its own key – later only the cluster or the pipeline has it.
# on argo
mkdir -p ~/sops/tag04 && cd ~/sops/tag04
age-keygen -o prod.key
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
PROD=$(age-keygen -y prod.key)
cd ~/sops/repo
cat > .sops.yaml <<EOF
creation_rules:
- path_regex: ^prod/.*\.yaml$
encrypted_regex: ^(data|stringData)$
age: $PROD
- path_regex: ^dev/.*\.yaml$
encrypted_regex: ^(data|stringData)$
age: $ICH
- path_regex: ^notes/.*
age: $ICH
EOF
cat .sops.yamlExpected:
Public key: age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
creation_rules:
- path_regex: ^prod/.*\.yaml$
encrypted_regex: ^(data|stringData)$
age: age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
- path_regex: ^dev/.*\.yaml$
encrypted_regex: ^(data|stringData)$
age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
- path_regex: ^notes/.*
age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
The heredoc is deliberately unquoted (<<EOF): the
shell substitutes $PROD and $ICH,
\. stays as it is. You separate multiple recipients in
age: with commas (Day 5).
Save your progress? Sign up for free
Drill 2: Encrypt without options
# on argo
cd ~/sops/repo
mkdir -p dev prod
for e in dev prod; do
kubectl create secret generic notes-db -n notes-$e \
--from-literal=password=lab-db-$e-123 --dry-run=client -o yaml > $e/notes-db.yaml
done
sops encrypt -i dev/notes-db.yaml
sops encrypt -i prod/notes-db.yaml
yq '.sops.age[].recipient' dev/notes-db.yaml prod/notes-db.yaml
yq '.kind, .data.password' prod/notes-db.yaml | cut -c1-40Expected:
age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
---
age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks
Secret
ENC[AES256_GCM,data:qJhs0Ct2YVuBXOKbIKY4
-i (in place) replaces the plaintext in the file with
the encrypted version. Recipients and encrypted_regex come
from the rule that matches the path.
Warning:
-ionly overwrites the file itself. Copies, editor backups and whatever was already staged withgit addstay plaintext – so never stage plaintext files (Day 9).
Save your progress? Sign up for free
Drill 3: The missing key and where sops looks
Symptom: prod cannot be opened with your key.
# on argo
cd ~/sops/repo
sops decrypt prod/notes-db.yaml 2>&1 | head -9Expected:
Failed to get the data key required to decrypt the SOPS file.
Group 0: FAILED
age1r025nv6kv083tjhqa7xpslvkc0m8qm2jljd0aqn55nzr859xqe7q4798ks: FAILED
- | failed to create reader for decrypting sops data key with
| age: no identity matched any of the recipients. Did not find
| keys in locations 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
| 'SOPS_AGE_SSH_PRIVATE_KEY_CMD', '/home/ubuntu/.ssh/id_rsa',
| 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', and 'SOPS_AGE_KEY_CMD'.
The message names the recipient for which a key is missing and all
the places where sops looked. It found only
~/.config/sops/age/keys.txt – and that one does not match.
Three ways to give sops a different key:
# on argo
cd ~/sops/repo
SOPS_AGE_KEY_FILE=~/sops/tag04/prod.key sops decrypt prod/notes-db.yaml \
| grep password
SOPS_AGE_KEY_CMD="cat $HOME/sops/tag04/prod.key" sops decrypt prod/notes-db.yaml \
| grep -c password
SOPS_AGE_KEY=$(grep AGE-SECRET ~/sops/tag04/prod.key) \
sops decrypt prod/notes-db.yaml | grep -c passwordExpected:
password: bGFiLWRiLXByb2QtMTIz
1
1
| Variable | Content | Typical use |
|---|---|---|
SOPS_AGE_KEY_FILE |
path to an identity file | laptop, pipeline with a file |
SOPS_AGE_KEY |
the identity itself | CI variable |
SOPS_AGE_KEY_CMD |
command that prints the identity | password manager |
sops reads the default file keys.txt in
addition – even when one of the variables is set. And an SSH
key at ~/.ssh/id_ed25519 or id_rsa also counts
as an identity (Day 1, Drill 5).
Save your progress? Sign up for free
Drill 4: The first matching rule wins
Break it: Someone adds a rule for all YAML files “just to be safe” – at the top.
# on argo
cd ~/sops/repo
cp .sops.yaml ~/sops/tag04/sops.yaml.gut
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
yq -i '.creation_rules = [{"path_regex": "\\.yaml$", "age": "'$ICH'"}]
+ .creation_rules' .sops.yaml
head -4 .sops.yaml
kubectl create secret generic z --from-literal=k=lab-z-123 --dry-run=client \
-o yaml > prod/z.yaml
sops encrypt -i prod/z.yamlSymptom:
# on argo
cd ~/sops/repo
yq '.sops.age[].recipient, .apiVersion' prod/z.yaml | cut -c1-40Expected:
creation_rules:
- path_regex: \.yaml$
age: age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
- path_regex: ^prod/.*\.yaml$
age126j96yd6te036uya83l76w0afmlaj9vyylux
ENC[AES256_GCM,data:Wbg=,iv:xyfhbqSe4rID
Cause: The new rule matches every YAML file and
comes first. The prod file is therefore encrypted for the wrong key (the
pipeline cannot read it) and fully, even apiVersion.
Fix and verify: Specific rules at the top, general ones at the bottom – or leave out the general one entirely.
# on argo
cd ~/sops/repo
rm prod/z.yaml
cp ~/sops/tag04/sops.yaml.gut .sops.yaml
yq '.creation_rules[].path_regex' .sops.yamlExpected:
^prod/.*\.yaml$
^dev/.*\.yaml$
^notes/.*
Save your progress? Sign up for free
Drill 5: No rule, no configuration
# on argo
cd ~/sops/repo
printf 'a: b\n' > lose.yaml
sops encrypt -i lose.yaml; echo "rc=$?"
rm lose.yaml
cd /tmp
kubectl create secret generic y --from-literal=k=lab-y-123 --dry-run=client \
-o yaml > ~/sops/repo/dev/y.yaml
sops encrypt -i ~/sops/repo/dev/y.yaml; echo "rc=$?"Expected:
error loading config: no matching creation rules found
rc=1
config file not found, or has no creation rules, and no keys provided through command
line options
rc=1
The first message is a safeguard: a file without a rule is not
encrypted with just any key. The second is a trap: from
/tmp, sops does not find the .sops.yaml,
because it searches from the current directory, not from the file path.
The remedy: work inside the repo or name the configuration –
--config is a global option and comes
before the subcommand.
# on argo
cd /tmp
sops --config ~/sops/repo/.sops.yaml encrypt -i ~/sops/repo/dev/y.yaml
yq '.sops.age[].recipient' ~/sops/repo/dev/y.yaml
rm ~/sops/repo/dev/y.yamlExpected:
age126j96yd6te036uya83l76w0afmlaj9vyyluxrmdeheew3u0wzdmslw69y2
From a subfolder of the repo (cd prod), on the other
hand, sops finds the file, and the path_regex is still
matched against prod/….
Save your progress? Sign up for free
Drill 6: Limit the MAC to encrypted values
Day 3 showed that even a change to the namespace breaks the MAC. In
repos where tools such as Renovate or kustomize edit adjust
plaintext fields (labels, annotations), that can get in the way. The
option mac_only_encrypted exists only in
.sops.yaml:
# on argo
cd ~/sops/repo
yq -i '.creation_rules[1].mac_only_encrypted = true' .sops.yaml
kubectl create secret generic m --from-literal=k=lab-m-123 --dry-run=client \
-o yaml > dev/m.yaml
sops encrypt -i dev/m.yaml
yq '.sops.mac_only_encrypted' dev/m.yaml
sed -i 's/name: m$/name: umbenannt/' dev/m.yaml
sops decrypt dev/m.yaml | grep -E 'name|k:'Expected:
true
k: bGFiLW0tMTIz
name: umbenannt
No MAC mismatch: the name is changed, the encrypted value unchanged. The price: nobody notices any more when an encrypted value moves into another namespace or under another name. That is why the book keeps the default.
# on argo
cd ~/sops/repo
rm dev/m.yaml
cp ~/sops/tag04/sops.yaml.gut .sops.yamlSave your progress? Sign up for free
Drill 7: Rules and files into the repo
# on argo
cd ~/sops/repo
for f in dev/notes-db.yaml prod/notes-db.yaml notes/app.enc.yaml; do
printf '%s ' $f; sops filestatus $f; done
git add .sops.yaml dev prod
git commit -q -m "tag04: creation_rules, notes-db für dev und prod"
git push -q
git log --oneline | head -1
git ls-filesExpected:
dev/notes-db.yaml {"encrypted":true}
prod/notes-db.yaml {"encrypted":true}
notes/app.enc.yaml {"encrypted":true}
6fb50ed tag04: creation_rules, notes-db für dev und prod
.sops.yaml
README.md
dev/notes-db.yaml
notes/app.enc.yaml
prod/notes-db.yaml
.sops.yaml belongs in the repo: it contains only public
keys and documents who may read what. prod.key does not
belong in it.
Save your progress? Sign up for free
Exercise: Speed Drills
- Show all
path_regexentries of the rules file. Target time: 10 s - Show the prod password in plaintext (base64-decoded). Target time: 40 s
- Show which rule would apply to
dev/x.yaml, without encrypting anything. Target time: 30 s
Solution
# on argo
cd ~/sops/repo
yq '.creation_rules[].path_regex' .sops.yaml
SOPS_AGE_KEY_FILE=~/sops/tag04/prod.key sops decrypt \
--extract '["data"]["password"]' prod/notes-db.yaml | base64 -d; echo
for r in $(yq '.creation_rules[].path_regex' .sops.yaml); do
echo dev/x.yaml | grep -qE "$r" && { echo "$r"; break; }; doneExpected:
^prod/.*\.yaml$
^dev/.*\.yaml$
^notes/.*
lab-db-prod-123
^dev/.*\.yaml$
The loop mimics “first matching rule wins” with grep -E
– sops uses Go regex; for simple patterns that is equivalent.
Save your progress? Sign up for free
Checkpoint
# on argo
cd ~/sops/repo && git status -sb
sops decrypt dev/notes-db.yaml | yq '.data.password' | base64 -d; echo
yq '.creation_rules | length' .sops.yaml
ls ~/sops/tag04/prod.keyExpected:
## main...origin/main
lab-db-dev-123
3
/home/ubuntu/sops/tag04/prod.key
Challenge
Add a rule that encrypts a dotenv file dev/app.env with
your key (all values), create the file with
API_KEY=lab-key-123, encrypt it in place and commit
both.
Hint:
encrypted_regex: ^(data|stringData)$ would encrypt nothing
in dotenv – the new rule needs no regex and does not have to come before
the YAML rule, because the patterns do not overlap.
Solution
# on argo
cd ~/sops/repo
ICH=$(age-keygen -y ~/.config/sops/age/keys.txt)
yq -i '.creation_rules += [{"path_regex": "^dev/.*\\.env$", "age": "'$ICH'"}]' \
.sops.yaml
printf 'API_KEY=lab-key-123\n' > dev/app.env
sops encrypt -i dev/app.env
cut -c1-40 dev/app.env | head -1
git add .sops.yaml dev/app.env
git commit -q -m "tag04: dotenv-Regel für dev" && git push -q
sops decrypt dev/app.envExpected:
API_KEY=ENC[AES256_GCM,data:N1skRIiKsUbbn
API_KEY=lab-key-123
Save your progress? Sign up for free
Interview Note
- What is in
.sops.yamland who reads it?creation_ruleswithpath_regex, keys and options. sops reads it when encrypting and onupdatekeys, not when decrypting. The first matching rule wins. - How do you separate environments? One rule per environment with its own recipients – the prod pipeline has only the prod key, developers only dev.
Enterprise Note
- Protect
.sops.yamlwith CODEOWNERS: whoever changes the rules changes who can read secrets in the future. - Instead of individual personal keys in prod: KMS or OpenBao Transit (Day 7) plus an emergency key in the vault.
- No catch-all rule
\.yaml$at the end without a reason – better a file fails with “no matching creation rules” than gets encrypted for the wrong people. SOPS_AGE_KEY_CMDconnects sops to the company password manager without the key lying on the laptop as a file.