Skip to content
Hands-on IT IT Drills

Day 8: sops in Everyday Work

Goal: You pass secrets to programs without them landing on disk, in the shell history or in your environment, read encrypted diffs in plain text and keep your key on one machine while you work on another. · Duration: approx. 60 min · Prerequisite: Day 7

Briefing

  • sops exec-env starts a command with the decrypted values as environment variables – for that process only.
  • sops exec-file hands the decrypted file to the command as a temporary path {}, by default as a FIFO (named pipe) that is empty after one read.
  • With a textconv driver, git diff shows plain text instead of ENC[…] – only on your machine, not in the repo.
  • sops keyservice keeps a key on one machine and answers decryption requests over the network.
Tool Plain text ends up …
sops decrypt > datei on disk (avoid)
sops decrypt \| befehl in a pipe
sops exec-env in the environment of a child process
sops exec-file in a FIFO under /tmp

Drill 1: Variables for one process only

# on argo
mkdir -p ~/sops/tag08 && cd ~/sops/repo
sops exec-env dev/app.env 'echo "API_KEY hat ${#API_KEY} Zeichen"'
sops exec-env dev/app.env 'env | grep -c API_KEY'
env | grep -c API_KEY
sops exec-env --pristine dev/app.env 'env'

Expected (hat 11 Zeichen means “has 11 characters”):

API_KEY hat 11 Zeichen
1
0
API_KEY=lab-key-123
PWD=/home/ubuntu/sops/repo

The command is in single quotes: otherwise your shell would substitute $API_KEY beforehand (empty). Outside the child process the variable does not exist. --pristine passes only the decrypted values – handy for seeing exactly what ends up in the environment.

Nested YAML files do not fit into environment variables:

# on argo
cd ~/sops/repo
sops exec-env notes/app.enc.yaml 'true'; echo "rc=$?"

Expected:

cannot use complex value in environment; offending key db
rc=1

Save your progress? Sign up for free

Drill 2: A file that exists only once

Some programs want a configuration file, not variables.

# on argo
cd ~/sops/repo
sops exec-file notes/app.enc.yaml 'ls -l {}; head -3 {}'
sops exec-file --no-fifo notes/app.enc.yaml 'ls -l {}'
sops exec-file --output-type json notes/app.enc.yaml 'jq -r .db.user {}'
ls -d /tmp/.sops* 2>/dev/null | wc -l

Expected:

prw------- 1 ubuntu ubuntu 0 Nov 20 16:37 /tmp/.sops1891905312/tmp-file
# Zugangsdaten der App notes (Laborwerte)
db:
    host: postgres.notes.svc
-rw------- 1 ubuntu ubuntu 174 Nov 20 16:37 /tmp/.sops1431033768/tmp-file2633480121
notes
0

The p in prw------- is the FIFO: the content flows through once and never sits on disk. A program that opens the file twice waits forever the second time – that is what --no-fifo is for (a regular file with mode 0600). sops cleans up both variants afterwards; --output-type converts the format along the way.

Save your progress? Sign up for free

Drill 3: Values in pipes

You can pass a single value directly to a program that reads from standard input.

# on argo
cd ~/sops/repo
sops decrypt --extract '["api"]["key"]' notes/app.enc.yaml \
  | kubectl create secret generic notes-api --from-file=key=/dev/stdin \
    --dry-run=client -o jsonpath='{.data.key}' | base64 -d; echo

Expected:

lab-key-123

--from-file=key=/dev/stdin instead of --from-literal=key=…: a value passed as an argument is visible to every user of the machine in ps and lands in the shell history.

Save your progress? Sign up for free

Drill 4: git diff in plain text

Symptom: A diff of an encrypted file only tells you that something changed (Day 2). For a review you want to see what.

# on argo
cd ~/sops/repo
git config diff.sopsdiffer.textconv "sops decrypt"
printf '*.yaml diff=sopsdiffer\n*.env diff=sopsdiffer\n' > .gitattributes
sops set notes/app.enc.yaml '["api"]["key"]' '"lab-key-789"'
git diff notes/app.enc.yaml
git diff --stat
git checkout notes/app.enc.yaml

Expected:

diff --git a/notes/app.enc.yaml b/notes/app.enc.yaml
index 902c4fc..ab13a47 100644
--- a/notes/app.enc.yaml
+++ b/notes/app.enc.yaml
@@ -5,5 +5,5 @@ db:
     user: notes
     password: lab-db-456
 api:
-    key: lab-key-123
+    key: lab-key-789
     debug: false
 notes/app.enc.yaml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Updated 1 path from the index

.gitattributes belongs in the repo and only says “compare these files with the sopsdiffer driver”. The driver itself lives in your local .git/config – anyone without a key still sees ENC[…]. --stat counts the real lines of the file.

# on argo
cd ~/sops/repo
git add .gitattributes
git commit -q -m "tag08: Klartext-Diff für sops-Dateien" && git push -q
git log --oneline | head -1

Expected:

4206ff1 tag08: Klartext-Diff für sops-Dateien

Save your progress? Sign up for free

Drill 5: Indentation

sops writes YAML with four spaces; kubectl and most linters expect two. --indent is a global option; for good, it belongs in .sops.yaml:

# on argo
cd ~/sops/repo
sops --indent 2 decrypt dev/notes-db.yaml | head -3
cp .sops.yaml ~/sops/tag08/sops.yaml.vorher
yq -i '.stores.yaml.indent = 2' .sops.yaml
sops decrypt dev/notes-db.yaml | head -3

Expected:

apiVersion: v1
data:
  password: bGFiLWRiLWRldi00NTY=
apiVersion: v1
data:
  password: bGFiLWRiLWRldi00NTY=

stores.yaml.indent applies to everything sops writes in this repo – including when it encrypts. Existing files only change the next time they are written.

# on argo
cd ~/sops/repo
git commit -qam "tag08: YAML mit zwei Leerzeichen" && git push -q
yq '.stores' .sops.yaml

Expected:

yaml:
  indent: 2

Save your progress? Sign up for free

Drill 6: The key stays where it is

Scenario: you work on a server via SSH, while the prod key lives on another machine. sops keyservice answers requests there, and the key never leaves that machine. In the lab both roles play out on argo, connected via 127.0.0.1:5000 – across machines you would tunnel it with ssh -R 5000:127.0.0.1:5000.

# on argo
cd ~/sops/repo
SOPS_AGE_KEY_FILE=~/sops/tag04/prod.key XDG_CONFIG_HOME=/nirgends \
  sops keyservice --addr 127.0.0.1:5000 --verbose > ~/sops/tag08/ks.log 2>&1 &
sleep 2
VAULT_TOKEN= XDG_CONFIG_HOME=/nirgends sops decrypt prod/notes-db.yaml 2>&1 \
  | head -1
VAULT_TOKEN= XDG_CONFIG_HOME=/nirgends sops decrypt \
  --keyservice tcp://127.0.0.1:5000 prod/notes-db.yaml | grep password
kill %1
cut -c1-72 ~/sops/tag08/ks.log | grep -E 'Listening|succeeded'

Expected:

Failed to get the data key required to decrypt the SOPS file.
  password: bGFiLWRiLXByb2QtMTIz
[KEYSERVICE]     time="2026-11-20T16:37:27Z" level=info msg="Listening on t
[AGE]    time="2026-11-20T16:37:29Z" level=info msg="Decryption succeeded"

VAULT_TOKEN= switches off the Transit path, XDG_CONFIG_HOME your own key – only the keyservice is left. The two-space indentation comes from Drill 5. The keyservice decrypts the data key and sends it back. With --prompt it asks for confirmation of every request.

Warning: The keyservice has no authentication. Bind it only to 127.0.0.1 or a Unix socket, never to a network interface.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Print the number of variables in dev/app.env without showing a value. Target time: 30 s
  2. Show the file notes/app.enc.yaml as JSON via exec-file. Target time: 30 s

Solution

# on argo
cd ~/sops/repo
sops exec-env --pristine dev/app.env 'env | grep -vc ^PWD='
sops exec-file --output-type json notes/app.enc.yaml 'jq -c .api {}'

Expected:

1
{"key":"lab-key-123","debug":false}

Save your progress? Sign up for free

Checkpoint

# on argo
cd ~/sops/repo && git status -sb
cat .gitattributes
git config --get diff.sopsdiffer.textconv

Expected:

## main...origin/main
*.yaml diff=sopsdiffer
*.env diff=sopsdiffer
sops decrypt

Challenge

Use git log -p to show all password changes of dev/notes-db.yaml in plain text. How many do you see, and why not all of them from Day 4 and 5?

Hint: git log -p uses textconv only with --textconv.

Solution

# on argo
cd ~/sops/repo
git log -p --textconv -- dev/notes-db.yaml 2>/dev/null \
  | grep -E '^[-+] +password'
git log --format=%h -- dev/notes-db.yaml | wc -l

Expected:

-  password: bGFiLWRiLWRldi0xMjM=
+  password: bGFiLWRiLWRldi00NTY=
5

Only the last change (…dev-123 → …dev-456) is readable. The older states are encrypted for your old key from Day 1, which you replaced on Day 5; sops decrypt fails there, and git shows nothing. For the colleague with her key it would be exactly the other way round.

Save your progress? Sign up for free

Cleanup

# on argo
pgrep -fc 'sops keyservice'
rm -f ~/sops/tag08/ks.log

Expected:

0

Interview Note

  • How do you pass a secret to a program without writing it to disk? sops exec-env (environment of a child process), sops exec-file (FIFO) or a pipe; never as a command-line argument.
  • How do you review changes to encrypted files? Structure in the diff (which key), content locally via a textconv driver – only people with the key can see it.

Enterprise Note

  • Environment variables are not secret: child processes inherit them, crash dumps and /proc/<pid>/environ (for the same user) reveal them. For long-running services, prefer files with tight permissions or a secret store.
  • Describe the textconv driver in an onboarding document; maintain .gitattributes centrally.
  • Use the keyservice only for exceptions (emergency, Shamir session); for everyday work a KMS with identity is better.