Skip to content
Hands-on IT IT Drills

Day 8: Files, Secret References and values.schema.json

Goal: Bring configuration files into the chart cleanly, accept secrets only by reference, and protect the chart’s values with a JSON schema. · Duration: approx. 60 min · Prerequisite: Day 7

Briefing

  • Long configurations live as real files next to templates/ and get into the chart via .Files. Anything inside templates/ or excluded by .helmignore is invisible to .Files – without an error message.
  • The chart does not ship secrets; it references a Secret that is created some other way (the existingSecret pattern).
  • values.schema.json checks all values before rendering – on lint, template, install and upgrade, with all errors at once.
 notes/
 ├─ config/*.conf, motd.txt   ──.Files──>  ConfigMap notes-files
 ├─ values.schema.json        ──checks──>  values (before rendering)
 └─ templates/deployment.yaml ──secretKeyRef──>  Secret notes-api-key (kubectl)

Drill 1: Files with Files.Glob and Files.Get

# on prod
mkdir -p ~/helm/tag08 && cd ~/helm/charts
mkdir -p notes/config
cat > notes/config/limits.conf <<'EOF'
# notes/config/limits.conf – Stand Tag 08
max_note_length = 2000
max_notes_per_minute = 60
EOF
cat > notes/config/logging.conf <<'EOF'
# notes/config/logging.conf – Stand Tag 08
format = text
EOF
cat > notes/config/motd.txt <<'EOF'
Willkommen bei {{ .Release.Name }} im Namespace {{ .Release.Namespace }}.
EOF
cat > notes/templates/files.yaml <<'EOF'
{{/* notes/templates/files.yaml – Stand Tag 08 */}}
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "notes.fullname" . }}-files
  labels:
    {{- include "notes.labels" . | nindent 4 }}
data:
  {{- (.Files.Glob "config/*.conf").AsConfig | nindent 2 }}
  motd.txt: |-
    {{- tpl (.Files.Get "config/motd.txt") . | nindent 4 }}
EOF
helm template notes ./notes -s templates/files.yaml -n notes | sed -n '/^data/,$p'

Expected:

data:
  limits.conf: |
    # notes/config/limits.conf – Stand Tag 08
    max_note_length = 2000
    max_notes_per_minute = 60
  logging.conf: |
    # notes/config/logging.conf – Stand Tag 08
    format = text
  motd.txt: |-
    Willkommen bei notes im Namespace notes.

The message of the day reads ‘Welcome to notes in namespace notes.’. .AsConfig turns each file into a key with a block string. motd.txt takes the detour through .Files.Get and tpl because it contains template expressions. The glob *.conf leaves out motd.txt on purpose; otherwise it would appear twice, once unrendered.

Save your progress? Sign up for free

Drill 2: When Files returns an empty string

# on prod
mkdir -p notes/entwurf
echo 'geheim = ja' > notes/entwurf/neu.conf
printf 'alice\nbob\n' > notes/config/admins.txt
cat > notes/templates/spielwiese.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: spiel
data:
  admins: {{ range .Files.Lines "config/admins.txt" }}{{ . }},{{ end }}
  anzahl: {{ len (.Files.Glob "config/*") | quote }}
  ausTemplates: {{ .Files.Get "templates/service.yaml" | quote }}
  ausEntwurf: {{ .Files.Get "entwurf/neu.conf" | quote }}
  tippfehler: {{ .Files.Get "config/limit.conf" | quote }}
---
apiVersion: v1
kind: Secret
metadata:
  name: spiel
data:
  {{- (.Files.Glob "config/logging.conf").AsSecrets | nindent 2 }}
EOF
helm template notes ./notes -s templates/spielwiese.yaml \
  | grep -v -E '^(---|#|apiVersion|kind|metadata|  name)'
rm -r notes/templates/spielwiese.yaml notes/entwurf notes/config/admins.txt

entwurf means ‘draft’. The keys mean ‘count’ (anzahl), ‘from templates’, ‘from draft’ and ‘typo’ (tippfehler).

Expected (abridged):

data:
  logging.conf: IyBub3Rlcy9jb25maWcvbG9nZ2luZy5jb25mIOKAkyBTdGFuZCBUYWcgMDgKZm9ybWF0…
data:
  admins: alice,bob,
  anzahl: "4"
  ausTemplates: ""
  ausEntwurf: ""
  tippfehler: ""

Three empty strings, no warning: templates/ is off limits for .Files, .helmignore has excluded entwurf/ since Day 2, and the typo limit.conf slips through just as silently. So check every new file with helm template -s. .AsSecrets encodes to Base64 for Secret.data; Helm sorts the Secret before the ConfigMap.

Save your progress? Sign up for free

Drill 3: existingSecret for the API key

So far lab-key-123 sits in plain text in values.yaml and therefore in Git, in the release Secret and in the manifest. From now on the chart only references a Secret. Prepared at the same time: the reference to a database password for Day 9.

# on prod
cat > notes/values.yaml <<'EOF'
# notes/values.yaml – Stand Tag 08

# -- Anzahl der notes-api-Pods (0 ist erlaubt, null = Standard 2)
replicaCount: 2

image:
  # -- Image ohne Tag
  repository: ghcr.io/thomaszachmann/notes-api
  # -- Tag des Images (leer = appVersion)
  tag: ""

apiKey:
  # -- Name eines vorhandenen Secrets mit dem API-Key (Pflicht)
  existingSecret: ""
  # -- Schlüssel im Secret
  key: api-key

database:
  # -- Vorhandenes Secret mit dem DB-Passwort (leer = Speicher statt Postgres)
  existingSecret: ""
  # -- Schlüssel im Secret
  passwordKey: password
  # -- Host der Datenbank (Pflicht, wenn existingSecret gesetzt ist)
  host: ""
  port: 5432
  name: notes
  user: notes

# -- Zusätzliche Umgebungsvariablen: Name -> Wert (Map, damit -f ergänzen kann)
env:
  NOTES_LOG_LEVEL: info
  NOTES_PAGE_SIZE: "50"

# -- Team-Label an Deployment und Pods (leer = kein Label)
team: ""

# -- Ressourcen des Containers
resources:
  requests:
    cpu: 20m
    memory: 32Mi
  limits:
    memory: 128Mi

# -- nodeSelector der Pods (leer = keiner)
nodeSelector: {}

service:
  # -- Port des Service
  port: 80

ingress:
  # -- Ingress anlegen
  enabled: false
  # -- IngressClass (k3s: traefik)
  className: traefik
  # -- Hostnamen, alle zeigen auf den Service des Charts
  hosts: []

# -- Ersetzt den Chart-Namen in Namen und Labels
nameOverride: ""
# -- Ersetzt den vollständigen Namen aller Objekte
fullnameOverride: ""

# -- Zeit zum sauberen Beenden (Go-Dauer wie 30s, 1m)
shutdownTimeout: 30s
EOF
cat > notes/templates/deployment.yaml <<'EOF'
{{/* notes/templates/deployment.yaml – Stand Tag 08 */}}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "notes.fullname" . }}
  labels:
    {{- include "notes.labels" . | nindent 4 }}
spec:
  replicas: {{ hasKey .Values "replicaCount" | ternary .Values.replicaCount 2 }}
  selector:
    matchLabels:
      {{- include "notes.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "notes.labels" . | nindent 8 }}
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/config.yaml") .
          | sha256sum }}
    spec:
      terminationGracePeriodSeconds: {{ durationSeconds .Values.shutdownTimeout }}
      containers:
        - name: notes-api
          image: {{ include "notes.image" . | quote }}
          ports: [{name: http, containerPort: 8080}]
          env:
            - name: NOTES_API_KEY
              valueFrom:
                secretKeyRef:
                  name: {{ required "apiKey.existingSecret fehlt"
                    .Values.apiKey.existingSecret }}
                  key: {{ .Values.apiKey.key }}
            {{- with .Values.database }}
            {{- if .existingSecret }}
            - name: NOTES_DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ .existingSecret }}
                  key: {{ .passwordKey }}
            - name: NOTES_DB_URL
              value: {{ printf "postgres://%s:$(NOTES_DB_PASSWORD)@%s:%v/%s?%s"
                .user (required "database.host fehlt" .host) .port .name
                "sslmode=disable" | quote }}
            {{- end }}
            {{- end }}
          envFrom:
            - configMapRef:
                name: {{ include "notes.fullname" . }}-config
          readinessProbe:
            httpGet: {path: /healthz, port: http}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
EOF
helm template notes ./notes -s templates/deployment.yaml 2>&1 | head -1
helm template notes ./notes -s templates/deployment.yaml \
  --set apiKey.existingSecret=notes-api-key | sed -n '/env:/,/envFrom/p'
helm template notes ./notes -s templates/deployment.yaml \
  --set apiKey.existingSecret=notes-api-key --set database.existingSecret=notes-db \
  --set database.host=pg | grep -A1 'name: NOTES_DB_URL'

The new doc comments: apiKey.existingSecret is the name of an existing Secret with the API key (required), key the key inside the Secret. database.existingSecret is an existing Secret with the DB password (empty = in-memory storage instead of Postgres), and host is required once existingSecret is set.

Expected:

Error: execution error at (notes/templates/deployment.yaml:30:27): apiKey.existingSecret
fehlt
          env:
            - name: NOTES_API_KEY
              valueFrom:
                secretKeyRef:
                  name: notes-api-key
                  key: api-key
          envFrom:
            - name: NOTES_DB_URL
              value: "postgres://notes:$(NOTES_DB_PASSWORD)@pg:5432/notes?sslmode=di…

fehlt means ‘missing’, as on Day 7. Kubernetes replaces $(NOTES_DB_PASSWORD) when the Pod starts, because the variable is defined further up in the same list. That way the password stays in the Secret, and the chart still builds the URL that notes-api expects.

Save your progress? Sign up for free

Drill 4: Create the Secret, upgrade, check for leaks

# on prod
kubectl -n notes create secret generic notes-api-key \
  --from-literal=api-key=lab-key-123
cat >> values/prod.yaml <<'EOF'
apiKey:
  existingSecret: notes-api-key
EOF
sed -i 's/Stand Tag 07/Stand Tag 08/' values/prod.yaml
sed -i 's/^version: .*/version: 0.7.0/; s/Tag 07/Tag 08/' notes/Chart.yaml
helm lint notes -f values/prod.yaml | tail -1
helm upgrade notes ./notes -n notes -f values/prod.yaml --wait | grep REVISION
helm get values notes -n notes | head -3
helm get manifest notes -n notes | grep -c lab-key
kubectl -n notes get cm notes-files -o jsonpath='{.data.motd\.txt}{"\n"}'

Expected:

secret/notes-api-key created
1 chart(s) linted, 0 chart(s) failed
REVISION: 19
USER-SUPPLIED VALUES:
apiKey:
  existingSecret: notes-api-key
0
Willkommen bei notes im Namespace notes.

The key appears neither in the values nor in the release manifest. Anyone who wants to read it needs read access to the Secret – and that is governed by RBAC, not by the chart.

Save your progress? Sign up for free

Drill 5: What the API key protects

A look at the interface before you rely on the key:

# on prod
curl -s -w ' %{http_code}\n' -H 'Host: api.notes.lab' localhost/notes \
  -d '{"text":"ohne Key"}'
curl -s -w ' %{http_code}\n' -H 'Host: api.notes.lab' -H 'X-API-Key: lab-key-123' \
  localhost/notes -d '{"text":"erste Notiz"}'
for i in 1 2 3 4; do
  curl -s -w ' %{http_code}\n' -H 'Host: api.notes.lab' localhost/notes; done

The note texts mean ‘without key’ and ‘first note’.

Expected (IDs abridged):

{"error":"invalid api key"} 401
{"id":"43d2d3b5-…","text":"erste Notiz"} 201
[{"id":"43d2d3b5-…","text":"erste Notiz"}] 200
[] 200
[{"id":"43d2d3b5-…","text":"erste Notiz"}] 200
[] 200

Two findings. First: notes-api only checks the key on writes (POST); GET /notes is open without a key – if you want to protect reads, you need an Ingress with authentication or a NetworkPolicy in front. Second: without a database each of the two Pods has its own storage, so the note only shows up on every second request. Day 9 gives the release a database. Right after an upgrade, Traefik may briefly still point to a terminated Pod and return 502.

Save your progress? Sign up for free

Drill 6: values.schema.json

The schema replaces the fail rule from Day 7 and also catches typos in key names.

# on prod
cat > notes/values.schema.json <<'EOF'
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "notes – Stand Tag 08",
  "type": "object",
  "additionalProperties": false,
  "required": ["image", "apiKey"],
  "properties": {
    "replicaCount": {"type": ["integer", "null"], "minimum": 0},
    "image": {
      "type": "object",
      "additionalProperties": false,
      "required": ["repository"],
      "properties": {
        "repository": {"type": "string", "minLength": 1},
        "tag": {"type": "string"}
      }
    },
    "apiKey": {
      "type": "object",
      "required": ["existingSecret", "key"],
      "properties": {
        "existingSecret": {"type": "string"},
        "key": {"type": "string", "minLength": 1}
      }
    },
    "database": {"type": "object"},
    "env": {
      "type": "object",
      "additionalProperties": {"type": "string"},
      "properties": {
        "NOTES_LOG_LEVEL": {"enum": ["debug", "info", "warn", "error"]}
      }
    },
    "team": {"type": "string"},
    "resources": {"type": "object"},
    "nodeSelector": {"type": "object"},
    "service": {"type": "object"},
    "ingress": {"type": "object"},
    "nameOverride": {"type": "string"},
    "fullnameOverride": {"type": "string"},
    "shutdownTimeout": {"type": "string", "pattern": "^([0-9]+(ms|s|m|h))+$"}
  }
}
EOF
cat > notes/templates/config.yaml <<'EOF'
{{/* notes/templates/config.yaml – Stand Tag 08 (Log-Level prüft das Schema) */}}
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "notes.fullname" . }}-config
  labels:
    {{- include "notes.labels" . | nindent 4 }}
data:
  {{- range $name, $value := .Values.env }}
  {{ $name }}: {{ $value | quote }}
  {{- end }}
EOF
helm lint notes -f values/prod.yaml | tail -1
helm lint notes --set replicaCount=abc | grep ERROR
helm template notes ./notes -f values/prod.yaml --set replicas=3 \
  --set env.NOTES_LOG_LEVEL=WARN --set replicaCount=-1 --set shutdownTimeout=30

The header of config.yaml now notes that the schema checks the log level.

Expected:

1 chart(s) linted, 0 chart(s) failed
[ERROR] values.yaml: - at '/replicaCount': got string, want null or integer
[ERROR] templates/: values don't meet the specifications of the schema(s) in the
following chart(s):
Error: values don't meet the specifications of the schema(s) in the following chart(s):
notes:
- at '/env/NOTES_LOG_LEVEL': value must be one of 'debug', 'info', 'warn', 'error'
- at '/shutdownTimeout': got number, want string
- at '/replicaCount': minimum: got -1, want 0
- at '': additional properties 'replicas' not allowed

Four errors in one run – required and fail would have stopped at the first one. Without a schema, replicas=3 would have silently fizzled out. "type": ["integer", "null"] allows the null from Day 3 that deletes the default.

Save your progress? Sign up for free

Drill 7: The schema on upgrade and its limits

# on prod
helm upgrade notes ./notes -n notes -f values/prod.yaml --set env.NOTES_LOG_LEVEL=laut
helm history notes -n notes --max 1 -o json | jq -r '.[0].revision'
helm template notes ./notes -f values/prod.yaml --set-string replicaCount=2 2>&1 \
  | grep 'at '

Expected:

Error: UPGRADE FAILED: values don't meet the specifications of the schema(s) in the
following chart(s):
notes:
- at '/env/NOTES_LOG_LEVEL': value must be one of 'debug', 'info', 'warn', 'error'
19
- at '/replicaCount': got string, want null or integer

The log level laut (‘loud’) fails before rendering, and revision 19 stays. --set-string is the trap here: 2 becomes "2", and the schema rejects it. Helm 4 also understands newer drafts and fetches remote $ref while rendering – a problem in an air-gapped CI:

# on prod
cd ~/helm/tag08 && mkdir -p s/templates
printf 'apiVersion: v2\nname: s\nversion: 0.1.0\n' > s/Chart.yaml
cat > s/values.schema.json <<'EOF'
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$defs": {"port": {"type": "integer", "minimum": 1, "maximum": 65535}},
  "type": "object",
  "properties": {"port": {"$ref": "#/$defs/port"}}
}
EOF
helm template x ./s --set port=70000 2>&1 | grep 'at '
cat > s/values.schema.json <<'EOF'
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {"port": {"$ref": "https://example.com/port.json"}}
}
EOF
helm template x ./s --set port=1 2>&1 | tail -1

Expected:

- at '/port': maximum: got 70,000, want 65,535
failing loading "https://example.com/port.json": HTTP request to
https://example.com/port.json returned status 404 (Not Found)

So keep schemas self-contained: definitions under $defs instead of foreign URLs.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Show only the schema error line for --set team=1. Target time: 20 s
  2. Check whether the release manifest contains the API key. Target time: 15 s
  3. List the keys of the ConfigMap notes-files. Target time: 20 s
  4. Read the API key from the Secret in plain text. Target time: 20 s

Solution

# on prod
cd ~/helm/charts
helm template notes ./notes -f values/prod.yaml --set team=1 2>&1 | grep 'at '
helm get manifest notes -n notes | grep -c lab-key
kubectl -n notes get cm notes-files -o json | jq -r '.data | keys[]'
kubectl -n notes get secret notes-api-key -o jsonpath='{.data.api-key}' \
  | base64 -d; echo

Expected:

- at '/team': got number, want string
0
limits.conf
logging.conf
motd.txt
lab-key-123

Save your progress? Sign up for free

Checkpoint

# on prod
helm list -n notes

Expected: Revision 19, notes-0.7.0, deployed.

# on prod
helm lint ~/helm/charts/notes --set replicas=3 | tail -1

Expected: Error: 1 chart(s) linted, 1 chart(s) failed

# on prod
kubectl -n notes get deploy notes \
  -o jsonpath='{.spec.template.spec.containers[0].env[0].valueFrom}{"\n"}'

Expected: {"secretKeyRef":{"key":"api-key","name":"notes-api-key"}}

Challenge

In a copy, replace the loose "ingress": {"type": "object"} with a real rule: enabled is a boolean, hosts a list of strings, and if enabled is true, hosts must have at least one entry.

Hint: draft-07 supports if/then at object level; const checks a fixed value.

Solution

# on prod
cd ~/helm/tag08 && rm -rf notes && cp -r ~/helm/charts/notes .
jq '.properties.ingress = {
  "type": "object",
  "properties": {
    "enabled": {"type": "boolean"},
    "className": {"type": "string"},
    "hosts": {"type": "array", "items": {"type": "string"}}
  },
  "if": {"properties": {"enabled": {"const": true}}},
  "then": {"required": ["hosts"], "properties": {"hosts": {"minItems": 1}}}
}' notes/values.schema.json > s.json && mv s.json notes/values.schema.json
helm template notes ./notes --set apiKey.existingSecret=x --set ingress.enabled=true \
  2>&1 | grep 'at '
helm template notes ./notes --set apiKey.existingSecret=x --set ingress.enabled=yes \
  2>&1 | grep 'at '
helm template notes ./notes -f ~/helm/charts/values/prod.yaml \
  -s templates/ingress.yaml | grep -c host:

Expected:

- at '/ingress/hosts': minItems: got 0, want 1
- at '/ingress/enabled': got string, want boolean
2

With --set, yes is a string, not a boolean – the schema notices; the template with if .Values.ingress.enabled would have created the Ingress.

Save your progress? Sign up for free

Cleanup

# on prod
cd ~/helm/charts
git add -A
git commit -qm "Tag 08: Dateien, existingSecret, values.schema.json, notes 0.7.0"

Interview Note

  • How do you get a large configuration file into a chart? As a file next to templates/, read with .Files.Get or (.Files.Glob "…").AsConfig; if it needs values, add tpl. Files in templates/ or listed in .helmignore are invisible.
  • How do you handle secrets in a chart? The chart references, via existingSecret, a Secret that something else creates (External Secrets, SOPS, Sealed Secrets). No plain text in values – values end up in the release Secret.
  • What is values.schema.json for? It checks types, required fields, value ranges and typos before rendering, reports everything at once and documents the interface.

Enterprise Note

  • The schema is part of the chart API: adding a new required field or additionalProperties: false later is a breaking change (major version).
  • Editors with the YAML language server check values.yaml against the schema and complete keys (comment # yaml-language-server: $schema=…). helm-docs, by contrast, reads only values.yaml with its # -- comments, not the schema – you maintain both separately.
  • Air-gapped CI environments: no remote $ref in the schema, otherwise every render fails without internet access.
  • An open GET like the one in notes-api is a finding for the security review, not for the chart – but the chart can offer a NetworkPolicy or Ingress auth.