Day 6: Files, io, JSON and YAML – manifestlint
Goal: A linter that checks Kubernetes YAMLs for
:latest, missing resources and missing labels – as text or JSON, with clean exit codes · Duration: approx. 120 min · Prerequisite: Day 5
Briefing
- Decode YAML with
go.yaml.in/yaml/v3into slim structs – only the fields you check. - Several documents per file (
---) via a decoder loop untilio.EOF. - Directories with
filepath.WalkDir, stdin viaio.Reader– the same code for file and pipe. - Output with
encoding/jsonand struct tags; exit code 0/1/2 for CI. - A golden file (Day 4) safeguards the JSON format.
PATH... --> Files() --> LintFile() --> Decode() --> Check() --> text | json
"-" = stdin os.Open/io.Reader yaml rules Exit 0/1/2
Drill 1: Module and sample manifests
mkdir -p ~/go-lab/tag06-manifestlint && cd ~/go-lab/tag06-manifestlint
go mod init example.com/manifestlint
go get go.yaml.in/yaml/[email protected]
mkdir -p cmd/manifestlint/testdata internal/lint manifests/app manifests/.cacheExpected:
go: creating new go.mod: module example.com/manifestlint
go: added go.yaml.in/yaml/v3 v3.0.5
Why go.yaml.in? The old gopkg.in/yaml.v3 is
no longer maintained; the YAML team of the Go community continues it
under this path.
Two manifests with deliberate mistakes, plus a decoy in the hidden
.cache folder and a non-YAML file:
cat > manifests/app/web.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
labels: {app.kubernetes.io/name: web}
spec:
replicas: 2
selector: {matchLabels: {app.kubernetes.io/name: web}}
template:
metadata: {labels: {app.kubernetes.io/name: web}}
spec:
initContainers:
- name: migrate
image: registry.local:5000/migrate
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {memory: 64Mi}
containers:
- name: web
image: nginx:latest
resources:
requests: {cpu: 100m}
---
apiVersion: v1
kind: Service
metadata:
name: web
spec:
selector: {app.kubernetes.io/name: web}
ports: [{port: 80}]
EOFcat > manifests/backup.yml <<'EOF'
apiVersion: batch/v1
kind: CronJob
metadata:
name: backup
labels: {app.kubernetes.io/name: backup}
spec:
schedule: "0 3 * * *"
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: "ghcr.io/example/backup@sha256:\
0b8a3f6c2d1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a"
resources:
requests: {cpu: 1, memory: 256Mi}
limits: {memory: 256Mi}
EOFThe \ at the end of the line in the double-quoted image
string wraps the long digest without inserting a space.
echo "kind: Pod" > manifests/.cache/ignoriert.yaml
echo "# Doku" > manifests/README.mdSave your progress? Sign up for free
Drill 2: YAML into structs
A Kubernetes object has hundreds of fields – you declare only the
ones you need. The decoder ignores unknown fields. Depending on the
kind, the containers sit in three different places.
cat > internal/lint/manifest.go <<'EOF'
// Package lint checks Kubernetes manifests for typical production mistakes.
package lint
import (
"errors"
"fmt"
"io"
"go.yaml.in/yaml/v3"
)
// Manifest contains only the fields the rules need.
type Manifest struct {
Kind string `yaml:"kind"`
Metadata struct {
Name string `yaml:"name"`
Labels map[string]string `yaml:"labels"`
} `yaml:"metadata"`
Spec struct {
PodSpec `yaml:",inline"` // Pod
Template struct {
Spec PodSpec `yaml:"spec"`
} `yaml:"template"` // Deployment, StatefulSet, DaemonSet, Job
JobTemplate struct {
Spec struct {
Template struct {
Spec PodSpec `yaml:"spec"`
} `yaml:"template"`
} `yaml:"spec"`
} `yaml:"jobTemplate"` // CronJob
} `yaml:"spec"`
}
// PodSpec is the part where the containers live.
type PodSpec struct {
Containers []Container `yaml:"containers"`
InitContainers []Container `yaml:"initContainers"`
}
// Container with image and resources.
type Container struct {
Name string `yaml:"name"`
Image string `yaml:"image"`
Resources struct {
Requests map[string]string `yaml:"requests"`
Limits map[string]string `yaml:"limits"`
} `yaml:"resources"`
}
// Pod returns the PodSpec depending on the kind, otherwise nil.
func (m *Manifest) Pod() *PodSpec {
switch m.Kind {
case "Pod":
return &m.Spec.PodSpec
case "Deployment", "StatefulSet", "DaemonSet", "ReplicaSet", "Job":
return &m.Spec.Template.Spec
case "CronJob":
return &m.Spec.JobTemplate.Spec.Template.Spec
}
return nil
}
// Decode reads all YAML documents (separated by ---) from r.
func Decode(r io.Reader) ([]Manifest, error) {
dec := yaml.NewDecoder(r)
var out []Manifest
for i := 1; ; i++ {
var m Manifest
err := dec.Decode(&m)
if errors.Is(err, io.EOF) {
return out, nil
}
if err != nil {
return out, fmt.Errorf("%d. Dokument: %w", i, err)
}
if m.Kind != "" { // skip empty documents
out = append(out, m)
}
}
}
EOFWhy? yaml:",inline" puts the fields of
PodSpec directly under spec (Pod), while
Deployments have them under spec.template.spec.
errors.Is(err, io.EOF) is the regular end of the loop, not
an error.
Save your progress? Sign up for free
Drill 3: The rules
cat > internal/lint/rules.go <<'EOF'
package lint
import (
"fmt"
"slices"
"strings"
)
// Finding is a violation; the JSON tags determine the output with -o json.
type Finding struct {
File string `json:"file"`
Kind string `json:"kind"`
Name string `json:"name"`
Rule string `json:"rule"`
Message string `json:"message"`
}
// Check applies all rules to a manifest.
func Check(m Manifest, requiredLabels []string) []Finding {
var fs []Finding
add := func(rule, format string, args ...any) {
fs = append(fs, Finding{Kind: m.Kind, Name: m.Metadata.Name,
Rule: rule, Message: fmt.Sprintf(format, args...)})
}
for _, l := range requiredLabels {
if _, ok := m.Metadata.Labels[l]; !ok {
add("label", "%s fehlt", l)
}
}
pod := m.Pod()
if pod == nil {
return fs
}
for _, c := range slices.Concat(pod.InitContainers, pod.Containers) {
if floatingTag(c.Image) {
add("image-tag", "%s: %s", c.Name, c.Image)
}
for _, k := range []string{"cpu", "memory"} {
if c.Resources.Requests[k] == "" {
add("resources", "%s: requests.%s fehlt", c.Name, k)
}
}
if c.Resources.Limits["memory"] == "" {
add("resources", "%s: limits.memory fehlt", c.Name)
}
}
return fs
}
// floatingTag detects :latest and missing tags; digests are always fixed.
func floatingTag(image string) bool {
if strings.Contains(image, "@sha256:") {
return false
}
last := image[strings.LastIndex(image, "/")+1:] // ignore the registry port
name, tag, ok := strings.Cut(last, ":")
return name == "" || !ok || tag == "latest"
}
EOFgo vet ./...Expected: no output.
Why slices.Concat instead of
append(pod.InitContainers, pod.Containers...)?
append may write into the array of
InitContainers if there is spare capacity – a classic
aliasing bug. Concat always creates a new slice.
Save your progress? Sign up for free
Drill 4: Find and read files
filepath.WalkDir calls your function for every entry;
filepath.SkipDir skips an entire subtree.
LintFile works only with io.Reader – file or
stdin, the decoder doesn’t care.
cat > internal/lint/files.go <<'EOF'
package lint
import (
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
)
// Files collects manifests: named files always, in directories only
// *.yaml/*.yml. Hidden directories (.git) are skipped.
func Files(paths []string) ([]string, error) {
var out []string
for _, root := range paths {
if root == "-" { // stdin
out = append(out, root)
continue
}
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
switch {
case err != nil:
return err
case d.IsDir() && p != root && strings.HasPrefix(d.Name(), "."):
return filepath.SkipDir
case !d.IsDir() && (p == root || isYAML(p)):
out = append(out, p)
}
return nil
})
if err != nil {
return nil, err
}
}
return out, nil
}
func isYAML(p string) bool {
ext := filepath.Ext(p)
return ext == ".yaml" || ext == ".yml"
}
// LintFile reads a file (or stdin) and checks all documents in it.
func LintFile(path string, labels []string) ([]Finding, error) {
var r io.Reader = os.Stdin
if path != "-" {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
r = f
}
ms, err := Decode(r)
if err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
var out []Finding
for _, m := range ms {
for _, f := range Check(m, labels) {
f.File = path
out = append(out, f)
}
}
return out, nil
}
EOFWhy defer f.Close() right after the error check? That
way the file is closed on every return path – even if
Decode fails. For files opened read-only, the error from
Close is meaningless.
Save your progress? Sign up for free
Drill 5: CLI, JSON and exit codes
run instead of logic in main: arguments and
writers come in, an exit code goes out – which makes the whole CLI
testable. flag.Func builds a repeatable flag without a type
of its own.
cat > cmd/manifestlint/main.go <<'EOF'
// manifestlint checks Kubernetes YAMLs. Exit 0 = clean, 1 = findings, 2 = error.
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"os"
"example.com/manifestlint/internal/lint"
)
func main() {
os.Exit(run(os.Args[1:], os.Stdout, os.Stderr))
}
func run(args []string, stdout, stderr io.Writer) int {
fl := flag.NewFlagSet("manifestlint", flag.ContinueOnError)
fl.SetOutput(stderr)
format := fl.String("o", "text", "Ausgabe: text oder json")
var labels []string
fl.Func("label", "Pflicht-Label (mehrfach; Default app.kubernetes.io/name)",
func(s string) error { labels = append(labels, s); return nil })
if err := fl.Parse(args); err != nil {
return 2
}
if len(labels) == 0 {
labels = []string{"app.kubernetes.io/name"}
}
if fl.NArg() == 0 {
fmt.Fprintln(stderr, "Aufruf: manifestlint [-o json] [-label k] PFAD...")
return 2
}
files, err := lint.Files(fl.Args())
if err != nil {
fmt.Fprintln(stderr, "manifestlint:", err)
return 2
}
findings := []lint.Finding{} // empty instead of nil: JSON becomes [] instead of null
for _, f := range files {
fs, err := lint.LintFile(f, labels)
if err != nil {
fmt.Fprintln(stderr, "manifestlint:", err)
return 2
}
findings = append(findings, fs...)
}
switch *format {
case "json":
enc := json.NewEncoder(stdout)
enc.SetIndent("", " ")
if err := enc.Encode(findings); err != nil {
fmt.Fprintln(stderr, "manifestlint:", err)
return 2
}
case "text":
for _, f := range findings {
fmt.Fprintf(stdout, "%s %s/%s %s: %s\n",
f.File, f.Kind, f.Name, f.Rule, f.Message)
}
fmt.Fprintf(stdout, "%d Datei(en), %d Befund(e)\n", len(files), len(findings))
default:
fmt.Fprintf(stderr, "manifestlint: unbekanntes Format %q\n", *format)
return 2
}
if len(findings) > 0 {
return 1
}
return 0
}
EOFgo build -o manifestlint ./cmd/manifestlint
./manifestlint manifests; echo "Exit: $?"
./manifestlint -o json manifests/backup.yml; echo "Exit: $?"Expected:
manifests/app/web.yaml Deployment/web image-tag: migrate: registry.local:5000/migrate
manifests/app/web.yaml Deployment/web image-tag: web: nginx:latest
manifests/app/web.yaml Deployment/web resources: web: requests.memory fehlt
manifests/app/web.yaml Deployment/web resources: web: limits.memory fehlt
manifests/app/web.yaml Service/web label: app.kubernetes.io/name fehlt
2 Datei(en), 5 Befund(e)
Exit: 1
[]
Exit: 0
README.md and .cache/ignoriert.yaml don’t
show up, the registry port :5000 is not mistaken for a tag,
and the digest in the CronJob counts as fixed. cpu: 1 is a
number in YAML – the decoder still stores it as the string
"1". (In the output, “fehlt” means “missing” and
“Datei(en), Befund(e)” means “file(s), finding(s)”.)
Now the error paths:
./manifestlint; echo "Exit: $?"
./manifestlint -o xml manifests; echo "Exit: $?"
printf 'kind: Pod\n---\nmetadata: [kaputt\n' > manifests/kaputt.yaml
./manifestlint manifests; echo "Exit: $?"
rm manifests/kaputt.yamlExpected (shortened):
Aufruf: manifestlint [-o json] [-label k] PFAD...
Exit: 2
manifestlint: unbekanntes Format "xml"
Exit: 2
manifestlint: manifests/kaputt.yaml: 2. Dokument: yaml: line 2: did not find expected…
Exit: 2
Why three exit codes? A CI pipeline has to be able to tell “manifest bad” (1) from “linter broken” (2) – otherwise a syntax error gets clicked away as a matter of style.
Save your progress? Sign up for free
Drill 6: Unit tests with strings and TempDir
You test rules and decoder without files:
strings.NewReader is an io.Reader. A map as
the test table is enough when the order doesn’t matter.
cat > internal/lint/rules_test.go <<'EOF'
package lint
import (
"strings"
"testing"
)
func TestFloatingTag(t *testing.T) {
tests := map[string]bool{
"nginx": true,
"nginx:latest": true,
"nginx:1.27": false,
"registry.local:5000/app": true,
"registry.local:5000/app:1.0": false,
"ghcr.io/x/app@sha256:0b8a3f6c2": false,
}
for image, want := range tests {
t.Run(image, func(t *testing.T) {
if got := floatingTag(image); got != want {
t.Errorf("floatingTag(%q) = %v, want %v", image, got, want)
}
})
}
}
const multiDoc = `
kind: Pod
metadata: {name: ok, labels: {app.kubernetes.io/name: ok}}
spec:
containers:
- name: app
image: nginx:1.27
resources: {requests: {cpu: 1, memory: 1Gi}, limits: {memory: 1Gi}}
---
# leeres Dokument
---
kind: CronJob
metadata: {name: nightly}
spec:
jobTemplate: {spec: {template: {spec: {containers: [{name: job, image: x}]}}}}
`
func TestDecodeUndCheck(t *testing.T) {
ms, err := Decode(strings.NewReader(multiDoc))
if err != nil {
t.Fatal(err)
}
if len(ms) != 2 {
t.Fatalf("%d Manifeste, want 2", len(ms))
}
if fs := Check(ms[0], []string{"app.kubernetes.io/name"}); len(fs) != 0 {
t.Errorf("Pod: unerwartete Befunde %v", fs)
}
var rules []string
for _, f := range Check(ms[1], []string{"app.kubernetes.io/name"}) {
rules = append(rules, f.Rule)
}
want := "label image-tag resources resources resources"
if got := strings.Join(rules, " "); got != want {
t.Errorf("CronJob: Regeln %q, want %q", got, want)
}
}
func TestDecodeFehler(t *testing.T) {
_, err := Decode(strings.NewReader("kind: Pod\n---\nmetadata: [kaputt\n"))
if err == nil || !strings.Contains(err.Error(), "2. Dokument") {
t.Errorf("err = %v, want Hinweis auf 2. Dokument", err)
}
}
EOFThe file search gets a real but disposable directory tree:
cat > internal/lint/files_test.go <<'EOF'
package lint
import (
"os"
"path/filepath"
"slices"
"testing"
)
func TestFiles(t *testing.T) {
dir := t.TempDir()
for _, p := range []string{"a.yaml", "sub/b.yml", "README.md", ".git/c.yaml"} {
full := filepath.Join(dir, p)
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, nil, 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Files([]string{dir, filepath.Join(dir, "README.md"), "-"})
if err != nil {
t.Fatal(err)
}
want := []string{
filepath.Join(dir, "a.yaml"), filepath.Join(dir, "sub/b.yml"),
filepath.Join(dir, "README.md"), "-", // named explicitly: always included
}
if !slices.Equal(got, want) {
t.Errorf("Files =\n%v\nwant\n%v", got, want)
}
}
EOFgo test -count=1 -v ./internal/lint 2>&1 | grep -E '^(---|ok|FAIL)'Expected:
--- PASS: TestFiles (0.00s)
--- PASS: TestFloatingTag (0.00s)
--- PASS: TestDecodeUndCheck (0.00s)
--- PASS: TestDecodeFehler (0.00s)
ok example.com/manifestlint/internal/lint 0.244s
Save your progress? Sign up for free
Drill 7: Golden file for the JSON
The JSON is an interface – other tools parse it. A golden file (Day
4) reports every format change. The test input is a copy of
web.yaml in testdata/, which
go build ignores.
cp manifests/app/web.yaml cmd/manifestlint/testdata/web.yamlcat > cmd/manifestlint/main_test.go <<'EOF'
package main
import (
"bytes"
"flag"
"io"
"os"
"testing"
)
var update = flag.Bool("update", false, "Golden Files neu schreiben")
func TestJSONGolden(t *testing.T) {
var out bytes.Buffer
code := run([]string{"-o", "json", "testdata/web.yaml"}, &out, io.Discard)
if code != 1 {
t.Errorf("Exit-Code %d, want 1", code)
}
golden := "testdata/web.golden.json"
if *update {
if err := os.WriteFile(golden, out.Bytes(), 0o644); err != nil {
t.Fatal(err)
}
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(out.Bytes(), want) {
t.Errorf("Ausgabe weicht ab:\n%s", out.String())
}
}
func TestExitCodes(t *testing.T) {
tests := []struct {
name string
args []string
want int
}{
{"ohne Pfad", nil, 2},
{"fehlt", []string{"gibt-es-nicht.yaml"}, 2},
{"Format", []string{"-o", "xml", "testdata/web.yaml"}, 2},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := run(tt.args, io.Discard, io.Discard); got != tt.want {
t.Errorf("Exit-Code %d, want %d", got, tt.want)
}
})
}
}
EOFgo test ./cmd/manifestlint -update
head -8 cmd/manifestlint/testdata/web.golden.json
go test -race -count=1 ./...Expected:
ok example.com/manifestlint/cmd/manifestlint 0.143s
[
{
"file": "testdata/web.yaml",
"kind": "Deployment",
"name": "web",
"rule": "image-tag",
"message": "migrate: registry.local:5000/migrate"
ok example.com/manifestlint/cmd/manifestlint 1.151s
ok example.com/manifestlint/internal/lint 1.307s
Warning: After
-update, always readgit diff testdata/. A blindly updated golden file no longer tests anything.
Save your progress? Sign up for free
Drill 8: The tool is done
Take over .golangci.yml as on Day 5, then check
everything and hook the tool into a pipe.
kubectl create --dry-run=client generates YAML without any
cluster; KUBECONFIG=/dev/null makes sure kubectl doesn’t
touch any real configuration.
cp ../tag05-kctxsafe/.golangci.yml .
gofmt -l . && go vet ./... && ~/go-lab/bin/lint run ./...
go build -o manifestlint ./cmd/manifestlint
KUBECONFIG=/dev/null kubectl create deployment demo --image=nginx:1.27 \
--dry-run=client -o yaml | ./manifestlint -label team -
./manifestlint -o json manifests | jq -r '.[].rule' | sort | uniq -cExpected:
0 issues.
- Deployment/demo label: team fehlt
- Deployment/demo resources: nginx: requests.cpu fehlt
- Deployment/demo resources: nginx: requests.memory fehlt
- Deployment/demo resources: nginx: limits.memory fehlt
1 Datei(en), 4 Befund(e)
2 image-tag
1 label
2 resources
With -label team you replace the default label; for
both, you pass -label twice.
kubectl kustomize overlays/prod | ./manifestlint - or
helm template … | ./manifestlint - work the same way.
Save your progress? Sign up for free
Checkpoint
cd ~/go-lab/tag06-manifestlint
ok() { echo "✔ $1"; }; nok() { echo "✘ $1"; }
go vet ./... && go test -count=1 ./... >/dev/null && ok "vet + Tests" || nok "Tests"
./manifestlint manifests >/dev/null; [ $? -eq 1 ] && ok "Exit 1 bei Befunden" \
|| nok "Exit-Code"
./manifestlint manifests/backup.yml >/dev/null && ok "Digest ist sauber" \
|| nok "backup.yml"
./manifestlint -o json manifests | jq -e 'length == 5' >/dev/null \
&& ok "JSON: 5 Befunde" || nok "JSON"
./manifestlint gibt-es-nicht 2>/dev/null; [ $? -eq 2 ] && ok "Exit 2 bei Fehler" \
|| nok "Fehlerpfad"Expected:
✔ vet + Tests
✔ Exit 1 bei Befunden
✔ Digest ist sauber
✔ JSON: 5 Befunde
✔ Exit 2 bei Fehler
Challenge
New rule probe: every container of a Deployment or
StatefulSet needs a readinessProbe. Init containers are
exempt.
Hint: You don’t have to understand the content of
the probe – a field of type any is nil if the
key is missing.
Solution
In internal/lint/manifest.go, extend the
Container:
type Container struct {
Name string `yaml:"name"`
Image string `yaml:"image"`
ReadinessProbe any `yaml:"readinessProbe"` // only: present or not
Resources struct { // gofmt -w aligns the columns
In internal/lint/rules.go, before the last
return fs in Check:
if m.Kind == "Deployment" || m.Kind == "StatefulSet" {
for _, c := range pod.Containers { // init containers don't need one
if c.ReadinessProbe == nil {
add("probe", "%s: readinessProbe fehlt", c.Name)
}
}
}
gofmt -w . && go build -o manifestlint ./cmd/manifestlint
./manifestlint manifests | grep probe
go test ./... 2>&1 | tail -3
go test ./cmd/manifestlint -update && go test ./...Expected (shortened):
manifests/app/web.yaml Deployment/web probe: web: readinessProbe fehlt
FAIL example.com/manifestlint/cmd/manifestlint 0.145s
ok example.com/manifestlint/internal/lint 0.273s
FAIL
ok example.com/manifestlint/cmd/manifestlint 0.151s
ok example.com/manifestlint/internal/lint 0.180s
The golden file fires – exactly right, the JSON has changed. Rewrite
it with -update only after checking.
Save your progress? Sign up for free
Enterprise Note
- For real policy enforcement in the cluster: Kyverno, OPA Gatekeeper
or ValidatingAdmissionPolicy (CEL);
manifestlintis the quick check before that in the pipeline. - Established linters like kube-linter, kube-score or Polaris come with dozens of rules – a tool of your own pays off for in-house rules (team labels, allowed registries).
- For SARIF or JUnit output, one more
-oformat is enough; GitLab and GitHub then show findings directly in the merge request. - Large monorepos: check files in parallel (Day 8) and pass only
changed files from
git diff --name-only. - Exceptions need a process: an annotation with a reason and an expiry date instead of silently switching off a rule.