Day 3: Managers and Datasources
Goal: For every file in the repo, you know which manager reads it, which datasource supplies the versions, and what you need to do when Renovate overlooks a file. · Duration: approx. 90 min · Prerequisite: Day 2
Briefing
- A manager finds files via
managerFilePatternsand extracts dependencies from them (npm,gomod,dockerfile,helmv3, …). - A datasource supplies the available versions for a
dependency (
npm,go,docker,helm,github-tags, …). - A versioning decides what is greater, stable, or
“in range” (
semver,docker,helm, …). - Managers with a lockfile (
go.sum,package-lock.json,Chart.lock) need the matching tool (go,npm,helm) to update it. - The
kubernetesmanager has no default file patterns. Without config it sees nothing.
File Manager Datasource Versioning
go.mod ──► gomod ──► go (Proxy) ──► semver
values.yaml──► helm-values──► docker ──► docker
Chart.yaml ──► helmv3 ──► helm ──► helm
k8s/*.yaml ──► kubernetes ──► docker ──► docker (only with config!)
extracts supplies versions compares and sorts
Drill 1: Two new repos
Today shop-api is joined by shop-worker
(Go) and shop-deploy (Helm chart, Kubernetes manifests,
docker-compose). Both already come with a renovate.json
with config:recommended, so there is no onboarding PR. Each
repo is created like shop-api on Day 0: copy,
git init, commit, push – the loop only saves you
typing.
source ~/renovate-lab/setup/env.sh
export RENOVATE_GITHUB_COM_TOKEN=$(gh auth token) # optional, see Day 0
mkdir -p ~/renovate-lab/tag03 && cd ~/renovate-lab/tag03
for r in shop-worker shop-deploy; do
cp -R ~/renovate-lab/code/tag03/$r start-$r
git -C start-$r init -q -b main
git -C start-$r add -A
git -C start-$r commit -qm "Initialer Stand"
git -C start-$r push -q forgejo:shop/$r.git main
done
cat start-shop-worker/go.modExpected:
module example.com/shop/shop-worker
go 1.21
require (
github.com/google/uuid v1.3.0
github.com/joho/godotenv v1.4.0
github.com/pkg/errors v0.9.0
)
The interesting spots in shop-deploy (all files under
~/renovate-lab/code/tag03/shop-deploy):
# charts/shop/Chart.yaml (excerpt)
dependencies:
- name: podinfo
version: 6.5.0
repository: https://stefanprodan.github.io/podinfo
# charts/shop/values.yaml (excerpt)
image:
repository: nginx
tag: 1.25.3
# docker-compose.yml: postgres:15.4, redis:7.0.12
# k8s/deployment.yaml: image: busybox:1.36.0
# k8s/kustomization.yaml: images: [{name: memcached, newTag: 1.6.21}]
# examples/docker-compose.yml: mailhog/mailhog:v1.0.0go.sum is real, generated with go mod tidy.
Renovate has to update it with every update (Drill 7).
Save your progress? Sign up for free
Drill 2: Which manager finds what?
--dry-run=extract stops after reading the files. With
--report-type=file, Renovate also writes a JSON report that
you evaluate with jq.
cd ~/renovate-lab/tag03
renovate --dry-run=extract --report-type=file --report-path=extract.json \
shop/shop-api shop/shop-worker shop/shop-deploy > extract.log 2>&1
jq -c '.repositories[].packageFiles | keys' extract.jsonExpected:
R03_KEYS
Per repo, the managers that kicked in. Under each manager is the list of files it found:
jq -r '.repositories[].packageFiles[][].packageFile' extract.jsonExpected:
R03_FILES
Two files are missing: k8s/deployment.yaml (no pattern
for kubernetes, Drill 4) and
examples/docker-compose.yml (ignored, Drill 5). The default
patterns are in the source code of each manager, for example:
M=~/renovate-lab/tools/node_modules/renovate/dist/modules/manager
for m in kubernetes helm-values docker-compose kustomize; do
p=$(grep -o 'managerFilePatterns: .*' "$M/$m/index.js" | head -1)
printf "%-15s %s\n" "$m" "$p"
done | cut -c1-88Expected:
kubernetes managerFilePatterns: [] };
helm-values managerFilePatterns: ["/(^|/)k8s/.+\\.yaml$/"]
|/)values\\.ya?ml$/"],
docker-compose managerFilePatterns: ["/(^|/)(?:docker-)?compose[^/]*\\.ya?ml$/"] };
kustomize managerFilePatterns: ["/(^|/)kustomization\\.ya?ml$/"],
The complete list of all managers and their patterns is at docs.renovatebot.com/modules/manager/.
Tip:
helm-valuespicks up everyvalues.yamland recognizesimage:withrepository/tag(optionallyregistry). It does not see other structures; for those there are custom managers (Day 7).
Save your progress? Sign up for free
Drill 3: Datasources and versioning
--dry-run=lookup additionally queries the datasources,
but does not create branches.
renovate --dry-run=lookup --report-type=file --report-path=lookup.json \
shop/shop-api shop/shop-worker shop/shop-deploy > lookup.log 2>&1
jq -c '.repositories["shop/shop-api"].packageFiles.npm[0].deps[0]
| {depName, currentValue, updates: [.updates[].newVersion]}' lookup.jsonExpected:
{"depName":"express","currentValue":"4.18.2","updates":["4.22.3","5.3.0"]}
Every dependency carries the updates found for it. The same for all repos, one line per dependency:
jq -r '.repositories[].packageFiles[][].deps[]
| "\(.depName) \(.currentValue) \([.updates[]?.newVersion])"' lookup.jsonExpected (your target versions will be newer):
node 20.11.0-alpine ["20.20.2","26.11.1"]
actions/checkout v3 ["v7.0.1"]
actions/setup-node v3 ["v7.1.0"]
ubuntu latest []
node 20 ["24.21.0"]
express 4.18.2 ["4.22.3","5.3.0"]
lodash ^4.17.20 []
prettier 3.0.0 ["3.9.10"]
go 1.21 []
github.com/google/uuid v1.3.0 ["v1.6.0"]
github.com/joho/godotenv v1.4.0 ["v1.5.1"]
github.com/pkg/errors v0.9.0 ["v0.9.1"]
postgres 15.4 ["15.19","18.6"]
redis 7.0.12 ["7.4.11","8.10.2"]
nginx 1.25.3 ["1.31.6"]
podinfo 6.5.0 ["6.15.0"]
memcached 1.6.21 ["1.6.45"]
Two target versions, as with postgres, mean: one minor
PR and one major PR (Day 4). lodash gets none because
4.17.21 is already within the range ^4.17.20
(Day 6). Which datasource and which versioning each dependency has is in
the same entries; sort -u turns that into the list of
combinations:
jq -r '.repositories[].packageFiles[][].deps[] | "\(.datasource) \(.versioning)"' \
lookup.json | sort -uExpected:
R03_DSRC
Four managers (dockerfile, docker-compose,
helm-values, kustomize) share the
docker datasource. One manager can use several datasources:
github-actions queries the tags for uses:, and
for node-version the releases of
actions/node-versions. Why does the go
directive not get an update?
jq '.repositories["shop/shop-worker"].packageFiles.gomod[0].deps[0]
| {depName, currentValue, datasource, versioning, currentVersion, updates}' \
lookup.jsonExpected:
{
"depName": "go",
"currentValue": "1.21",
"datasource": "golang-version",
"versioning": "go-mod-directive",
"currentVersion": "1.27.2",
"updates": []
}The go-mod-directive versioning reads
go 1.21 as a minimum version (^1.21). Every
newer 1.x satisfies that, so there is nothing to replace. Day 6 shows
how to raise it anyway with rangeStrategy.
Save your progress? Sign up for free
Drill 4: kubernetes needs managerFilePatterns
From now on you change shop-deploy in a local clone and
push as lab-admin.
cd ~/renovate-lab/tag03
git clone -q forgejo:shop/shop-deploy.git
cd shop-deploy
cat > renovate.json <<'EOF'
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"kubernetes": {
"managerFilePatterns": ["/(^|/)k8s/.+\\.yaml$/"]
}
}
EOF
renovate-config-validator
git commit -qam "kubernetes-Manager für k8s/ einschalten"
git push -q
renovate --dry-run=lookup --report-type=file --report-path=../k8s.json \
shop/shop-deploy > ../k8s.log 2>&1
jq -c '.repositories[].packageFiles.kubernetes[].deps[]
| {depName, currentValue, datasource}' ../k8s.jsonExpected:
INFO: Validating renovate.json
INFO: Config validated successfully against 1 file(s)
R03_K8S
Between /…/ is a regex (re2); without slashes it is a
glob. In JSON, \. becomes \\.. The manager
reads only files with apiVersion and kind, and
besides images it also finds the API version (datasource
kubernetes-api). kustomization.yaml does match
the pattern, but gives the kubernetes manager nothing; that
is kustomize’s job.
Warning: In older Renovate versions the option was called
fileMatchand was always a regex without slashes. Old guides are full of it. The validator migrates it, but aborts with--strict.
mkdir -p ~/renovate-lab/tag03/alt && cd ~/renovate-lab/tag03/alt
cat > renovate.json <<'EOF'
{
"kubernetes": {
"fileMatch": ["(^|/)k8s/.+\\.yaml$"]
}
}
EOF
renovate-config-validator --strict > val.log 2>&1; echo "Exit: $?"
grep -A2 "migration necessary" val.log | cut -c1-88
cd ~/renovate-lab/tag03/shop-deployExpected:
Exit: 1
WARN: Config migration necessary
"oldConfig": {"kubernetes": {"fileMatch": ["(^|/)k8s/.+\\.yaml$"]}},
"newConfig": {"kubernetes": {"managerFilePatterns": ["/(^|/)k8s/.+\\.yaml$/"]}}
Save your progress? Sign up for free
Drill 5: ignorePaths
Why is examples/docker-compose.yml missing? The debug
log shows the effective ignorePaths.
LOG_LEVEL=debug renovate --dry-run=extract shop/shop-deploy 2>&1 \
| grep -A9 "Found repo ignorePaths"Expected:
DEBUG: Found repo ignorePaths (repository=shop/shop-deploy)
"ignorePaths": [
"**/node_modules/**",
"**/bower_components/**",
"**/vendor/**",
"**/examples/**",
"**/__tests__/**",
"**/test/**",
"**/tests/**",
"**/__fixtures__/**"
Renovate’s own default is only **/node_modules/** and
**/bower_components/**. The rest comes from
config:recommended through the preset
:ignoreModulesAndTests. ignorePaths is
not mergeable: your own list completely replaces the
inherited one. Try it out locally without pushing.
--platform=local reads the working directory including
unsaved changes (Day 2).
jq '.ignorePaths = ["charts/**"]' renovate.json > tmp.json
mv tmp.json renovate.json
renovate --platform=local --dry-run=extract --report-type=file \
--report-path=../local.json > ../local.log 2>&1
jq -r '.repositories[].packageFiles[][].packageFile' ../local.json
git checkout -q renovate.jsonExpected:
docker-compose.yml
examples/docker-compose.yml
k8s/deployment.yaml
k8s/kustomization.yaml
The chart is gone, but examples/ is back. If you want to
extend instead of replace, include the inherited entries in your
list.
Save your progress? Sign up for free
Drill 6: enabledManagers
enabledManagers is an allowlist: all other managers are
off.
jq '.enabledManagers = ["kubernetes", "kustomize"]' renovate.json > tmp.json
mv tmp.json renovate.json
renovate --platform=local --dry-run=extract --report-type=file \
--report-path=../local.json > ../local.log 2>&1
jq -r '.repositories[].packageFiles[][].packageFile' ../local.json
jq '.enabledManagers = ["helm", "kubernetes"]' renovate.json > tmp.json
mv tmp.json renovate.json
renovate-config-validator > ../val.log 2>&1; echo "Exit: $?"
grep -o 'enabledManagers are not supported: .*' ../val.log
git checkout -q renovate.jsonExpected:
k8s/deployment.yaml
k8s/kustomization.yaml
Exit: 1
enabledManagers are not supported: \"helm\""
There is no manager called helm; it is called
helmv3 (Chart.yaml) or helm-values.
enabledManagers does not magically turn on the
kubernetes manager either. Without
managerFilePatterns it stays blind.
Tip: You can turn off individual managers more precisely with
"<manager>": {"enabled": false}, for example"docker-compose": {"enabled": false}. That stays readable when Renovate gets new managers.
Save your progress? Sign up for free
Drill 7: Lockfile update with gomod
Now a real run against shop-worker. Renovate changes
go.mod, then calls go and commits the new
go.sum along with it.
cd ~/renovate-lab/tag03
LOG_LEVEL=debug renovate shop/shop-worker > worker.log 2>&1
grep -E "^ INFO: PR created|Falling back|\"command\"|updatedArtifacts" worker.log \
| cut -c1-80
curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/pulls?state=open" \
| jq -r '.[] | "#\(.number) \(.title)"'
curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/pulls/1.diff" \
| grep -E '^[-+][^-+]' | cut -c1-72Expected:
DEBUG: Falling back to binarySource=global (repository=shop/shop-worker, branch=
"command": "go get -t ./...",
"updatedArtifacts": ["go.sum"]
INFO: PR created (repository=shop/shop-worker, branch=renovate/github.com-pkg-e
DEBUG: Falling back to binarySource=global (repository=shop/shop-worker, branch=
"command": "go get -t ./...",
"updatedArtifacts": ["go.sum"]
INFO: PR created (repository=shop/shop-worker, branch=renovate/github.com-googl
#2 Update module github.com/google/uuid to v1.6.0
#1 Update module github.com/pkg/errors to v0.9.1
- github.com/pkg/errors v0.9.0
+ github.com/pkg/errors v0.9.1
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLG
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDR
Three observations:
- Only two PRs, although three updates are pending: that is
prHourlyLimit2 from the defaults (Day 4). Issue #3 is the Dependency Dashboard. binarySourcedefaults toinstall. Outside the official Renovate image (containerbase), Renovate falls back toglobaland uses thegofrom yourPATH, no matter which Go versiongo.modrequires.- Renovate calls
go get, notgo mod tidy. The old checksums forv0.9.0stay ingo.sum. Day 13 shows how to clean up withpostUpdateOptions: ["gomodTidy"].
Save your progress? Sign up for free
Drill 8: No tool, no lockfile
What happens if go is missing? A placeholder script in
front of the real go in your PATH simulates
that. --pr-hourly-limit=0 lifts the limit for this one
run.
mkdir -p nogo
cat > nogo/go <<'EOF'
#!/bin/sh
echo "go: command not found (simuliert)" >&2
exit 127
EOF
chmod +x nogo/go
PATH="$PWD/nogo:$PATH" renovate --pr-hourly-limit=0 shop/shop-worker 2>&1 \
| grep -A1 -E "PR created|WARN" | cut -c1-88
PR=$(curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/pulls?state=open" \
| jq '[.[] | select(.title | test("godotenv"))][0].number'); echo "PR #$PR"
SHA=$(curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/pulls/$PR" \
| jq -r .head.sha)
curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/commits/$SHA/status" \
| jq -r '.statuses[] | "\(.context): \(.status)"'
curl -s -u "$ADMIN" "$FORGEJO/api/v1/repos/shop/shop-worker/issues/$PR/comments" \
| jq -r '.[0].body' | head -1Expected:
INFO: PR created (repository=shop/shop-worker, branch=renovate/github.com-joho-godotenv
"pr": 4,
--
WARN: artifactErrors (repository=shop/shop-worker, branch=renovate/github.com-joho-godo
"artifactErrors": [{"fileName": "go.sum", "stderr": "go: command not found (simul
PR #4
renovate/artifacts: failure
### ⚠️ Artifact update problem
The PR exists anyway, but with a red renovate/artifacts
status and a comment with the error message. Renovate retries only on
certain occasions, for example when the PR title starts with
rebase!:
curl -s -u "$ADMIN" -X PATCH -H 'Content-Type: application/json' \
-d '{"title":"rebase!"}' "$FORGEJO/api/v1/repos/shop/shop-worker/pulls/$PR" \
| jq -r .title
renovate shop/shop-worker 2>&1 | grep -E "Branch updated|PR updated" | cut -c1-60
R="$FORGEJO/api/v1/repos/shop/shop-worker"
curl -s -u "$ADMIN" "$R/pulls/$PR" | jq -r .title
curl -s -u "$ADMIN" "$R/raw/go.sum?ref=renovate/github.com-joho-godotenv-1.x" \
| grep -c 'godotenv v1.5.1'
curl -s -u "$ADMIN" "$R/issues/$PR/comments" | jq lengthExpected:
rebase!
INFO: Branch updated (repository=shop/shop-worker, branch=r
INFO: PR updated (repository=shop/shop-worker, branch=renov
Update module github.com/joho/godotenv to v1.5.1
2
0
Title repaired, go.sum contains two new lines, and the
error comment is gone. Which managers need a tool at all:
| Manager | Artifact | Tool |
|---|---|---|
npm |
package-lock.json |
npm (with Node) |
gomod |
go.sum, possibly vendor/ |
go |
helmv3 |
Chart.lock, only if present |
helm |
kustomize |
Helm charts, only with kustomizeInflateHelmCharts |
helm |
dockerfile, docker-compose,
kubernetes, helm-values,
github-actions |
none | none |
binarySource is a global option (Day 9):
global uses the PATH, install
installs the matching version on demand in the official image, and
docker starts sidecar containers.
Save your progress? Sign up for free
Checkpoint
go is in your PATH, and
shop-deploy has a pattern for the kubernetes
manager:
cd ~/renovate-lab/tag03
R="$FORGEJO/api/v1/repos/shop"
command -v go
curl -s -u "$ADMIN" "$R/shop-deploy/raw/renovate.json" \
| jq -c .kubernetes.managerFilePatternsExpected:
R03_KP1
PR #1 in shop-worker also updates
go.sum:
curl -s -u "$ADMIN" "$R/shop-worker/pulls/1/files" | jq -r '.[].filename'Expected:
go.mod
go.sum
PR #4 has no failed status (artifact error):
SHA=$(curl -s -u "$ADMIN" "$R/shop-worker/pulls/4" | jq -r .head.sha)
curl -s -u "$ADMIN" "$R/shop-worker/commits/$SHA/status" | jq -r .stateExpected:
R03_KP2
Challenge
From now on, the platform team stores CronJobs under
deploy/manifests/*.yml (extension .yml!). Copy
cronjob.yml from the companion code there and make sure
Renovate finds the image alpine:3.18.4 without losing the
existing pattern for k8s/.
Hint: managerFilePatterns takes several
entries, regex and glob mixed.
Solution
Check the starting point:
cd ~/renovate-lab/tag03/shop-deploy
mkdir -p deploy/manifests
cp ~/renovate-lab/code/tag03/challenge/deploy/manifests/cronjob.yml deploy/manifests/
renovate --platform=local --dry-run=extract --report-type=file \
--report-path=../local.json > ../local.log 2>&1
jq -r '.repositories[].packageFiles[][].packageFile' ../local.json | grep -c manifestsExpected: no match.
0
Add a second pattern, this time as a glob:
cat > renovate.json <<'EOF'
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"kubernetes": {
"managerFilePatterns": [
"/(^|/)k8s/.+\\.yaml$/",
"deploy/manifests/*.yml"
]
}
}
EOF
renovate-config-validator
git add -A
git commit -qm "CronJob in deploy/manifests erfassen"
git push -q
renovate --dry-run=lookup --report-type=file --report-path=../ch.json \
shop/shop-deploy > ../ch.log 2>&1
jq -c '.repositories[].packageFiles.kubernetes[].deps[]
| {depName, currentValue, datasource}' ../ch.jsonExpected:
INFO: Validating renovate.json
INFO: Config validated successfully against 1 file(s)
R03_CH
A pattern "/\\.ya?ml$/" for everything would be more
convenient, but would make the manager parse every YAML file, including
Helm templates with {{ … }}. Keep the patterns narrow.
Save your progress? Sign up for free
Enterprise Note
- For Kubernetes, Kustomize, and Argo CD repos, create an org preset
with
managerFilePatterns(Day 11) instead of copying the patterns into every repo. - After onboarding, check with
--dry-run=extractwhether really all files are detected. Otherwise nobody notices an overlooked manifest folder. - Run Renovate in the official image with
binarySource=install. Then the Go, Node, or Helm version matches the project instead of the runner. - Your own
ignorePathsreplace the inherited list. Deliberately addvendor/,test/, and so on back in, otherwise Renovate opens PRs against test data. enabledManagersis suitable for pilot phases (“Docker images only at first”). Keep the list in the preset so that expanding it remains a central step.