Skip to content
Hands-on IT IT Drills

Day 13: Automation with REST, kcadm.sh and Terraform

Goal: Manage Keycloak without the console: with a service account via the Admin REST API, and declaratively with Terraform – including drift and importing existing objects. · Duration: approx. 90 min · Prerequisite: Day 12

Briefing

  • Everything the admin console can do is available through the Admin REST API under /admin/realms/<realm>/…. The console itself is just a client of this API.
  • Automation does not log in as a human, but as the service account of a client (client credentials grant). What it may do is determined by client roles of realm-management: view-users, manage-users, manage-clients, manage-realm …
  • kcadm.sh is a convenient wrapper around this API – good for scripts and troubleshooting.
  • Terraform describes the desired state. plan shows the difference to the actual state, apply establishes it. Manual changes (drift) show up in the next plan.
 curl / kcadm.sh / Terraform ─Client Credentials─► Token (automation, realm-management)
        └──────────────── Admin REST API /admin/realms/lab/… ──────────────────┘

Drill 1: A Service Account

source ~/kc-lab/setup/env.sh
mkdir -p ~/kc-lab/tag13
kclogin
kcadm create clients -r lab -s clientId=automation -s secret=lab-client-secret-123 \
  -s serviceAccountsEnabled=true -s standardFlowEnabled=false \
  -s directAccessGrantsEnabled=false
kcadm add-roles -r lab --uusername service-account-automation \
  --cclientid realm-management --rolename view-users --rolename view-realm
bot() { curl -s -d grant_type=client_credentials -d client_id=automation \
  -d client_secret=lab-client-secret-123 \
  $KC_URL/realms/lab/protocol/openid-connect/token | jq -r .access_token; }
jwt "$(bot)" | jq -c '.resource_access["realm-management"]'

Expected:

Logging into http://localhost:8080 as user kcadmin of realm master
Created new client with id '…'
{"roles":["view-realm","view-users","query-groups","query-users"]}

The client has no login flow, only a service account. Keycloak creates the corresponding user itself: service-account-automation. view-users brings query-users and query-groups along as a composite.

Save your progress? Sign up for free

Drill 2: The Admin REST API with curl

A=$KC_URL/admin/realms/lab
T=$(bot)
curl -s -H "Authorization: Bearer $T" "$A/users?username=anna&exact=true" \
  | jq -c '.[] | {username, email}'
curl -s -H "Authorization: Bearer $T" "$A/users/count"; echo
RITA='{"username":"rita","enabled":true,"email":"[email protected]","emailVerified":true,
  "firstName":"Rita","lastName":"Rest"}'
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $T" \
  -H 'Content-Type: application/json' -d "$RITA" $A/users

Expected:

{"username":"anna","email":"[email protected]"}
10
403

The bot may read, but not create. We give it manage-users – and then need a new token, because roles are in the token:

kcadm add-roles -r lab --uusername service-account-automation \
  --cclientid realm-management --rolename manage-users
T=$(bot)
curl -s -D - -o /dev/null -H "Authorization: Bearer $T" \
  -H 'Content-Type: application/json' -d "$RITA" $A/users | grep -i -E '^HTTP|^location'
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $T" \
  -H 'Content-Type: application/json' -d "$RITA" $A/users

Expected:

HTTP/2 201 
location: https://kc.192.168.64.5.sslip.io/admin/realms/lab/users/…
409
  • 201 Created returns no body; the ID is in the Location header.
  • The second attempt returns 409 Conflict: rita already exists. A script that may run twice checks first or treats 409 as “already done”.
  • kcadm does the same: kcadm create users sends exactly this POST.

Save your progress? Sign up for free

Drill 3: Setting Up Terraform

sudo apt-get install -y unzip >/dev/null
V=1.17.1; ARCH=$(dpkg --print-architecture)
cd ~/kc-lab/tag13
U=https://releases.hashicorp.com/terraform/${V}
curl -sSLO $U/terraform_${V}_linux_${ARCH}.zip
curl -sSL $U/terraform_${V}_SHA256SUMS \
  | grep "linux_${ARCH}.zip" | sha256sum -c
sudo unzip -o -q terraform_${V}_linux_${ARCH}.zip terraform -d /usr/local/bin
terraform version | head -1

Expected:

terraform_1.17.1_linux_amd64.zip: OK
Terraform v1.17.1

On a Mac with Apple silicon, it says arm64 there. Terraform manages roles, clients and users – for that, the bot needs more rights:

kcadm add-roles -r lab --uusername service-account-automation \
  --cclientid realm-management --rolename manage-clients --rolename manage-realm
cp -R ~/kc-lab/code/terraform ~/kc-lab/terraform && cd ~/kc-lab/terraform
ls
sed -n '/^provider/,/^}/p' versions.tf

Expected:

main.tf
variables.tf
versions.tf
provider "keycloak" {
  url           = var.kc_url
  realm         = "lab"
  client_id     = "automation"
  client_secret = var.client_secret
}

The provider logs in as automation, just like our curl. The address and the secret come from variables, so the secret is not in the code. Terraform reads them from environment variables with the prefix TF_VAR_:

export TF_VAR_kc_url=$KC_URL TF_VAR_client_secret=lab-client-secret-123
terraform init -no-color | grep -E 'Installed|successfully'

Expected:

- Installed keycloak/keycloak v5.10.0 (self-signed, key ID A63FAED146E27E2A)
Terraform has been successfully initialized!

Save your progress? Sign up for free

Drill 4: Plan and Apply

main.tf describes reports for notes: a role notes-berichte (notes reports), a group berichte with this role, a client notes-reports and the user theo in the group.

grep -E '^(resource|data)' main.tf
terraform plan -no-color | grep -E 'will be created|Plan:'

Expected:

data "keycloak_realm" "lab" {
resource "keycloak_role" "berichte" {
resource "keycloak_group" "berichte" {
resource "keycloak_group_roles" "berichte" {
resource "keycloak_openid_client" "reports" {
resource "keycloak_user" "theo" {
resource "keycloak_user_groups" "theo" {
  # keycloak_group.berichte will be created
  # keycloak_group_roles.berichte will be created
  # keycloak_openid_client.reports will be created
  # keycloak_role.berichte will be created
  # keycloak_user.theo will be created
  # keycloak_user_groups.theo will be created
Plan: 6 to add, 0 to change, 0 to destroy.
terraform apply -auto-approve -no-color | grep -E 'Apply complete'
terraform plan -no-color | tail -2
jwt "$(token theo lab-user-pw-theo)" \
  | jq -c '[.realm_access.roles[] | select(startswith("notes"))]'

Expected:

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.
["notes-berichte"]

Terraform sorted out the dependencies itself: first role and group, then the mapping. The second plan finds nothing left to do. What Terraform manages is recorded in the file terraform.tfstate – including secrets and passwords in plain text. In projects, the state belongs in a protected backend, never in Git.

Save your progress? Sign up for free

Drill 5: Someone Changes Something by Hand

RC=$(terraform output -raw reports_client_uuid)
kcadm update clients/$RC -r lab -s 'name=von Hand geändert'
terraform plan -no-color | grep -E '~ name|will be updated|Plan:'
terraform apply -auto-approve -no-color | grep -E 'Apply complete'
kcadm get clients/$RC -r lab | jq -r .name

Expected:

  # keycloak_openid_client.reports will be updated in-place
      ~ name                                           = "von Hand geändert" -> "notes…
Plan: 0 to add, 1 to change, 0 to destroy.
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
notes Berichte

The code wins (“von Hand geändert”, changed by hand, is gone). Whoever manages Keycloak with Terraform no longer changes these objects in the console – or brings the change into the code.

Save your progress? Sign up for free

Drill 6: Taking Over Existing Objects

The group leser from Day 2 already exists. Terraform can take it over instead of creating it anew. An import block names the ID, and -generate-config-out writes the matching code right away:

GID=$(kcadm get groups -r lab -q search=leser \
  | jq -r '.[] | select(.name=="leser") | .id')
printf 'import {\n  to = keycloak_group.leser\n  id = "lab/%s"\n}\n' "$GID" > import.tf
terraform plan -no-color -generate-config-out=leser.tf \
  | grep -E 'will be imported|Plan:'
grep -v '^#' leser.tf
terraform apply -auto-approve -no-color | grep -E 'Apply complete'
terraform state list | grep leser

Expected:

  # keycloak_group.leser will be imported
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
resource "keycloak_group" "leser" {
  attributes      = {}
  description     = null
  name            = "leser"
  organization_id = null
  parent_id       = null
  realm_id        = "lab"
}
Apply complete! Resources: 1 imported, 0 added, 0 changed, 0 destroyed.
keycloak_group.leser

For the Keycloak provider, the import ID has the form <realm>/<id>. You clean up the generated code afterwards, for example realm_id to data.keycloak_realm.lab.id.

Save your progress? Sign up for free

Checkpoint

terraform state list | wc -l

Expected: data source, six resources and the imported group.

8
terraform plan -no-color -detailed-exitcode >/dev/null; echo "exit=$?"

Expected: Code and Keycloak match (0 = no changes).

exit=0
kcadm get users -r lab -q username=rita -q exact=true | jq -r '.[0].email'

Expected:

[email protected]

Challenge

The client notes-reports should fetch reports without a user: Its service account needs the role notes-berichte. Solve it with Terraform and show the role in the token.

Hint: The resource is called keycloak_openid_client_service_account_realm_role; keycloak_openid_client.reports provides the ID of the service account user.

Solution

cat > challenge.tf <<'EOF'
resource "keycloak_openid_client_service_account_realm_role" "reports" {
  realm_id                = data.keycloak_realm.lab.id
  service_account_user_id = keycloak_openid_client.reports.service_account_user_id
  role                    = keycloak_role.berichte.name
}
EOF
terraform apply -auto-approve -no-color | grep -E 'Apply complete'
T=$(curl -s -d grant_type=client_credentials -d client_id=notes-reports \
  -d client_secret=lab-client-secret-123 \
  $KC_URL/realms/lab/protocol/openid-connect/token | jq -r .access_token)
jwt "$T" | jq -c '[.realm_access.roles[] | select(startswith("notes"))]'

Expected:

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
["notes-berichte"]

Save your progress? Sign up for free

Enterprise Note

  • Least privilege: A dedicated automation client per realm and purpose, with only the roles it needs. manage-realm is almost admin.
  • Creating realms requires an account in the realm master (role create-realm). For whole environments, Terraform gets a service account of its own there.
  • State: a remote backend with encryption and locking (S3, GitLab, Terraform Cloud). Better to read secrets from a vault than to write them into .tf files.
  • Alternatives: keycloak-config-cli applies realm JSON idempotently; the operator (Day 14) can create realms via KeycloakRealmImport, but not maintain them.
  • OpenTofu understands the same configuration and the same provider.