Day 14: Practice – an OpenBao policy operator
Goal: In one day you build a real operator that manages OpenBao policies from Kubernetes objects – with CEL, a finalizer, drift detection, events, a metric, and an envtest suite against a real
bao server -dev· Duration: approx. 180 min · Prerequisite: Day 13
Briefing
BaoPolicydescribes an ACL policy: as HCL text (spec.policy) or as rules (spec.rules[]). The CEL rule enforces exactly one.- The operator writes the policy to OpenBao as
k8s.<namespace>.<name>. A periodic resync detects drift and restores the desired state. - Address and token come from the Secret
bao-operator-tokenin the operator namespace. The operator watches this Secret. - The finalizer deletes the policy in OpenBao before the object disappears.
- Everything from Days 3–13 in one project: markers, status, events, watches, metrics, tests.
kubectl apply Reconcile OpenBao (dev, :18200)
BaoPolicy -----------> 1. set finalizer
spec.rules / policy 2. read Secret bao-operator-token
3. GetPolicy k8s.<ns>.<name> <---- sys/policies/acl/...
4. missing/different -> PutPolicy -> (created/updated/drift)
5. status + event + metric
6. RequeueAfter resync-interval (drift detection)
kubectl delete -----> finalizer: DeletePolicy --------> policy gone
Drill 1: Project and OpenBao client
The operator stands on its own – a new project next to the
site-operator.
export PATH=~/op-lab/bin:$PATH GOTOOLCHAIN=go1.26.9
kubebuilder version | head -1
bao version
mkdir -p ~/op-lab/tag14-bao-operator && cd ~/op-lab/tag14-bao-operator
kubebuilder init --domain lab.example --repo example.com/bao-operator
kubebuilder create api --group bao --version v1alpha1 --kind BaoPolicy \
--resource --controller
go get github.com/openbao/openbao/api/[email protected]Expected (abridged):
KubeBuilder: v4.16.0
OpenBao v2.5.4 (4f6d47246a053375271a5fd8af85c3b75695aa46), built 2026-05-20T15:53:48Z
...
Next: define a resource with:
$ kubebuilder create api
...
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests
go: added github.com/openbao/openbao/api/v2 v2.7.1
The client’s Go module is called
github.com/openbao/openbao/api/v2 – the package inside it
is simply called api. As always, the envtest assets come
from ~/op-lab/envtest:
make envtest
export KUBEBUILDER_ASSETS=$(bin/setup-envtest use 1.37.x \
--bin-dir ~/op-lab/envtest -p path)
echo $KUBEBUILDER_ASSETS
grep -E "controller-runtime |openbao" go.modExpected:
/Users/<du>/op-lab/envtest/k8s/1.37.0-darwin-arm64
github.com/openbao/openbao/api/v2 v2.7.1
sigs.k8s.io/controller-runtime v0.25.0
Save your progress? Sign up for free
Drill 2: The API with a CEL rule
has(self.policy) != has(self.rules) is only true if
exactly one of the fields is set. Capability as an enum
keeps typos like lesen out of the API in the first
place.
cat > api/v1alpha1/baopolicy_types.go <<'EOF'
package v1alpha1
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
)
// Capability is an OpenBao capability.
// +kubebuilder:validation:Enum=create;read;update;patch;delete;list;sudo;deny
type Capability string
// PolicyRule describes one path block of the policy.
type PolicyRule struct {
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=512
Path string `json:"path"`
// +kubebuilder:validation:MinItems=1
// +listType=set
Capabilities []Capability `json:"capabilities"`
}
// BaoPolicySpec: exactly one of policy (HCL) or rules.
// +kubebuilder:validation:XValidation:rule="has(self.policy) != has(self.rules)"
type BaoPolicySpec struct {
// HCL text of the policy.
// +optional
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=65536
Policy string `json:"policy,omitempty"`
// Structured rules; the operator renders HCL from them.
// +optional
// +kubebuilder:validation:MinItems=1
// +kubebuilder:validation:MaxItems=64
// +listType=atomic
Rules []PolicyRule `json:"rules,omitempty"`
}
// BaoPolicyStatus is the observed state.
type BaoPolicyStatus struct {
// Name of the policy in OpenBao: k8s.<namespace>.<name>
// +optional
PolicyName string `json:"policyName,omitempty"`
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// +listType=map
// +listMapKey=type
// +optional
Conditions []metav1.Condition `json:"conditions,omitempty"`
}
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:resource:shortName=bp
// +kubebuilder:printcolumn:name="Policy",type=string,JSONPath=`.status.policyName`
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
// BaoPolicy manages an ACL policy in OpenBao.
type BaoPolicy struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitzero"`
Spec BaoPolicySpec `json:"spec"`
Status BaoPolicyStatus `json:"status,omitzero"`
}
// +kubebuilder:object:root=true
// BaoPolicyList contains a list of BaoPolicy.
type BaoPolicyList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitzero"`
Items []BaoPolicy `json:"items"`
}
func init() {
SchemeBuilder.Register(func(s *runtime.Scheme) error {
s.AddKnownTypes(SchemeGroupVersion, &BaoPolicy{}, &BaoPolicyList{})
return nil
})
}
EOF
make manifests generate
grep -A1 "x-kubernetes-validations" config/crd/bases/bao.lab.example_baopolicies.yamlExpected:
x-kubernetes-validations:
- rule: has(self.policy) != has(self.rules)
Why no message? The marker only fits on one line; with a
message it becomes unreadably long. In return, the API error message
quotes the rule itself (Drill 8).
Save your progress? Sign up for free
Drill 3: From rules to HCL – the unit test first
Rendering is pure logic without a cluster – so a classic table test.
strconv.Quote makes sure that a path containing
" cannot smuggle in a second HCL statement.
cat > internal/controller/render.go <<'EOF'
package controller
import (
"fmt"
"strconv"
"strings"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
// policyName is unique: namespaces contain no dots.
func policyName(bp *baov1alpha1.BaoPolicy) string {
return "k8s." + bp.Namespace + "." + bp.Name
}
// desiredPolicy returns the HCL that should be in OpenBao.
func desiredPolicy(spec baov1alpha1.BaoPolicySpec) string {
if spec.Policy != "" {
return spec.Policy
}
return renderRules(spec.Rules)
}
// renderRules produces deterministic HCL; Quote prevents HCL injection.
func renderRules(rules []baov1alpha1.PolicyRule) string {
var b strings.Builder
for i, r := range rules {
if i > 0 {
b.WriteString("\n")
}
caps := make([]string, len(r.Capabilities))
for j, c := range r.Capabilities {
caps[j] = strconv.Quote(string(c))
}
fmt.Fprintf(&b, "path %s {\n capabilities = [%s]\n}\n",
strconv.Quote(r.Path), strings.Join(caps, ", "))
}
return b.String()
}
EOF
cat > internal/controller/render_test.go <<'EOF'
package controller
import (
"testing"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
const capRead baov1alpha1.Capability = "read"
func TestRenderRules(t *testing.T) {
tests := []struct {
name string
rules []baov1alpha1.PolicyRule
want string
}{
{"eine Regel", []baov1alpha1.PolicyRule{
{Path: "secret/data/app/*", Capabilities: []baov1alpha1.Capability{
capRead, "list"}},
}, "path \"secret/data/app/*\" {\n capabilities = [\"read\", \"list\"]\n}\n"},
{"Anführungszeichen werden escaped", []baov1alpha1.PolicyRule{
{Path: `a" {`, Capabilities: []baov1alpha1.Capability{"deny"}},
}, "path \"a\\\" {\" {\n capabilities = [\"deny\"]\n}\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := renderRules(tt.rules); got != tt.want {
t.Errorf("renderRules() =\n%q\nwant\n%q", got, tt.want)
}
})
}
}
EOF
go test ./internal/controller/ -run TestRenderRules -vExpected (abridged):
--- PASS: TestRenderRules (0.00s)
--- PASS: TestRenderRules/eine_Regel (0.00s)
--- PASS: TestRenderRules/Anführungszeichen_werden_escaped (0.00s)
PASS
ok example.com/bao-operator/internal/controller 0.366s
Why k8s.<ns>.<name>? Namespaces are DNS
labels without dots – the first dot after the namespace separates
unambiguously. And the k8s. prefix allows a narrowly scoped
policy for the operator token (Drill 8).
Save your progress? Sign up for free
Drill 4: Token Secret and metric
The client is built fresh from the Secret on every reconcile. A rotated token therefore takes effect immediately, without a restart.
cat > internal/controller/bao.go <<'EOF'
package controller
import (
"context"
"fmt"
"time"
"github.com/openbao/openbao/api/v2"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
)
// TokenSecretName lives in the operator namespace, keys: address, token.
const TokenSecretName = "bao-operator-token"
// baoClient builds a client from the Secret on every reconcile –
// this way a rotated token takes effect without a restart.
func (r *BaoPolicyReconciler) baoClient(ctx context.Context) (*api.Client, error) {
var s corev1.Secret
key := types.NamespacedName{Namespace: r.OperatorNamespace, Name: TokenSecretName}
if err := r.Get(ctx, key, &s); err != nil {
return nil, fmt.Errorf("secret %s: %w", key, err)
}
addr, token := string(s.Data["address"]), string(s.Data["token"])
if addr == "" || token == "" {
return nil, fmt.Errorf("secret %s: address oder token fehlt", key)
}
cfg := api.DefaultConfig()
cfg.Address = addr
cfg.Timeout = 10 * time.Second
c, err := api.NewClient(cfg)
if err != nil {
return nil, err
}
c.SetToken(token)
return c, nil
}
EOF
cat > internal/controller/metrics.go <<'EOF'
package controller
import (
"github.com/prometheus/client_golang/prometheus"
"sigs.k8s.io/controller-runtime/pkg/metrics"
)
// resultError is the label for failed syncs.
const resultError = "error"
// policySyncTotal counts syncs by result:
// created, updated, drift, unchanged, deleted, error.
var policySyncTotal = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "bao_operator_policy_sync_total",
Help: "Anzahl der Policy-Syncs gegen OpenBao nach Ergebnis.",
}, []string{"result"})
func init() {
metrics.Registry.MustRegister(policySyncTotal)
}
EOFExpected: no output. The build only happens at the end of Drill 5 –
bao.go needs the reconciler’s fields.
Security: The token never ends up in the log, the status, or an event. Error messages only name the Secret.
Save your progress? Sign up for free
Drill 5: The reconciler
Reconcile itself stays short: finalizer, sync, status, requeue.
RequeueAfter is the drift detection – OpenBao doesn’t
report changes, so we check regularly.
cat > internal/controller/baopolicy_controller.go <<'EOF'
package controller
import (
"context"
"time"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/equality"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/tools/events"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/handler"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
const finalizerName = "bao.lab.example/finalizer"
// BaoPolicyReconciler synchronizes BaoPolicy objects to OpenBao.
type BaoPolicyReconciler struct {
client.Client
Scheme *runtime.Scheme
Recorder events.EventRecorder
OperatorNamespace string
ResyncInterval time.Duration
}
// +kubebuilder:rbac:groups=bao.lab.example,resources=baopolicies,verbs=get;list
// +kubebuilder:rbac:groups=bao.lab.example,resources=baopolicies,verbs=watch;update
// +kubebuilder:rbac:groups=bao.lab.example,resources=baopolicies/status,verbs=update
// +kubebuilder:rbac:groups="",resources=secrets,verbs=get;list;watch
// +kubebuilder:rbac:groups=events.k8s.io,resources=events,verbs=create;patch
func (r *BaoPolicyReconciler) Reconcile(
ctx context.Context, req ctrl.Request,
) (ctrl.Result, error) {
log := logf.FromContext(ctx)
var bp baov1alpha1.BaoPolicy
if err := r.Get(ctx, req.NamespacedName, &bp); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
name := policyName(&bp)
if !bp.DeletionTimestamp.IsZero() {
return ctrl.Result{}, r.finalize(ctx, &bp, name)
}
if controllerutil.AddFinalizer(&bp, finalizerName) {
if err := r.Update(ctx, &bp); err != nil {
return ctrl.Result{}, err
}
}
before := bp.Status.DeepCopy()
bp.Status.PolicyName = name
result, reason, err := r.sync(ctx, &bp, name)
policySyncTotal.WithLabelValues(result).Inc()
if err != nil {
log.Error(err, "Sync fehlgeschlagen", "policy", name)
r.setConditions(&bp, metav1.ConditionFalse, reason, err.Error())
r.Recorder.Eventf(&bp, nil, corev1.EventTypeWarning, reason,
"SyncPolicy", "%v", err)
} else {
bp.Status.ObservedGeneration = bp.Generation
r.setConditions(&bp, metav1.ConditionTrue, reason, "Policy "+name+" aktiv")
}
if !equality.Semantic.DeepEqual(before, &bp.Status) {
if uerr := r.Status().Update(ctx, &bp); uerr != nil {
return ctrl.Result{}, uerr
}
}
if err != nil {
return ctrl.Result{}, err
}
log.V(1).Info("Policy synchronisiert", "policy", name, "result", result)
return ctrl.Result{RequeueAfter: r.ResyncInterval}, nil
}
// SetupWithManager: watch BaoPolicy plus the token Secret.
func (r *BaoPolicyReconciler) SetupWithManager(mgr ctrl.Manager) error {
isToken := predicate.NewPredicateFuncs(func(o client.Object) bool {
return o.GetNamespace() == r.OperatorNamespace &&
o.GetName() == TokenSecretName
})
return ctrl.NewControllerManagedBy(mgr).
For(&baov1alpha1.BaoPolicy{}).
Watches(&corev1.Secret{},
handler.EnqueueRequestsFromMapFunc(r.allPolicies),
builder.WithPredicates(isToken)).
Named("baopolicy").
Complete(r)
}
// allPolicies: new/rotated token -> reconcile all policies again.
func (r *BaoPolicyReconciler) allPolicies(
ctx context.Context, _ client.Object,
) []reconcile.Request {
var list baov1alpha1.BaoPolicyList
if err := r.List(ctx, &list); err != nil {
logf.FromContext(ctx).Error(err, "BaoPolicies auflisten")
return nil
}
reqs := make([]reconcile.Request, 0, len(list.Items))
for _, bp := range list.Items {
reqs = append(reqs, reconcile.Request{
NamespacedName: client.ObjectKeyFromObject(&bp),
})
}
return reqs
}
EOFCompared to the scaffold, the RBAC markers are deliberately narrow:
no create/delete on BaoPolicies, no
/finalizers (we set no OwnerReferences), but reading
Secrets and writing events (events.k8s.io, the new events
API behind GetEventRecorder, Day 11).
Now the actual work: compare desired against actual state, and write
only on a difference. Whether a difference is an update or drift is
revealed by observedGeneration (Day 5): if the spec hasn’t
changed, someone has been writing around in OpenBao.
cat > internal/controller/sync.go <<'EOF'
package controller
import (
"context"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
// reasonBaoError: OpenBao reported an error.
const reasonBaoError = "BaoError"
// sync writes the policy if it is missing or differs (drift).
func (r *BaoPolicyReconciler) sync(
ctx context.Context, bp *baov1alpha1.BaoPolicy, name string,
) (result, reason string, err error) {
bao, err := r.baoClient(ctx)
if err != nil {
return resultError, "TokenUnavailable", err
}
want := desiredPolicy(bp.Spec)
got, err := bao.Sys().GetPolicyWithContext(ctx, name)
if err != nil {
return resultError, reasonBaoError, err
}
switch {
case got == want:
return "unchanged", "InSync", nil
case got == "":
result, reason = "created", "Created"
case bp.Generation != bp.Status.ObservedGeneration:
result, reason = "updated", "Updated"
default:
result, reason = "drift", "DriftCorrected"
}
if err := bao.Sys().PutPolicyWithContext(ctx, name, want); err != nil {
return resultError, reasonBaoError, err
}
typ, note := corev1.EventTypeNormal, "Policy %s geschrieben"
if result == "drift" {
typ, note = corev1.EventTypeWarning, "Drift in %s korrigiert"
}
r.Recorder.Eventf(bp, nil, typ, reason, "PutPolicy", note, name)
return result, reason, nil
}
func (r *BaoPolicyReconciler) setConditions(
bp *baov1alpha1.BaoPolicy, s metav1.ConditionStatus, reason, msg string,
) {
ready := "PolicyActive"
if s == metav1.ConditionFalse {
ready = "SyncFailed"
}
for _, c := range [][2]string{{"Synced", reason}, {"Ready", ready}} {
meta.SetStatusCondition(&bp.Status.Conditions, metav1.Condition{
Type: c[0], Status: s, Reason: c[1], Message: msg,
ObservedGeneration: bp.Generation,
})
}
}
// finalize deletes the policy in OpenBao before the object disappears.
func (r *BaoPolicyReconciler) finalize(
ctx context.Context, bp *baov1alpha1.BaoPolicy, name string,
) error {
if !controllerutil.ContainsFinalizer(bp, finalizerName) {
return nil
}
bao, err := r.baoClient(ctx)
if err != nil {
return err
}
if err := bao.Sys().DeletePolicyWithContext(ctx, name); err != nil {
policySyncTotal.WithLabelValues(resultError).Inc()
return err
}
policySyncTotal.WithLabelValues("deleted").Inc()
r.Recorder.Eventf(bp, nil, corev1.EventTypeNormal, "Deleted",
"DeletePolicy", "Policy %s gelöscht", name)
controllerutil.RemoveFinalizer(bp, finalizerName)
return r.Update(ctx, bp)
}
EOF
go build ./... && echo build okExpected:
build ok
Save your progress? Sign up for free
Drill 6: Wiring up main.go
cmd/main.go stays the scaffold. Add four places.
Imports:
"time"
corev1 "k8s.io/api/core/v1"
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/client"Two flags directly before opts := zap.Options{:
var operatorNamespace string
var resyncInterval time.Duration
flag.StringVar(&operatorNamespace, "operator-namespace", "bao-operator-system",
"Namespace mit dem Secret bao-operator-token.")
flag.DurationVar(&resyncInterval, "resync-interval", time.Minute,
"Wie oft jede Policy gegen OpenBao geprüft wird (Drift).")In ctrl.Options after
HealthProbeBindAddress – without these lines the manager
caches all Secrets of the cluster (Day 13):
// Cache Secrets only from the operator namespace (Day 13).
Cache: cache.Options{ByObject: map[client.Object]cache.ByObject{
&corev1.Secret{}: {Namespaces: map[string]cache.Config{
operatorNamespace: {},
}},
}},And at the reconciler:
if err := (&controller.BaoPolicyReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
Recorder: mgr.GetEventRecorder("bao-operator"),
OperatorNamespace: operatorNamespace,
ResyncInterval: resyncInterval,
}).SetupWithManager(mgr); err != nil {make manifests generate fmt vet
go build -o bin/manager ./cmd && bin/manager --help 2>&1 \
| grep -A1 -E "operator-namespace|resync-interval"Expected:
-operator-namespace string
Namespace mit dem Secret bao-operator-token. (default "bao-operator-system")
-resync-interval duration
Wie oft jede Policy gegen OpenBao geprüft wird (Drift). (default 1m0s)
Save your progress? Sign up for free
Drill 7: envtest suite with a real OpenBao
The suite starts a bao server -dev on
127.0.0.1:18200 by itself (lab token
lab-root), then envtest and the manager with a 2 s resync.
No mock: the tests see exactly what OpenBao stores.
cat > internal/controller/suite_test.go <<'EOF'
package controller
import (
"context"
"net/http"
"os/exec"
"path/filepath"
"testing"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/openbao/openbao/api/v2"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/scheme"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/envtest"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
const (
baoAddr = "http://127.0.0.1:18200"
baoRoot = "lab-root" // Lab-Wert, nur für den Dev-Server
opNS = "bao-operator-system"
)
var (
ctx context.Context
cancel context.CancelFunc
testEnv *envtest.Environment
k8sClient client.Client
baoCmd *exec.Cmd
baoc *api.Client
)
func TestControllers(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Controller Suite")
}
var _ = BeforeSuite(func() {
logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true)))
ctx, cancel = context.WithCancel(context.TODO())
By("OpenBao-Dev-Server starten")
if resp, err := http.Get(baoAddr + "/v1/sys/health"); err == nil {
_ = resp.Body.Close()
Fail("Port 18200 ist belegt – läuft noch ein bao server -dev?")
}
baoCmd = exec.Command("bao", "server", "-dev",
"-dev-root-token-id="+baoRoot, "-dev-listen-address=127.0.0.1:18200")
Expect(baoCmd.Start()).To(Succeed())
Eventually(func() error {
resp, err := http.Get(baoAddr + "/v1/sys/health")
if err == nil {
_ = resp.Body.Close()
}
return err
}, 15*time.Second, 200*time.Millisecond).Should(Succeed())
cfg := api.DefaultConfig()
cfg.Address = baoAddr
var err error
baoc, err = api.NewClient(cfg)
Expect(err).NotTo(HaveOccurred())
baoc.SetToken(baoRoot)
By("envtest starten")
Expect(baov1alpha1.AddToScheme(scheme.Scheme)).To(Succeed())
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
ErrorIfCRDPathMissing: true,
}
restCfg, err := testEnv.Start()
Expect(err).NotTo(HaveOccurred())
k8sClient, err = client.New(restCfg, client.Options{Scheme: scheme.Scheme})
Expect(err).NotTo(HaveOccurred())
Expect(k8sClient.Create(ctx, &corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{Name: opNS},
})).To(Succeed())
By("Manager mit dem Reconciler starten")
mgr, err := ctrl.NewManager(restCfg, ctrl.Options{
Scheme: scheme.Scheme,
Metrics: metricsserver.Options{BindAddress: "0"},
})
Expect(err).NotTo(HaveOccurred())
Expect((&BaoPolicyReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
Recorder: mgr.GetEventRecorder("bao-operator"),
OperatorNamespace: opNS,
ResyncInterval: 2 * time.Second,
}).SetupWithManager(mgr)).To(Succeed())
go func() {
defer GinkgoRecover()
Expect(mgr.Start(ctx)).To(Succeed())
}()
})
var _ = AfterSuite(func() {
cancel()
if testEnv != nil {
Expect(testEnv.Stop()).To(Succeed())
}
if baoCmd != nil && baoCmd.Process != nil {
_ = baoCmd.Process.Kill()
_ = baoCmd.Wait()
}
})
EOFThe specs run Ordered: first the Secret is missing, then
it arrives – that way you test the Secret watch along the way.
cat > internal/controller/baopolicy_controller_test.go <<'EOF'
package controller
import (
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/prometheus/client_golang/prometheus/testutil"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
var _ = Describe("BaoPolicy", Ordered, func() {
const timeout, tick = 10 * time.Second, 200 * time.Millisecond
key := client.ObjectKey{Namespace: "default", Name: "app-read"}
const name = "k8s.default.app-read"
readPolicy := func() (string, error) { return baoc.Sys().GetPolicy(name) }
It("lehnt policy und rules gleichzeitig ab (CEL)", func() {
bp := &baov1alpha1.BaoPolicy{
ObjectMeta: metav1.ObjectMeta{Name: "beides", Namespace: "default"},
Spec: baov1alpha1.BaoPolicySpec{Policy: "x", Rules: []baov1alpha1.PolicyRule{
{Path: "a", Capabilities: []baov1alpha1.Capability{capRead}},
}},
}
err := k8sClient.Create(ctx, bp)
Expect(apierrors.IsInvalid(err)).To(BeTrue())
Expect(err.Error()).To(ContainSubstring("has(self.policy) != has(self.rules)"))
})
It("meldet TokenUnavailable ohne Secret", func() {
bp := &baov1alpha1.BaoPolicy{
ObjectMeta: metav1.ObjectMeta{Name: key.Name, Namespace: key.Namespace},
Spec: baov1alpha1.BaoPolicySpec{Rules: []baov1alpha1.PolicyRule{{
Path: "secret/data/app/*",
Capabilities: []baov1alpha1.Capability{capRead},
}}},
}
Expect(k8sClient.Create(ctx, bp)).To(Succeed())
Eventually(func(g Gomega) {
g.Expect(k8sClient.Get(ctx, key, bp)).To(Succeed())
c := meta.FindStatusCondition(bp.Status.Conditions, "Synced")
g.Expect(c).NotTo(BeNil())
g.Expect(c.Reason).To(Equal("TokenUnavailable"))
}, timeout, tick).Should(Succeed())
})
It("schreibt die Policy, sobald das Secret da ist", func() {
Expect(k8sClient.Create(ctx, &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{Name: TokenSecretName, Namespace: opNS},
StringData: map[string]string{"address": baoAddr, "token": baoRoot},
})).To(Succeed())
Eventually(readPolicy, timeout, tick).Should(ContainSubstring(
`path "secret/data/app/*"`))
bp := &baov1alpha1.BaoPolicy{}
Eventually(func(g Gomega) {
g.Expect(k8sClient.Get(ctx, key, bp)).To(Succeed())
g.Expect(meta.IsStatusConditionTrue(bp.Status.Conditions, "Ready")).
To(BeTrue())
g.Expect(bp.Status.PolicyName).To(Equal(name))
g.Expect(bp.Status.ObservedGeneration).To(Equal(bp.Generation))
}, timeout, tick).Should(Succeed())
Expect(bp.Finalizers).To(ContainElement(finalizerName))
})
It("übernimmt Spec-Änderungen", func() {
bp := &baov1alpha1.BaoPolicy{}
Expect(k8sClient.Get(ctx, key, bp)).To(Succeed())
bp.Spec.Rules[0].Capabilities = append(bp.Spec.Rules[0].Capabilities, "list")
Expect(k8sClient.Update(ctx, bp)).To(Succeed())
Eventually(readPolicy, timeout, tick).Should(ContainSubstring(`"read", "list"`))
})
It("repariert Drift beim Resync", func() {
before := testutil.ToFloat64(policySyncTotal.WithLabelValues("drift"))
Expect(baoc.Sys().PutPolicy(name, `path "*" { capabilities = ["sudo"] }`)).
To(Succeed())
Eventually(readPolicy, timeout, tick).Should(ContainSubstring(`"read", "list"`))
Expect(testutil.ToFloat64(policySyncTotal.WithLabelValues("drift"))).
To(BeNumerically(">", before))
})
It("löscht die Policy über den Finalizer", func() {
bp := &baov1alpha1.BaoPolicy{}
Expect(k8sClient.Get(ctx, key, bp)).To(Succeed())
Expect(k8sClient.Delete(ctx, bp)).To(Succeed())
Eventually(func() bool {
return apierrors.IsNotFound(k8sClient.Get(ctx, key, bp))
}, timeout, tick).Should(BeTrue())
Expect(readPolicy()).To(BeEmpty())
})
})
EOF
go mod tidy
go test ./internal/... -count=1 -v | grep -E "^(---|ok|PASS|FAIL)|Ran "Expected (abridged):
--- PASS: TestRenderRules (0.00s)
Ran 6 of 6 Specs in 8.095 seconds
--- PASS: TestControllers (8.10s)
PASS
ok example.com/bao-operator/internal/controller 8.466s
Warning: If a Bao is already running on port 18200 (say, from Drill 8), the suite aborts immediately with “Port 18200 ist belegt” (port 18200 is in use) – otherwise it would talk to the foreign server.
Save your progress? Sign up for free
Drill 8: Live – kubectl apply, bao policy read
For the live run, the manager needs an API server.
labenv starts envtest permanently and writes an admin
kubeconfig – your real context stays untouched.
mkdir -p hack/labenv
cat > hack/labenv/main.go <<'EOF'
// labenv starts envtest (etcd + kube-apiserver) with our CRDs and
// writes an admin kubeconfig. Lab only – Ctrl+C stops everything.
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
"sigs.k8s.io/controller-runtime/pkg/envtest"
)
func main() {
env := &envtest.Environment{
CRDDirectoryPaths: []string{"config/crd/bases"},
ErrorIfCRDPathMissing: true,
}
if _, err := env.Start(); err != nil {
fmt.Fprintln(os.Stderr, "envtest:", err)
os.Exit(1)
}
defer func() { _ = env.Stop() }()
user, err := env.AddUser(envtest.User{
Name: "lab-admin", Groups: []string{"system:masters"},
}, nil)
if err == nil {
var kc []byte
if kc, err = user.KubeConfig(); err == nil {
err = os.WriteFile("labenv.kubeconfig", kc, 0o600)
}
}
if err != nil {
fmt.Fprintln(os.Stderr, "kubeconfig:", err)
return
}
fmt.Println("envtest läuft, KUBECONFIG=labenv.kubeconfig")
sig := make(chan os.Signal, 1)
signal.Notify(sig, os.Interrupt, syscall.SIGTERM)
<-sig
}
EOF
echo labenv.kubeconfig >> .gitignoreTerminal 1 (OpenBao) and terminal 2 (envtest):
bao server -dev -dev-root-token-id=lab-root -dev-listen-address=127.0.0.1:18200cd ~/op-lab/tag14-bao-operator && go run ./hack/labenvExpected in terminal 2:
envtest läuft, KUBECONFIG=labenv.kubeconfig
Terminal 3: the operator does not get the root
token, but its own token with least privilege – only policies with the
prefix k8s..
cd ~/op-lab/tag14-bao-operator
export KUBECONFIG=$PWD/labenv.kubeconfig
export BAO_ADDR=http://127.0.0.1:18200 BAO_TOKEN=lab-root # Lab-Wert
cat > operator-policy.hcl <<'EOF'
# The operator may only manage policies with the prefix k8s.
path "sys/policies/acl/k8s.*" {
capabilities = ["create", "read", "update", "delete"]
}
EOF
bao policy write bao-operator operator-policy.hcl
OP_TOKEN=$(bao token create -policy=bao-operator -period=24h -orphan -field=token)
kubectl create namespace bao-operator-system
kubectl -n bao-operator-system create secret generic bao-operator-token \
--from-literal=address=$BAO_ADDR --from-literal=token=$OP_TOKEN
echo 'path "x" { capabilities = ["read"] }' \
| BAO_TOKEN=$OP_TOKEN bao policy write admin - 2>&1 | grep -E "Code|denied"Expected:
Success! Uploaded policy: bao-operator
namespace/bao-operator-system created
secret/bao-operator-token created
Code: 403. Errors:
* permission denied
The last command is the counter-check: with the operator token,
nobody can overwrite the admin policy.
Terminal 4 (manager, 10 s resync, metrics without TLS on localhost only):
cd ~/op-lab/tag14-bao-operator && export KUBECONFIG=$PWD/labenv.kubeconfig
go run ./cmd/main.go --operator-namespace=bao-operator-system \
--resync-interval=10s --metrics-bind-address=127.0.0.1:18080 \
--metrics-secure=false --health-probe-bind-address=127.0.0.1:18081Expected (abridged):
INFO setup Starting manager
INFO Starting EventSource {"controller": "baopolicy", ... "kind source: *v1.Secret"}
INFO Starting workers {"controller": "baopolicy", ... "worker count": 1}
Back in terminal 3 – the real moment:
kubectl create namespace shop
cat > config/samples/bao_v1alpha1_baopolicy.yaml <<'EOF'
apiVersion: bao.lab.example/v1alpha1
kind: BaoPolicy
metadata:
name: webshop-read
namespace: shop
spec:
rules:
- path: secret/data/webshop/*
capabilities: [read, list]
- path: secret/data/webshop/admin
capabilities: [deny]
EOF
kubectl apply -f config/samples/bao_v1alpha1_baopolicy.yaml
kubectl wait -n shop bp/webshop-read --for=condition=Ready --timeout=30s
kubectl get bp -n shop
bao policy read k8s.shop.webshop-readExpected:
baopolicy.bao.lab.example/webshop-read created
baopolicy.bao.lab.example/webshop-read condition met
NAME POLICY AGE
webshop-read k8s.shop.webshop-read 0s
path "secret/data/webshop/*" {
capabilities = ["read", "list"]
}
path "secret/data/webshop/admin" {
capabilities = ["deny"]
}
Now you play the attacker: someone with the root token loosens the policy directly in OpenBao. After one resync interval at the latest, it is repaired.
echo 'path "*" { capabilities = ["sudo", "read"] }' \
| bao policy write k8s.shop.webshop-read -
sleep 11
bao policy read k8s.shop.webshop-read | head -3
kubectl get events -n shop
curl -s 127.0.0.1:18080/metrics | grep '^bao_operator'Expected (abridged):
Success! Uploaded policy: k8s.shop.webshop-read
path "secret/data/webshop/*" {
capabilities = ["read", "list"]
}
LAST SEEN TYPE REASON OBJECT MESSAGE
16s Normal Created baopolicy/webshop-read Policy k8s.shop.web...
2s Warning DriftCorrected baopolicy/webshop-read Drift in k8s.shop.w...
bao_operator_policy_sync_total{result="created"} 1
bao_operator_policy_sync_total{result="drift"} 1
bao_operator_policy_sync_total{result="unchanged"} 4
An alert on
increase(bao_operator_policy_sync_total{result="drift"}[1h]) > 0
is a real security signal: someone is bypassing GitOps.
The API rejects nonsense before the operator ever sees it:
kubectl apply -f - <<'EOF'
apiVersion: bao.lab.example/v1alpha1
kind: BaoPolicy
metadata: {name: kaputt, namespace: shop}
spec:
policy: 'path "x" { capabilities = ["read"] }'
rules: [{path: x, capabilities: [read]}]
EOF
kubectl apply -f - <<'EOF'
apiVersion: bao.lab.example/v1alpha1
kind: BaoPolicy
metadata: {name: kaputt, namespace: shop}
spec:
rules: [{path: x, capabilities: [lesen]}]
EOFExpected (abridged):
The BaoPolicy "kaputt" is invalid: spec: Invalid value: failed rule:
has(self.policy) != has(self.rules)
The BaoPolicy "kaputt" is invalid:
* spec.rules[0].capabilities[0]: Unsupported value: "lesen": supported values:
"create", "read", "update", "patch", "delete", "list", "sudo", "deny"
Finally, the finalizer:
kubectl get bp -n shop webshop-read -o jsonpath='{.metadata.finalizers}{"\n"}'
kubectl delete bp -n shop webshop-read
bao policy read k8s.shop.webshop-readExpected:
["bao.lab.example/finalizer"]
baopolicy.bao.lab.example "webshop-read" deleted from shop namespace
No policy named: k8s.shop.webshop-read
Tip: As a test, delete the Secret and create a new BaoPolicy: it gets
Synced=False (TokenUnavailable). As soon as the Secret is back, the Secret watch triggers the reconcile andkubectl wait --for=condition=Readyreturns immediately.
Save your progress? Sign up for free
Checkpoint
Create webshop-read from Drill 8 again
(kubectl apply -f config/samples/), then:
cat > hack/kontrollpunkt.sh <<'EOF'
#!/usr/bin/env bash
# Checkpoint Day 14 – expects labenv, bao dev and a running manager.
export KUBECONFIG=$PWD/labenv.kubeconfig BAO_ADDR=http://127.0.0.1:18200
export BAO_TOKEN=lab-root # Lab-Wert
ok() { echo "✔ $1"; }
nok() { echo "✘ $1"; }
check() { if eval "$2" >/dev/null 2>&1; then ok "$1"; else nok "$1"; fi; }
check "OpenBao Dev-Server erreichbar" "bao status"
check "CRD baopolicies installiert" \
"kubectl get crd baopolicies.bao.lab.example"
check "Token-Secret vorhanden" \
"kubectl -n bao-operator-system get secret bao-operator-token"
check "webshop-read ist Ready" "kubectl wait -n shop bp/webshop-read \
--for=condition=Ready --timeout=10s"
check "Policy steht in OpenBao" \
"bao policy read k8s.shop.webshop-read | grep -q 'secret/data/webshop'"
check "Finalizer gesetzt" "kubectl get bp -n shop webshop-read \
-o jsonpath='{.metadata.finalizers}' | grep -q bao.lab.example/finalizer"
check "Metrik bao_operator_policy_sync_total" \
"curl -s 127.0.0.1:18080/metrics | grep -q bao_operator_policy_sync_total"
check "CEL lehnt policy+rules ab" "! kubectl apply --dry-run=server -f - <<'Y'
apiVersion: bao.lab.example/v1alpha1
kind: BaoPolicy
metadata: {name: x, namespace: shop}
spec: {policy: 'path \"x\" {}', rules: [{path: x, capabilities: [read]}]}
Y"
EOF
bash hack/kontrollpunkt.shExpected:
✔ OpenBao Dev-Server erreichbar
✔ CRD baopolicies installiert
✔ Token-Secret vorhanden
✔ webshop-read ist Ready
✔ Policy steht in OpenBao
✔ Finalizer gesetzt
✔ Metrik bao_operator_policy_sync_total
✔ CEL lehnt policy+rules ab
Challenge
Extend BaoPolicy with an optional field
spec.authRole.serviceAccounts. If it is set, the operator
additionally creates the Kubernetes auth role
auth/kubernetes/role/k8s.<ns>.<name>, which
binds exactly these ServiceAccounts in their own
namespace to the policy – and the finalizer cleans it up as
well.
Hint: bao.Logical().WriteWithContext
with bound_service_account_names,
bound_service_account_namespaces and
token_policies; for this, the operator token needs a second
path block.
Solution
In api/v1alpha1/baopolicy_types.go, add a type and a
field in BaoPolicySpec:
// AuthRole binds the policy to ServiceAccounts in the object's own namespace.
type AuthRole struct {
// +kubebuilder:validation:MinItems=1
// +listType=set
ServiceAccounts []string `json:"serviceAccounts"`
}
// (in BaoPolicySpec, after Rules)
// Optional: create the Kubernetes auth role k8s.<ns>.<name>.
// +optional
AuthRole *AuthRole `json:"authRole,omitempty"`cat > internal/controller/authrole.go <<'EOF'
package controller
import (
"context"
baov1alpha1 "example.com/bao-operator/api/v1alpha1"
)
const roleMount = "auth/kubernetes/role/"
// syncRole writes the role idempotently; the namespace is fixed to the
// object's own – so a team cannot bind foreign ServiceAccounts.
func (r *BaoPolicyReconciler) syncRole(
ctx context.Context, bp *baov1alpha1.BaoPolicy, name string,
) error {
if bp.Spec.AuthRole == nil {
return nil
}
bao, err := r.baoClient(ctx)
if err != nil {
return err
}
_, err = bao.Logical().WriteWithContext(ctx, roleMount+name, map[string]any{
"bound_service_account_names": bp.Spec.AuthRole.ServiceAccounts,
"bound_service_account_namespaces": []string{bp.Namespace},
"token_policies": []string{name},
"token_ttl": "1h",
})
return err
}
EOFIn Reconcile directly after r.sync(...),
and in finalize before
DeletePolicyWithContext:
if err == nil {
if rerr := r.syncRole(ctx, &bp, name); rerr != nil {
result, reason, err = resultError, reasonBaoError, rerr
}
} if bp.Spec.AuthRole != nil {
if _, err := bao.Logical().DeleteWithContext(ctx, roleMount+name); err != nil {
return err
}
}Extend the operator policy, enable the auth method, re-apply the CRD, restart the manager (terminal 4), and try it out:
cat >> operator-policy.hcl <<'EOF'
path "auth/kubernetes/role/k8s.*" {
capabilities = ["create", "update", "delete"]
}
EOF
bao policy write bao-operator operator-policy.hcl
bao auth enable kubernetes
make manifests generate && kubectl apply -f config/crd/bases/
kubectl apply -f - <<'EOF'
apiVersion: bao.lab.example/v1alpha1
kind: BaoPolicy
metadata: {name: webshop-read, namespace: shop}
spec:
rules: [{path: secret/data/webshop/*, capabilities: [read, list]}]
authRole: {serviceAccounts: [webshop]}
EOF
kubectl wait -n shop bp/webshop-read --for=condition=Ready --timeout=20s
bao read auth/kubernetes/role/k8s.shop.webshop-read \
| grep -E "_names |_namespaces |token_policies|token_ttl"
kubectl delete bp -n shop webshop-read
bao read auth/kubernetes/role/k8s.shop.webshop-readExpected (abridged):
Success! Enabled kubernetes auth method at: kubernetes/
baopolicy.bao.lab.example/webshop-read condition met
bound_service_account_names [webshop]
bound_service_account_namespaces [shop]
token_policies [k8s.shop.webshop-read]
token_ttl 1h
baopolicy.bao.lab.example "webshop-read" deleted from shop namespace
No value found at auth/kubernetes/role/k8s.shop.webshop-read
The operator sets the namespace itself – so a team cannot bind
ServiceAccounts from foreign namespaces to its policy. Open edge: if
authRole is removed later, the role stays in place until
the BaoPolicy is deleted.
Save your progress? Sign up for free
Enterprise Note
- Real OpenBao: TLS with the CA from the Secret
(
api.Config.ConfigureTLS); instead of a static token, better use Kubernetes auth for the operator itself (its own ServiceAccount, short-lived tokens, renewal). - Multi-tenancy: prefix
k8s.<ns>.plus an operator policy only onsys/policies/acl/k8s.*; additionally admission (ValidatingAdmissionPolicy) againstsudo/path "*"in team namespaces. - Tighter Secrets RBAC: instead of a ClusterRole, a Role in the
operator namespace (marker with
namespace=), matching the cache filter from Drill 6. - Drift alert on
bao_operator_policy_sync_total{result="drift"}andresult="error"; weigh the resync interval against the load on OpenBao (Day 13). - Stuck finalizer when OpenBao is gone for good (Day 7): a runbook
with a deliberate
kubectl patch ... finalizers: nulland manual policy cleanup.