Day 7: HTTP Client, context, Timeouts, Retries – ghrel
Goal: A CLI that queries GitHub releases – with a timeout per attempt, an overall budget via context and retries with backoff · Duration: approx. 120 min · Prerequisite: Day 6
Briefing
net/httpis enough: build the request, set headers, decode JSON into structs.- Two time limits:
http.Client.Timeoutper attempt,context.WithTimeoutfor everything including retries. - Retry only when it makes sense: 429, 5xx, network errors – never on 404 or an expired context.
Retry-Afterfrom the server beats your own backoff.- Tests run against
httptest.Server– no network, no rate limit, deterministic.
ghrel ---ctx (15 s total)----------------------------------------------.
attempt 1 (max 5 s) -> 503 | wait 0.5 s | attempt 2 -> 503 | wait 1 s |
attempt 3 -> 200 -> JSON -> table |
'------------------------------------------ ctx expired? -> abort at once'
Drill 1: Module without dependencies
mkdir -p ~/go-lab/tag07-ghrel && cd ~/go-lab/tag07-ghrel
go mod init example.com/ghrel
mkdir -p cmd/ghrel internal/github
curl -s https://api.github.com/repos/openbao/openbao/releases/latest \
| jq '{tag_name, name, prerelease, published_at}'Expected (shortened, values change with every release):
go: creating new go.mod: module example.com/ghrel
{
"tag_name": "v2.7.1",
"name": "v2.7.1",
"prerelease": false,
"published_at": "2026-10-01T14:03:19Z"
}
Why curl first? That way you see the real shape of the JSON before you write structs for it. Without a token, GitHub allows 60 requests per hour and IP – enough for today.
Save your progress? Sign up for free
Drill 2: Types and client
JSON tags connect tag_name with TagName;
time.Time understands RFC 3339 directly.
encoding/json ignores fields you don’t declare.
cat > internal/github/client.go <<'EOF'
// Package github is a minimal client for the GitHub releases API.
package github
import (
"context"
"fmt"
"net/http"
"time"
)
// Release contains the fields that ghrel displays.
type Release struct {
TagName string `json:"tag_name"`
Name string `json:"name"`
Prerelease bool `json:"prerelease"`
PublishedAt time.Time `json:"published_at"`
HTMLURL string `json:"html_url"`
}
// APIError is a response with status >= 400.
type APIError struct {
Status int
Message string
}
func (e *APIError) Error() string {
return fmt.Sprintf("GitHub-API: %d %s", e.Status, e.Message)
}
// Client talks to api.github.com (or a test server).
type Client struct {
BaseURL string // e.g. https://api.github.com
Token string // optional; raises the rate limit from 60 to 5000/h
HTTP *http.Client // with a timeout per attempt
Retries int // additional attempts on 429/5xx/network errors
Backoff time.Duration // wait before the 1st retry, doubled after that
}
// Releases returns the latest n releases of owner/repo.
func (c *Client) Releases(ctx context.Context, repo string, n int) ([]Release, error) {
var rs []Release
err := c.get(ctx, fmt.Sprintf("/repos/%s/releases?per_page=%d", repo, n), &rs)
return rs, err
}
// Latest returns the latest stable release (no drafts/prereleases).
func (c *Client) Latest(ctx context.Context, repo string) (Release, error) {
var r Release
err := c.get(ctx, "/repos/"+repo+"/releases/latest", &r)
return r, err
}
EOFWhy a custom error type? The caller wants to get at the status with
errors.As (Day 3) to report a 404 in a friendly way – a
string comparison on the error message would be fragile.
Save your progress? Sign up for free
Drill 3: One attempt, then retries
cat > internal/github/retry.go <<'EOF'
package github
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strconv"
"time"
)
// get fetches path and decodes JSON into v – with retries and backoff.
func (c *Client) get(ctx context.Context, path string, v any) error {
wait := c.Backoff
for attempt := 0; ; attempt++ {
retryAfter, err := c.do(ctx, path, v)
if err == nil || attempt >= c.Retries || ctx.Err() != nil || !retryable(err) {
return err
}
if retryAfter > 0 {
wait = retryAfter // the server's value beats our own estimate
}
select {
case <-ctx.Done(): // overall budget used up or Ctrl+C
return fmt.Errorf("%w (letzter Fehler: %v)", ctx.Err(), err)
case <-time.After(wait):
}
wait *= 2
}
}
// do makes exactly one attempt.
func (c *Client) do(ctx context.Context, path string, v any) (time.Duration, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.BaseURL+path, nil)
if err != nil {
return 0, err
}
req.Header.Set("Accept", "application/vnd.github+json")
req.Header.Set("X-GitHub-Api-Version", "2022-11-28")
req.Header.Set("User-Agent", "ghrel-lab") // GitHub rejects requests without a User-Agent
if c.Token != "" {
req.Header.Set("Authorization", "Bearer "+c.Token)
}
resp, err := c.HTTP.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
var body struct{ Message string }
_ = json.NewDecoder(io.LimitReader(resp.Body, 64<<10)).Decode(&body)
secs, _ := strconv.Atoi(resp.Header.Get("Retry-After"))
return time.Duration(secs) * time.Second,
&APIError{Status: resp.StatusCode, Message: body.Message}
}
return 0, json.NewDecoder(resp.Body).Decode(v)
}
// retryable: 429, 5xx and network errors yes, other 4xx no.
func retryable(err error) bool {
var ae *APIError
if errors.As(err, &ae) {
return ae.Status == http.StatusTooManyRequests || ae.Status >= 500
}
return true // network error: connection refused, reset, timeout per attempt
}
EOFWhy? NewRequestWithContext binds the request to the
context: when the budget runs out, even a hanging connection is aborted
immediately. The select waits for the backoff
or the end of the context – a bare
time.Sleep would ignore Ctrl+C. io.LimitReader
protects against a huge error page filling up memory.
Tip: Retries are only safe for idempotent requests. You may repeat a GET, but not a POST that creates an order – not without an idempotency key.
Save your progress? Sign up for free
Drill 4: The CLI
run gets the context, arguments, environment and writers
– as on Day 6. GHREL_API redirects the base URL (tests,
GitHub Enterprise), cmp.Or takes the first non-empty
value.
cat > cmd/ghrel/main.go <<'EOF'
// ghrel shows the GitHub releases of a repo: ghrel [-n 5] [-latest] owner/repo
package main
import (
"cmp"
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"net/http"
"os"
"os/signal"
"strings"
"text/tabwriter"
"time"
"example.com/ghrel/internal/github"
)
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
defer stop()
os.Exit(run(ctx, os.Args[1:], os.Getenv, os.Stdout, os.Stderr))
}
func run(ctx context.Context, args []string, getenv func(string) string,
stdout, stderr io.Writer) int {
fl := flag.NewFlagSet("ghrel", flag.ContinueOnError)
fl.SetOutput(stderr)
n := fl.Int("n", 5, "Anzahl Releases")
latest := fl.Bool("latest", false, "nur das neueste stabile Release")
asJSON := fl.Bool("json", false, "JSON statt Tabelle")
timeout := fl.Duration("timeout", 15*time.Second, "Gesamtbudget inkl. Retries")
if err := fl.Parse(args); err != nil {
return 2
}
repo := fl.Arg(0)
if fl.NArg() != 1 || strings.Count(repo, "/") != 1 {
fmt.Fprintln(stderr, "Aufruf: ghrel [-n 5] [-latest] [-json] owner/repo")
return 2
}
c := &github.Client{
BaseURL: cmp.Or(getenv("GHREL_API"), "https://api.github.com"),
Token: getenv("GITHUB_TOKEN"),
HTTP: &http.Client{Timeout: 5 * time.Second}, // per attempt
Retries: 3,
Backoff: 500 * time.Millisecond,
}
ctx, cancel := context.WithTimeout(ctx, *timeout)
defer cancel()
var rs []github.Release
var err error
if *latest {
var r github.Release
r, err = c.Latest(ctx, repo)
rs = []github.Release{r}
} else {
rs, err = c.Releases(ctx, repo, *n)
}
var ae *github.APIError
switch {
case errors.As(err, &ae) && ae.Status == http.StatusNotFound:
fmt.Fprintf(stderr, "ghrel: %s nicht gefunden (oder keine Releases)\n", repo)
return 1
case err != nil:
fmt.Fprintln(stderr, "ghrel:", err)
return 1
}
if *asJSON {
enc := json.NewEncoder(stdout)
enc.SetIndent("", " ")
_ = enc.Encode(rs)
return 0
}
tw := tabwriter.NewWriter(stdout, 0, 4, 2, ' ', 0)
fmt.Fprintln(tw, "TAG\tDATUM\tNAME")
for _, r := range rs {
name := r.Name
if r.Prerelease {
name += " (pre)"
}
date := r.PublishedAt.Format(time.DateOnly)
fmt.Fprintf(tw, "%s\t%s\t%s\n", r.TagName, date, name)
}
_ = tw.Flush()
return 0
}
EOFgo vet ./... && go build -o ghrel ./cmd/ghrel
./ghrel -n 3 kubernetes/kubernetes
./ghrel -latest openbao/openbao
./ghrel nobody-xyz/nope-123; echo "Exit: $?"Expected (shortened, values change):
TAG DATUM NAME
v1.38.0-alpha.2 2026-10-07 v1.38.0-alpha.2 (pre)
v1.38.0-alpha.1 2026-09-29 v1.38.0-alpha.1 (pre)
v1.37.1 2026-09-23 v1.37.1
TAG DATUM NAME
v2.7.1 2026-10-01 v2.7.1
ghrel: nobody-xyz/nope-123 nicht gefunden (oder keine Releases)
Exit: 1
Why time.DateOnly? Go formats times using the reference
date 2006-01-02 15:04:05; the constant saves you from
memorizing it. (“DATUM” is German for “date”, “nicht gefunden (oder
keine Releases)” means “not found (or no releases)”.)
Security: For a bigger quota:
GITHUB_TOKEN=$(gh auth token) ./ghrel …– the token comes only from the environment, never into code, flags or shell history.
Save your progress? Sign up for free
Drill 5: Watch retries live
Port 9 (discard) is closed on the Mac – every connection is refused immediately, so it’s a network error, so it’s a retry.
time GHREL_API=http://127.0.0.1:9 ./ghrel acme/tool
time GHREL_API=http://127.0.0.1:9 ./ghrel -timeout 1s acme/toolExpected (shortened):
ghrel: Get "http://127.0.0.1:9/repos/acme/tool/releases?per_page=5": dial tcp …
… 3,591 total
ghrel: context deadline exceeded (letzter Fehler: Get "http://127.0.0.1:9/repos…
… 1,014 total
Why 3.6 s? One attempt plus three retries with 0.5 + 1 + 2 s of
backoff. With -timeout 1s, the overall budget kicks in in
the middle of waiting – and the error names both: the reason for
aborting (“letzter Fehler” = “last error”) and the last cause.
Save your progress? Sign up for free
Drill 6: Tests against httptest
httptest.NewServer starts a real HTTP server on a free
port. The handler plays GitHub – including errors you could never
trigger on purpose with the real GitHub.
cat > internal/github/client_test.go <<'EOF'
package github
import (
"context"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
)
// newTestClient starts a fake GitHub; a tiny backoff keeps tests fast.
func newTestClient(t *testing.T, h http.HandlerFunc) *Client {
t.Helper()
srv := httptest.NewServer(h)
t.Cleanup(srv.Close)
return &Client{BaseURL: srv.URL, Token: "lab-token", HTTP: srv.Client(),
Retries: 3, Backoff: time.Millisecond}
}
func TestReleasesHeaderUndJSON(t *testing.T) {
c := newTestClient(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/repos/acme/tool/releases" ||
r.URL.Query().Get("per_page") != "2" {
t.Errorf("falsche URL: %s", r.URL)
}
if got := r.Header.Get("Authorization"); got != "Bearer lab-token" {
t.Errorf("Authorization = %q", got)
}
fmt.Fprint(w, `[{"tag_name":"v1.2.0","prerelease":false,
"published_at":"2026-10-01T12:00:00Z"},{"tag_name":"v1.1.0"}]`)
})
rs, err := c.Releases(context.Background(), "acme/tool", 2)
if err != nil {
t.Fatal(err)
}
if len(rs) != 2 || rs[0].TagName != "v1.2.0" || rs[0].PublishedAt.Day() != 1 {
t.Errorf("Releases = %+v", rs)
}
}
func TestRetry(t *testing.T) {
tests := []struct {
name string
failWith int // status of the first two responses
wantCalls int32
wantErr bool
}{
{"503 wird wiederholt", http.StatusServiceUnavailable, 3, false},
{"429 wird wiederholt", http.StatusTooManyRequests, 3, false},
{"404 sofort Fehler", http.StatusNotFound, 1, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var calls atomic.Int32 // handlers run in their own goroutines
c := newTestClient(t, func(w http.ResponseWriter, r *http.Request) {
if calls.Add(1) <= 2 {
w.Header().Set("Retry-After", "0")
http.Error(w, `{"message":"kaputt"}`, tt.failWith)
return
}
fmt.Fprint(w, `{"tag_name":"v1.0.0"}`)
})
_, err := c.Latest(context.Background(), "acme/tool")
if (err != nil) != tt.wantErr || calls.Load() != tt.wantCalls {
t.Errorf("err=%v calls=%d, want err=%v calls=%d",
err, calls.Load(), tt.wantErr, tt.wantCalls)
}
})
}
}
func TestAPIErrorPerErrorsAs(t *testing.T) {
c := newTestClient(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"message":"Not Found"}`)
})
_, err := c.Latest(context.Background(), "acme/fehlt")
var ae *APIError
if !errors.As(err, &ae) || ae.Status != 404 || ae.Message != "Not Found" {
t.Errorf("err = %v", err)
}
}
func TestGesamtTimeout(t *testing.T) {
c := newTestClient(t, func(w http.ResponseWriter, r *http.Request) {
select { // hangs until the client gives up
case <-r.Context().Done():
case <-time.After(2 * time.Second):
}
})
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
start := time.Now()
_, err := c.Latest(ctx, "acme/tool")
if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("err = %v, want DeadlineExceeded", err)
}
if d := time.Since(start); d > time.Second {
t.Errorf("hat %v gedauert – Context ignoriert?", d)
}
}
EOFgo test -race -count=1 -v ./internal/github 2>&1 | grep -E '^(---|ok|\s+---)'Expected:
--- PASS: TestReleasesHeaderUndJSON (0.00s)
--- PASS: TestRetry (0.01s)
--- PASS: TestRetry/503_wird_wiederholt (0.00s)
--- PASS: TestRetry/429_wird_wiederholt (0.00s)
--- PASS: TestRetry/404_sofort_Fehler (0.00s)
--- PASS: TestAPIErrorPerErrorsAs (0.00s)
--- PASS: TestGesamtTimeout (0.05s)
ok example.com/ghrel/internal/github 1.250s
Why atomic.Int32? The handler runs in a different
goroutine than the test. A plain int would be a data race –
-race would report it (more on that on Day 8).
Save your progress? Sign up for free
Drill 7: Test the CLI
Via GHREL_API, run points to the test
server. You check the table byte for byte – tabwriter
aligns to the longest cell.
cat > cmd/ghrel/main_test.go <<'EOF'
package main
import (
"bytes"
"context"
"fmt"
"net/http"
"net/http/httptest"
"testing"
)
func TestRunTabelle(t *testing.T) {
h := func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, `[{"tag_name":"v2.0.0-rc.1","name":"RC","prerelease":true,
"published_at":"2026-10-02T08:00:00Z"}]`)
}
srv := httptest.NewServer(http.HandlerFunc(h))
defer srv.Close()
env := map[string]string{"GHREL_API": srv.URL}
var out bytes.Buffer
code := run(context.Background(), []string{"acme/tool"},
func(k string) string { return env[k] }, &out, &out)
want := "TAG DATUM NAME\nv2.0.0-rc.1 2026-10-02 RC (pre)\n"
if code != 0 || out.String() != want {
t.Errorf("Exit %d, Ausgabe:\n%s\nwant:\n%s", code, out.String(), want)
}
}
func TestRunFalscherAufruf(t *testing.T) {
var out bytes.Buffer
getenv := func(string) string { return "" }
code := run(context.Background(), []string{"nurname"}, getenv, &out, &out)
if code != 2 {
t.Errorf("Exit %d, want 2", code)
}
}
EOFgo test -race -count=1 ./...Expected:
ok example.com/ghrel/cmd/ghrel 1.177s
ok example.com/ghrel/internal/github 1.365s
Save your progress? Sign up for free
Drill 8: The tool is done
cp ../tag06-manifestlint/.golangci.yml .
gofmt -l . && go vet ./... && ~/go-lab/bin/lint run ./...
go build -o ghrel ./cmd/ghrel
./ghrel -n 2 -json cli/cli | jq -r '.[] | "\(.tag_name) \(.html_url)"'
./ghrel; echo "Exit: $?"Expected (shortened, values change):
0 issues.
v2.102.0 https://github.com/cli/cli/releases/tag/v2.102.0
v2.101.0 https://github.com/cli/cli/releases/tag/v2.101.0
Aufruf: ghrel [-n 5] [-latest] [-json] owner/repo
Exit: 2
ghrel is now a building block for scripts: “Is there a
newer OpenBao version than the one in the cluster?” is a one-liner with
-latest -json | jq -r '.[0].tag_name'.
Save your progress? Sign up for free
Checkpoint
cd ~/go-lab/tag07-ghrel
ok() { echo "✔ $1"; }; nok() { echo "✘ $1"; }
go vet ./... && go test -race -count=1 ./... >/dev/null && ok "vet + Tests (race)" \
|| nok "Tests"
./ghrel -latest openbao/openbao | grep -q '^v2\.' && ok "echtes GitHub" || nok "API"
./ghrel nobody-xyz/nope-123 2>/dev/null; [ $? -eq 1 ] && ok "404 -> Exit 1" \
|| nok "404"
s=$(date +%s); GHREL_API=http://127.0.0.1:9 ./ghrel -timeout 1s a/b 2>/dev/null
[ $(( $(date +%s) - s )) -le 2 ] && ok "Gesamtbudget greift" || nok "Timeout"Expected:
✔ vet + Tests (race)
✔ echtes GitHub
✔ 404 -> Exit 1
✔ Gesamtbudget greift
Challenge
When the quota without a token is exhausted, GitHub answers with
403, X-RateLimit-Remaining: 0 and the reset
time as Unix seconds in X-RateLimit-Reset. Extend
APIError so that the error message names the time from
which it works again – and write a test for it.
Hint: time.Unix(sek, 0) turns Unix
seconds into a time.Time; a retry would be wrong here,
because the reset is often minutes away.
Solution
In client.go, replace APIError and
Error():
type APIError struct {
Status int
Message string
Reset time.Time // set when the rate limit is exhausted
}
func (e *APIError) Error() string {
msg := fmt.Sprintf("GitHub-API: %d %s", e.Status, e.Message)
if !e.Reset.IsZero() {
msg += fmt.Sprintf(" (Limit wieder frei um %s)", e.Reset.Format(time.TimeOnly))
}
return msg
}
In retry.go, replace the last three lines of the error
branch in do:
ae := &APIError{Status: resp.StatusCode, Message: body.Message}
if resp.Header.Get("X-RateLimit-Remaining") == "0" {
reset, _ := strconv.ParseInt(resp.Header.Get("X-RateLimit-Reset"), 10, 64)
ae.Reset = time.Unix(reset, 0)
}
secs, _ := strconv.Atoi(resp.Header.Get("Retry-After"))
return time.Duration(secs) * time.Second, ae
cat >> internal/github/client_test.go <<'EOF'
func TestRateLimit(t *testing.T) {
c := newTestClient(t, func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-RateLimit-Remaining", "0")
w.Header().Set("X-RateLimit-Reset", "1791640800")
w.WriteHeader(http.StatusForbidden)
fmt.Fprint(w, `{"message":"API rate limit exceeded"}`)
})
_, err := c.Latest(context.Background(), "acme/tool")
var ae *APIError
if !errors.As(err, &ae) || ae.Reset.Unix() != 1791640800 {
t.Fatalf("err = %v", err)
}
t.Log(err)
}
EOF
go test -count=1 -v -run RateLimit ./internal/github | grep -v '^=== 'Expected (the time depends on your time zone):
client_test.go:118: GitHub-API: 403 API rate limit exceeded (Limit wieder frei…
--- PASS: TestRateLimit (0.01s)
PASS
ok example.com/ghrel/internal/github 0.242s
Save your progress? Sign up for free
Enterprise Note
- Behind corporate proxies:
http.DefaultTransportreadsHTTPS_PROXY/NO_PROXYautomatically; your own CAs come via the system trust store ortls.Config.RootCAs. - GitHub Enterprise Server has the API under
https://<host>/api/v3– that is exactly whatGHREL_APIis for. - For automation, use a GitHub App or fine-grained tokens with only
contents:readinstead of personal tokens; tokens from Vault/OpenBao instead of.envfiles. - Many clients at once: add jitter to the backoff, otherwise all retries come back in the same beat (thundering herd).
- Retries, status codes and durations belong in metrics and logs (Day 9) – silent retries hide outages at the provider.