Skip to content
Hands-on IT IT Drills

Day 6: HTTP and APIs – ghrel

Goal: build a CLI ghrel that robustly queries the latest GitHub releases · Duration: approx. 120 min · Prerequisite: Day 5

Briefing

  • httpx is the modern HTTP client: sync and async with the same API, timeouts mandatory.
  • Responses never end up in the code as a bare dict, but as a validated pydantic model.
  • Retries only on 429/5xx and network errors, with exponential backoff, jitter and Retry-After.
  • Tests run without a network via httpx.MockTransport.
  • The token comes from GITHUB_TOKEN, never from arguments.
ghrel owner/repo ... --> retry.get() --> httpx.Client --> api.github.com
                              |                              |
                         Backoff/Jitter            JSON --> Release (pydantic)

Drill 1: Project and first request

mkdir -p ~/py-lab && cd ~/py-lab
uv init -q --package --name ghrel tag06-ghrel
cd tag06-ghrel && mkdir tests
uv add -q httpx pydantic typer
uv add -q --dev pytest ruff
uv run python - <<'EOF'
import httpx

r = httpx.get(
    "https://api.github.com/repos/astral-sh/uv/releases/latest",
    headers={"Accept": "application/vnd.github+json"},
    timeout=10,
)
print(r.status_code, r.headers["x-ratelimit-remaining"], r.json()["tag_name"])
open("release.json", "wb").write(r.content)
EOF

Expected (version and counter change):

200 55 0.13.0

Why? Without a token, GitHub allows 60 requests per hour and IP; the header shows what is left. release.json (around 100 KB) serves as illustration material right away.

uv run python -c 'import json; print(sorted(json.load(open("release.json"))))'

Expected (shortened):

['assets', 'assets_url', 'author', 'body', 'created_at', 'draft', 'html_url', ...

Save your progress? Sign up for free

Drill 2: The response as a pydantic model

We only model the fields we need. pydantic ignores everything else by default.

cat > src/ghrel/__init__.py <<'EOF'
"""ghrel – neueste GitHub-Releases im Terminal."""
EOF
cat > src/ghrel/models.py <<'EOF'
"""Pydantic-Modelle: nur die Felder, die wir wirklich brauchen."""

from datetime import UTC, datetime

from pydantic import BaseModel, HttpUrl


class Release(BaseModel):
    tag_name: str
    name: str | None = None
    published_at: datetime
    html_url: HttpUrl
    prerelease: bool = False

    def age_days(self, now: datetime | None = None) -> int:
        now = now or datetime.now(UTC)
        return (now - self.published_at).days
EOF
uv run python - <<'EOF'
import json

from pydantic import ValidationError

from ghrel.models import Release

rel = Release.model_validate(json.load(open("release.json")))
print(repr(rel.tag_name), rel.published_at, type(rel.published_at).__name__)
print(rel.model_dump_json())
try:
    Release.model_validate({"tag_name": "v1", "published_at": "gestern"})
except ValidationError as exc:
    print(exc)
EOF

Expected (shortened):

'0.13.0' 2026-10-09 19:49:55+00:00 datetime
{"tag_name":"0.13.0","name":"0.13.0","published_at":"2026-10-09T19:49:55Z",...}
2 validation errors for Release
published_at
  Input should be a valid datetime or date, input is too short [type=datetime_...
html_url
  Field required [type=missing, input_value={'tag_name': 'v1', ...}, ...]

Why? The ISO string becomes a timezone-aware datetime, missing or broken fields stand out immediately with a precise message – not three functions later as a KeyError.

Tip: uv init --package created a hello-world main() in src/ghrel/__init__.py. For now we replace it with a docstring; in Drill 5, main() will then start the typer CLI.

Save your progress? Sign up for free

Drill 3: Timeouts, retries, backoff

First see what a timeout is. 10.255.255.1 is a non-routable address.

uv run python - <<'EOF'
import httpx

try:
    httpx.get("http://10.255.255.1", timeout=httpx.Timeout(5.0, connect=1.0))
except httpx.TransportError as exc:
    print(type(exc).__name__, "->", exc)
EOF

Expected (after approx. 1 s):

ConnectTimeout -> timed out

Now the retry logic. sleep is a parameter so that tests don’t really wait.

cat > src/ghrel/retry.py <<'EOF'
"""GET mit Timeouts, Retries, exponentiellem Backoff und Retry-After."""

import logging
import random
import time
from collections.abc import Callable

import httpx

log = logging.getLogger(__name__)

RETRY_STATUS = {429, 500, 502, 503, 504}


def backoff(attempt: int, base: float = 0.5, cap: float = 8.0) -> float:
    """Full Jitter: zufällig zwischen 0 und min(cap, base * 2**attempt)."""
    return random.uniform(0, min(cap, base * 2**attempt))


def get(
    client: httpx.Client,
    url: str,
    *,
    retries: int = 3,
    sleep: Callable[[float], None] = time.sleep,
) -> httpx.Response:
    for attempt in range(retries + 1):
        try:
            resp = client.get(url)
        except httpx.TransportError as exc:  # Connect-Fehler, Timeouts
            if attempt == retries:
                raise
            wait, reason = backoff(attempt), type(exc).__name__
        else:
            if resp.status_code not in RETRY_STATUS or attempt == retries:
                return resp
            ra = resp.headers.get("Retry-After", "")
            wait = float(ra) if ra.isdigit() else backoff(attempt)
            reason = str(resp.status_code)
        log.warning("Versuch %d: %s, warte %.2fs", attempt + 1, reason, wait)
        sleep(wait)
    raise AssertionError("unreachable")
EOF
uv run python -c 'from ghrel.retry import backoff
print([round(backoff(a), 2) for a in range(6)])'

Expected (random, but never above 0.5, 1, 2, 4, 8, 8):

[0.3, 0.2, 1.82, 2.37, 5.07, 3.51]

Why? Full jitter spreads the retries of many clients randomly – otherwise they all hammer a recovering server at the same time. 404 or 401 are never retried: a second attempt changes nothing.

The test builds a fake server with MockTransport that returns 503, 429 and 200 in sequence:

cat > tests/test_retry.py <<'EOF'
import httpx

from ghrel import retry


def client_for(*codes: int) -> httpx.Client:
    it = iter(codes)
    transport = httpx.MockTransport(
        lambda req: httpx.Response(next(it), headers={"Retry-After": "2"})
    )
    return httpx.Client(base_url="https://lab.invalid", transport=transport)


def test_retry_until_success():
    waits: list[float] = []
    resp = retry.get(client_for(503, 429, 200), "/x", sleep=waits.append)
    assert resp.status_code == 200
    assert waits == [2.0, 2.0]  # Retry-After schlägt Backoff


def test_retry_gives_up_after_retries():
    resp = retry.get(client_for(502, 502, 502, 502), "/x", sleep=lambda s: None)
    assert resp.status_code == 502


def test_no_retry_on_404():
    waits: list[float] = []
    resp = retry.get(client_for(404), "/x", sleep=waits.append)
    assert resp.status_code == 404
    assert waits == []
EOF
uv run pytest -q tests/test_retry.py

Expected:

...                                                                      [100%]
3 passed in 0.13s

Save your progress? Sign up for free

Drill 4: The GitHub client

An httpx.Client with base_url, fixed headers and a timeout – created once, reused for all requests (connection pooling).

cat > src/ghrel/github.py <<'EOF'
"""Dünner GitHub-Client: Header, Token, Fehlerbehandlung."""

import re

import httpx

from ghrel import retry
from ghrel.models import Release

API = "https://api.github.com"
REPO_RE = re.compile(r"^[\w.-]+/[\w.-]+$")


class GhError(Exception):
    """Fachlicher Fehler mit verständlicher Meldung."""


def make_client(
    token: str | None = None, transport: httpx.BaseTransport | None = None
) -> httpx.Client:
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "User-Agent": "ghrel/0.1",
    }
    if token:
        headers["Authorization"] = f"Bearer {token}"
    return httpx.Client(
        base_url=API,
        headers=headers,
        timeout=httpx.Timeout(10.0, connect=3.0),
        transport=transport,
    )


def latest_release(client: httpx.Client, repo: str) -> Release:
    if not REPO_RE.match(repo):
        raise GhError(f"{repo}: erwartet owner/name")
    resp = retry.get(client, f"/repos/{repo}/releases/latest")
    if resp.status_code == 404:
        raise GhError(f"{repo}: kein Release oder Repo nicht gefunden")
    remaining = resp.headers.get("x-ratelimit-remaining")
    if resp.status_code in (403, 429) and remaining == "0":
        raise GhError("Rate-Limit erreicht – GITHUB_TOKEN setzen")
    resp.raise_for_status()
    return Release.model_validate(resp.json())
EOF

Why? transport is the lever for tests: production uses the network, tests a MockTransport. The repo name is checked by regex before the request – this way no ../ ends up in the URL.

The test response lives as a fixture in conftest.py, which makes it available to all test files:

cat > tests/conftest.py <<'EOF'
import pytest


@pytest.fixture
def release() -> dict:
    """Gekürzte echte Antwort von /repos/astral-sh/uv/releases/latest."""
    return {
        "tag_name": "0.13.0",
        "name": "0.13.0",
        "published_at": "2026-10-09T19:49:55Z",
        "html_url": "https://github.com/astral-sh/uv/releases/tag/0.13.0",
        "prerelease": False,
        "assets": [],  # Extra-Felder ignoriert pydantic
    }
EOF
cat > tests/test_github.py <<'EOF'
from datetime import UTC, datetime

import httpx
import pytest

from ghrel.github import GhError, latest_release, make_client
from ghrel.models import Release


def fake(handler) -> httpx.Client:
    return make_client(transport=httpx.MockTransport(handler))


def test_model_parses_and_ages(release):
    rel = Release.model_validate(release)
    assert rel.published_at.tzinfo is not None
    assert rel.age_days(datetime(2026, 10, 19, 20, tzinfo=UTC)) == 10


def test_latest_release_sends_token(release):
    def handler(req):
        assert req.url.path == "/repos/astral-sh/uv/releases/latest"
        assert req.headers["Authorization"] == "Bearer lab-token-123"  # Lab-Wert
        return httpx.Response(200, json=release)

    client = make_client("lab-token-123", httpx.MockTransport(handler))
    assert latest_release(client, "astral-sh/uv").tag_name == "0.13.0"


@pytest.mark.parametrize(
    ("status", "headers", "msg"),
    [
        (404, {}, "nicht gefunden"),
        (403, {"x-ratelimit-remaining": "0"}, "Rate-Limit"),
    ],
)
def test_latest_release_errors(status, headers, msg):
    client = fake(lambda r: httpx.Response(status, headers=headers))
    with pytest.raises(GhError, match=msg):
        latest_release(client, "acme/nope")


def test_invalid_repo_name():
    with pytest.raises(GhError, match="owner/name"):
        latest_release(fake(lambda r: httpx.Response(200)), "kein-slash")
EOF
uv run pytest -q

Expected:

........                                                                 [100%]
8 passed in 0.72s

Security: lab-token-123 is a lab value. Never put real tokens in tests, code or shell history – always take them from the environment.

Save your progress? Sign up for free

Drill 5: The CLI with typer

typer builds a CLI including --help from type hints and Annotated (Day 3).

cat > src/ghrel/cli.py <<'EOF'
"""CLI: ghrel owner/repo [owner/repo ...]"""

import json
import logging
import os
from typing import Annotated

import httpx
import typer

from ghrel.github import GhError, latest_release, make_client

app = typer.Typer(add_completion=False)


@app.command()
def main(
    repos: Annotated[list[str], typer.Argument(help="owner/name, mehrere möglich")],
    as_json: Annotated[bool, typer.Option("--json", help="JSON statt Tabelle")] = False,
    verbose: Annotated[bool, typer.Option("-v", help="Retries loggen")] = False,
) -> None:
    """Zeigt das neueste Release je GitHub-Repo."""
    logging.basicConfig(level=logging.INFO if verbose else logging.ERROR)
    rows, failed = [], 0
    with make_client(os.environ.get("GITHUB_TOKEN")) as client:
        for repo in repos:
            try:
                rel = latest_release(client, repo)
            except (GhError, httpx.HTTPError) as exc:
                typer.echo(f"✘ {exc}", err=True)
                failed += 1
                continue
            rows.append(
                {
                    "repo": repo,
                    "tag": rel.tag_name,
                    "published": rel.published_at.date().isoformat(),
                    "age_days": rel.age_days(),
                    "url": str(rel.html_url),
                }
            )
    if as_json:
        typer.echo(json.dumps(rows, indent=2))
    else:
        for r in rows:
            typer.echo(
                f"{r['repo']:<24} {r['tag']:<12} {r['published']}  "
                f"vor {r['age_days']} Tagen"
            )
    raise typer.Exit(code=1 if failed else 0)
EOF
cat > src/ghrel/__init__.py <<'EOF'
"""ghrel – neueste GitHub-Releases im Terminal."""

from ghrel.cli import app


def main() -> None:
    app()
EOF
cat > src/ghrel/__main__.py <<'EOF'
from ghrel import main

main()
EOF
uv run ghrel --help

Expected (shortened):

 Usage: ghrel [OPTIONS] {repos}...
 Zeigt das neueste Release je GitHub-Repo.
│ *    repos      <str>  owner/name, mehrere möglich [required]                │
│ --json            JSON statt Tabelle                                         │
│         -v        Retries loggen                                             │

Why? [project.scripts] still points to ghrel:main – which now starts the typer app, and __main__.py additionally enables python -m ghrel. An error for one repo doesn’t abort everything: message on stderr, on to the next one, exit code 1 at the end. That is the behavior you want in shell pipelines and CI.

Save your progress? Sign up for free

Drill 6: CLI test and lint

The CLI test swaps out only make_client via monkeypatch – the rest runs for real.

cat > tests/test_cli.py <<'EOF'
import json

import httpx
from typer.testing import CliRunner

from ghrel import cli
from ghrel.github import make_client


def test_cli_json_and_exit_code(monkeypatch, release):
    def handler(req):
        if "missing" in req.url.path:
            return httpx.Response(404)
        return httpx.Response(200, json=release)

    transport = httpx.MockTransport(handler)
    monkeypatch.setattr(cli, "make_client", lambda tok: make_client(tok, transport))
    result = CliRunner().invoke(cli.app, ["astral-sh/uv", "acme/missing", "--json"])
    assert result.exit_code == 1
    assert json.loads(result.stdout)[0]["tag"] == "0.13.0"
    assert "acme/missing" in result.stderr
EOF
cat >> pyproject.toml <<'EOF'

[tool.ruff.lint]
extend-select = ["E501", "B", "UP", "SIM"]

[tool.pytest.ini_options]
testpaths = ["tests"]
EOF
uv run pytest -q && uv run ruff check . && uv run ruff format --check .

Expected:

.........                                                                [100%]
9 passed in 0.21s
All checks passed!
11 files already formatted

Save your progress? Sign up for free

Drill 7: The tool is done

With a token, the limit rises to 5000 requests per hour. If you have the GitHub CLI, you get the token without ever displaying it:

export GITHUB_TOKEN="$(gh auth token)"
uv run ghrel astral-sh/uv astral-sh/ruff openbao/openbao acme/gibtsnicht-xyz
echo "exit=$?"
uv run ghrel --json pydantic/pydantic
uv run ghrel -v astral-sh/uv

Expected (versions and dates change):

✘ acme/gibtsnicht-xyz: kein Release oder Repo nicht gefunden
astral-sh/uv             0.13.0       2026-10-09  vor 0 Tagen
astral-sh/ruff           0.17.0       2026-10-09  vor 0 Tagen
openbao/openbao          v2.7.1       2026-10-01  vor 8 Tagen
exit=1
[
  {
    "repo": "pydantic/pydantic",
    "tag": "v2.14.0",
    "published": "2026-10-08",
    "age_days": 1,
    "url": "https://github.com/pydantic/pydantic/releases/tag/v2.14.0"
  }
]
INFO:httpx:HTTP Request: GET https://api.github.com/repos/astral-sh/uv/releases/...
astral-sh/uv             0.13.0       2026-10-09  vor 0 Tagen

Why is the error message at the top? ghrel reports errors immediately, the table only after the loop. With --json and jq, ghrel becomes a building block: uv run ghrel --json astral-sh/uv | jq -r '.[0].tag'.

Tip: No gh? Then create a fine-grained token without permissions (public repos only) and set it with read -s GITHUB_TOKEN; export GITHUB_TOKEN – this way it doesn’t end up in the history.

Save your progress? Sign up for free

Checkpoint

#!/usr/bin/env bash
check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
cd ~/py-lab/tag06-ghrel
check "Tests grün" "uv run pytest -q"
check "ruff sauber" "uv run ruff check . && uv run ruff format --check ."
check "Mock statt Netz in Tests" "grep -q MockTransport tests/test_github.py"
check "kein Token im Code" "! grep -rqE 'gh[pousr]_[A-Za-z0-9]{20,}' src tests"
check "falscher Name -> Exit 1" "! uv run ghrel kein-slash"
check "Live-Abfrage" "uv run ghrel astral-sh/uv | grep -q astral-sh/uv"

Expected:

✔ Tests grün
✔ ruff sauber
✔ Mock statt Netz in Tests
✔ kein Token im Code
✔ falscher Name -> Exit 1
✔ Live-Abfrage

Challenge

Build an ETag cache: ghrel remembers the ETag and the release per repo in ~/.cache/ghrel.json and sends If-None-Match on the next run. For authenticated requests, a 304 Not Modified does not count against the rate limit.

Hint: retry.get needs an additional parameter headers, latest_release an optional cache: dict.

Solution

In src/ghrel/retry.py, add the parameter and pass it through:

    retries: int = 3,
    headers: dict[str, str] | None = None,
    sleep: Callable[[float], None] = time.sleep,
...
            resp = client.get(url, headers=headers)

In src/ghrel/github.py, replace the function latest_release:

def latest_release(
    client: httpx.Client, repo: str, cache: dict | None = None
) -> Release:
    if not REPO_RE.match(repo):
        raise GhError(f"{repo}: erwartet owner/name")
    cached = (cache or {}).get(repo)
    headers = {"If-None-Match": cached["etag"]} if cached else {}
    resp = retry.get(client, f"/repos/{repo}/releases/latest", headers=headers)
    if resp.status_code == 304 and cached:  # zählt nicht gegen das Rate-Limit
        return Release.model_validate(cached["data"])
    if resp.status_code == 404:
        raise GhError(f"{repo}: kein Release oder Repo nicht gefunden")
    remaining = resp.headers.get("x-ratelimit-remaining")
    if resp.status_code in (403, 429) and remaining == "0":
        raise GhError("Rate-Limit erreicht – GITHUB_TOKEN setzen")
    resp.raise_for_status()
    rel = Release.model_validate(resp.json())
    if cache is not None and "etag" in resp.headers:
        cache[repo] = {
            "etag": resp.headers["etag"],
            "data": rel.model_dump(mode="json"),
        }
    return rel

In src/ghrel/cli.py, load the cache, pass it through and save it:

from pathlib import Path
...
CACHE = Path(os.environ.get("GHREL_CACHE", Path.home() / ".cache" / "ghrel.json"))
...
    cache = json.loads(CACHE.read_text()) if CACHE.exists() else {}
    with make_client(os.environ.get("GITHUB_TOKEN")) as client:
        for repo in repos:
            try:
                rel = latest_release(client, repo, cache)
...
    CACHE.parent.mkdir(parents=True, exist_ok=True)
    CACHE.write_text(json.dumps(cache))

So that the CLI test doesn’t write into your real ~/.cache, extend the signature in tests/test_cli.py with tmp_path and add as the first line in the test:

def test_cli_json_and_exit_code(monkeypatch, release, tmp_path):
    monkeypatch.setattr(cli, "CACHE", tmp_path / "ghrel.json")

Append a test to tests/test_github.py:

def test_etag_cache_uses_304(release):
    seen: list[str | None] = []

    def handler(req):
        seen.append(req.headers.get("If-None-Match"))
        if req.headers.get("If-None-Match") == '"abc"':
            return httpx.Response(304)
        return httpx.Response(200, json=release, headers={"ETag": '"abc"'})

    cache: dict = {}
    client = fake(handler)
    assert latest_release(client, "astral-sh/uv", cache).tag_name == "0.13.0"
    assert latest_release(client, "astral-sh/uv", cache).tag_name == "0.13.0"
    assert seen == [None, '"abc"']
uv run pytest -q
export GHREL_CACHE=$PWD/cache.json
uv run ghrel -v astral-sh/uv 2>&1 | grep HTTP
uv run ghrel -v astral-sh/uv 2>&1 | grep HTTP

Expected (shortened):

10 passed in 0.34s
INFO:httpx:HTTP Request: GET https://api.github.com/.../latest "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET https://api.github.com/.../latest "HTTP/1.1 304 Not ...

Save your progress? Sign up for free

Enterprise Note

  • GitHub Enterprise Server has a different base URL (https://ghe.example.com/api/v3) – make API configurable via an environment variable.
  • Corporate proxy and your own CA: httpx reads HTTPS_PROXY; the CA via SSL_CERT_FILE or httpx.Client(verify=ssl_context). Never verify=False.
  • For automation, use a GitHub App or a fine-grained token with minimal permissions instead of personal tokens; take the token from Vault/OpenBao or the CI secret store.
  • Retries need a budget: with many clients, circuit breakers and global rate limits cap how much load retries generate.
  • Logs without the Authorization header: httpx doesn’t log headers, your own debug output must mask secrets.