Day 4: Testing – semver
Goal: Build a
semvertool that compares, sorts and bumps versions according to semver.org – backed by table tests, subtests, examples, golden files, fuzzing, benchmarks and coverage · Duration: approx. 150 min · Prerequisite: Day 3
Briefing
- Tests live next to the code in
*_test.go;go testfindsTestXxx(t *testing.T),BenchmarkXxx,FuzzXxxandExampleXxxon its own. No framework needed. - Table-driven: a slice of test cases plus a loop;
t.Runturns them into named subtests that you can run individually. - Golden files keep expected output in
testdata/;-updaterewrites them. - Fuzzing generates inputs you would never have
thought of; findings end up as regression tests in
testdata/fuzz/. - Semver rules you test:
1.10.0 > 1.2.0,1.0.0-rc.1 < 1.0.0,beta.11 > beta.2, build metadata (+…) doesn’t count.
internal/semver cmd/semver
semver.go Parse, String ◄──┐ main.go run(args, stdin)
compare.go Compare, Sort │ main_test.go TestRun, TestSortGolden
bump.go Bump │ testdata/ sort.input, sort.golden
*_test.go Table, Example, Fuzz, Bench ──► go test -run / -fuzz / -bench / -cover
Drill 1: Parse and String
mkdir -p ~/go-lab/tag04-semver && cd ~/go-lab/tag04-semver
go mod init example.com/semver
mkdir -p cmd/semver internal/semver
cat > internal/semver/semver.go <<'EOF'
// Package semver parses, compares and bumps versions according to semver.org 2.0.0.
package semver
import (
"errors"
"fmt"
"strconv"
"strings"
)
// ErrInvalid reports an invalid version.
var ErrInvalid = errors.New("ungültige Version")
// Version is a semantic version like v1.4.2-rc.1+build.7.
type Version struct {
Major, Minor, Patch int
Pre []string // prerelease, e.g. ["rc", "1"]
Build string // build metadata, doesn't count when comparing
V bool // written with a leading "v"
}
// Parse splits s. The leading "v" is optional.
func Parse(s string) (Version, error) {
var v Version
rest, hasV := strings.CutPrefix(s, "v")
v.V = hasV
rest, build, hasBuild := strings.Cut(rest, "+")
core, pre, hasPre := strings.Cut(rest, "-")
parts := strings.Split(core, ".")
if len(parts) != 3 {
return Version{}, fmt.Errorf("%w %q: brauche MAJOR.MINOR.PATCH", ErrInvalid, s)
}
nums := [3]*int{&v.Major, &v.Minor, &v.Patch}
for i, p := range parts {
n, err := number(p)
if err != nil {
return Version{}, fmt.Errorf("%w %q: %v", ErrInvalid, s, err)
}
*nums[i] = n
}
if hasPre {
v.Pre = strings.Split(pre, ".")
for _, id := range v.Pre {
if err := identifier(id); err != nil {
return Version{}, fmt.Errorf("%w %q: %v", ErrInvalid, s, err)
}
}
}
if hasBuild {
for id := range strings.SplitSeq(build, ".") {
if id == "" || strings.Trim(id, idChars) != "" {
return Version{}, fmt.Errorf("%w %q: Build %q", ErrInvalid, s, id)
}
}
v.Build = build
}
return v, nil
}
const idChars = "0123456789" +
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-"
// number checks a number without leading zeros.
func number(s string) (int, error) {
if s == "" || strings.Trim(s, "0123456789") != "" {
return 0, fmt.Errorf("%q ist keine Zahl", s)
}
if len(s) > 1 && s[0] == '0' {
return 0, fmt.Errorf("%q hat führende Null", s)
}
return strconv.Atoi(s)
}
// identifier checks a prerelease part like "rc" or "1".
func identifier(id string) error {
if id == "" || strings.Trim(id, idChars) != "" {
return fmt.Errorf("prerelease %q ungültig", id)
}
if isNum(id) && len(id) > 1 && id[0] == '0' {
return fmt.Errorf("prerelease %q hat führende Null", id)
}
return nil
}
func isNum(s string) bool { return strings.Trim(s, "0123456789") == "" }
// String puts the version back together.
func (v Version) String() string {
s := fmt.Sprintf("%d.%d.%d", v.Major, v.Minor, v.Patch)
if v.V {
s = "v" + s
}
if len(v.Pre) > 0 {
s += "-" + strings.Join(v.Pre, ".")
}
if v.Build != "" {
s += "+" + v.Build
}
return s
}
EOF
go vet ./...Expected:
go: creating new go.mod: module example.com/semver
Why? strings.Cut splits at the first occurrence and
tells you with the third return value whether the separator was there –
that cleanly separates + (build) and -
(prerelease). nums is an array of pointers
to the three fields, so one loop fills Major, Minor and Patch.
strings.SplitSeq (Go 1.24) returns an iterator instead of a
slice; go fix suggests that on its own.
Save your progress? Sign up for free
Drill 2: The first table test
cat > internal/semver/semver_test.go <<'EOF'
package semver
import (
"errors"
"reflect"
"testing"
)
func TestParse(t *testing.T) {
tests := []struct {
in string
want Version
wantErr bool
}{
{in: "1.2.3", want: Version{Major: 1, Minor: 2, Patch: 3}},
{in: "v0.10.0", want: Version{Minor: 10, V: true}},
{in: "1.0.0-rc.1", want: Version{Major: 1, Pre: []string{"rc", "1"}}},
{in: "1.0.0+git.abc", want: Version{Major: 1, Build: "git.abc"}},
{in: "1.2", wantErr: true},
{in: "1.02.3", wantErr: true},
{in: "1.2.3-", wantErr: true},
{in: "1.2.3-rc..1", wantErr: true},
{in: "1.2.3-01", wantErr: true},
{in: "1.2.3+", wantErr: true},
{in: "x.y.z", wantErr: true},
}
for _, tt := range tests {
got, err := Parse(tt.in)
if tt.wantErr {
if !errors.Is(err, ErrInvalid) {
t.Errorf("Parse(%q) err = %v, will ErrInvalid", tt.in, err)
}
continue
}
if err != nil {
t.Errorf("Parse(%q) unerwarteter Fehler: %v", tt.in, err)
continue
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("Parse(%q)\n got: %+v\nwant: %+v", tt.in, got, tt.want)
}
}
}
EOF
go test ./...
go test -v -run TestParse ./internal/semverExpected:
? example.com/semver/cmd/semver [no test files]
ok example.com/semver/internal/semver 0.164s
=== RUN TestParse
--- PASS: TestParse (0.00s)
PASS
ok example.com/semver/internal/semver 0.129s
Now break it on purpose – you want to know what a failure looks like:
perl -pi -e 's/Minor: 2, Patch: 3\}/Minor: 2, Patch: 4}/' internal/semver/semver_test.go
go test ./internal/semver
perl -pi -e 's/Minor: 2, Patch: 4\}/Minor: 2, Patch: 3}/' internal/semver/semver_test.goExpected:
--- FAIL: TestParse (0.00s)
semver_test.go:40: Parse("1.2.3")
got: 1.2.3
want: 1.2.4
FAIL
FAIL example.com/semver/internal/semver 0.166s
FAIL
Why? The message names the file, line, input, actual and expected
value – that is how your test messages should always look.
%+v uses the String() method, because that
makes Version satisfy fmt.Stringer
(interfaces, Day 2). t.Errorf reports and carries on,
t.Fatalf aborts the test. reflect.DeepEqual
also compares slices and nested structs. (Test messages like “will” mean
“want”.)
Save your progress? Sign up for free
Drill 3: Compare, Bump and subtests
cat > internal/semver/compare.go <<'EOF'
package semver
import (
"cmp"
"slices"
)
// Compare returns -1, 0 or +1 – suitable for slices.SortFunc.
func Compare(a, b Version) int {
if c := cmp.Or(
cmp.Compare(a.Major, b.Major),
cmp.Compare(a.Minor, b.Minor),
cmp.Compare(a.Patch, b.Patch),
); c != 0 {
return c
}
// Without a prerelease is greater: 1.0.0-rc.1 < 1.0.0
switch {
case len(a.Pre) == 0 && len(b.Pre) == 0:
return 0
case len(a.Pre) == 0:
return 1
case len(b.Pre) == 0:
return -1
}
for i := range min(len(a.Pre), len(b.Pre)) {
if c := compareID(a.Pre[i], b.Pre[i]); c != 0 {
return c
}
}
return cmp.Compare(len(a.Pre), len(b.Pre)) // rc < rc.1
}
// compareID: numbers numerically, numbers < text, text by ASCII.
func compareID(a, b string) int {
an, bn := isNum(a), isNum(b)
switch {
case an && bn: // no leading zeros: shorter = smaller, no overflow
return cmp.Or(cmp.Compare(len(a), len(b)), cmp.Compare(a, b))
case an:
return -1
case bn:
return 1
}
return cmp.Compare(a, b)
}
// Sort sorts vs in ascending order.
func Sort(vs []Version) { slices.SortStableFunc(vs, Compare) }
EOF
cat > internal/semver/bump.go <<'EOF'
package semver
import "fmt"
// Bump erhöht part ("major", "minor" oder "patch") wie npm und Cargo:
// Eine Prerelease wird zuerst "fertig", 1.3.0-rc.1 +minor = 1.3.0.
func Bump(v Version, part string) (Version, error) {
pre := len(v.Pre) > 0
switch part {
case "major":
if !pre || v.Minor != 0 || v.Patch != 0 {
v.Major, v.Minor, v.Patch = v.Major+1, 0, 0
}
case "minor":
if !pre || v.Patch != 0 {
v.Minor, v.Patch = v.Minor+1, 0
}
case "patch":
if !pre {
v.Patch++
}
default:
return v, fmt.Errorf("unbekannter Teil %q (major, minor, patch)", part)
}
v.Pre, v.Build = nil, ""
return v, nil
}
EOF
cat > internal/semver/compare_test.go <<'EOF'
package semver
import "testing"
// mustParse is a test helper: t.Helper() reports errors at the caller.
func mustParse(t *testing.T, s string) Version {
t.Helper()
v, err := Parse(s)
if err != nil {
t.Fatalf("Parse(%q): %v", s, err)
}
return v
}
func TestCompare(t *testing.T) {
tests := []struct {
a, b string
want int
}{
{"1.2.3", "1.2.3", 0},
{"1.2.3", "1.10.0", -1},
{"2.0.0", "1.99.99", 1},
{"1.0.0-rc.1", "1.0.0", -1},
{"1.0.0-alpha", "1.0.0-alpha.1", -1},
{"1.0.0-alpha.beta", "1.0.0-beta", -1},
{"1.0.0-beta.2", "1.0.0-beta.11", -1},
{"1.0.0-1", "1.0.0-alpha", -1},
{"1.0.0+build.1", "1.0.0+build.2", 0},
{"v1.0.0", "1.0.0", 0},
}
for _, tt := range tests {
t.Run(tt.a+"_vs_"+tt.b, func(t *testing.T) {
a, b := mustParse(t, tt.a), mustParse(t, tt.b)
if got := Compare(a, b); got != tt.want {
t.Errorf("Compare = %d, will %d", got, tt.want)
}
if got := Compare(b, a); got != -tt.want {
t.Errorf("umgekehrt = %d, will %d", got, -tt.want)
}
})
}
}
func TestBump(t *testing.T) {
tests := []struct{ in, part, want string }{
{"1.2.3", "patch", "1.2.4"},
{"1.2.3", "minor", "1.3.0"},
{"v1.2.3", "major", "v2.0.0"},
{"1.2.3-rc.1", "patch", "1.2.3"},
{"1.3.0-rc.1", "minor", "1.3.0"},
{"1.3.1-rc.1", "minor", "1.4.0"},
{"2.0.0-beta", "major", "2.0.0"},
{"1.2.3+git.abc", "patch", "1.2.4"},
}
for _, tt := range tests {
t.Run(tt.in+"_"+tt.part, func(t *testing.T) {
got, err := Bump(mustParse(t, tt.in), tt.part)
if err != nil {
t.Fatal(err)
}
if got.String() != tt.want {
t.Errorf("Bump = %s, will %s", got, tt.want)
}
})
}
if _, err := Bump(Version{}, "micro"); err == nil {
t.Error("Bump(micro) ohne Fehler")
}
}
EOF
go test ./internal/semver
go test -v -run 'TestCompare/beta' ./internal/semverExpected:
ok example.com/semver/internal/semver 0.164s
=== RUN TestCompare
=== RUN TestCompare/1.0.0-alpha.beta_vs_1.0.0-beta
=== RUN TestCompare/1.0.0-beta.2_vs_1.0.0-beta.11
--- PASS: TestCompare (0.00s)
--- PASS: TestCompare/1.0.0-alpha.beta_vs_1.0.0-beta (0.00s)
--- PASS: TestCompare/1.0.0-beta.2_vs_1.0.0-beta.11 (0.00s)
PASS
ok example.com/semver/internal/semver 0.176s
Why? -run takes one regex per level, separated by
/ – TestCompare/beta runs only the subtests
with “beta” in their name. Every case also checks the reverse direction:
a comparison that is not antisymmetric breaks any sort.
mustParse calls t.Helper() so that an error
shows the line in the test, not the one in the helper.
compareID compares numeric prerelease parts by their length
– so even beta.99999999999999999999 doesn’t overflow.
Save your progress? Sign up for free
Drill 4: Examples – documentation that gets tested
cat > internal/semver/example_test.go <<'EOF'
package semver_test
import (
"fmt"
"example.com/semver/internal/semver"
)
// Examples are documentation AND tests: go test compares the Output lines.
func ExampleBump() {
v, _ := semver.Parse("v1.4.2")
next, _ := semver.Bump(v, "minor")
fmt.Println(next)
// Output: v1.5.0
}
func ExampleSort() {
var vs []semver.Version
for _, s := range []string{"1.10.0", "1.2.0", "1.2.0-rc.1"} {
v, _ := semver.Parse(s)
vs = append(vs, v)
}
semver.Sort(vs)
fmt.Println(vs)
// Output: [1.2.0-rc.1 1.2.0 1.10.0]
}
EOF
go test -v -run Example ./internal/semverExpected:
=== RUN ExampleBump
--- PASS: ExampleBump (0.00s)
=== RUN ExampleSort
--- PASS: ExampleSort (0.00s)
PASS
ok example.com/semver/internal/semver 0.163s
Why? go test compares everything an example prints with
the // Output: comment. go doc and pkg.go.dev
show the same examples as documentation – so they can never go stale.
The package semver_test tests from the outside, like a real
user: exported names only.
Save your progress? Sign up for free
Drill 5: The CLI and golden files
main gets a function run that knows nothing
about os.Args, os.Stdin or
os.Exit – which makes the whole tool testable:
cat > cmd/semver/main.go <<'EOF'
// semver compares, sorts and bumps versions according to semver.org.
// Day 4 from "Go hands-on".
package main
import (
"bufio"
"errors"
"fmt"
"io"
"os"
"strings"
"example.com/semver/internal/semver"
)
const usage = `Aufruf:
semver compare A B gibt A<B, A=B oder A>B aus
semver bump PART VERSION PART: major, minor, patch
semver sort sortiert Versionen von stdin`
var errUsage = errors.New("falscher Aufruf")
func main() {
out, err := run(os.Args[1:], os.Stdin)
fmt.Print(out)
switch {
case errors.Is(err, errUsage):
fmt.Fprintln(os.Stderr, usage)
os.Exit(2)
case err != nil:
fmt.Fprintln(os.Stderr, "semver:", err)
os.Exit(1)
}
}
// run is main without os.*: arguments and stdin in, output and error out.
// That way tests can call the whole tool.
func run(args []string, stdin io.Reader) (string, error) {
if len(args) == 0 {
return "", errUsage
}
switch cmd := args[0]; {
case cmd == "compare" && len(args) == 3:
a, err := semver.Parse(args[1])
if err != nil {
return "", err
}
b, err := semver.Parse(args[2])
if err != nil {
return "", err
}
op := [...]string{"<", "=", ">"}[semver.Compare(a, b)+1]
return fmt.Sprintf("%s %s %s\n", a, op, b), nil
case cmd == "bump" && len(args) == 3:
v, err := semver.Parse(args[2])
if err != nil {
return "", err
}
v, err = semver.Bump(v, args[1])
if err != nil {
return "", err
}
return v.String() + "\n", nil
case cmd == "sort" && len(args) == 1:
return sortVersions(stdin)
}
return "", errUsage
}
// sortVersions reads one version per line; empty lines are skipped.
func sortVersions(r io.Reader) (string, error) {
var vs []semver.Version
sc := bufio.NewScanner(r)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" {
continue
}
v, err := semver.Parse(line)
if err != nil {
return "", err
}
vs = append(vs, v)
}
if err := sc.Err(); err != nil {
return "", err
}
semver.Sort(vs)
var b strings.Builder
for _, v := range vs {
b.WriteString(v.String() + "\n")
}
return b.String(), nil
}
EOF
mkdir -p cmd/semver/testdata
cat > cmd/semver/testdata/sort.input <<'EOF'
1.0.0
1.0.0-rc.1
v0.9.12
1.0.0-beta.11
1.0.0-alpha
1.0.0-beta.2
1.0.0-alpha.beta
1.0.0-alpha.1
0.10.0
1.0.0-beta
EOF
cat > cmd/semver/main_test.go <<'EOF'
package main
import (
"errors"
"flag"
"os"
"path/filepath"
"strings"
"testing"
"example.com/semver/internal/semver"
)
var update = flag.Bool("update", false, "Golden Files neu schreiben")
func TestSortGolden(t *testing.T) {
in, err := os.Open(filepath.Join("testdata", "sort.input"))
if err != nil {
t.Fatal(err)
}
defer in.Close() //nolint:errcheck // read only
got, err := run([]string{"sort"}, in)
if err != nil {
t.Fatal(err)
}
golden := filepath.Join("testdata", "sort.golden")
if *update {
if err := os.WriteFile(golden, []byte(got), 0o644); err != nil {
t.Fatal(err)
}
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if got != string(want) {
t.Errorf("Ausgabe weicht ab:\n%s\nwill (%s):\n%s", got, golden, want)
}
}
func TestRun(t *testing.T) {
tests := []struct {
name string
args []string
want string
wantErr error
}{
{"compare", []string{"compare", "1.2.3", "1.10.0"}, "1.2.3 < 1.10.0\n", nil},
{"bump", []string{"bump", "minor", "v1.2.3"}, "v1.3.0\n", nil},
{"leer", nil, "", errUsage},
{"zu wenig", []string{"compare", "1.0.0"}, "", errUsage},
{"kaputt", []string{"bump", "patch", "1.x.0"}, "", semver.ErrInvalid},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := run(tt.args, strings.NewReader(""))
if !errors.Is(err, tt.wantErr) { // errors.Is(nil, nil) is true
t.Fatalf("err = %v, will %v", err, tt.wantErr)
}
if got != tt.want {
t.Errorf("Ausgabe = %q, will %q", got, tt.want)
}
})
}
}
EOF
go test ./cmd/semverExpected:
--- FAIL: TestSortGolden (0.00s)
main_test.go:35: open testdata/sort.golden: no such file or directory
FAIL
FAIL example.com/semver/cmd/semver 0.242s
FAIL
The golden file doesn’t exist yet. Have it generated, check it with your own eyes and test again:
go test ./cmd/semver -run Golden -update
cat cmd/semver/testdata/sort.golden
go test ./cmd/semverExpected:
ok example.com/semver/cmd/semver 0.129s
v0.9.12
0.10.0
1.0.0-alpha
1.0.0-alpha.1
1.0.0-alpha.beta
1.0.0-beta
1.0.0-beta.2
1.0.0-beta.11
1.0.0-rc.1
1.0.0
ok example.com/semver/cmd/semver 0.140s
Why? The order
alpha < alpha.1 < alpha.beta < beta < beta.2 < beta.11 < rc.1 < 1.0.0
is exactly the example from the specification on semver.org. Golden
files pay off for longer output (reports, generated YAML): if the output
changes, the diff in the Git review shows exactly what.
-update is a custom test flag, defined with
flag.Bool in the test. Go tests always run in their
package’s folder, which is why the relative path testdata/…
works.
Save your progress? Sign up for free
Drill 6: Fuzzing – inputs you never think of
Two fuzz tests: the first checks a property
(whatever Parse accepts, String returns unchanged). The second compares
differentially with
golang.org/x/mod/semver, the implementation the Go tooling
itself uses:
go get golang.org/x/[email protected]
cat > internal/semver/fuzz_test.go <<'EOF'
package semver
import (
"strings"
"testing"
xsemver "golang.org/x/mod/semver"
)
// FuzzParse: whatever Parse accepts, String() must return exactly.
func FuzzParse(f *testing.F) {
for _, s := range []string{"1.2.3", "v1.0.0-rc.1+b.7", "0.0.0-0.a-b"} {
f.Add(s) // seed corpus
}
f.Fuzz(func(t *testing.T, s string) {
v, err := Parse(s)
if err != nil {
return // invalid inputs are ok, just no panic
}
if got := v.String(); got != s {
t.Errorf("Parse(%q).String() = %q", s, got)
}
if Compare(v, v) != 0 {
t.Errorf("Compare(%q, %q) != 0", s, s)
}
})
}
// FuzzCompare compares with golang.org/x/mod/semver (differential fuzzing).
func FuzzCompare(f *testing.F) {
f.Add("1.0.0-alpha.1", "1.0.0-alpha.beta")
f.Add("1.0.0-beta.11", "1.0.0-beta.2")
f.Fuzz(func(t *testing.T, a, b string) {
va, errA := Parse(a)
vb, errB := Parse(b)
if errA != nil || errB != nil {
return
}
ref := xsemver.Compare(withV(a), withV(b))
if got := Compare(va, vb); got != ref {
t.Errorf("Compare(%q, %q) = %d, x/mod sagt %d", a, b, got, ref)
}
})
}
func withV(s string) string { return "v" + strings.TrimPrefix(s, "v") }
EOF
go mod tidy
go test -run '^$' -fuzz '^FuzzParse$' -fuzztime 10s ./internal/semverExpected (shortened, numbers vary):
go: added golang.org/x/mod v0.42.0
fuzz: elapsed: 0s, gathering baseline coverage: 0/3 completed
fuzz: elapsed: 3s, execs: 144061 (48002/sec), new interesting: 98 (total: 101)
fuzz: elapsed: 6s, execs: 309666 (55213/sec), new interesting: 131 (total: 134)
fuzz: elapsed: 9s, execs: 508360 (66239/sec), new interesting: 150 (total: 153)
fuzz: elapsed: 11s, execs: 583914 (37592/sec), new interesting: 159 (total: 162)
PASS
ok example.com/semver/internal/semver 11.386s
Nothing found – good. So you can see what fuzzing achieves, you
switch off the check for leading zeros in prerelease parts
(1.0.0-01 is invalid according to the specification):
perl -pi -e 's/if isNum\(id\) &&/if false \&\&/' internal/semver/semver.go
go test -run '^$' -fuzz '^FuzzCompare$' -fuzztime 30s ./internal/semverExpected (shortened; your input and hash will differ):
fuzz: minimizing 63-byte failing input file
fuzz: elapsed: 0s, minimizing
--- FAIL: FuzzCompare (0.07s)
--- FAIL: FuzzCompare (0.00s)
fuzz_test.go:41: Compare("1.0.0-c.00", "0.0.0+0") = 1, x/mod sagt -1
Failing input written to testdata/fuzz/FuzzCompare/2ec74667cff9214b
To re-run:
go test -run=FuzzCompare/2ec74667cff9214b
FAIL
Found in milliseconds: 1.0.0-c.00 is invalid, but your
code takes it for a version and sorts it above 0.0.0.
Switch the check back on and keep the finding:
perl -pi -e 's/if false &&/if isNum(id) \&\&/' internal/semver/semver.go
go test -v -run FuzzCompare ./internal/semver | grep -- '--- PASS'
ls internal/semver/testdata/fuzz/FuzzCompare/Expected:
--- PASS: FuzzCompare (0.00s)
--- PASS: FuzzCompare/seed#0 (0.00s)
--- PASS: FuzzCompare/seed#1 (0.00s)
--- PASS: FuzzCompare/2ec74667cff9214b (0.00s)
2ec74667cff9214b
Why? A normal go test runs the seeds
and the saved findings – without -fuzz, so
fast enough for any CI. You commit the testdata/fuzz/
folder; it is your growing regression test. Fuzzing itself runs on only
one package at a time and needs -fuzz with exactly one
matching test.
Save your progress? Sign up for free
Drill 7: Benchmarks, coverage, race
cat > internal/semver/bench_test.go <<'EOF'
package semver
import "testing"
func BenchmarkParse(b *testing.B) {
for b.Loop() { // since Go 1.24: measures only the loop body
if _, err := Parse("v1.42.7-rc.3+build.2026"); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkCompare(b *testing.B) {
x, _ := Parse("1.0.0-beta.11")
y, _ := Parse("1.0.0-beta.2")
for b.Loop() {
Compare(x, y)
}
}
EOF
go test -run '^$' -bench . -benchmem ./internal/semver
go test -cover ./...
go test -coverprofile=cover.out ./internal/semver >/dev/null
go tool cover -func=cover.out | tail -4
go test -race -count=1 ./...Expected (times depend on your hardware):
goos: darwin
goarch: arm64
pkg: example.com/semver/internal/semver
cpu: Apple M4
BenchmarkParse-10 1203819 922.0 ns/op 112 B/op 3 allocs/op
BenchmarkCompare-10 25468334 48.37 ns/op 0 B/op 0 allocs/op
PASS
ok example.com/semver/internal/semver 2.494s
ok example.com/semver/cmd/semver 0.137s coverage: 72.7% of statements
ok example.com/semver/internal/semver 0.264s coverage: 100.0% of statements
example.com/semver/internal/semver/semver.go:76: identifier 100.0%
example.com/semver/internal/semver/semver.go:86: isNum 100.0%
example.com/semver/internal/semver/semver.go:89: String 100.0%
total: (statements) 100.0%
ok example.com/semver/cmd/semver 1.275s
ok example.com/semver/internal/semver 1.149s
Why? b.Loop() (since Go 1.24) replaces the old
for i := 0; i < b.N; i++: setup before the loop is not
measured, and the compiler may not optimize the call away.
-benchmem shows allocations – often the bigger lever than
CPU (Day 13). 100 % coverage only means “every line ran once”, not
“everything is correct” – that is what you have fuzzing for.
cmd/semver is at 73 % because main itself is
not tested; that is exactly why the logic lives in run.
-race finds data races as soon as goroutines are involved
(Day 8); get used to it now.
Tip:
go tool cover -html=cover.outopens a colored view in the browser: green tested, red untested.
Save your progress? Sign up for free
Drill 8: The tool is done
cd ~/go-lab/tag04-semver
gofmt -l . && go vet ./... && ~/go-lab/bin/lint run ./...
go build -o bin/semver ./cmd/semver
./bin/semver compare 1.2.3 1.10.0
./bin/semver compare v1.0.0-rc.1 v1.0.0
./bin/semver bump minor v1.2.3
./bin/semver bump major 2.0.0-rc.1
printf 'v1.31.2\nv1.32.0-rc.1\nv1.32.0\nv1.31.10\n' > tags.txt
./bin/semver sort < tags.txt | tail -1
sort -V tags.txt | tail -1
./bin/semver compare 1.02.0 1.0.0; echo "exit=$?"
./bin/semver; echo "exit=$?"Expected:
0 issues.
1.2.3 < 1.10.0
v1.0.0-rc.1 < v1.0.0
v1.3.0
2.0.0
v1.32.0
v1.32.0-rc.1
semver: ungültige Version "1.02.0": "02" hat führende Null
exit=1
Aufruf:
semver compare A B gibt A<B, A=B oder A>B aus
semver bump PART VERSION PART: major, minor, patch
semver sort sortiert Versionen von stdin
exit=2
Do you see the two v1.32.0 lines? sort -V
considers v1.32.0-rc.1 newer than v1.32.0 –
according to semver it’s the other way around. Exactly for cases like
this, a tested tool of your own pays off. In release scripts (Day 14),
git tag | semver sort | tail -1 returns the latest tag and
semver bump patch the next one.
Your module:
tag04-semver/
├── cmd/semver/
│ ├── main.go, main_test.go
│ └── testdata/sort.input, sort.golden
├── internal/semver/
│ ├── semver.go, compare.go, bump.go
│ ├── semver_test.go, compare_test.go, example_test.go
│ ├── fuzz_test.go, bench_test.go
│ └── testdata/fuzz/FuzzCompare/…
├── go.mod
└── go.sum
Save your progress? Sign up for free
Checkpoint
#!/usr/bin/env bash
check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
cd ~/go-lab/tag04-semver
check "gofmt sauber" "test -z \"\$(gofmt -l .)\""
check "go vet" "go vet ./..."
check "golangci-lint" "~/go-lab/bin/lint run ./..."
check "Tests grün (mit Race)" "go test -race -count=1 ./..."
check "Golden File vorhanden" "test -s cmd/semver/testdata/sort.golden"
check "Fuzz-Fund gespeichert" "ls internal/semver/testdata/fuzz/FuzzCompare/*"
check "Coverage Paket 100 %" \
"go test -cover ./internal/semver | grep -q 'coverage: 100.0%'"
check "kurzer Fuzz-Lauf" \
"go test -run '^$' -fuzz '^FuzzParse$' -fuzztime 5s ./internal/semver"Expected:
✔ gofmt sauber
✔ go vet
✔ golangci-lint
✔ Tests grün (mit Race)
✔ Golden File vorhanden
✔ Fuzz-Fund gespeichert
✔ Coverage Paket 100 %
✔ kurzer Fuzz-Lauf
Challenge
Write a fuzz test FuzzBump for the property “a bumped
version is always greater than before”
(Compare(next, v) == 1). Does it find anything? Then add a
seed that goes to the limits of int – and fix what it
finds.
Hint: On your Mac, math.MaxInt is
9223372036854775807, and Parse accepts this number.
Solution
cat > internal/semver/bump_fuzz_test.go <<'EOF'
package semver
import "testing"
// FuzzBump: every bumped version must be greater than before.
func FuzzBump(f *testing.F) {
f.Add("1.2.3", "minor")
f.Add("2.0.0-rc.1", "major")
f.Add("9223372036854775807.0.0", "major") // math.MaxInt
f.Fuzz(func(t *testing.T, s, part string) {
v, err := Parse(s)
if err != nil {
return
}
next, err := Bump(v, part)
if err != nil {
return // unknown part or overflow
}
if Compare(next, v) != 1 {
t.Errorf("Bump(%q, %q)\n= %q ist nicht größer", s, part, next)
}
})
}
EOF
go test -run FuzzBump ./internal/semverExpected:
--- FAIL: FuzzBump (0.00s)
--- FAIL: FuzzBump/seed#2 (0.00s)
bump_fuzz_test.go:20: Bump("9223372036854775807.0.0", "major")
= "-9223372036854775808.0.0" ist nicht größer
FAIL
Without the third seed, the fuzzer runs for minutes without a finding
– it rarely guesses 19-digit numbers. But the overflow is real: Go
integers overflow silently, without a panic. The fix: the existing
function becomes the internal bump, and the new
Bump checks first – no caller has to change.
perl -0pi -e 's/import "fmt"/import (\n\t"fmt"\n\t"math"\n)/' internal/semver/bump.go
perl -pi -e 's/^func Bump\(/func bump(/; s/^\/\/ Bump erh/\/\/ bump erh/' \
internal/semver/bump.go
cat >> internal/semver/bump.go <<'EOF'
// Bump increases like bump, but refuses parts that would overflow.
func Bump(v Version, part string) (Version, error) {
if max(v.Major, v.Minor, v.Patch) == math.MaxInt {
return v, fmt.Errorf("%w: %s läuft über", ErrInvalid, v)
}
return bump(v, part)
}
EOF
gofmt -l . ; go test ./internal/semverExpected:
ok example.com/semver/internal/semver 0.171s
So seeds are not an afterthought: boundary values (0, empty strings,
MaxInt, very long inputs) always belong in
f.Add.
Save your progress? Sign up for free
Enterprise Note
- In CI:
go test -race -count=1 ./...on every push, fuzzing at night with a longer-fuzztimeon a dedicated runner; new findings as a PR with the corpus entry. - Coverage targets per package instead of global: 100 % for parsers and money calculations, much less for glue code. Coverage is a hint, not proof of quality.
- Take golden files seriously in review: a blind
-updateis an unchecked approval. - Compare benchmarks only on quiet hardware; for reliable statements,
run several times with
-count=10andbenchstatfromgolang.org/x/perf. - On the subject of versions: Go modules use
golang.org/x/mod/semver, Helm charts and Kubernetes tools oftengithub.com/Masterminds/semverwith constraints like>=1.28 <1.31. Write your own implementation only if the rules really are different.