feat(runtime/go): smart ShouldRebuild via go list -deps#6823
feat(runtime/go): smart ShouldRebuild via go list -deps#6823alessandrolattao wants to merge 2 commits intoanomalyco:devfrom
Conversation
The Go runtime today rebuilds every Lambda in the same go.mod tree whenever any file changes. Fine when you have one go.mod per Lambda (the doc'd layout), but unworkable when you choose a single-module monorepo with many handlers — in that case `sst dev` rebuilds every Lambda on every save and the dev loop falls apart. This change captures each handler's transitive import graph at Build time (`go list -deps -json`) and uses it as the rebuild scope. ShouldRebuild fires only on files actually in that graph — same idea the Node runtime already uses with its esbuild metafile. Works in every layout (per-Lambda, per-domain, single-module). Also fixes a `go build` bug exposed by sub-path handlers: the source arg now has a `./` prefix so a non-`.` source path doesn't get treated as a stdlib package import. Tests cover ShouldRebuild edges, the GOMODCACHE/stdlib filter, a race-detector concurrency probe, and hermetic `go list` integration fixtures (skipped under -short).
There was a problem hiding this comment.
Pull request overview
This PR improves the Go runtime’s incremental rebuild behavior in sst dev by capturing each handler’s transitive dependency file set via go list -deps -json during Build, and using that set in ShouldRebuild to rebuild only the functions actually affected by a file change. It also fixes go build invocation for sub-path handlers by forcing ./-prefixed relative package paths, and adds unit + integration tests for the new dependency-capture and rebuild-scope logic.
Changes:
- Capture per-function dependency graphs at build time (
go list -deps -json) and use them to scopeShouldRebuild. - Fix
go buildargument to use./-prefixed relative handler paths to avoid stdlib-package misresolution. - Add unit tests for graph parsing / path helpers / concurrency and hermetic integration tests against the real Go toolchain.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| pkg/runtime/golang/golang.go | Implements dependency capture, new rebuild decision logic, GOMODCACHE filtering, and path fix for go build. |
| pkg/runtime/golang/golang_test.go | Adds unit tests for Properties contract, arch mapping, ShouldRebuild, isUnderDir, and go-list parsing filters. |
| pkg/runtime/golang/build_test.go | Adds integration tests exercising captureDeps with real go list in a scrubbed environment. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The previous sync.Once made the GOMODCACHE filter silently inert if the very first Build had a degenerate env: a transient `go env` failure locked the cached value to "" for the lifetime of the process. Replace it with a Mutex + resolved flag, slog.Warn on failure, and re-try on the next call. Tag rebuild log lines with a `reason` (file_in_graph, gomod_change, new_file_in_pkg) so dev-loop traces show why each rebuild fires. Demote the per-build "running go build" line to Debug. Drop the unused per-handler `directories` map now that the import graph covers it. Add GOWORK=off and GOSUMDB=off to the integration test env so `go list` can't pick up the host's go.work or hit the checksum DB.
|
I've just realized that the current version fails to trigger a rebuild for Lambdas when code shared via workspaces/replaces is modified, even if the Lambda depends on it. This update also fixes that issue. |
Why
The Go runtime today rebuilds every Lambda in the same
go.modtree whenever any file changes. That's fine with the documented layout (onego.modper handler), but as soon as a project chooses a single-module monorepo with many handlers — share aservices/sharedpackage across N Lambdas, keep one set of dependencies, onego.mod tidy, etc. —sst devbecomes unusable: editing one handler triggers a rebuild of every other handler in the module.Side effect of one-go.mod-per-Lambda also pays a recurring cost on every CI lint and CI test pass that scales linearly with the Lambda count, since each handler is its own
golangci-lint+go testinvocation.This is the same problem the Node runtime already solved: it uses esbuild's metafile to know exactly which files compile into each bundle and rebuilds only the Lambdas whose actual deps changed.
What
Capture each handler's transitive import graph at
Buildtime viago list -deps -jsonand use it as the rebuild scope.ShouldRebuildfires only when the changed file is in that handler's graph. Works in every layout (per-Lambda, per-domain, single-module monorepo).Side fix:
go buildarguments now use a./prefix on the source path so layouts where the source is a sub-path don't fail withpackage <path> is not in std. With the legacy one-go.mod-per-Lambda layoutsrcwas always., so the bug never surfaced.The previous directory-scope behavior was deliberately removed rather than kept as a fallback. In a single-module repo it would silently fan out to every Lambda when
captureDepsfailed, hiding a real problem behind a worse one. AcaptureDepsfailure now logs a warning with full context (goflags,gomodcache) and disablesShouldRebuildfor that function until the next successfulBuild.Tests
`pkg/runtime/golang/golang_test.go` — pure unit tests for `Properties` JSON contract, the goarch mapping, `ShouldRebuild` edge cases (non-go file, missing graph, file hit/miss, relative path resolution), the `GOMODCACHE`/stdlib filter via a fixture JSON stream, a 50×50 race-detector concurrency probe, and a cross-OS `isUnderDir` helper.
`pkg/runtime/golang/build_test.go` — hermetic integration tests against the real `go list` toolchain (env scrubbed: `GOPROXY=off`, `GOTOOLCHAIN=local`, redirected `GOMODCACHE`/`GOCACHE`/`HOME`). Skipped under `-short` and when `go` is not in `PATH`.
`go test ./pkg/runtime/golang/... -race`: 1.6s full, 1.0s under `-short`. `go vet` clean.
Tested manually
Validated end-to-end on a real repo with ~60 Go Lambdas consolidated into one root `services/go.mod`. With this patch: