[go-fan] Go Module Review: itchyny/gojq

[github-actions]

🐹 Go Fan Report: itchyny/gojq

Module Overview

github.com/itchyny/gojq is a pure-Go implementation of the jq command-line JSON processor. It provides a full jq-compatible parser and compiler, supports running queries with context/cancellation, custom Go functions, and produces idiomatic Go values. The project is at v0.12.19, which brings enhanced array handling (up to 2^29 elements), improved concurrent execution, and better type error messages.

Current Usage in gh-aw

• Files: 2 files (internal/middleware/jqschema.go, internal/middleware/jqschema_bench_test.go)
• Import Count: 2 imports
• Key APIs Used:
  • gojq.Parse(query string) (*Query, error) — parses a jq filter at startup
  • gojq.Compile(query *Query) (*Code, error) — compiles the parsed query once at init
  • (*Code).RunWithContext(ctx, input) — runs the pre-compiled query with context support per request
  • (*gojq.HaltError) — type-asserted for distinguishing clean halts from errors

The module is used exclusively in jqschema.go to implement a "schema inference" transformation: large MCP tool responses are saved to disk, and a compact schema representation (leaf values replaced with their type names, arrays collapsed to one element) is returned to the agent instead. The compiled *gojq.Code is stored in a package-level variable and reused on every request.

Research Findings

Architecture Quality

The current usage already follows gojq best practices almost perfectly:

• Compile-once pattern: gojq.Parse + gojq.Compile at init() time, then Code.RunWithContext per call (10–100x faster than parse-per-request, as measured in the bench tests)
• Context propagation: RunWithContext is used, so the query respects cancellation and deadlines
• HaltError handling: Correctly type-asserts *gojq.HaltError to distinguish clean halts (nil value) from error exits

Recent Updates (v0.12.x)

• v0.12.19: Enhanced array limit (2^29), concurrent performance improvements, better type error messages — already on this version ✅
• v0.12.x introduced gojq.WithFunction for binding native Go functions into jq programs
• Context cancellation via RunWithContext was added in earlier v0.12.x — already used ✅

Best Practices (from gojq maintainer)

• Prefer gojq.Compile + Code.Run over Query.Run for repeated executions — already done ✅
• Use RunWithContext rather than Run for server/long-running use — already done ✅
• Custom Go functions can be registered via gojq.WithFunction to supplement or replace complex jq recursion

Improvement Opportunities

🏃 Quick Wins

1. Drain iterator completely in tests: In jqschema_bench_test.go the "parse every time" benchmark uses query.RunWithContext (bypassing Compile). This is intentional for comparison, but the benchmark benchmark could add a comment explicitly noting it simulates a degraded path, since it skips Compile.

2. Type switch completeness: In WrapToolHandler, the type switch that decides whether to unmarshal from JSON:

   case map[string]interface{}, []interface{}, string, float64, bool:
       jsonData = data

   This is missing nil and json.Number. If upstream SDK ever surfaces json.Number values (which some JSON decoders emit when UseNumber() is set), it would fall through to the unmarshal path unexpectedly. Adding case nil: with an early return and case json.Number: (converting to float64) would make this more robust.

3. HaltError.Value() nil check comment: The comment // HaltError with nil value means clean halt (not an error) is accurate, but the gojq docs phrase it as "halt with no error argument". A minor documentation alignment.

✨ Feature Opportunities

1. Native Go walk_schema via gojq.WithFunction: The current walk_schema jq filter is a pure-jq recursive function. gojq supports registering native Go functions via gojq.WithFunction(name, minArgs, maxArgs, fn). A Go-native implementation of the schema walk would bypass jq's interpreter overhead for the recursion, potentially giving 2–5x speedup on deeply-nested payloads. Example:

   // walkSchemaFunc is a native Go implementation of walk_schema
   func walkSchemaFn(_ interface{}, args []interface{}) interface{} {
       return inferSchema(args[0])
   }
   
   // At Compile time:
   gojq.Compile(query, gojq.WithFunction("walk_schema", 0, 0, walkSchemaFn))

   This would let the jq filter become simply walk_schema (calling the Go implementation), removing all the def walk_schema: ... jq code.

2. Streaming / early-abort for very large payloads: Currently the full data object is marshaled to JSON (json.Marshal(data)) before applying jq. For very large responses, a streaming approach that checks the size threshold before full marshaling could save allocations. This is not a gojq issue per se, but gojq's iterator model supports early abort via context.WithCancel.

📐 Best Practice Alignment

1. gojq compile options: gojq.Compile accepts functional options. Consider explicitly passing gojq.WithEnvironment(nil) (or relevant options) if custom variable scoping is ever needed. Currently no options are passed, which is fine for the current static filter.

2. WithFunction for type: The filter uses jq's built-in type function, which is implemented natively in gojq already. No action needed.

3. Error wrapping: Errors from applyJqSchema are wrapped with fmt.Errorf("%w", ...) throughout. This is correct and idiomatic.

🔧 General Improvements

1. Separate schema inference from gojq: The schema walk logic (inferSchema) could live as a pure Go function independent of gojq, making it testable without a jq runtime and removing the indirection through the jq interpreter entirely. The jqSchemaFilter would then be retired. This is a larger refactor but would eliminate the jq dependency for this specific use case.

2. Make PayloadPreviewSize configurable: Currently a compile-time constant. Exposing it as a server config option would let operators tune it without code changes.

Recommendations

Priority	Action	Complexity
Low	Add nil and json.Number cases to type switch in WrapToolHandler	Trivial
Medium	Implement native Go walk_schema via gojq.WithFunction for performance	Small
Low	Make PayloadPreviewSize a configurable parameter	Small
High (future)	Decouple schema inference from jq entirely (pure Go walk)	Medium

Next Steps

• Add nil and json.Number to the type switch in WrapToolHandler (defensive)
• Prototype gojq.WithFunction("walk_schema", ...) and benchmark against current jq-recursive approach
• Consider making PayloadPreviewSize runtime-configurable

---

Generated by Go Fan 🐹
Module summary saved to: specs/mods/gojq.md (pending write access)
Run: §25040727568

References:

• §25040727568
• https://github.com/itchyny/gojq
• https://pkg.go.dev/github.com/itchyny/gojq

Note

🔒 Integrity filter blocked 10 items

The following items were blocked because they don't meet the GitHub integrity level.

• search_repositories search_repositories: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• https://github.com/santhosh-tekuri/jsonschema search_repositories: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• https://github.com/golang/term search_repositories: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• https://github.com/open-telemetry/opentelemetry-go search_repositories: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• https://github.com/itchyny/gojq search_repositories: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• itchyny/gojq@b7ebffb list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• open-telemetry/opentelemetry-go@fb02c76 list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• spf13/pflag@3d32e71 list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• santhosh-tekuri/jsonschema@180cde3 list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".
• https://github.com/itchyny/gojq/releases/tag/v0.12.19 get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Go Fan · ● 717.6K · ◷

• expires on May 5, 2026, 7:57 AM UTC