Skip to content

Validate generated OpenAPI 3.x documents against the official meta-schemas - #1005

Open
numbata wants to merge 1 commit into
pr/6-oas31-extrasfrom
pr/7-validation-and-parity
Open

numbata wants to merge 1 commit into
pr/6-oas31-extrasfrom
pr/7-validation-and-parity

Conversation

@numbata

@numbata numbata commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Generated OpenAPI 3 documents are now checked against the official 3.0 and 3.1 meta-schemas, and the 3.x output is compared to Swagger 2.0 output for the decisions both share. This gives maintainers evidence that the new output is valid and consistent before it is released. It adds tests and one development dependency, nothing at runtime.

Problem solved

Hand-written expectations can agree with a wrong document. Only the standard's own schemas can say whether the output is valid, and nothing otherwise stops the two generators from disagreeing on shared behavior.

What changes

A be_valid_openapi matcher validates documents with JSONSchemer.openapi, using the OpenAPI 3.0 and 3.1 meta-schemas bundled in the json_schemer development dependency, and specs validate documents from a wide range of APIs. Parity specs pin hidden routes, content types, add_root, array_use_braces (flat query and form arrays keep their braces, validated against the meta-schemas) and token_owner to the same answer in both formats (parameter-level hidden procs give the same, wrong, answer in both versions and those examples are pending); the way tags are derived and merged is shared too, except for one documented difference.

Drawbacks and nuances

  • New development dependency: json_schemer; the meta-schemas come from the installed gem version, so a gem upgrade can change what counts as valid.
  • Meta-schema validation checks structure only. It cannot detect semantic errors such as a dangling $ref, a duplicate operationId, or a path parameter missing from the path template, nor whether the description matches the API's real behavior.
  • Slightly longer test runs.
  • Tag collection is not identical: Swagger 2.0 reads the first operation of each path and emits no user tags for an API without routes, while 3.x collects from every operation and always emits user tags. A parity example pins this documented difference.

Alternatives considered

  • Vendor the official schema files: about 3,000 lines of JSON to review and keep current, plus a license notice, for what the dev dependency already ships.
  • Download the meta-schemas in CI: depends on network, flaky and not reproducible offline.
  • External linters such as Redocly or Spectral: would catch semantic errors, but add a non-Ruby toolchain, version drift and rule configuration to the project.
  • Skip validation and rely on snapshot specs: cheaper, but snapshots enshrine mistakes.

Stacks on: pr/6-oas31-extras.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Danger Report

No issues found.

View run

@numbata
numbata added this pull request to stack #1007 October 14, 2026 16:31
@numbata
numbata force-pushed the pr/7-validation-and-parity branch from 5a606c6 to ef0e11a Compare October 14, 2026 18:28
@numbata
numbata force-pushed the pr/7-validation-and-parity branch from ef0e11a to 84d4ec3 Compare October 14, 2026 19:31
@numbata
numbata force-pushed the pr/7-validation-and-parity branch from 84d4ec3 to b99e9dd Compare October 14, 2026 19:41
…hemas

The OpenAPI 3 output is only useful if standards-conforming tools accept
it, and hand-written assertions cannot prove that. This adds a `be_valid_openapi` matcher backed by the
OpenAPI 3.0 and 3.1 meta-schemas bundled in the json_schemer development
dependency, and checks documents produced from a broad range of APIs against them.

It also pins the relationship between the two generators: shared route
decisions, and option handling such as `add_root`, `array_use_braces` and
`token_owner`, must give the same answer in Swagger 2.0 and OpenAPI 3.x
output.
@numbata
numbata force-pushed the pr/7-validation-and-parity branch from b99e9dd to acca5c1 Compare October 14, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant