diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..ad711d9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,55 @@ +--- +name: Bug report +about: A parsing or emission bug — wrong, missing, or invalid OpenAPI output +title: 'bug: ' +labels: bug +--- + + + +## Minimal reproduction + + + +```go +// ShowAccount godoc +// +// @Summary ... +// @Router /accounts/{id} [get] +func ShowAccount(w http.ResponseWriter, r *http.Request) {} +``` + +## Command + + + +```sh +oasgen -d . -o openapi.json -oas 3.2.0 +``` + +## Actual output + + + +```json +``` + +## Expected output + + + +```json +``` + +## Environment + +- `openapi-parser` version / commit: +- Go version (`go version`): +- OS: diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..c1373e1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,6 @@ +--- +blank_issues_enabled: true +contact_links: + - name: Security report + url: https://github.com/indykite/openapi-parser/blob/master/responsible_disclosure.md + about: Please report security-sensitive issues privately — see the responsible disclosure policy, not a public issue. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..5f59be9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,34 @@ +--- +name: Feature request +about: A new annotation, directive, type-resolution rule, or OpenAPI construct +title: 'feat: ' +labels: enhancement +--- + +## What should be supported + + + +## Example input + + + +```go +``` + +## Expected output + + + +```json +``` + +## Alternatives / workarounds + + diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..9e64434 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,50 @@ +# Pull request + + + +## What this PR does + + + +## Effect on annotations and output + + + +## How it was tested + + + +- [ ] `make test` passes +- [ ] `make lint` passes +- [ ] Tests added/updated for the new behavior (positive case + edge case) +- [ ] `make parity` run (if output-affecting) — repo and result: + +## Known gaps or follow-ups + + + +## Checklist + +- [ ] One fix or feature per PR — no unrelated changes bundled in. +- [ ] PR title follows the existing convention (e.g. `feat: add @tag.parent support`, `fix: nullable pointer emission for 3.1`). +- [ ] No new entries in `go.mod` — the module is stdlib-only by design (see [`CONTRIBUTING.md`](../CONTRIBUTING.md#ground-rules)); if a dependency is unavoidable, it was agreed in an issue first. +- [ ] Breaking changes to annotation interpretation or emitted output are called out in the title and body. +- [ ] No secrets, internal URLs, or personal data committed. +- [ ] I have the right to contribute this content under the repo's [LICENSE](../LICENSE). diff --git a/CODEOWNERS b/CODEOWNERS new file mode 100644 index 0000000..2282b82 --- /dev/null +++ b/CODEOWNERS @@ -0,0 +1,20 @@ +# Each line is a file pattern followed by one or more owners. +# Later match takes precedence over earlier rule. + +# Global rule set for everything +* @indykite/devx + +# +# CI/CD, SAST & linting configs +/.github/ @indykite/devx @indykite/ops +.pre-commit-config.yaml @indykite/devx @indykite/ops +.golangci.yml @indykite/devx @indykite/ops +.checkov.yaml @indykite/devx @indykite/ops +.deepsource.toml @indykite/devx @indykite/ops +.hadolint.yaml @indykite/devx @indykite/ops +.shellcheckrc @indykite/devx @indykite/ops +.trivy.yaml @indykite/devx @indykite/ops +.trivy_data.yaml @indykite/devx @indykite/ops +.trivyignore.yaml @indykite/devx @indykite/ops +.yamlfmt.yml @indykite/devx @indykite/ops +.yamllint.yml @indykite/devx @indykite/ops diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..a4b964b --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,81 @@ +# Contributing + +Thanks for your interest in improving `openapi-parser`. When contributing to this repository, please first discuss any non-trivial change via an issue before opening a pull request — a 30-second sanity check from a maintainer can save an afternoon of work. + +## Before you start + +- Read the [README](./README.md) to understand the pipeline (`extract.go` → `lexer.go` → `parse.go` → `schema.go` → `emit.go`) and the version-agnostic model in the middle. +- Check open issues and PRs to make sure the change isn't already in flight. +- Read [`responsible_disclosure.md`](./responsible_disclosure.md) if your contribution touches anything security-sensitive. Do **not** open a public issue for security reports. + +## Ground rules + +- **Stdlib only, by design.** The module has no third-party dependencies and no swag dependency: parsing uses `go/parser` + `go/ast`, emission uses `encoding/json` plus the minimal internal + YAML encoder (`gen/yaml.go`). PRs that add a dependency to `go.mod` will be rejected unless discussed and agreed in an issue first. +- **swag compatibility matters.** Existing swag-style annotations must keep working. If a change alters how an annotation is interpreted, call that out explicitly in the PR. +- **Spec-version separation.** Annotation syntax and output spec version evolve independently. Parser/model changes belong in `parse.go`/`schema.go`; version-specific output belongs in the emitter. + +## Development setup + +You need Go 1.26 or later. + +```sh +git clone https://github.com/indykite/openapi-parser +cd openapi-parser + +make install-tools # gci + golangci-lint +go build ./... +make test +``` + +Optionally install the pre-commit hooks (`pre-commit install`) so formatting and lint run automatically. + +## Making changes + +1. Fork the repository and branch off `master`. +2. Make your change, keeping it focused: one fix or feature per PR. +3. Add or update tests. Parser and emitter behavior is covered by tests under `gen/` with fixtures in `testdata/`; new directives or type-resolution rules need both a positive case and an edge case. +4. Run the checks locally before pushing: + + ```sh + make fmt # gofmt + make gci # import ordering + make lint # golangci-lint + make test # go test ./... + ``` + +5. If your change affects generated output, run the parity gate against a repo you have access to and note the result in the PR: + + ```sh + make parity REPO=path/to/annotated-repo + ``` + + Any repo with swag-style annotations and a checked-in baseline works: the tool speaks of "services", but that just means any directory with its own + `docs/swagger.yaml`; a single-binary repo counts as one service at its root (see [README § Parity gate](./README.md#parity-gate-cmdoasparity)). + +## Pull requests + +- Keep one fix or feature per PR; reviewers should not have to evaluate unrelated changes together. +- Use a descriptive title following the existing convention, e.g. `feat: add @tag.parent support` or `fix: nullable pointer emission for 3.1`. +- The body should answer: what does this change do, why is it needed, and how was it tested? +- Breaking changes to annotation interpretation or emitted output must be called out explicitly in the PR title and body. +- By submitting a PR you confirm you have the right to contribute the content under this repo's [Apache-2.0 LICENSE](./LICENSE). + +## Reporting bugs + +A good parser bug report contains: + +1. A minimal annotated Go snippet that reproduces the issue. +2. The `oasgen` invocation used (flags, target OpenAPI version). +3. The output you got and the output you expected (ideally referencing the [OpenAPI specification](https://spec.openapis.org/) or swag's behavior). + +## Code of conduct + +Be kind. We follow the spirit of the [Contributor Covenant](https://www.contributor-covenant.org/) v2.1: + +- Assume good faith; critique the work, not the person. +- Be specific and actionable in reviews: vague negativity wastes everyone's time. +- Welcome newcomers; point them at this guide rather than dismissing the PR. +- Harassment, discriminatory language, doxxing, or sustained bad-faith argument are not tolerated and are grounds for a ban. + +To report a conduct issue privately, use the contact address in [`responsible_disclosure.md`](./responsible_disclosure.md); reports are kept confidential among the maintainers handling the case, and good-faith reports will never be held against you. diff --git a/GNUmakefile b/GNUmakefile index 7b18c36..b8bbf5a 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -35,8 +35,9 @@ tidy: go mod tidy # Compare oasgen output against swag-generated docs in another repository. -# Services are auto-discovered by their docs/swagger.yaml; pass SERVICES to -# restrict the list. Example: make parity REPO=../my-platform +# A "service" is any directory with its own docs/swagger.yaml baseline; they +# are auto-discovered, and a single-binary repo has one at its root. Pass +# SERVICES to restrict the list. Example: make parity REPO=../my-platform parity: @test -n "$(REPO)" || { echo "usage: make parity REPO=path/to/repo [SERVICES=dir1,dir2]"; exit 2; } @echo "==> Checking op/schema parity against swag output in $(REPO)..." diff --git a/README.md b/README.md index f5a4186..aef9f28 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,101 @@ # oasgen Parse Go files carrying **swag-style annotations** and emit **OpenAPI 3.2** -(or 3.1, for backward compatibility) — **without depending on swaggo/swag**. +(or 3.1, for backward compatibility) **without depending on swaggo/swag**. -It's a from-scratch replacement for swag's front end: existing annotations -keep working, but the parser, schema resolver, and emitter are all here. -Because we own the parser, 3.2 constructs are parsed -natively (`@Router /x [query]`, `@tag.parent`, …) instead of being smuggled -through `x-` hacks in a post-processor. +**You write** (unchanged swag-style annotations): + +```go +package accounts + +// @title Accounts API +// @version 1.0.0 + +type Account struct { + ID int64 `json:"id"` + Email string `json:"email" example:"ada@corp.io"` + Plan string `json:"plan" binding:"oneof=free pro"` + Trial *string `json:"trial_ends"` +} + +// GetAccount godoc +// @Summary Fetch one account +// @Produce json +// @Param id path int true "account id" minimum(1) +// @Success 200 {object} Account +// @Router /accounts/{id} [get] +func GetAccount(w http.ResponseWriter, r *http.Request) {} +``` + +**You get** (OpenAPI 3.2): + +```yaml +openapi: "3.2.0" +info: + title: "Accounts API" + version: "1.0.0" +paths: + "/accounts/{id}": + get: + operationId: GetAccount + summary: "Fetch one account" + parameters: + - name: id + in: path + required: true + description: "account id" + schema: + type: integer + format: int32 + minimum: 1 + responses: + "200": + description: OK + content: + application/json: + schema: + "$ref": "#/components/schemas/accounts.Account" +components: + schemas: + accounts.Account: + type: object + properties: + id: {type: integer, format: int64} + email: + type: string + examples: ["ada@corp.io"] + plan: + type: string + enum: [free, pro] + trial_ends: + type: [string, "null"] +``` + +## Why + +Tired of swaggo/swag forcing you into Swagger 2.0? Generate OpenAPI 3.2 +documents directly from your existing annotations - zero intermediate +conversion steps. + +`oasgen` is a from-scratch replacement for swag's front end: existing +annotations keep working, but the parser, schema resolver, and emitter are +all here. Because we own the parser, 3.2 constructs are parsed natively +(`@Router /x [query]`, `@tag.parent`, webhooks, streaming `itemSchema`, …) +instead of being smuggled through `x-` hacks in a post-processor. And there +is no conversion pipeline to babysit: annotations go in, a valid 3.2 (or +3.1) document comes out. + +## Quick start + +```sh +go install github.com/indykite/openapi-parser/cmd/oasgen@latest + +oasgen -d . -o openapi.yaml +``` + +`-d` selects the directories to scan for annotations (recursively), `-o` the +output file - format inferred from the extension. Add `-oas 3.1.0` if you +need 3.1 output. > **Naming:** the repository and Go module are `openapi-parser` > (`github.com/indykite/openapi-parser`); `oasgen` is the name of the command @@ -56,7 +144,7 @@ How `-d` is scoped: one `-d` root, they merge into a single document and their general-info blocks overwrite each other. - **Types** resolve against the whole Go module enclosing the first `-d` dir, - so cross-package references (`model.Account`) work without extra flags — no + so cross-package references (`model.Account`) work without extra flags: no `--parseDependency`/`--parseInternal` equivalents needed. Outside a module, only the given dirs are indexed. Types from other modules aren't resolved and degrade to a permissive `object` schema. @@ -74,33 +162,33 @@ data, err := api.EmitJSON(gen.EmitOptions{Version: "3.2.0"}) // or api.EmitYAML( Beyond swag's annotation set, the resolver understands: -- **Generics** — swaggo bracket syntax `listResponse[AccountResponse]` in +- **Generics**: swaggo bracket syntax `listResponse[AccountResponse]` in annotations and Go generic types in struct fields; instantiations are registered as `Base-Arg` components. -- **Embedded structs** — anonymous fields (including pointer embeds) promote +- **Embedded structs**: anonymous fields (including pointer embeds) promote their properties and required lists into the parent schema, like Go does. -- **Named non-struct types** — `type Labels []*Account` resolves inline to its +- **Named non-struct types**: `type Labels []*Account` resolves inline to its underlying type. -- **Cross-package references** — the whole module is indexed, so +- **Cross-package references**: the whole module is indexed, so `model.Account` resolves through the referencing file's imports. -- **Multi-word security scheme names** — `@securityDefinitions.apikey Bearer +- **Multi-word security scheme names**: `@securityDefinitions.apikey Bearer Token` and `@Security Bearer Token` keep the full name. -- **`@schemes`** — expands host-derived servers into one entry per scheme. -- **Constraints** — `@Param` attributes (`minimum(1) maximum(100) default(20) +- **`@schemes`**: expands host-derived servers into one entry per scheme. +- **Constraints**: `@Param` attributes (`minimum(1) maximum(100) default(20) pattern(...)`) and struct tags (`minimum`, `maximum`, `minLength`, `maxLength`, `multipleOf`, `default`) emit real JSON Schema keywords, and `binding:`/`validate:` rules map too: `oneof` → `enum`, `gte`/`lte`/`min`/ `max`/`len` → `minimum`/`maxLength`/`minItems`/… by type, `gt`/`lt` → `exclusiveMinimum`/`exclusiveMaximum`. -- **Typed scalars** — examples, defaults, and enums are coerced to the field's +- **Typed scalars**: examples, defaults, and enums are coerced to the field's type (`example:"5"` on an int emits `5`, not `"5"`). -- **Nullable pointers** — `*T` adds `"null"` to the type array; pointer struct +- **Nullable pointers**: `*T` adds `"null"` to the type array; pointer struct refs wrap as `anyOf: [$ref, null]` ($ref allows no sibling keywords). -- **Request body content type** — derived from `@Accept` (one entry per MIME +- **Request body content type**: derived from `@Accept` (one entry per MIME type) instead of hardcoding `application/json`. -- **Response descriptions** — default to the HTTP status text when the +- **Response descriptions**: default to the HTTP status text when the annotation carries none (matches swag). -- **Schema `x-` extensions** — swag's `extensions:"x-nullable,x-unit=kg,!x-hidden"` +- **Schema `x-` extensions**: swag's `extensions:"x-nullable,x-unit=kg,!x-hidden"` struct tag (bare = true, `!` = false, `x-` prefix added if missing). ## Beyond swag's directive set @@ -128,15 +216,15 @@ Beyond swag's annotation set, the resolver understands: ## Native 3.2 constructs -- **Streaming** — a `@Produce` of a sequential media type (`event-stream`/ +- **Streaming**: a `@Produce` of a sequential media type (`event-stream`/ `sse`, `jsonl`, `ndjson`, `json-seq`) emits the response schema as 3.2 `itemSchema` (each event's shape); 3.1 output falls back to `schema`. -- **Webhooks** — `@Webhook account.created [post]` in place of `@Router` +- **Webhooks**: `@Webhook account.created [post]` in place of `@Router` documents an outgoing event under the top-level `webhooks` map (3.1+); the method defaults to `post`. -- **`$self`** — `@self https://.../openapi.json` sets the document's own URI +- **`$self`**: `@self https://.../openapi.json` sets the document's own URI (3.2 only). -- **OAuth2 device flow** — `@securityDefinitions.oauth2.device` with +- **OAuth2 device flow**: `@securityDefinitions.oauth2.device` with `@deviceAuthorizationUrl`/`@tokenUrl`; plus `@refreshUrl`, `@oauth2MetadataUrl`, and `@deprecated` on any scheme. 3.2-only pieces are dropped cleanly from 3.1 output. @@ -148,15 +236,18 @@ Beyond swag's annotation set, the resolver understands: `oasparity` is a spec-parity regression gate: it parses each service's annotations with this module and compares the operation and schema inventories against the checked-in baseline (`docs/swagger.yaml`), without needing swag -or a YAML dependency. The baseline may be swag's Swagger 2.0 output (the +or a YAML dependency. "Service" is just the word for a check unit so any +directory with annotated Go code and its own baseline document counts as one, +whatever it is in your repo: a monorepo has many, a single-binary repo has +exactly one (its root). The baseline may be swag's Swagger 2.0 output (the migration case) or a previously generated OpenAPI 3.x document (the regression case). Missing/extra operations or missing schemas fail with exit 1; extra -oasgen components are allowed (unreferenced components are valid — swag +oasgen components are allowed (unreferenced components are valid so swag doesn't register embedded base types, we do). Against a 2.0 baseline, constructs swag cannot express (webhooks, the `query` method, `additionalOperations` verbs) are excluded from the comparison; against -a 3.x baseline they are compared too. 3.x baselines should be 3.2 output — +a 3.x baseline they are compared too. 3.x baselines should be 3.2 output: 3.1 downgrades those methods to `post`, which would diff against the annotations. @@ -169,6 +260,9 @@ go run ./cmd/oasparity -repo ../my-platform -services services/a,services/b -v # A different swag output location: go run ./cmd/oasparity -repo ../my-platform -docs api/swagger.yaml + +# A single-binary repo: the repo root is the one "service": +go run ./cmd/oasparity -repo ../my-api ``` Run it against any repo you're migrating before releasing a parser change; diff --git a/cmd/oasparity/main.go b/cmd/oasparity/main.go index ce9e066..67855fe 100644 --- a/cmd/oasparity/main.go +++ b/cmd/oasparity/main.go @@ -22,7 +22,10 @@ // oasparity -repo path/to/repo -services svc/a,svc/b # explicit list // // Without -services, every directory under -repo containing the baseline -// (the -docs relative path) is checked. A service fails when an operation +// (the -docs relative path) is checked. A "service" is just such a +// directory — any subtree with annotated Go code and its own baseline; a +// single-binary repo is one service at its root. A service fails when an +// operation // differs in either direction, or when a baseline schema has no oasgen // counterpart. Extra oasgen components are reported but allowed // (unreferenced components are valid). Exits 1 on any failure so it can diff --git a/responsible_disclosure.md b/responsible_disclosure.md new file mode 100644 index 0000000..8cf34fe --- /dev/null +++ b/responsible_disclosure.md @@ -0,0 +1,53 @@ +# Responsible Disclosure + +Thank you for taking the time to help keep this project — and the toolchains that consume its output — safe. + +## Why a parser needs a disclosure process + +`oasgen` parses untrusted input: Go source files and their comment annotations. Its output (OpenAPI documents) is consumed by code generators, gateways, and validation middleware. +A parsing bug can therefore be more than a crash: a crafted annotation or source file could produce a document that misleads downstream security tooling +(e.g. dropping a declared security scheme, mangling a constraint) or hang/exhaust the parser in CI. + +## In scope + +- Crafted Go source or annotations that crash the parser, cause unbounded memory/CPU use, or hang it. +- Inputs that make the emitter silently drop or alter security-relevant output (security schemes, `security` requirements, constraints, enum restrictions). +- Path handling issues: `-d`/`-o` flags reading or writing files outside the intended directories. +- Bugs in the internal YAML encoder (`gen/yaml.go`) that produce output parsed differently by downstream YAML consumers than intended. + +## Out of scope + +- Vulnerabilities in Go itself, `golangci-lint`, or other development tooling: report those upstream. +- Issues that require an already-compromised machine or a malicious operator. +- Theoretical concerns without a demonstrable trigger. +- Incorrect-but-honest output for exotic annotations (that's a regular bug so open an issue). + +## How to report + +**Contact:** + +When reporting, include: + +1. The affected component (file or package) and commit hash. +2. A minimal reproduction: ideally a small annotated Go file plus the `oasgen` invocation. +3. The impact: what an attacker could achieve, and under what assumptions. +4. Any suggested mitigation, if you have one. + +Please **do not** open a public GitHub issue for security-sensitive reports. + +## What to expect + +- **Acknowledgement** within 5 business days. +- **Triage and severity assessment** within 10 business days. +- **Fix or mitigation** on a timeline proportional to severity. +- **Credit** in the changelog or release notes if you would like it. Let us know your preferred name/handle. + +## Safe harbor + +We will not pursue legal action against researchers who: + +- Make a good-faith effort to avoid privacy violations, data destruction, and service disruption while testing. +- Report findings privately and give a reasonable window for remediation before public disclosure. +- Do not exploit a finding beyond what is necessary to demonstrate it. + +If in doubt about whether your testing falls within these guidelines, ask first.