Skip to content

fix: treat function names and keywords as member names in dot notation per RFC 9535 - #27

Merged
TristanSpeakEasy merged 1 commit into
mainfrom
fix/member-name-shorthand-keywords
Nov 16, 2026
Merged

TristanSpeakEasy merged 1 commit into
mainfrom
fix/member-name-shorthand-keywords

Conversation

@TristanSpeakEasy

Copy link
Copy Markdown
Member

Why

The tokenizer treated length, count, match, search and value as function names whenever any non-identifier character followed them, and treated true, false and null as keywords everywhere. That meant valid RFC 9535 queries using those words as member names were rejected:

Query Before After
$.value[0], $.value[*], $.value[?@.id == 'x'] rejected accepted
$.value.id, $..value[0], $[?@.value[0] == 1] rejected accepted
$.length[0], $.count.x rejected accepted
$.true, $.null.x rejected accepted
$.value accepted accepted

$.value only worked because the end-of-input branch skipped the function check. This was reported against openapi arazzo validate in speakeasy-api/openapi#301. Microsoft Graph and OData list responses put their arrays under value, so it hits a lot of real Arazzo workflows.

What changed

  • A name directly after . or .. is always tokenized as a member name, which matches RFC 9535 member-name-shorthand.
  • Otherwise a function name only becomes a FUNCTION token when ( follows it directly, matching function-expr = function-name "(" ... with no whitespace allowed.
  • parseFunctionExpr now requires a FUNCTION token instead of accepting any token followed by (.

Review notes

  • The parser change is needed so whitespace between a function name and ( is still rejected (compliance suite whitespace, functions, * cases). Before, the tokenizer caught that indirectly.
  • Side effect worth knowing: before this change, an unknown function such as $[?foo(@.a) > 1] parsed silently as length(), because functionTypeLength is the zero value of the enum. It's now a parse error, which is what RFC 9535 expects.
  • The four Function call ... tokenizer tests used a method-call form (@.length()), which isn't RFC 9535 syntax and never got through the parser. I rewrote them as real function calls (length(), count(search(...))) so they still cover FUNCTION tokenization.

Testing

  • go build ./... and go test ./... (including the RFC 9535 compliance suite) pass locally.
  • Added parser round-trip cases, tokenizer cases and query evaluation cases for function names and keywords used as member names, plus the whitespace-before-( rejection.
  • Against speakeasy-api/openapi using a local go.work: the repro from #301 now validates, and the overlay, arazzo and openapi package tests pass. One Arazzo stress test lists an expected $.length == 5 error in its ignore list. It's still rejected, but now at the == (column 9) rather than the space (column 8), so that ignore string needs updating when openapi bumps to the release with this fix.

@TristanSpeakEasy TristanSpeakEasy added the bug Something isn't working label Nov 16, 2026
@vercel

vercel Bot commented Nov 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
jsonpath Ready Ready Preview Nov 16, 2026 12:23am UTC

Request Review

@TristanSpeakEasy

Copy link
Copy Markdown
Member Author

@cubic-dev-ai

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 16, 2026

Copy link
Copy Markdown

@cubic-dev-ai

@TristanSpeakEasy I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Fixes RFC 9535 compliance by tokenizing names after ./.. as member names even when they match keywords or function names, requiring a direct ( for function calls, and making unknown functions parse errors. Focused bug fix with parser, tokenizer, and query tests.

View guided diff | Turn on auto-fix | Re-trigger cubic

@TristanSpeakEasy
TristanSpeakEasy merged commit a36ba47 into main Nov 16, 2026
5 checks passed
@TristanSpeakEasy
TristanSpeakEasy deleted the fix/member-name-shorthand-keywords branch November 16, 2026 00:38

This branch was successfully deployed

1 active deployment
Preview — ec14d5f5 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant