Skip to content

Gemini 4 Hydrogen / Neon: runtime strips xhigh from supportedReasoningEfforts despite CAPI advertising it #5527

Backlog

Description

@mingley

Describe the bug

Google publicly supports thinking_level: "xhigh" (default high, alongside low, medium, high, xhigh) on both gemini-4-hydrogen and gemini-4-neon in the Gemini API and on Vertex AI for Gemini 4 Hydrogen.

GitHub Copilot's /models catalog already advertises "xhigh" in capabilities.supports.reasoning_effort for gemini-4-hydrogen:

{
  "id": "gemini-4-hydrogen",
  "vendor": "Google",
  "capabilities": {
    "supports": {
      "reasoning_effort": ["low", "medium", "high", "xhigh"],
      "reasoningEffort": true
    }
  },
  "supportedReasoningEfforts": ["low", "medium", "high"]
}

However, Copilot CLI's native runtime (runtime.node — modelSupportedReasoningEfforts / rt_model_client_policy::reasoning_validation) filters vendor == "Google" and gemini-* model IDs against a hardcoded ["none", "minimal", "low", "medium", "high"] allowlist that omits "xhigh". As a result:

  1. supportedReasoningEfforts is truncated to ["low", "medium", "high"], so "Extra High" (xhigh) cannot be selected in the /model picker for gemini-4-hydrogen or gemini-4-neon.
  2. Switching to gemini-4-hydrogen from a model running at effortLevel: "xhigh" clamps the session's reasoningEffort to "high".
  3. Calling session.model.switchTo({ modelId: "gemini-4-hydrogen", reasoningEffort: "xhigh" }) fails with:
    Reasoning effort 'xhigh' is not supported for model 'gemini-4-hydrogen'.

Related: #5518 (which noted xhigh missing from /model on gemini-4-neon and questioned whether CAPI advertised xhigh).

Affected version

GitHub Copilot CLI 1.0.114 and GitHub Copilot CLI 1.0.115-0

Steps to reproduce the behavior

  1. Interactive /model picker:

    • Start copilot (1.0.114 or 1.0.115-0), run /model, highlight Gemini 4 Hydrogen (or Gemini 4 Neon), and press Left/Right arrow keys to cycle reasoning effort.
    • Only Low, Medium, and High are offered; Extra High (xhigh) is missing.
  2. Inspect models.list via bundled copilot-sdk:

    const raw = await client.rpc.models.list();
    const hydrogen = raw.models.find(m => m.id === "gemini-4-hydrogen");
    console.log({
      capiSupports: hydrogen.capabilities.supports.reasoning_effort,
      cliSupported: hydrogen.supportedReasoningEfforts,
    });
    // Output:
    // {
    //   capiSupports: [ 'low', 'medium', 'high', 'xhigh' ],
    //   cliSupported: [ 'low', 'medium', 'high' ]
    // }
  3. Verify vendor/ID allowlist in runtime.node:

    const w = require("<copilot-pkg>/prebuilds/darwin-arm64/runtime.node");
    const makeEntry = (id, vendor) => [{
      id,
      name: id,
      vendor,
      supportedReasoningEfforts: ["none", "minimal", "low", "medium", "high", "xhigh", "max"],
      capabilities: {
        family: id,
        supports: {
          reasoning_effort: ["none", "minimal", "low", "medium", "high", "xhigh", "max"],
          reasoningEffort: true,
        },
      },
    }];
    
    w.modelSupportedReasoningEfforts("gemini-4-hydrogen", JSON.stringify(makeEntry("gemini-4-hydrogen", "Google")));
    // -> [ 'none', 'minimal', 'low', 'medium', 'high' ]  ('xhigh' stripped)
    
    w.modelSupportedReasoningEfforts("gemini-4-neon", JSON.stringify(makeEntry("gemini-4-neon", "Google")));
    // -> [ 'none', 'minimal', 'low', 'medium', 'high' ]  ('xhigh' stripped)
    
    w.modelSupportedReasoningEfforts("custom-id", JSON.stringify(makeEntry("custom-id", "OpenAI")));
    // -> [ 'none', 'low', 'medium', 'high', 'xhigh', 'max' ]
  4. Attempt explicit switchTo with reasoningEffort: "xhigh":

    await session.rpc.model.switchTo({
      modelId: "gemini-4-hydrogen",
      reasoningEffort: "xhigh",
      source: "model_picker",
    });
    // Throws: Request session.model.switchTo failed with message:
    // Reasoning effort 'xhigh' is not supported for model 'gemini-4-hydrogen'.

Expected behavior

When CAPI advertises "xhigh" in capabilities.supports.reasoning_effort for Google Gemini models that support it (gemini-4-hydrogen and gemini-4-neon), Copilot CLI's Google/Gemini reasoning-effort allowlist should permit "xhigh" so it can be selected in /model, --reasoning-effort xhigh, effortLevel, and session.model.switchTo, while continuing to cap models whose CAPI capabilities only list ["low", "medium", "high"] (such as gemini-4-carbon, gemini-4-argon, and gemini-3.8-flash).

Additional context

  • Operating system: macOS (arm64)
  • Versions tested: 1.0.114 and 1.0.115-0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions