Gemini: Support mixing provider tools with custom function tools

[cainydev]

Description

Gemini 3+ models now support using built-in tools (google_search, url_context) alongside custom function declarations in the same request via toolConfig.includeServerSideToolInvocations.

Currently, Prism throws an exception when attempting this:

Use of provider tools with custom tools is not currently supported by Gemini.

This restriction was originally correct, when search grounding was added in #265, the Gemini REST API did not support mixing built-in tools with function calling (it required the Live/WebSocket API). Google has since added toolConfig.includeServerSideToolInvocations to the standard REST API for Gemini 3+ models. The throw was carried over into #405 without re-testing.

Model compatibility

Tested every available Gemini model via the REST API with includeServerSideToolInvocations: true:

Model	Mixed tools support
gemini-3.1-pro-preview	✅
gemini-3.1-pro-preview-customtools	✅
gemini-3.1-flash-lite-preview	✅
gemini-3-pro-preview	✅
gemini-3-flash-preview	✅
gemini-pro-latest	✅ (alias → 3.x)
gemini-flash-latest	✅ (alias → 3.x)
gemini-flash-lite-latest	❌ (alias → 2.5)
gemini-2.5-pro	❌ "not enabled for model"
gemini-2.5-flash	❌ "not enabled for model"
gemini-2.5-flash-lite	❌ "not enabled for model"
gemini-2.0-flash	❌ deprecated (404)

All Gemini 3+ models support it. Gemini 2.5 and below do not. When the flag is sent to an unsupported model, Gemini returns a clear 400 error: "Tool call context circulation is not enabled for models/gemini-2.5-flash".

When is the flag needed?

Scenario	includeServerSideToolInvocations	Result
Provider tools only	Not set	✅ 200
Custom tools only	Not set	✅ 200
Both mixed	Not set	❌ 400 "Please enable tool_config.include_server_side_tool_invocations"
Both mixed	true	✅ 200 (Gemini 3+)

The flag is only needed when both provider tools and custom function tools are present in the same request.

Structured output interaction

Separately tested structured output (response_mime_type: application/json) combined with tools:

Combination	Gemini 3.x	Gemini 2.5
Structured output only (no tools)	✅	✅
Structured + custom tools	✅	❌ "Function calling with response mime type 'application/json' is unsupported"
Structured + provider tools	✅	❌ "Tool use with response mime type 'application/json' is unsupported"
Structured + both tools + flag	✅	❌

This is a separate Gemini 2.5 limitation (structured output + any tools is unsupported) and not directly related to this issue, but worth noting since the Gemini Structured handler currently has no guard for this and will pass through the API error.

Proposed fix

1. Remove the throw in all three Gemini handlers (Text, Structured, Stream)
2. When both tool types are present, add includeServerSideToolInvocations: true to tool_config
3. No model version checking needed, unsupported models return a clear API error naturally

Minimal diff for the Text handler:

- if ($request->tools() !== [] && $request->providerTools() != []) {
-     throw new PrismException('Use of provider tools with custom tools is not currently supported by Gemini.');
- }
+ $hasBothToolTypes = $request->tools() !== [] && $request->providerTools() !== [];

  // ... existing tool building code (no changes needed) ...

  // When building the request payload:
+ if ($hasBothToolTypes) {
+     $toolConfig = array_merge($toolConfig ?? [], [
+         'includeServerSideToolInvocations' => true,
+     ]);
+ }

Verified working

Tested with a local patch on Prism v0.99.21:

• google_search + custom function tools ✅
• url_context + custom function tools ✅
• google_search + url_context + custom function tools ✅
• All of the above with ->structured() mode ✅
• Tested across three providers: Anthropic and OpenAI already support mixing provider tools with custom tools without issues.

Gemini docs reference

• Built-in tools with function calling

Happy to submit a PR for this.