feat: enable to will_continue-like function response with streaming mode.

[Lin-Nikaido]

Linked Issue

close #2014

Abstruction

• feat: enable to will_continue-like function response in BaseLlmFlow.run_async method with streaming mode.
  It expected the tool returns generator, and the runner.async_run method when streaming_mode: StreamingMode.SSE yields the generator result as each Event. also, the streaming_mode is not SSE there is no change.

I expect this usecase is the function-tool will take few minutes total, and the user want to notice user its progress.
e.g. Like this function.

def search_tool(query: str):
    """ Example
    The query gives like " Answer to the Ultimate Question of Life, the Universe, and Everything".
    """
    yield {
        {
            "status": "inProgress",
            "message": f"searching Google with `{query}`"
        },
    }
    # Searching will take few second...
    link_list = ["https://hitsite1.com", "https://hitsite2.com", "https://hitsite3.com"]
    yield {
        {
            "status": "inProgress",
            "message": f"accessing each sites.\n{json.dumps(link_list, indent=2, ensure_ascii=False)}"
        },
    }
    # access these, retrieve information and analyze.
    yield {
        {
            "status": "inProgress",
            "message": f"summarizing the information."
        },
    }
    # summarizing each analyzed information.
    yield {
        {
            "status": "ok",
            "message": "42"
        },
    }

testing plan

Added /tests/unittests/tools/test_tools_generative_call.py and it passed

[gemini-code-assist]

Summary of Changes

Hello @Lin-Nikaido, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant feature that allows function tools to provide streaming responses, enabling real-time progress updates for long-running operations. It refactors the function call handling mechanism to support asynchronous generators, ensuring that intermediate results from tools can be streamed back to the user when Server-Sent Events (SSE) mode is enabled. This enhances the user experience by providing transparency into ongoing tool executions.

Highlights

• Streaming Function Tool Responses: Introduced the capability for function tools to return streaming responses, allowing for real-time progress updates during long-running operations. This is achieved by enabling tools to return generator or async generator objects.
• Asynchronous Generator Integration: Refactored the core function call handling in BaseLlmFlow and functions.py to process and yield intermediate events from asynchronous generators returned by tools, specifically when StreamingMode.SSE is active.
• Parallel Streaming Support: Implemented a mechanism (_concat_function_call_generators) to merge and stream events from multiple parallel function calls, ensuring that progress updates from concurrent tool executions are handled correctly.
• Function Schema Updates: Updated the function parameter parsing utility to correctly recognize and represent Python's Generator, Iterator, Iterable, AsyncGenerator, AsyncIterator, and AsyncIterable types as ARRAY in the generated tool schemas.
• Enhanced Tool Context: Added run_config to ToolContext to provide tools with direct access to the current run's configuration, including the streaming mode, enabling conditional streaming behavior within tools.
• Comprehensive Unit Tests: Added a new, extensive suite of unit tests (test_tools_generative_call.py) to validate the streaming functionality for various types of generator and async generator functions, covering both single and parallel tool executions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a significant feature to enable streaming responses from function tools, allowing them to yield progress updates. The implementation involves changes across the flow, function handling, and tool execution logic to support generators and async generators. A comprehensive set of unit tests has been added to validate the new functionality. My review focuses on improving code quality by addressing an unnecessary import, removing commented-out code, and identifying several areas with code duplication that could be refactored for better maintainability.

[Lin-Nikaido]

@hangfei
I notice this request is similar to #2698 . Could you confirm if they are related?

[Lin-Nikaido]

Hi, I really appreciate the work on this project.

I’ve been eagerly waiting for this feature to be merged, as it would be very helpful for my use case.
If there are any issues or changes needed in this PR, please let me know — I’d be happy to update it right away.

Thank you for your time!

[hangfei]

@hangfei I notice this request is similar to #2698 . Could you confirm if they are related?

Thanks for the contribution. It's similar. I will evaluate how to merge these to PRs. Stay tuned.

[Lin-Nikaido]

I resolve conflict and follow latest version.
I resubmitted PR as #4170