feat(tools): add Scavio Search API tools (web, Amazon, YouTube)#5987
Closed
scavio-ai wants to merge 3 commits into
Closed
feat(tools): add Scavio Search API tools (web, Amazon, YouTube)#5987scavio-ai wants to merge 3 commits into
scavio-ai wants to merge 3 commits into
Conversation
Add ScavioSearchTool, ScavioAmazonSearchTool, and ScavioYouTubeSearchTool powered by the Scavio Search API (scavio.dev). Enables CrewAI agents to perform real-time web searches, Amazon product lookups, and YouTube video searches with structured JSON results. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds Scavio integration: dependency entries and package exports, a ScavioBaseTool handling client init and API-key validation, three concrete tools (web, Amazon, YouTube) with sync/async implementations and result truncation/formatting, README, and comprehensive tests. ChangesScavio Tools Implementation
Sequence Diagram(s)sequenceDiagram
participant ScavioSearchTool
participant ScavioBaseTool
participant ScavioClient
participant AsyncScavioClient
ScavioSearchTool->>ScavioBaseTool: instantiate (api_key, max_results)
ScavioBaseTool->>ScavioClient: ScavioClient(api_key)
ScavioBaseTool->>AsyncScavioClient: AsyncScavioClient(api_key)
ScavioSearchTool->>ScavioClient: google.search(query, params)
ScavioClient-->>ScavioSearchTool: raw response
ScavioSearchTool->>ScavioBaseTool: _truncate_results(raw, "results")
ScavioBaseTool-->>ScavioSearchTool: truncated dict
ScavioSearchTool->>ScavioBaseTool: _format_response(truncated)
ScavioBaseTool-->>ScavioSearchTool: JSON string
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
lib/crewai-tools/pyproject.toml (1)
19-19: ⚡ Quick winMake
scaviooptional or drop the lazy-install path.Right now the packaging story is internally inconsistent:
base.pytreats Scavio as an optional dependency with lazy import/missing-package handling, but Line 19 makes it a hard dependency for everycrewai-toolsinstall. Please either move this into an extra like the other provider SDKs, or simplifyScavioBaseToolto assume the package is always present.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-tools/pyproject.toml` at line 19, The project currently lists "scavio" as a hard dependency in pyproject.toml while base.py/ScavioBaseTool treats it as optional; resolve by either making scavio an optional extra or by removing lazy-missing handling: Option A — move "scavio" from the main dependencies into an extras group (e.g., add it under [project.optional-dependencies] with a name like "scavio" so consumers can install crewai-tools[scavio]); Option B — simplify ScavioBaseTool in base.py to assume scavio is always installed (remove try/except lazy-import/missing-package logic and use a normal import), and keep "scavio" as a required dependency; pick one approach and update pyproject.toml and/or ScavioBaseTool accordingly so the packaging and runtime behavior are consistent.lib/crewai-tools/tests/tools/scavio_tool_test.py (1)
77-267: ⚡ Quick winExercise the public sync and async entry points.
These tests only call private
_run()methods, so they never coverBaseTool.run()validation/usage handling or any_arun()branch. A regression inargs_schemawiring or the async client path would still ship green.Example additions
class TestScavioSearchTool: + `@patch`("crewai_tools.tools.scavio_tool.base.ScavioClient") + `@patch`("crewai_tools.tools.scavio_tool.base.AsyncScavioClient") + `@patch`("crewai_tools.tools.scavio_tool.base.SCAVIO_AVAILABLE", True) + def test_public_run_path(self, mock_async_client, mock_client_cls): + mock_client = MagicMock() + mock_client.google.search.return_value = _mock_search_response() + mock_client_cls.return_value = mock_client + + tool = ScavioSearchTool(api_key=MOCK_API_KEY, max_results=2) + parsed = json.loads(tool.run(query="test query")) + + assert len(parsed["results"]) == 2 + + `@pytest.mark.asyncio` + `@patch`("crewai_tools.tools.scavio_tool.base.ScavioClient") + `@patch`("crewai_tools.tools.scavio_tool.base.AsyncScavioClient") + `@patch`("crewai_tools.tools.scavio_tool.base.SCAVIO_AVAILABLE", True) + async def test_arun_path(self, mock_async_client_cls, mock_client_cls): + mock_async_client = MagicMock() + mock_async_client.google.search = MagicMock(return_value=_mock_search_response()) + mock_async_client_cls.return_value = mock_async_client + + tool = ScavioSearchTool(api_key=MOCK_API_KEY, max_results=2) + parsed = json.loads(await tool._arun(query="test query")) + + assert len(parsed["results"]) == 2🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-tools/tests/tools/scavio_tool_test.py` around lines 77 - 267, Tests only exercise private _run methods so public sync/async entry points and BaseTool.run validation aren't covered; update the tests for each tool class (ScavioSearchTool, ScavioAmazonSearchTool, ScavioYouTubeSearchTool) to call the public run(...) method and the async entry point (await tool.arun(...) or use asyncio.run(tool.arun(...))) in addition to existing _run checks, and assert the same JSON results and that the mocked ScavioClient methods (e.g., mock_client.google.search, mock_client.amazon.search, mock_client.youtube.search) were called with the expected params; ensure you use the same mocks/patches and handle the event loop correctly in the async tests so the _arun / arun code paths and BaseTool.run validation are exercised.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai-tools/src/crewai_tools/tools/scavio_tool/base.py`:
- Around line 34-37: The max_results Field currently allows negatives which
breaks its contract; update the Field declaration for max_results (in class
where max_results is defined in base.py) to enforce a positive integer (e.g.,
use Field(..., ge=1) or Field(..., gt=0) or replace with pydantic.conint(ge=1))
so any assigned value must be >=1; ensure imports are adjusted if you switch to
conint and run tests to verify validation behavior.
- Around line 52-93: The __init__ of the Scavio tool (check SCAVIO_AVAILABLE and
client/async_client initialization) should not prompt or mutate the environment;
remove the interactive click.confirm branch and the subprocess.run("uv add
scavio") call and instead fail fast by raising a clear ImportError when
SCAVIO_AVAILABLE is False (mention installation instructions like "pip install
scavio" or "uv add scavio" in the message). Ensure the code still imports
ScavioClient/AsyncScavioClient when available and that only a deterministic
ImportError is raised from __init__ (do not import or call click/subprocess or
attempt to install).
- Around line 27-33: ScavioBaseTool currently defaults api_key from
os.getenv("SCAVIO_API_KEY") but still constructs ScavioClient/AsyncScavioClient
with api_key=None; update the __init__ of ScavioBaseTool so that inside the
SCAVIO_AVAILABLE branch you explicitly check self.api_key and raise a clear
configuration error (e.g., ValueError or a custom ConfigError) if it's missing,
before attempting to instantiate ScavioClient or AsyncScavioClient; reference
the api_key Field, the SCAVIO_AVAILABLE flag, and the
ScavioClient/AsyncScavioClient instantiation points so the check and error are
added only when the package is available.
---
Nitpick comments:
In `@lib/crewai-tools/pyproject.toml`:
- Line 19: The project currently lists "scavio" as a hard dependency in
pyproject.toml while base.py/ScavioBaseTool treats it as optional; resolve by
either making scavio an optional extra or by removing lazy-missing handling:
Option A — move "scavio" from the main dependencies into an extras group (e.g.,
add it under [project.optional-dependencies] with a name like "scavio" so
consumers can install crewai-tools[scavio]); Option B — simplify ScavioBaseTool
in base.py to assume scavio is always installed (remove try/except
lazy-import/missing-package logic and use a normal import), and keep "scavio" as
a required dependency; pick one approach and update pyproject.toml and/or
ScavioBaseTool accordingly so the packaging and runtime behavior are consistent.
In `@lib/crewai-tools/tests/tools/scavio_tool_test.py`:
- Around line 77-267: Tests only exercise private _run methods so public
sync/async entry points and BaseTool.run validation aren't covered; update the
tests for each tool class (ScavioSearchTool, ScavioAmazonSearchTool,
ScavioYouTubeSearchTool) to call the public run(...) method and the async entry
point (await tool.arun(...) or use asyncio.run(tool.arun(...))) in addition to
existing _run checks, and assert the same JSON results and that the mocked
ScavioClient methods (e.g., mock_client.google.search,
mock_client.amazon.search, mock_client.youtube.search) were called with the
expected params; ensure you use the same mocks/patches and handle the event loop
correctly in the async tests so the _arun / arun code paths and BaseTool.run
validation are exercised.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 6643e07d-02c0-4aec-8f07-4ce41ff17a89
📒 Files selected for processing (10)
lib/crewai-tools/pyproject.tomllib/crewai-tools/src/crewai_tools/__init__.pylib/crewai-tools/src/crewai_tools/tools/__init__.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/README.mdlib/crewai-tools/src/crewai_tools/tools/scavio_tool/__init__.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/base.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_amazon_search_tool.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_search_tool.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_youtube_search_tool.pylib/crewai-tools/tests/tools/scavio_tool_test.py
… install Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyproject.toml`:
- Line 233: The pyproject.toml dependency for scavio currently tracks branch =
"main" which can drift; update the scavio entry in pyproject.toml to pin to a
fixed ref by replacing branch = "main" with either rev =
"c3d359c1099c2a009f56e75bb1885dbf41cf6469" (the commit in the lock) or a stable
tag like v0.1.0 so installs are deterministic; locate the scavio dependency
declaration in pyproject.toml and change the git source to use rev (or tag)
instead of branch, then run the lock/update workflow to verify dependencies
resolve.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 217ccff6-8b60-4076-b33e-d2cb1f0b8829
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
lib/crewai-tools/pyproject.tomllib/crewai-tools/src/crewai_tools/tools/scavio_tool/README.mdlib/crewai-tools/src/crewai_tools/tools/scavio_tool/base.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_amazon_search_tool.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_search_tool.pylib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_youtube_search_tool.pylib/crewai-tools/tests/tools/scavio_tool_test.pypyproject.toml
💤 Files with no reviewable changes (3)
- lib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_search_tool.py
- lib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_youtube_search_tool.py
- lib/crewai-tools/src/crewai_tools/tools/scavio_tool/scavio_amazon_search_tool.py
✅ Files skipped from review due to trivial changes (1)
- lib/crewai-tools/src/crewai_tools/tools/scavio_tool/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
- lib/crewai-tools/tests/tools/scavio_tool_test.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new tools powered by the Scavio Search API, enabling CrewAI agents to perform real-time structured searches across web, Amazon, and YouTube:
Architecture
brave_search_tool— sharedScavioBaseToolbase class inbase.pywith individual tool filesscavioPython SDK as apackage_dependency(lazy-imported, auto-install prompt if missing)_run) and async (_arun) execution supportedmax_resultsFiles Added
Files Modified
lib/crewai-tools/src/crewai_tools/tools/__init__.py— added Scavio tool imports +__all__entrieslib/crewai-tools/src/crewai_tools/__init__.py— added Scavio tool exports +__all__entrieslib/crewai-tools/pyproject.toml— addedscaviodependencyTest plan
pytest tests/tools/scavio_tool_test.py -v)crewai_toolspackage levelUsage Example
Scavio provides real-time structured data from Google, Amazon, Walmart, YouTube, Reddit, and TikTok. Free API keys at dashboard.scavio.dev.
Summary by CodeRabbit
New Features
Documentation
Tests