Skip to content

feat: add shell tool#1107

Open
akihikokuroda wants to merge 19 commits into
generative-computing:mainfrom
akihikokuroda:shell
Open

feat: add shell tool#1107
akihikokuroda wants to merge 19 commits into
generative-computing:mainfrom
akihikokuroda:shell

Conversation

@akihikokuroda
Copy link
Copy Markdown
Member

@akihikokuroda akihikokuroda commented May 20, 2026

Pull Request

Issue

Part of #1024

Description

This PR doesn't include issues mentioned in #1087

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used: claudecode

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda akihikokuroda requested a review from a team as a code owner May 20, 2026 16:18
@github-actions github-actions Bot added the enhancement New feature or request label May 20, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

This comment is managed by a bot. Editing it is fine — checking off boxes, adding notes — but please leave the HTML comment marker on the first line alone, otherwise checklist updates will break.

Tool PR Checklist

Use this checklist when adding or modifying tools in mellea/stdlib/tools/.

Protocol Compliance

  • Ensure compatibility with existing backends and providers
    • For most tools being added as functions, this means that calling convert_function_to_tool works

Integration

  • Tool exported in mellea/stdlib/tools/__init__.py or, if you are adding a library of tools, from your sub-module

Copy link
Copy Markdown
Contributor

@planetf1 planetf1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the design is solid — subprocess.run(shell=False) as the foundation, shlex.split() for tokenisation, and working-dir restriction are the right primitives. Two confirmed bugs block merge; a third issue means the _bash_patterns.py test suite gives false assurance about what's actually blocked at runtime.

if cmd not in SAFE_WRAPPER_COMMANDS or arg_cmd not in (
"bash",
"sh",
"zsh",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

env bash -c <payload> bypasses the denylist

The code-execution check (lines 176–182) only fires when argv[0] is the interpreter. When argv[0] is env, that path is never reached — and lines 225–231 explicitly allow shells as nested arguments inside safe wrappers, so -c is never re-checked.

_is_dangerous_command(["env", "bash", "-c", "id"])          # (False, '')
_is_dangerous_command(["timeout", "60", "bash", "-c", "id"])  # (False, '')

Fix: re-check for -c/-e on the nested interpreter, or restrict the shell allowlist here to script-file invocations only (no -c/-e in the remaining argv).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

additional check is added. The tests are added.

Comment thread mellea/stdlib/tools/_bash_audit.py Outdated
with self._storage_lock:
results = self._violations[:]

for violation in results:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classic iterate-and-remove bug — filter results will be incomplete

results.remove() inside for violation in results causes Python's iterator to skip the element immediately after each removal. Filters will silently return the wrong set.

if session_id:
    results = [v for v in results if v.session_id == session_id]
if pattern:
    results = [v for v in results if v.pattern == pattern]
if category:
    results = [v for v in results if v.category == category]
if severity:
    results = [v for v in results if v.severity == severity]

Comment thread mellea/stdlib/tools/shell.py Outdated
Returns:
A tuple of (has_dangerous_paths, reason_message).
"""
write_commands = {"rm", "touch", "cp", "mv", "mkdir", "mkfifo", "mknod", "tee"}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write_commands is missing several filesystem-write commands

chmod, chown, ln, and dd aren't in this set, so the dangerous-path checks never fire for them. A few things that currently slip through:

  • chmod 777 /etc/passwd
  • ln -sf /etc/passwd /allowed/path/link (symlink escape out of the allowed fence)
  • dd if=/dev/urandom of=/boot/vmlinuz

Same set is duplicated on line 415 — both need updating.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional write commands are added. Add "ln" command handling.

]


def check_all_patterns(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This framework isn't connected to the execution path

shell.py only imports record_bash_violation from _bash_auditcheck_all_patterns() is never called from bash_executor(). The tests in test_bash_guardrails.py all pass, but none of it affects whether a command is actually blocked at runtime.

If this is intended as the extensible pattern layer, wire check_all_patterns(argv) into _is_dangerous_command(). If it's documentation of the threat model, the disconnect from the execution path should be made explicit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wire check_all_patterns is wired into the execution path by updating _is_dangerous_command to use the pattern framework.

Comment thread docs/examples/tools/shell_example.py Outdated


def example_3_with_working_dir() -> None:
"""Example 3: Restrict write validation and execution cwd to a directory."""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two functions are both labelled "Example 3" — example_3_llm_with_forced_tool_use (line 85) already has that number. This one should be Example 4, with the subsequent examples renumbered.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda akihikokuroda requested a review from planetf1 May 26, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants