Initial implementation of AI Assistant#732
Open
yuripourre wants to merge 1 commit intoBabylonJS:masterfrom
Open
Initial implementation of AI Assistant#732yuripourre wants to merge 1 commit intoBabylonJS:masterfrom
yuripourre wants to merge 1 commit intoBabylonJS:masterfrom
Conversation
Open
Contributor
|
@yuripourre did you try the idea you had to have more "common way" to handle actions? |
Contributor
Author
|
@julien-moreau no, I thought about it a lot and I think the current implementation would be better long-term. The reason is, the tools would be separate by the type of element. We spoke about having this as an external plugin, maybe it's the way to go. |
Contributor
julien-moreau
left a comment
julien-moreau
left a comment
There was a problem hiding this comment.
I just improved the support of plugins in the editor. I have plans to make this AI a plugin that you'll take the control
Contributor
Author
I really appreciate it, please tell me how I can help. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title
Initial implementation of AI Assistant. I recommend using the qwen family but any model with tools support should work.
Screencast.From.2025-11-17.13-08-09.mp4
Summary
Add a new tab called AI Assistant where the user can send messages to a LLM to modify the scene. Currently only local Ollama is supported and it can only modify meshes, materials, lights and cameras.
It is still in very early stages but shouldn't be hard to add more features from there.
Changes Made
AI Assistant tab
Creates a new tab AI Assistant with a chat interface.
Integration with Ollama using langgchain
Benefits
Users can use LLMs to modify the scene