Add Xquik read-only X data tools#6385
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces the Xquik X Data toolset, enabling the retrieval of read-only public X/Twitter data such as tweets, users, and trends, complete with provenance metadata for RAG workflows. The implementation includes a new credential class, core tool logic, and unit tests. Feedback from the review focuses on improving the robustness of API response handling; specifically, it is recommended to read response bodies as text before parsing to prevent stream consumption errors in node-fetch and to adopt a fail-fast strategy by throwing errors when encountering invalid payload formats.
| async function readResponsePayload(response: Response): Promise<unknown> { | ||
| try { | ||
| return await response.json() | ||
| } catch { | ||
| try { | ||
| return await response.text() | ||
| } catch { | ||
| return undefined | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
In node-fetch, the response body can only be consumed once. If response.json() fails because the response is not valid JSON, the body stream is already consumed. It is safer to read the body as text first and then attempt to parse it as JSON. Additionally, per our fail-fast policy for external data, we should throw an error if the response cannot be parsed rather than returning a default or empty value.
async function readResponsePayload(response: Response): Promise<unknown> {
const text = await response.text()
try {
return JSON.parse(text)
} catch (err) {
throw new Error("Failed to parse response as JSON from external source")
}
}References
- When handling potentially invalid data from external sources (like an API response), prefer throwing an error for invalid input types rather than silently returning a default or empty value. This promotes fail-fast behavior.
|
Updated in bab3c0d.
|
Description
Adds a Flowise Tools node for Xquik read-only public X/Twitter data, scoped to the discussion feedback in #6374 (comment).
Validation
Note: full flowise-components lint still reports existing unused eslint-disable directives outside this change in ChatFireworks, GoogleGenerativeAIEmbedding, CustomTool, OpenAPIToolkit, and validator. The new Xquik files pass targeted lint.