Open
Conversation
Contributor
Adds the AgenticCheck construct that lets users define agentic checks in their Checkly project via checkly.config.ts. Agentic checks use a prompt string (no scripts/bundling) to define what the AI agent should verify. - Add AGENTIC to CheckTypes enum - Create AgenticCheck extending Check (not RuntimeCheck) - Validate prompt (required, max 10000 chars) and frequency (min 30) - Add construct export and analytics mapping - Add 6 tests covering synthesis, defaults, groups, and validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Agentic checks always run in parallel for better reporting UX. The construct omits runParallel from props (users can't override it) and hardcodes it to true in synthesize(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This reverts commit a77f117.
Lock down AgenticCheckProps to only the subset of CheckProps the Checkly platform currently honors for agentic checks. Omits locations, privateLocations, runParallel, retryStrategy, shouldFail, doubleCheck, triggerIncident and groupId at the type level, and restricts frequency to the discrete set exposed in the webapp builder. Defensive overrides in the constructor ensure project-level config defaults for any of these fields are also ignored, so the construct never claims to support what the platform won't honor. These restrictions can be relaxed additively (without breaking changes) once the platform supports the corresponding capabilities for agentic checks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cess
Adds an `agentRuntime` prop to AgenticCheck that declares the runtime
context the agent has access to during execution: which skills it can
use and which environment variables it is permitted to read.
agentRuntime is the explicit security boundary for the agent. Anything
not declared here is unavailable at runtime, which keeps the blast
radius of any prompt injection as small as possible. Environment
variables accept either bare names or `{ name, description }` objects;
descriptions are passed to the agent so it can decide when to read each
variable, and are validated to stay within the runner's 200-char limit.
The construct always synthesizes a complete agentRuntime payload (with
empty arrays when unset), so the backend has unambiguous source-of-truth
semantics: omitted skills/env vars mean "the agent should not have
them", not "preserve whatever was there before". The backend translates
this clean shape into the existing agenticCheckData storage blob.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
thebiglabasky
force-pushed
the
herve/AI-114/agentic-check-cli-construct
branch
from
9e02521 to
e24774d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AgenticCheckconstruct for defining AI-powered monitoring checks viacheckly.config.tsCheck(no runtime/bundling needed) with apromptfield (required, max 10,000 chars)AGENTICtoCheckTypesenum and analytics mappingsUsage
Backend dependency
Requires the companion backend PR to update the CLI deploy schema: https://github.com/checkly/monorepo/pull/1098
Test plan
checkly deploy --previewagainst local backend🤖 Generated with Claude Code