Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
|
Could you add a link from it in the footer? |
Member
Author
|
@atinux in which section? |
Member
|
Explore > AI Evals Another suggestion is to remove Modules in the footer as already in the header now and have:
|
📝 WalkthroughWalkthroughAdds an AI Evals feature: footer navigation removes an external Modules link, updates Explore links to internal paths (/templates, /showcase), and adds an AI Evals item (/evals). Introduces a new page at app/pages/evals.vue that loads public agent-results data and renders an agent evaluation dashboard with filters, computed model/agent summary rows, expandable per-evaluation details, and SEO metadata. Adds a content collection evals in content.config.ts and content/evals.yml metadata, and adds public/agent-results.json with static evaluation results. Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
content.config.ts (1)
425-433:githubUrlshould use.url()validation for consistency.Other URL fields in this config (e.g.,
demo,purchase,website) usez.string().url(). Apply the same validation here.♻️ Suggested fix
evals: defineCollection({ type: 'data', source: 'evals.yml', schema: z.object({ title: z.string(), description: z.string(), - githubUrl: z.string() + githubUrl: z.string().url() }) })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content.config.ts` around lines 425 - 433, The evals collection schema currently validates githubUrl with z.string(); update the schema in the defineCollection call for evals to use z.string().url() for githubUrl so it matches other URL fields (demo, purchase, website) and enforces proper URL validation; locate the evals defineCollection and replace the githubUrl schema entry accordingly.app/pages/evals.vue (2)
50-53: Fetching own static asset via full site URL is suboptimal for SSR.
$fetch(joinURL(url, '/agent-results.json'))makes an external HTTP request to the app's own origin during SSR. Since this is a file inpublic/, you can simplify to$fetch('/agent-results.json')— Nuxt's$fetchhandles this internally during SSR without a network round-trip.♻️ Suggested fix
const [{ data: page }, { data: rawData }] = await Promise.all([ useAsyncData('evals', () => queryCollection('evals').first()), - useAsyncData('agent-results', () => $fetch<AgentResultsData>(joinURL(url, '/agent-results.json'))) + useAsyncData('agent-results', () => $fetch<AgentResultsData>('/agent-results.json')) ])This also removes the need for the
useSiteConfig()call on line 48 and thejoinURLimport on line 4 (if unused elsewhere).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` around lines 50 - 53, The $fetch call is making an external request via joinURL(url, '/agent-results.json') during SSR; change the useAsyncData invocation that fetches agent-results to call $fetch('/agent-results.json') instead (e.g., inside the second useAsyncData for 'agent-results') so Nuxt serves the public file without a network round-trip, and remove the now-unused useSiteConfig() call and joinURL import if they are no longer referenced.
122-134: Model avatar mapping is brittle and will need manual updates for every new model.This works for the current dataset but consider externalizing this mapping (e.g., into the JSON data or a config file) so new models don't require code changes.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` around lines 122 - 134, getModelAvatar currently hardcodes string checks for model names making it brittle; refactor by moving the model-to-avatar mapping out of the function into a configurable data source (e.g., a JSON config or a constant imported from a config module) and update getModelAvatar to look up the lowercased model key in that mapping with a sensible default fallback; specifically, create a mapping object (or load JSON) keyed by normalized model identifiers, import/use it inside getModelAvatar, replace the series of if checks with a single map lookup, and ensure the function still returns undefined or a default avatar when no match is found.app/composables/useNavigation.ts (1)
136-145: Inconsistent URL format: relative vs absolute paths in the same footer section.
TemplatesandShowcaseuse absolute URLs (https://nuxt.com/...) while the newAI Evalsentry uses a relative path (/evals). Consider using consistent path formats within the same section.♻️ Suggested fix
}, { label: 'AI Evals', - to: '/evals' + to: 'https://nuxt.com/evals' }]Or alternatively, make Templates and Showcase relative too.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/composables/useNavigation.ts` around lines 136 - 145, The footer navigation "children" array in useNavigation.ts mixes absolute URLs ('Templates' and 'Showcase' use https://nuxt.com/...) with a relative path for 'AI Evals' ('/evals'), causing inconsistent link behavior; update the entry for 'AI Evals' (the object with label 'AI Evals' inside the children array) to use the same URL format as the other items (e.g., change to 'https://nuxt.com/evals') or alternatively make the 'Templates' and 'Showcase' entries relative to match '/evals'—apply the change in the children array where these objects are defined.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/pages/evals.vue`:
- Around line 85-99: The computed allResults can divide by zero when evals is
empty; update the loop inside the allResults computed to guard against
evals.length === 0 by computing successRate only when evals.length > 0 (e.g.,
set successRate to 0 or a safe fallback), i.e. compute successes from evals, set
const denom = evals.length, then use denom ? Math.round((successes/denom)*100) :
0 when building the ModelRow object (referencing allResults, ModelRow, evals,
successes, and successRate).
---
Nitpick comments:
In `@app/composables/useNavigation.ts`:
- Around line 136-145: The footer navigation "children" array in
useNavigation.ts mixes absolute URLs ('Templates' and 'Showcase' use
https://nuxt.com/...) with a relative path for 'AI Evals' ('/evals'), causing
inconsistent link behavior; update the entry for 'AI Evals' (the object with
label 'AI Evals' inside the children array) to use the same URL format as the
other items (e.g., change to 'https://nuxt.com/evals') or alternatively make the
'Templates' and 'Showcase' entries relative to match '/evals'—apply the change
in the children array where these objects are defined.
In `@app/pages/evals.vue`:
- Around line 50-53: The $fetch call is making an external request via
joinURL(url, '/agent-results.json') during SSR; change the useAsyncData
invocation that fetches agent-results to call $fetch('/agent-results.json')
instead (e.g., inside the second useAsyncData for 'agent-results') so Nuxt
serves the public file without a network round-trip, and remove the now-unused
useSiteConfig() call and joinURL import if they are no longer referenced.
- Around line 122-134: getModelAvatar currently hardcodes string checks for
model names making it brittle; refactor by moving the model-to-avatar mapping
out of the function into a configurable data source (e.g., a JSON config or a
constant imported from a config module) and update getModelAvatar to look up the
lowercased model key in that mapping with a sensible default fallback;
specifically, create a mapping object (or load JSON) keyed by normalized model
identifiers, import/use it inside getModelAvatar, replace the series of if
checks with a single map lookup, and ensure the function still returns undefined
or a default avatar when no match is found.
In `@content.config.ts`:
- Around line 425-433: The evals collection schema currently validates githubUrl
with z.string(); update the schema in the defineCollection call for evals to use
z.string().url() for githubUrl so it matches other URL fields (demo, purchase,
website) and enforces proper URL validation; locate the evals defineCollection
and replace the githubUrl schema entry accordingly.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/pages/evals.vue`:
- Around line 48-60: Replace the absolute fetch with a relative path and
propagate async errors before turning missing data into 404s: change the $fetch
call to use '/agent-results.json' instead of joinURL(url,
'/agent-results.json'), and destructure useAsyncData responses to include error
refs (e.g. const [{ data: page, error: pageError }, { data: rawData, error:
rawDataError }] = await Promise.all([...])). First check and throw the async
errors (throw pageError.value or rawDataError.value / createError with their
message/status) if present, then only after no errors check for missing
page.value or rawData.value and throw the 404 createError for not-found.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
app/composables/useNavigation.tsapp/pages/evals.vuecontent.config.tspublic/agent-results.json
✅ Files skipped from review due to trivial changes (1)
- public/agent-results.json
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
app/pages/evals.vue (1)
239-239: Consider simplifying the guard condition.Since
rawDatais now a static import (always defined at build time), therawDatacheck inv-if="page && rawData"is redundant. You could simplify tov-if="page".♻️ Proposed simplification
- <div v-if="page && rawData"> + <div v-if="page">🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` at line 239, The template guard currently uses v-if="page && rawData" but rawData is a static import and always defined; update the template to remove the redundant check by changing the conditional to v-if="page" (modify the element that currently uses v-if="page && rawData" to use only v-if="page"), ensuring no other logic depends on rawData being falsy.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/pages/evals.vue`:
- Around line 32-38: Remove the unused AgentResultsData interface declaration:
locate the interface named AgentResultsData and delete its full declaration
block, and verify there are no remaining references to AgentResultsData
elsewhere (if any, replace with the inferred/static type or the concrete type
from the imported JSON). Ensure the file compiles after removal.
---
Nitpick comments:
In `@app/pages/evals.vue`:
- Line 239: The template guard currently uses v-if="page && rawData" but rawData
is a static import and always defined; update the template to remove the
redundant check by changing the conditional to v-if="page" (modify the element
that currently uses v-if="page && rawData" to use only v-if="page"), ensuring no
other logic depends on rawData being falsy.
atinux
approved these changes
Feb 26, 2026
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
app/pages/evals.vue (1)
91-96: Reset expanded state when the agent filter changes.Keeping old expansion state after filtering can leave expansion UI out of sync with the newly visible rows.
♻️ Suggested adjustment
-import { h, resolveComponent } from 'vue' +import { h, resolveComponent, watch } from 'vue' @@ -const expanded = ref({}) +const expanded = ref<Record<string, boolean>>({}) + +watch(selectedAgents, () => { + expanded.value = {} +})Also applies to: 127-128
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` around lines 91 - 96, filteredResults currently filters allResults by selectedAgents but doesn't reset the UI expansion state, causing stale expanded rows after filtering; add a watcher on selectedAgents (or incorporate logic where filteredResults is computed) to clear the expansion state ref (the ref that tracks expanded rows — e.g., expanded or expandedRows) whenever selectedAgents changes, and apply the same pattern to the other agent-filter computed at lines 127-128 so expansion is cleared any time the agent filter is updated.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/pages/evals.vue`:
- Around line 99-103: The computed formattedDate can render "Invalid Date" for
malformed rawData.metadata.exportedAt; update formattedDate to validate the
parsed Date (use new Date(rawData.metadata.exportedAt) then check date.getTime()
or isNaN(date.getTime())/isFinite) and return an empty string (or fallback) when
the date is invalid before calling toLocaleDateString; reference the
formattedDate computed and rawData.metadata.exportedAt when making the change.
---
Nitpick comments:
In `@app/pages/evals.vue`:
- Around line 91-96: filteredResults currently filters allResults by
selectedAgents but doesn't reset the UI expansion state, causing stale expanded
rows after filtering; add a watcher on selectedAgents (or incorporate logic
where filteredResults is computed) to clear the expansion state ref (the ref
that tracks expanded rows — e.g., expanded or expandedRows) whenever
selectedAgents changes, and apply the same pattern to the other agent-filter
computed at lines 127-128 so expansion is cleared any time the agent filter is
updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
❓ Type of change
📚 Description