Skip to content

v0.8.27: search MCP, landing, file editor improvements - #7646

Merged
waleedlatif1 merged 24 commits into
mainfrom
staging
Sep 9, 2026
Merged

v0.8.27: search MCP, landing, file editor improvements#7646
waleedlatif1 merged 24 commits into
mainfrom
staging

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

waleedlatif1 and others added 24 commits September 8, 2026 15:58
… schema (#7612)

* fix(parallel): migrate search and extract to V1 and fix deep research schema

- Search and Extract targeted the retired /v1beta endpoints; both now use /v1 with the
  advanced_settings request shape, V1 modes (turbo, fast, basic, advanced), and mode-aware
  hosted-key pricing. Retired one-shot/agentic modes are remapped and an objective-only
  workflow falls back to the objective as its query so saved blocks keep working
- Extract drops the removed excerpts toggle (V1 always returns excerpts), nests full_content
  under advanced_settings, surfaces the per-URL errors array, and fails cleanly when every URL
  errored
- Deep research sent output_schema as a bare string, which Parallel treats as a text schema,
  and read status from the wrong path so it always reported completed. It now offers an
  Output Format choice (markdown report by default, structured JSON on Pro+), sends the
  object-form schema, reads run.status, and passes a timeout to the blocking result endpoint
- Processor dropdown lists the documented tiers; subblock migration for the removed toggle;
  regenerated tool metadata and docs; unit tests for request shapes, pricing, and block mapping

* fix(parallel): move the Pi web_search Parallel request to V1 and drop any in tests

- The Pi web_search host and sandbox request builders still targeted /v1beta/search with the
  beta header and an objective-only body, which the V1 tool now rejects because search_queries
  is required. Both send the query as the single search query plus objective, with max_results
  under advanced_settings, so the parity test holds again
- Test helpers return Record<string, unknown> instead of any
* feat(search-mcp): connect clients with Sim OAuth

* fix(search-mcp): preserve existing API OAuth grants
* fix(copilot): resolve folder mentions across resource families

* fix(copilot): preserve organization folder delegation restrictions
…, catalogs, and workgroups (#7614)

* feat(athena): complete SQL-workload coverage with prepared statements, catalogs, and workgroups

Adds 15 tools so the Athena block covers the full SQL workload surface: prepared statements (create/get/batch get/update/list/delete), get query runtime statistics, batch get and update named queries, list/get data catalogs, get database, get table metadata, and list/get workgroups. Start query now accepts execution parameters and result reuse configuration. Contracts share one connection schema with region validation, and list databases now returns database parameters.

Claude-Session: https://claude.ai/code/session_01AtPwvDwhkhed2MwcZudrPN

* fix(athena): keep zero-minute result reuse age, enforce name patterns, declare table parameters output

Claude-Session: https://claude.ai/code/session_01AtPwvDwhkhed2MwcZudrPN
* fix(execution): wait for Redis readiness before subscribing

* fix(execution): preserve pending subscriptions through reconnects
* fix(knowledge): make indexing and connector recovery durable

* fix(helm): bump chart version for indexing job limits

* fix(knowledge): protect connector uploads during attachment
* fix(landing): show footer painting previews while loading

* fix(landing): include static image types in clean checkouts
* fix(landing): disable native dragging of navbar logos

* fix(landing): cover navbar logo drag prevention
* fix(knowledge): require workspace or organization ownership

* fix(knowledge): update processing tests for scoped billing
* fix(landing): improve mobile hero and workflow previews

* fix(landing): remove blank preview intro and close mobile QA gaps

* fix(landing): clarify swipe discovery and share interactive logs

* fix(landing): use absolute logs graph export

---------

Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
* fix(file-editor): preserve pasted content and literal markdown

* fix(file-editor): retain rich fragments through image uploads
* fix(files): preserve valid zero-byte tool outputs

* fix(files): keep url fallback and validate base64 canonically

Empty inline data no longer shadows a url. `''` and an empty Buffer now
produce a zero-byte buffer, which made `if (!buffer && data.url)` skip a
download that previously ran, so a tool emitting a placeholder alongside a
real url would have stored an empty file.

Replace the hand-rolled zero-length base64 check with the shared
`isCanonicalBase64`. `Buffer.from(x, 'base64')` silently drops characters
outside the alphabet, so `a!b!c!AAAA` and the non-canonical `AAAAA` decoded
to plausible-looking bytes instead of failing. Normalize first so the
payloads that already worked keep working: line-wrapped MIME base64,
unpadded base64url, and a base64 `data:` URI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VxZCGfFjZnokR8uFeWrfLp

* fix(files): reject whitespace-only base64 payloads

Normalization strips whitespace, so a nonempty payload of only spaces or
newlines collapsed to the empty encoding and was stored as a zero-byte file.
Only a payload that is empty before normalization is a legitimate zero-byte
file, so compare against the value after the data: URI prefix is removed —
which keeps an empty `data:text/plain;base64,` accepted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VxZCGfFjZnokR8uFeWrfLp

---------

Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
#7643)

* improvement(helm): lint the chart with ct instead of hand-rolled checks

Replaces `helm lint` and the bespoke version-bump job with chart-testing,
the CNCF chart linter that ingress-nginx, prometheus-community and
external-secrets all gate on. It subsumes both: `helm lint` plus yamllint
over Chart.yaml and every values file, Chart.yaml schema validation, and
--check-version-increment, which is on by default and is exactly what the
33 lines of bash were reimplementing.

One trap worth recording. Passing `--charts` silently DISABLES the version
check -- it prints "Version increment checking disabled." and still exits 0,
so wiring it that way would have swapped a working gate for one that can
never fail. The PR path uses `--chart-dirs helm --target-branch <base>`; a
push has no base to diff, so `--charts` is correct there. Verified both
directions in a scratch repo with a real remote: content changed without a
bump gives "chart version not ok. Needs a version bump!" and exit 1, the
same change with a bump gives "Chart version ok."

Three chart fixes ct's yamllint required: 66 lines of trailing whitespace in
values.yaml, one inline comment a space short of the two yamllint wants, and
brace spacing in ci/full-values.yaml. Nothing but whitespace -- no
non-comment line changed, and both rendered manifest sets are byte-identical
before and after.

Renames ci/kind-values.yaml to ci/kind-overlay.yaml. ct treats every
ci/*-values.yaml as a standalone values set, but that file is a partial
layered on default-values.yaml, so linting it alone tripped the chart's own
required-secret guards. The new name is outside the glob and the header says
why.

Maintainer validation stays off: it resolves maintainers[].name against real
forge accounts and ours is the display name "Sim Team", so enabling it would
change what Artifact Hub shows.

* feat(helm): derive chart appVersion from the release instead of gating on it

appVersion is what the chart's image tags default to, so a stale one publishes
a chart that installs an older Sim than the release it ships with, and
images.yaml -- the list an operator mirrors into a disconnected registry --
names the wrong tags with it. Published chart versions are immutable, so each
stale value is frozen the moment it ships.

It was bumped by hand, drifted forty releases, and drifted twice more after a
check started catching it. That is the tell: the check could refuse to publish
but could not supply the value, so the only thing it reliably produced was a
red build on every release and a chart that never shipped. Only 1.11.0 was
ever published for exactly this reason.

The publish jobs now derive it. On a release merge the tag does not exist yet
-- this commit is what cuts it -- so the version comes from the commit
subject, and from the latest release for every other push. Both publish paths
run the same script before packaging, so the OCI artifact and the HTTP repo
cannot disagree.

Removes the post-merge check, which is now unreachable by construction, and
does not replace it with the pre-merge title gate that was considered: that
one guarded ground truth with a heuristic, and its failure mode was passing
silently.

Precedent is cert-manager, whose chart also lives in its application repo and
which injects the version at package time. The projects that commit the value
and bump by hand -- argo-cd, ingress-nginx, prometheus-community -- all keep
the chart in a separate repository, where a human is already editing
Chart.yaml as the unit of change. We are the former shape.

Verified the resolution across six subjects: release commit, multi-line body
carrying a decoy version, ordinary push, leading whitespace, shell
metacharacters (no expansion), and a version that is not at the start. Also
verified the script's exit codes directly rather than through a pipe, since a
gate that cannot fail is the thing being replaced here.

Sets appVersion to v0.8.26, two releases ahead of where it was stuck, and
regenerates images.yaml with it.

* fix(helm): match ci.yml's release predicate and resolve the version once

Two review findings, both real.

The leading-whitespace tolerance was wrong here. detect-version in ci.yml is
what actually cuts the tag and builds the images, and it anchors the version
at the first character. Accepting a subject it rejects meant the chart could
publish naming a release that was never created, pointing at images that do
not exist. The pattern is now identical to ci.yml's, with a comment saying it
has to stay that way. The tolerance came from a PR-title parsing problem in an
earlier design; commit subjects have a different authority.

The two publish jobs also resolved the version independently, and they run in
parallel. A release becoming public between those two API calls would package
the same immutable chart version with different appVersions, so the OCI
artifact and the HTTP repo would install different Sims. Resolution moved to
one job both consume, which also removes the duplicated logic.

Separately, --check only inspected Chart.yaml while the script writes both
that and the inventory derived from it, so it could report success over a
stale inventory. It now verifies both halves; confirmed it exits 1 when the
inventory is stale and 0 once regenerated.
* feat(knowledge): export knowledge bases as bundle archives

Adds a streaming zip export for a knowledge base across the v2 API, the
internal API, the CLI, and the UI. The bundle carries the base's
configuration and tag definitions, each workspace-visible document's original
file and chunk text, and optionally the chunk vectors so an import into a
deployment with the same embedding model can reuse them. Access-control lists,
connector links, credentials, uploader identity, and storage keys never leave.

- knowledge.export operation gated by a new Knowledge Base Export permission
  group setting; exports record an audit event
- entries are appended one at a time so a large base never fans out storage
  reads and the manifest is written last with the counts actually produced
- sim knowledge export <knowledgeBaseId> --output-file kb.zip [--no-vectors]
- Export in the knowledge base row menu and base page header

* fix(knowledge): expose authorize on knowledge use cases and gate export on describable bundles

- defineAuthorizedKnowledgeUseCase now answers authorize() through the same
  resolution execute() uses, which headSafe: false routes require at module
  init; defineAuthorizedWorkspaceUseCase's return type states that authorize
  is always present
- the export use case validates the stored base against the bundle manifest
  schema before any byte streams, so a value the import side would refuse
  surfaces as a 409 rather than a truncated archive
- Export is hidden in the base menu and header when a permission group
  withholds knowledge.export
- operations id snapshot includes knowledge.export

* chore(docs): count the knowledge export path in the OpenAPI download test

* improvement(knowledge): read inline document payloads lazily and scope chunk reads to the base

Follow-ups from an independent review pass over the export path.

- the document listing no longer selects fileUrl: a data: document can hold
  megabytes in that column, so the listing carries a flag and the archive
  reads one payload at a time when it reaches that entry
- iterateDocumentChunks filters on knowledgeBaseId as well as documentId, so
  a future caller cannot reach another base's chunks through the bundle
- tag definitions are validated by the single bundle gate, which answers 409
  like every other undescribable value rather than throwing a bare ZodError
- a consumer that abandons the download aborts the append loop and destroys
  the in-flight source instead of leaving it pending
- direct tests for the knowledge use-case builder's authorize() on the
  workspace, organization, and legacy personal branches

* improvement(knowledge): validate stored chunk counts and scope inline reads to the base

- the bundle gate checks each document's stored chunk count against the
  format's per-document ceiling, so a base holding more chunks than a bundle
  can describe is refused up front rather than written into an invalid manifest
- the inline payload read carries its knowledge base id, matching the chunk
  reads, so neither can reach a row outside the base being exported
- an append is refused once the consumer has aborted, since a destroyed
  archive has no listener left to receive the error it would emit

* improvement(knowledge): bound the streamed chunks and validate the written manifest

Findings from a reuse, simplification, efficiency, and altitude review.

- the chunk stream stops at the per-document ceiling and raises afterwards, and
  the manifest that actually ships is the one validated: chunkCount is
  denormalized, so a document re-chunked mid-export could previously pass the
  pre-flight gate and land in a manifest no importer would accept
- a real archive failure is no longer mistaken for the consumer walking away;
  archiver emits close for both, so only the abort's own error is swallowed
- the inline-file probe reads a 5-byte prefix instead of matching the whole
  column, which was detoasting every document's fileUrl and defeating the
  optimization the listing exists for
- storedChunkCount replaces the redundant hasChunks pair
- the CLI reads the RFC 5987 filename first, so a knowledge base named
  "Suporte tecnico" no longer saves under a mangled ASCII name
- drops three bundle ceilings the import half will introduce, breaks a
  circular type import, and states the authorize guarantee in the type so a
  headSafe route is proven at compile time rather than at module load
* improvement(search): bound indexing and simplify source recovery

* fix(search): prune idle dispatch owners safely

* test(search): use static invitation mocks

* improvement(search): use source settings pages and searchable lists

* improvement(search): align source actions and load people concurrently

* improvement(search): use standard source row navigation

* fix(search): verify setup recovery and refresh guides

* fix(settings): gate organization view with search rollout

* improvement(search): organize integration management around providers and sources

* chore(search): preserve staging indexing implementation after rebase

* fix(search): keep account views consistent and avoid false unsaved edits

* improvement(search): clarify member connections and source administration

* fix(search): reconcile sync progress and finish source guides

* fix(sidebar): restore shared collapse animation
* fix(landing): support swiping customer stories

* fix(landing): support horizontal wheel story navigation
@waleedlatif1
waleedlatif1 requested a review from a team as a code owner September 9, 2026 04:46
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 9, 2026 4:46am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review (696 files, 100 file limit).

@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into main yet (aggregated across the feature PRs in this release). Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ⚠️ simstudioai/mothership#485 — merged into staging (this PR targets main) — fix(context): discover contents of attached folders

@waleedlatif1
waleedlatif1 merged commit bc540db into main Sep 9, 2026
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants