Speed up search by ranking from a title map - #3408
Draft
enf0rc3 wants to merge 1 commit into
Draft
Conversation
Ranking needed a url and a title, and Pagefind keeps both in the per-page fragment — so ordering thirty results meant fetching thirty files before the panel could draw, and a page ranked past thirty could not be reached at all. The build now writes a map of result id to url and title beside the index, which is the join Pagefind's own result stub already carries. So the whole result set is ranked before anything is fetched, and fragments are fetched only for the rows being drawn: ten per batch, against thirty to thirty-five for every settled query before. On Slow 4G with the map served uncompressed, first results arrive in 7.2s against 8.9s; the map is 28 KB gzipped, so most of that 122 KB is transfer a CDN removes. The shallow-page search this replaces is gone with it — the second Pagefind query, the landing filter, LANDING_DEPTH and the attribute it needed on every page's content div. A page the query names now wins from anywhere in the list rather than from a shortlist of 227. Relevance holds on both traffic-weighted sets: real-searches 57% w-S@1 and 84% w-S@5, top-pages 90% and 98%, unchanged either side. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Pull request environment is available at https://stoctodocspr3408.z22.web.core.windows.net. You can view the ephemeral environment status in Octopus Deploy. This environment will be automatically deprovisioned when the pull request is closed, or after 7 days of inactivity. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Search ranking needed a URL and a title for every result it ordered, and Pagefind keeps both inside the per-page fragment. So ordering thirty results meant downloading thirty files before the panel could draw a single row — which is what a reviewer noticed on the last PR, watching them trickle through six at a time on Slow 4G.
The build now writes a small map of result id to URL and title beside the index. Pagefind's own result stub already carries the id of its fragment, so that map is all ranking needs.
What this gives
How this was measured
Both traffic-weighted query sets replayed through the real overlay against the merged
main, same machine, same method:Relevance holding exactly is the point. Widening the ranking window from thirty rows to the whole result set moved nothing, so the shortlist it replaces was doing its job, and this does the same job for a third of the fetches.
219 Playwright tests pass, including the existing ranking, paging and tab-count cases unchanged.
Worth a reviewer's opinion
The map is fetched eagerly alongside the index, so someone who opens the overlay and never types now pays 28.5 KB they did not before. Moving it to the first keystroke would avoid that, at the cost of putting it on the path of the first search.