Implement sort logic for winget list output #6191
Implement sort logic for winget list output
#6191Madhusudhan-MSFT wants to merge 14 commits intomicrosoft:masterfrom
Conversation
This comment has been minimized.
This comment has been minimized.
|
Is this only applicable to |
This seems like it would be counter to what a user would want. If I did |
The current implementation is scoped to |
This was discussed and agreed upon during the initial design. The intent is to preserve source relevance ordering for queries since a settings-based sort could push the best match down without the user realizing why. Explicit @denelon — I believe this aligns with what we discussed during the design phase, would you mind confirming? |
I think there is, but agree with the approach of limiting the scope for now and letting Demitrius figure out when to bring the rest in |
From the design portion, my feedback was that the default behavior should be to remain unchanged with a query but could have a new ordering without a query. If the user does supply a setting value, my expectation would be that it would be used with or without a query unless overridden with a command line argument. What constitutes a query? Probably only the actual |
Thanks for the clarification, John. Updated in
Removed |
d92410d to
5aae88f
Compare
| std::vector<SortablePackageEntry> entries; | ||
| entries.reserve(items.size()); | ||
| for (size_t i = 0; i < items.size(); ++i) | ||
| { | ||
| entries.push_back(converter(items[i], i)); | ||
| } | ||
|
|
||
| SortEntries(entries, sortFields, direction); | ||
|
|
||
| std::vector<T> sorted; | ||
| sorted.reserve(items.size()); | ||
| for (const auto& entry : entries) | ||
| { | ||
| sorted.push_back(std::move(items[entry.OriginalIndex])); | ||
| } | ||
| items = std::move(sorted); |
There was a problem hiding this comment.
Why not sort items directly? I imagine the idea is that we can avoid doing the case folding multiple times for each string if we do it all upfront, but I'm not sure if the performance improvement is worth the added complexity. The bottleneck is most likely going to be the search, and sorting at most a few hundred items here isn't going to take that long regardless of how slow each comparison is.
I would also think that having a case-insensitive comparison function could be way better in many cases. For example, if the first letters of each string are all different, we would end up only folding those, instead of the full strings.
There was a problem hiding this comment.
Pre-fold was requested by JohnMcPMS in an earlier review round. At this scale (~100-200 items) the difference is negligible. Happy to revisit if profiling shows otherwise.
There was a problem hiding this comment.
At 200 items, a merge sort would expect 2125 comparisons (1.39 * n * log2(n)). Each comparison is two folds (4250), so the equal number of characters folds would occur at an average comparison length of 200 / 4250, or 4.7% of the average length. If you have to compare more characters than that on average, pre-folding will be better. At 100 items, this goes to 10.8%. And this doesn't take into account the relative cost of N function calls vs 2 * 1.39 * log2(N) function calls, which might be on the low end due to needing to potentially use the ICU break iterators to actually find the UTF-8 character sequences to individually fold.
I don't expect that we will have this many items regularly, but at the small numbers that are likely with a query I don't think any choice matters much. I would rather optimize for the worst case even if the best case gets a little worse.
| size_t sortLimit = 0; | ||
| for (const auto& arg : args) | ||
| { | ||
| if (arg.ExecArgType() == Execution::Args::Type::Sort) | ||
| { | ||
| sortLimit = arg.Limit(); | ||
| break; | ||
| } | ||
| } | ||
| REQUIRE(sortLimit > 0); | ||
|
|
||
| // Every valid sort field string must convert successfully. | ||
| std::vector<std::string_view> allFieldNames = { | ||
| "relevance", "name", "id", "version", "source", "available" | ||
| }; | ||
|
|
||
| REQUIRE(allFieldNames.size() == sortLimit); |
There was a problem hiding this comment.
This only ensures that the count limit matches the vector declared here. We still could easily add another sort field and update neither this list nor the count limit.
I think we have helper functions to get the list of all possible values of an enum that we could use here.
There was a problem hiding this comment.
GetAllExponentialEnumValues requires an E::Max sentinel which SortField currently lacks. The current test provides the necessary baseline coverage; we can extend it to use the enum reflection helper in a follow-up once the sentinel is added.
There was a problem hiding this comment.
I agree that this test does not cover anything useful. You have ensured that a list defined in the test is as long as the argument limit.
… add comprehensive tests
…pp file ## Summary Every other Workflow header in the project follows the declarations-only convention with a matching .cpp file. ListSortHelper.h was the only outlier using inline implementations. This refactor aligns it with the codebase convention and eliminates the duplicated sort loop between ListSortHelper.h and WorkflowBase.cpp. ## Changes - **ListSortHelper.h → declarations only**: Remove inline implementations of CompareByField and SortEntries, keeping only the struct definition and function signatures - **ListSortHelper.cpp**: New compilation unit with the extracted implementations of CompareByField and SortEntries - **WorkflowBase.cpp sort integration**: Replace the duplicated inline sort loop with an index-based permutation sort that projects InstalledPackagesTableLine into SortablePackageEntry and delegates field comparison to the shared CompareByField - Remove the local CompareByField adapter that was creating temporary SortablePackageEntry copies per comparison call - Align include path and grouping with repo conventions (bare sibling name, local before system) - Simplify GetArgs access to match established MultiQueryFlow pattern ## Impact - Sort logic now lives in a single shared location, reusable by future commands (search, upgrade, font list) without duplication - Follows the established Workflows/ header convention (24/24 other headers use .h + .cpp) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add --sort, --ascending, and --descending options to the options table. Add new 'Sorting output' section documenting: - CLI sort arguments with multi-field examples - Settings-based sortOrder configuration - Resolution order (CLI > settings > default) - Relevance protection behavior for queries
Validate --sort argument values in Command::ValidateArguments following the same pattern as --scope and --authentication-mode validation. Invalid values produce a clear error message listing valid options (name, id, version, source, available, relevance). Centralized in Command.cpp so any future command adding --sort gets validation automatically.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…edback - Remove HasRelevanceAffectingArgs function (overly broad filter) - Narrow relevance preservation to only the free-text query argument - Settings output.sortOrder now applies even when query is present (user explicitly configured sort preference takes priority) - Filter args (--id, --name, --tag etc.) no longer block settings sort since they use exact/substring matching with no meaningful relevance - Update list.md documentation to reflect new behavior - Resolution: CLI --sort > settings (always) > default relevance (query only)
- Fix comment: 'enable' -> 'ease' unit testing (not impossible, just decoupled) - Remove stale comment from deleted HasRelevanceAffectingArgs function - Add THROW_HR(E_UNEXPECTED) assertion for invalid sort values that bypass validation - Change --sort count limit from 7 to 6 (matches actual SortField enum count)
When no sort preferences are configured (no CLI --sort, no settings) and no positional query is present, default to sorting by name ascending for deterministic, user-friendly output. With a positional query, relevance ordering is preserved unless the user explicitly configures sort preferences. Update list.md to document this conditional default behavior.
Restructure ListSortHelper to own the production sort path: - SortablePackageEntry now precomputes case-folded strings and parsed versions at construction time, avoiding repeated FoldCase/Version parsing during comparisons. - Add OriginalIndex field so entries track their source position. - Add SortBy<T> template that converts arbitrary item vectors into SortablePackageEntry projections, sorts, then reorders the source vector to match. This is the production code path. - WorkflowBase calls SortBy with a conversion lambda, replacing the inline permutation sort. - SortEntries is no longer test-only — it is the same code path that production uses via SortBy. - Add SortBy template tests to validate the end-to-end pipeline with custom source types.
Rename to reflect broader scope — the helper sorts package table rows for any table-based command (list, search, upgrade), not just list.
Change SortField to flag-bit enum (uint32_t) and add DEFINE_ENUM_FLAG_OPERATORS so sort fields compose into a bitmask. SortBy computes the mask once and passes it through the converter to SortablePackageEntry's constructor, which now only precomputes fields that are actually needed for the requested sort.
- Add static_assert on SortField::Available at enum definition site to catch new values that bypass constructor handling. - Replace invalid-sort assertion with FAIL_FAST_MSG for clearer diagnostics. - Collapse two consecutive sortFields.empty() checks into a single branch. - Format SortablePackageEntry constructor arguments one-per-line. - Add --query example to list.md Example queries section.
- `WI_IsAnyFlagSet` was used for single-flag checks where `WI_IsFlagSet` is the correct API - `ComputeSortFieldMask` was defined inline in the header instead of the .cpp file - `SortInstalledPackagesTableLines` was called at three separate call sites instead of being encapsulated inside `OutputInstalledPackages` - Default-sort bypass only checked `Query` argument, missing the `MultiQuery` path - Test assertions validated only a single field (name or ID), providing no cross-field verification of sort correctness - Replace all five `WI_IsAnyFlagSet` calls with `WI_IsFlagSet` for single-flag conditionals - Move `ComputeSortFieldMask` body to PackageTableSortHelper.cpp, leaving only the declaration in the header - Consolidate sort invocation inside `OutputInstalledPackages`, reducing three call sites to one - Add `MultiQuery` argument check alongside `Query` for relevance-preserving default - Replace `GetNames`/`GetIds` test helpers with `ValidateSortResult` that asserts all six precomputed fields (FoldedName, FoldedId, FoldedSource, HasAvailableVersion, ParsedInstalledVersion, ParsedAvailableVersion) per entry - Add release notes entry for sortable list output - No behavioral change to end users — fixes are code quality improvements and correctness for the `--query` multi-value path - Test suite now validates complete sort state rather than a single projection, catching regressions in any field computation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
e8ba2d5 to
c19ce84
Compare
| Utility::Version ParsedAvailableVersion; | ||
| bool HasAvailableVersion = false; |
There was a problem hiding this comment.
| Utility::Version ParsedAvailableVersion; | |
| bool HasAvailableVersion = false; | |
| std::optional<Utility::Version> ParsedAvailableVersion; |
|
|
||
| Settings::SortField ComputeSortFieldMask(const std::vector<Settings::SortField>& sortFields) | ||
| { | ||
| Settings::SortField mask = Settings::SortField::Relevance; |
There was a problem hiding this comment.
Why is Relevance always included? There should be a None or similar for the 0 value.
| return a.FoldedSource.compare(b.FoldedSource); | ||
| case SortField::Available: | ||
| { | ||
| if (a.HasAvailableVersion != b.HasAvailableVersion) |
There was a problem hiding this comment.
Converting to an optional gets automatic empty vs not-empty comparison along with the value comparison. Look it up to ensure it does what you want though, as it might put the empties at the wrong end from where you want.
| { | ||
| // Invalid values should not reach here; ValidateArguments | ||
| // rejects them with a CommandException before workflow execution begins. | ||
| FAIL_FAST_MSG("Unexpected sort field value reached workflow; validation should have caught this."); |
There was a problem hiding this comment.
nit: I was thinking an assert, not a process exit. Is this the kind of thing that warrants that?
| return; | ||
| } | ||
|
|
||
| // 1. Determine sort fields: CLI --sort overrides everything |
There was a problem hiding this comment.
nit: Eventually all of this "determine the sort fields and order" will end up in the helper file as well.
| size_t sortLimit = 0; | ||
| for (const auto& arg : args) | ||
| { | ||
| if (arg.ExecArgType() == Execution::Args::Type::Sort) | ||
| { | ||
| sortLimit = arg.Limit(); | ||
| break; | ||
| } | ||
| } | ||
| REQUIRE(sortLimit > 0); | ||
|
|
||
| // Every valid sort field string must convert successfully. | ||
| std::vector<std::string_view> allFieldNames = { | ||
| "relevance", "name", "id", "version", "source", "available" | ||
| }; | ||
|
|
||
| REQUIRE(allFieldNames.size() == sortLimit); |
There was a problem hiding this comment.
I agree that this test does not cover anything useful. You have ensured that a list defined in the test is as long as the argument limit.
| { | ||
| Args args; | ||
| TestCommand command({ | ||
| Argument{ "sort", 's', Args::Type::Sort, DefaultDesc, ArgumentType::Standard }.SetCountLimit(6), |
There was a problem hiding this comment.
Similarly, this test seems to be checking that the count limit on arguments is implemented properly in the general sense. I honestly don't care if the argument takes in additional values over the total number of fields, the user is just wasting their own compute time recomparing. The only test that I care about existing is that you can provide all of the fields. Removing the limit would ensure that was possible, but if we are going to have the limit, the test needs to generate the set dynamically and ensure that they can all be provided.
| Version, | ||
| Source, | ||
| Available, | ||
| Relevance = 0x0, // Preserves current natural order (source-defined relevance ranking) |
There was a problem hiding this comment.
As said in another comment, we shouldn't treat relevance as "none".
Summary
Implements the sort logic for
winget listoutput, completing the feature requested in #4238. Builds on PR #6177 (settings parsing + CLI handling).Changes
Sort orchestration (
WorkflowBase.cpp):SortInstalledPackagesTableLines— resolves sort fields from CLI--sort> settingsoutput.sortOrder> query-aware default, determines direction, sorts viastd::stable_sort, applies permutation back.-q) is used and no sort preference is configured, relevance ordering is preserved. Ifoutput.sortOrderis configured, it is respected even with queries.FAIL_FAST_MSGfor unrecognized sort field values — ensures invalid enum values crash loudly in debug builds.Sort helper (
PackageTableSortHelper.h/.cpp):SortablePackageEntry— holds pre-computed case-folded sortable fields, computed only for fields needed viaComputeSortFieldMaskbitmask optimization.CompareByField— case-insensitive ordinal comparison using pre-computedFoldCasevalues.SortBy— template sort executor used by the workflow; standaloneSortEntrieswrapper available for unit tests.SortFieldenum uses flag-bit values (uint32_t) withDEFINE_ENUM_FLAG_OPERATORSfor efficient bitmask field tracking.CLI integration (
ListCommand.cpp):--sort(repeatable, limit 6),--ascending/--asc,--descending/--descarguments.Argument validation (
Command.cpp):--sortvalues inValidateArguments, following the--scope/--authentication-modepattern.Documentation (
list.md):--sort,--ascending,--descendingin options table. New "Sorting output" section.Previously,
winget listoutput preserved the source's natural ordering (source-determined relevance ranking). This PR changes the default to sort alphabetically by name (ascending) when no--sortargument, no-qquery, and nooutput.sortOrdersetting is configured.--sort-qquery with no sort preference--sort relevanceor"sortOrder": ["relevance"]"sortOrder": [](empty array)output.sortOrderconfigured +-qqueryTo restore previous (source-determined) ordering:
winget list --sort relevance{ "output": { "sortOrder": ["relevance"] } }Design decisions
-q) without configured sort preserves source relevance ranking. Settings override this if configured — explicit user preference takes priority.stable_sort--sort source --sort namegroups by source, alphabetical within).SortFieldenumComputeSortFieldMaskto skip unused field extraction — measurable win for large package lists.FoldCaseFAIL_FAST_MSGon invalid sortHow validated
Unit tests — 20 test cases:
PackageTableSortHelper.cpp: single/multi-field sorting, both directions, edge cases, case-insensitive comparison, relevance no-opCommand.cpp: readsListCommand::GetArguments()limit and validates against knownConvertToSortFieldstrings — breaks if enum changes without updating limitCommand.cpp: verifiesSetCountLimit(6)rejects 7--sortvalues withTooManyArgErrorManual testing — 21 scenarios on
wingetdev(Debug x64): defaults, single/multi-field sorts, direction flags, query + sort interaction, settings precedence, edge cases, invalid input. All passing.Fixes #4238