fix(learn): gate lessons on the learner's code, not the hint comments - #1259
Merged
Conversation
Every lesson checks "did the learner use this API", but the template each lesson writes carries that name in its own TODO/Hint lines. A plain grep matched the hint, so all nine gates passed on an untouched file -- and a lesson could be completed with an unrelated passing assertion: # TODO: Check that message contains "bashunit" # Hint: assert_contains "substring" "$message" assert_same 1 1 gate: 1 match (on the comment), run: exit 0, "Lesson 2 completed" -- without ever using assert_contains, assert_matches or assert_not_empty, which is the whole lesson. One shared helper counts matches outside comment lines, used by all nine gates. The guard is scoped to gates that name an assertion. The others look for `function set_up()` or `function data_provider_`, which the template declares in code as a skeleton on purpose -- those cannot distinguish template from solution and are not meant to; --fail-on-risky (#1257) is what makes an unfilled skeleton fail. An earlier version asserted the invariant over every gate and passed only because it stopped after the first file, where it would have been false for five of them. It now also asserts it compared something. Closes #1258
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Related #1258
Every lesson checks "did the learner use this API", but the template each lesson writes carries that name in its own TODO/Hint lines. All nine gates passed on their own untouched template — and a lesson could be completed with an unrelated passing assertion:
without ever using
assert_contains,assert_matchesorassert_not_empty, which is the whole lesson. Pairs with #1257: that closed the "no work at all" path, this closes the "wrong work" path.💡 Changes
bashunit::learn::count_in_codehelper counting matches outside comment lines, used by all nine gates rather than nine copiesOn the guard's scope
It covers gates naming an assertion. The others look for
function set_up()orfunction data_provider_, which the template declares in code as a skeleton on purpose — those cannot distinguish template from solution and are not meant to;--fail-on-riskyis what makes an unfilled skeleton fail.An earlier version asserted the invariant over every gate and passed only because a stray
breakstopped it after the first file, where it would have been false for five of them. The version here also asserts it compared something, so an empty set cannot fake a pass. Mutation-checked: reverting the helper to a plain grep fails both tests.