Skip to content

🐛 add a DoubleColon to the Token struct in the lexer.rs#243

Open
LesterEvSe wants to merge 1 commit intoBlockstreamResearch:masterfrom
LesterEvSe:fix/double-colon
Open

🐛 add a DoubleColon to the Token struct in the lexer.rs#243
LesterEvSe wants to merge 1 commit intoBlockstreamResearch:masterfrom
LesterEvSe:fix/double-colon

Conversation

@LesterEvSe
Copy link
Collaborator

This prevents the lexer from allowing spaces between colons (e.g., a: :b),

@LesterEvSe LesterEvSe requested a review from KyrylR March 17, 2026 12:39
@LesterEvSe LesterEvSe self-assigned this Mar 17, 2026
@LesterEvSe LesterEvSe requested a review from delta1 as a code owner March 17, 2026 12:39
@LesterEvSe LesterEvSe added the bug Something isn't working label Mar 17, 2026
src/lexer.rs Outdated
Comment on lines +24 to +25
/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`),
/// ensuring we strictly parse valid paths.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`),
/// ensuring we strictly parse valid paths.
/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`)

doc_link_with_quotes = "warn"
doc_markdown = "warn"
empty_enum = "warn"
empty_enums = "warn"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain in description why this is changed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After updating to Rust 1.94.0, cargo clippy started complaining about these lints. I just applied the automatic suggestions from Clippy to fix the warnings.

transmute_ptr_to_ptr = "warn"
trivially_copy_pass_by_ref = "warn"
unchecked_duration_subtraction = "warn"
unchecked_time_subtraction = "warn"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The solution is the same as in the previous answer.

src/lexer.rs Outdated
Comment on lines +153 to +170
just("->").to(Token::Arrow),
just("=>").to(Token::FatArrow),
just("=").to(Token::Eq),
just("::").to(Token::DoubleColon),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing, is param:::: now valid?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. It's working as usual; the error is being shown by jet::::. So I think it's behaving similarly to param::::.

Copy link
Contributor

@stringhandler stringhandler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4fe0378
Seems fine but would be useful to have some tests that show the problem. Have a look here for a simple way to create one https://github.com/BlockstreamResearch/SimplicityHL/pull/213/changes#diff-5d5fd55cd63fa75e4c8902fa6b6d63da3f8da03742454636a8d9c092f6f6eeb4R346

@LesterEvSe LesterEvSe force-pushed the fix/double-colon branch 2 times, most recently from 917a87a to 24f4e44 Compare March 17, 2026 14:12
@LesterEvSe
Copy link
Collaborator Author

4fe0378 Seems fine but would be useful to have some tests that show the problem. Have a look here for a simple way to create one https://github.com/BlockstreamResearch/SimplicityHL/pull/213/changes#diff-5d5fd55cd63fa75e4c8902fa6b6d63da3f8da03742454636a8d9c092f6f6eeb4R346

Done

Copy link
Contributor

@stringhandler stringhandler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

24f4e44

I think let's add some tests for witness:: and witness::::

src/lexer.rs Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove the double colons here now. The correct lexer should create [Token::Witness, Token::DoubleColon]

Copy link
Collaborator Author

@LesterEvSe LesterEvSe Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this OK?

    let jet = just("jet")
        .padded()
        .ignore_then(just("::"))
        .padded()
        .ignore_then(text::ident())
        .map(Token::Jet);
    let witness = just("witness")
        .padded()
        .ignore_then(just("::"))
        .padded()
        .ignore_then(text::ident())
        .map(Token::Witness);
    let param = just("param")
        .padded()
        .ignore_then(just("::"))
        .padded()
        .ignore_then(text::ident())
        .map(Token::Param);

Or do I need to change this part of the code?

let keyword = text::ident().map(|s| match s {
        "fn" => Token::Fn,
        "let" => Token::Let,
        "type" => Token::Type,
        "mod" => Token::Mod,
        "const" => Token::Const,
        "match" => Token::Match,
        "true" => Token::Bool(true),
        "false" => Token::Bool(false),
        _ => Token::Ident(s),
    });

If I add these kinds of words here, then I need to use Token::Jet("") or delete the (&'src str) payload here:

    // Jets, witnesses, and params
    Jet(&'src str),
    Witness(&'src str),
    Param(&'src str),

So, which approach should I choose?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose the option with the fewest changes (.padded(), ignore_then("::"))

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So those are valid now

jet :: add_32, witness :: PK, and witness:: PK

?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's an analogy to the Rust syntax.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants