Rust SQL parser benchmarks, second edition

[LucaCappelletti94]

Hi all,

I have just completed the second edition of the SQL Parsers, this time around including several different dialects and not only PostgreSQL like last time, and I have included the errors that the various parser make, plus the actual distributions of the time & memory requirements per statement. Hope it is helpful.

Website: https://sql-ast-benchmark.luca.phd/
Repo: https://github.com/LucaCappelletti94/sql_ast_benchmark

Best,
Luca

[LucaCappelletti94]

I have now also added historical benchmarks, going back many versions. I think it is rather interesting from a software archeology perspective.

[LucaCappelletti94]

I have now actually added differential checks between versions, going all the way back to version 0.6.1, so that any regressions on statements support are clearly listed with examples. In version 0.62 for instance there have been several regressions. I do not know how to best share these regressions other than just go look the latest up to date list I have prepared here: https://sql-ast-benchmark.luca.phd/parser/sqlparser_rs

It would not be reasonable to open issues for each one of them. I hope it is useful, and that the change of the software over time is informative.

[c2main]

Thanks for the study @LucaCappelletti94

In case they went under the radar, here are 2 (old and maintained) projects which have proven being useful at finding bugs, here and there:

• https://github.com/anse1/sqlsmith - generate (lot of possibly complex) queries from schema
• https://github.com/credativ/sqlreduce - reduce complex queries to minimum case for bugged expressions.

[LucaCappelletti94]

Thank you for the pointer @c2main, those look very interesting especially to feed fuzzers. For my benchmark, I believe that both the amount of bugs I am reporting / the size of the dataset is sufficient without also adding on top SQL generation, but open to suggestions if you believe that it would be a useful addition.

[c2main]

Thank you for the pointer @c2main, those look very interesting especially to feed fuzzers. For my benchmark, I believe that both the amount of bugs I am reporting / the size of the dataset is sufficient without also adding on top SQL generation, but open to suggestions if you believe that it would be a useful addition.

I agree with you, maybe food for later.