Skip to content

Quick modes pytest-cov results are misleading #1

@gvee-uk

Description

@gvee-uk

Should the pre-commit fail, you get loads of output in the terminal which includes the output from pytest-cov/coverage.

If you run the pre-commit hook without the --quick argument, this output is misleading.

For an example, here's the output from a repository that has significant coverage, when running in the slower mode the output:

Name                                                                      Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------
...
-------------------------------------------------------------------------------------------------------
TOTAL                                                                     12277    135    99%

And then when running in --quick mode:

Name                                                                      Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------
...
-------------------------------------------------------------------------------------------------------
TOTAL                                                                     12277   5106    58%

This output is very misleading to the end user, as they probably aren't expecting such a low coverage percentage!

It looks like one-or-more of the arguments appended to pytest[-cov] when running in quick mode skips a bunch of tests, causing the number of misses to increase (which makes logical sense when looking at the args).

I wonder if the output from pytest should be supressed in all instances? What happens if you have a failing test in your suite then?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions