Where should I report coverage-pytest-attrs behavior?

[kpblackatverisk]

Context

I use coverage run -m pytest to calculate coverage stats on my code. Today, the coverage report failed because some temporary representation files named ./repr, ./rep-1, ./rep-2 and so on were no longer available.

The source of the temporary representation files is the attrs library version 21.3.0 published 28 December 2021. pytest depends on attrs.

Obviously, one way I could have and will avoid surprises in the future is to freeze all dependencies, including the transitive ones.

coverage report suggested adding the --ignore-errors flag, but I don't want to adopt that as a permanent solution in case it hides other errors.

Questions

Where should I log an issue?

• attrs - though perhaps the temporary representation file is intended?
• pytest - as a request to avoid use of attrs==21.3.0 specifically?
• coverage - to automatically ignore failures for seeming temporary representation files?

Or does it not merit an issue and I should find a wiki knowledge base to document this known behavior?

[bluetech]

What are "temporary representation files"?

[kpblackatverisk]

I know pytest uses attrs, likely to simplify class implementations. I'm guessing, though I have not confirmed, that this is part of rewriting assertions. Whatever the case, attrs 21.3.0 seems to save a temporary file to the current directory named repr for the first tile, then rep-1 for the next one, and so on. The files are immediately removed though, so I cannot inspect them.

The only way I know the files existed is because the coverage testing tool takes a snapshot of files as it generates the .coverage file. When I go to run coverage report the representation files referenced in the .coverage file are no longer on disk.

[bluetech]

As far as I know, attrs doesn't create any files, and I don't see which change in attrs 21.3.0 might have caused this. pytest also doesn't create any files with such names. I'm less familiar with coverage.

If you're able to reproduce this in a standalone minimal project that would help a lot.

[dahiro]

I had the same issue. It disappeared by downgrading attrs to 21.2.0
Minified the example to this :

test_allinclusive.py

import unittest
from flask import Flask

def initialize_app():
    app = Flask(__name__)
    app.config["RESTX_MASK_SWAGGER"] = False
    return app

app = initialize_app()

class empfehlungstypen_api_test(unittest.TestCase):
    def setUp(self):
        self.app = app.test_client()
    def test_empfehlungstypen(self):
        response = self.app.get(
            "/dummyurl",
            headers={"x-api-key": "pleaseChangeMe_env:API_KEY"},
        )
        self.assertEqual("404 NOT FOUND", response.status)

I then ran this test with coverage run -m unittest test_allinclusive followed by a coverage xml
which resulted in
No source for code: '/Users/.../src/rep-1>'.

Ran this with python 3.8.9 with a virtualenv and the following dependencies installed:

wheel
Flask==2.0.2
Werkzeug==2.0.2
waitress
pyodbc
Flask-SQLAlchemy
flask-restx==0.5.1
coverage==6.2.0
attrs==21.3.0 # <---- switching versions here
pylint
autopep8
black

when I switch to attrs==21.2.0 the result of the xml-command is Wrote XML report to coverage.xml

[diazona]

I'm just an interested observer, but FWIW I was not able to reproduce the failure with attrs==21.3.0.

Here's my console transcript, in case it helps you identify some difference that accounts for why you're having the problem and I'm not 🤷

$ mkdir pytest-9459-example && cd pytest-9459-example
$ cat requirements.txt
wheel
Flask==2.0.2
Werkzeug==2.0.2
waitress
pyodbc
Flask-SQLAlchemy
flask-restx==0.5.1
coverage==6.2.0
attrs==21.3.0 # <---- switching versions here
pylint
autopep8
black
$ cat test_allinclusive.py 
import unittest
from flask import Flask

def initialize_app():
    app = Flask(__name__)
    app.config["RESTX_MASK_SWAGGER"] = False
    return app

app = initialize_app()

class empfehlungstypen_api_test(unittest.TestCase):
    def setUp(self):
        self.app = app.test_client()
    def test_empfehlungstypen(self):
        response = self.app.get(
            "/dummyurl",
            headers={"x-api-key": "pleaseChangeMe_env:API_KEY"},
        )
        self.assertEqual("404 NOT FOUND", response.status)
$ python3.8 -m venv venv
$ source venv/bin/activate
(venv) $ pip install -r requirements.txt
...[omitted]...
Successfully installed Flask-2.0.2 Flask-SQLAlchemy-2.5.1 Jinja2-3.0.3 MarkupSafe-2.0.1 SQLAlchemy-1.4.29 Werkzeug-2.0.2 aniso8601-9.0.1 astroid-2.9.0 attrs-21.3.0 autopep8-1.6.0 black-21.12b0 click-8.0.3 coverage-6.2 flask-restx-0.5.1 greenlet-1.1.2 importlib-resources-5.4.0 isort-5.10.1 itsdangerous-2.0.1 jsonschema-4.3.2 lazy-object-proxy-1.7.1 mccabe-0.6.1 mypy-extensions-0.4.3 pathspec-0.9.0 platformdirs-2.4.1 pycodestyle-2.8.0 pylint-2.12.2 pyodbc-4.0.32 pyrsistent-0.18.0 pytz-2021.3 six-1.16.0 toml-0.10.2 tomli-1.2.3 typing-extensions-4.0.1 waitress-2.0.0 wheel-0.37.1 wrapt-1.13.3 zipp-3.6.0
WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/home/diazona/tmp/pytest-9459-example/venv/bin/python3.8 -m pip install --upgrade pip' command.
(venv) $ coverage run -m unittest test_allinclusive
.
----------------------------------------------------------------------
Ran 1 test in 0.006s

OK
(venv) $ coverage xml
Wrote XML report to coverage.xml
(venv) $ pip freeze
aniso8601==9.0.1
astroid==2.9.0
attrs==21.3.0
autopep8==1.6.0
black==21.12b0
click==8.0.3
coverage==6.2
Flask==2.0.2
flask-restx==0.5.1
Flask-SQLAlchemy==2.5.1
greenlet==1.1.2
importlib-resources==5.4.0
isort==5.10.1
itsdangerous==2.0.1
Jinja2==3.0.3
jsonschema==4.3.2
lazy-object-proxy==1.7.1
MarkupSafe==2.0.1
mccabe==0.6.1
mypy-extensions==0.4.3
pathspec==0.9.0
platformdirs==2.4.1
pycodestyle==2.8.0
pylint==2.12.2
pyodbc==4.0.32
pyrsistent==0.18.0
pytz==2021.3
six==1.16.0
SQLAlchemy==1.4.29
toml==0.10.2
tomli==1.2.3
typing-extensions==4.0.1
waitress==2.0.0
Werkzeug==2.0.2
wrapt==1.13.3
zipp==3.6.0

[dahiro]

hmm, strange, the only difference in the pip freeze is the following

-typing_extensions==4.0.1
+typing-extensions==4.0.1

what OS are you running on? I'm on macOS 12.1 and also had in on our pipeline (node:lts-alpine)

[diazona]

Interesting, I don't know why that difference would show up, but it shouldn't matter - pip treats hyphens and underscores in package names identically, so typing-extensions and typing_extensions are the same package.

Anyway, I'm on Linux, Gentoo if it matters. But if you're getting it to occur in a Docker image, that's an easier way to compare configurations. Maybe you can give the Dockerfile below a try; I found that it did not cause the error to occur. If you can try to narrow down the differences between that and the image used by your pipeline, you might find out what makes the difference - or maybe you'll find a Docker image you can share which does reproduce the issue.

FROM python:3.8-alpine

RUN apk add --no-cache build-base unixodbc-dev
RUN adduser -D python-user
RUN mkdir /pytest-9459-example
RUN chown python-user /pytest-9459-example
WORKDIR /pytest-9459-example
USER python-user
COPY requirements.txt requirements.txt
COPY test_allinclusive.py test_allinclusive.py
RUN python3.8 -m venv venv
RUN venv/bin/pip install --no-cache-dir -r requirements.txt
RUN venv/bin/coverage run -m unittest test_allinclusive
RUN venv/bin/coverage xml

[bluetech]

It's a bug in attrs: python-attrs/attrs#895. They have a PR up which fixes it.

[diazona]

oh lol after I went to all that effort 😛 (actually it wasn't that much)

I suppose that still doesn't explain why @dahiro was able to reproduce the problem and I wasn't, despite using the same commands and (as far as I could tell) basically the same environment, but it doesn't matter now.

[nicoddemus]

It has been released in 21.4.0: https://github.com/python-attrs/attrs/releases/tag/21.4.0

[dahiro]

this solves it, thanks everybody!

[kpblackatverisk]

Thanks!

[Icaro0310]

The right tracker is the coverage.py one: nedbat/coveragepy/issues — the failure happens inside coverage report, and pytest only appears because it is how the suite was invoked. attrs is a red herring here: those repr/rep-N files are temporary coverage data artifacts; when coverage later tries to include them in the report and they have been cleaned up, you get this error.

Two practical workarounds while you do not pin transitive deps:

• coverage report --ignore-errors (as suggested) — acceptable for CI visibility.
• Exclude them explicitly: coverage report --omit="repr*,rep-*" or add a [report] omit section in .coveragerc/pyproject.toml, so a stray artifact in the project root never breaks the report.

If you do file the issue, a minimal reproducer (the sequence of coverage invocations that leaves repr files behind) is what the maintainers will ask for first.