`log_cli_level` does not accept an integer

[stdedos]

• a detailed description of the bug or problem you are having
• [-] output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

According to the documentation https://docs.pytest.org/en/stable/reference/reference.html#confval-log_cli_level

The integer value or the names of the levels can be used.

However, setting

[tool.pytest]
log_cli = true
log_cli_level = 0

causes

$ pytest -k test_timeout
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/main.py", line 314, in wrap_session
INTERNALERROR>     config._do_configure()
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1159, in _do_configure
INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 534, in call_historic
INTERNALERROR>     res = self._hookexec(self.name, self._hookimpls.copy(), kwargs, False)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     raise exception
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 121, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/logging.py", line 638, in pytest_configure
INTERNALERROR>     config.pluginmanager.register(LoggingPlugin(config), "logging-plugin")
INTERNALERROR>                                   ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/logging.py", line 689, in __init__
INTERNALERROR>     self.log_cli_level = get_log_level_for_setting(
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/logging.py", line 616, in get_log_level_for_setting
INTERNALERROR>     log_level = config.getini(setting_name)
INTERNALERROR>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1662, in getini
INTERNALERROR>     self._inicache[canonical_name] = val = self._getini(canonical_name)
INTERNALERROR>                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1706, in _getini
INTERNALERROR>     return self._getini_toml(name, canonical_name, type, value, default)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "~/p/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1847, in _getini_toml
INTERNALERROR>     raise TypeError(
INTERNALERROR> TypeError: ~/p/pyproject.toml: config option 'log_cli_level' expects a string, got int: 0

pytest 9.0.2

[thrasymache]

I have looked into this and found that the code for parsing configurations does not actually support something being either an integer or a string. If you want to make the code consistent with the documentation at least the following steps will be needed.

1. Testing of everything below
2. _getini_ini will need to support this mixed type (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/config/__init__.py currently line 1719)
3. _getini_toml will need to support this mixed type (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/config/__init__.py currently line 1780)
4. log_cli_level will need to be configured to accept this new type. This will be a modification to pytest_addoption in logging.py (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/logging.py currently line 278)

add_option_ini("--log-cli-level", dest="log_cli_level", default=None, help="CLI logging level")

will become something like

add_option_ini("--log-cli-level", dest="log_cli_level", type="int_or_string", default=None, help="CLI logging level")

The simpler task is to fix the documentation to match the code.

[thrasymache]

If you go further down in the reference page you linked to the "Complete Help Output" section it specifies that the value for this field must be a string.

[thrasymache]

On further testing it turns out that both parts are right. You can use integer log levels as long as they are quoted strings. Your problem is that the TOML parser distinguishes between "real integers" and integers that are quoted as strings. So 0 is rejected, but "0" would likely work.

(pytest) bellerophon@pop-os:~/virtualenvs$ pytest --log-cli-level=debug
============================================ test session starts ============================================
platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/bellerophon/virtualenvs
collected 0 items

=========================================== no tests ran in 0.00s ===========================================
(pytest) bellerophon@pop-os:~/virtualenvs$ pytest --log-cli-level=nonsense
ERROR: 'NONSENSE' is not recognized as a logging level name for 'log_cli_level'. Please consider passing the logging level num instead.

(pytest) bellerophon@pop-os:~/virtualenvs$ pytest --log-cli-level=0
============================================ test session starts ============================================
platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/bellerophon/virtualenvs
collected 0 items

=========================================== no tests ran in 0.00s ===========================================

[stdedos]

If you go further down in the reference page you linked to the "Complete Help Output" section it specifies that the value for this field must be a string.

I guess you mean those ones

  log_cli (bool):       Enable log display during test run (also known as
                        "live logging")
  log_cli_level (string):
                        Default value for --log-cli-level

That's pytest -h - I guess one can do the jump.

The simpler task is to fix the documentation to match the code.

... but that is much easier. And tbh I wasn't necessarily asking that it would be supported.

Maaybe one could do so that, after reading from the toml file, the value is stringified - as the rest of the places (ini, cli) are by definition "strings only".
But I have no idea how any of this is working 😅

[thrasymache]

The simpler task is to fix the documentation to match the code.

... but that is much easier. And tbh I wasn't necessarily asking that it would be supported.

The point is that it IS supported. If you try specifying the log level as an integer on the command line it should work for you like it did for me. Is your TOML file working if you put the integer in quotes?

The implementation will need to change for it to support log levels in TOML being specified directly as integers as opposed to strings that can be cast to integers, but since there is a work-around that seems excessive, especially when the test suite doesn't currently have any coverage for integer log levels.

If you go further down in the reference page you linked to the "Complete Help Output" section it specifies that the value for this field must be a string.

I guess you mean those ones

  log_cli (bool):       Enable log display during test run (also known as
                        "live logging")
  log_cli_level (string):
                        Default value for --log-cli-level

Yes

Maaybe one could do so that, after reading from the toml file, the value is stringified - as the rest of the places (ini, cli) are by definition "strings only". But I have no idea how any of this is working 😅

Your stack trace identifies files inside your virtual environment along with line numbers. You can view those files to see how these features are implemented. Some people might even try editing those files and seeing how that changed behaviors. Of course if the files were edited the virtual environment would no longer correspond to released versions of the packages. There is an awful lot that I don't know how it is working, but the parts that I do know are from taking a look under the hood.