Skip to content
Merged
28 changes: 27 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,32 @@ $env:OPENML_TEST_SERVER_ADMIN_KEY = "admin-key"
export OPENML_TEST_SERVER_ADMIN_KEY="admin-key"
```

#### Diagnosing Slow Tests

If you suspect a test (or the suite as a whole) is running too slowly, `pytest` already exposes everything you need to investigate it. A few invocations that are useful when looking into test runtimes:

```bash
# Show the 20 slowest tests (use 0 to list every test's duration)
pytest tests --durations=20

# Fail any test that exceeds the given timeout (requires pytest-timeout)
pytest tests --timeout=600

# Investigate only fixture/setup costs without actually running the tests
pytest tests --setup-only

# Profile a specific module, class, or test
pytest tests/test_datasets/test_dataset.py --durations=0

# Skip the slow live-server tests while profiling locally
pytest tests --durations=0 -m "not production_server and not test_server"

# Run the suite in parallel to reproduce CI behaviour (requires pytest-xdist)
pytest tests -n 4 --dist=load --durations=0
```

Combining these with the marker filters (`production_server`, `test_server`, `sklearn`) makes it straightforward to narrow the investigation down to the slow tests without changing project configuration.

### Pull Request Checklist

You can go to the `openml-python` GitHub repository to create the pull request by [comparing the branch](https://github.com/openml/openml-python/compare) from your fork with the `main` branch of the `openml-python` repository. When creating a pull request, make sure to follow the comments and structured provided by the template on GitHub.
Expand Down Expand Up @@ -214,4 +240,4 @@ When dependencies are installed, run
```bash
mkdocs serve
```
This will open a preview of the website.
This will open a preview of the website.
Loading