Conversation
|
One thing of note: This would be backwards incompatible for anyone depending on the behavior of resolving first-party This change also makes the behavior consistent with configuring a settings path. |
| potential_config_file.path, CONFIG_SECTIONS[potential_config_file.name] | ||
| ) | ||
| if "directory" not in config_data: | ||
| config_data["directory"] = os.path.dirname(potential_config_file.path) |
There was a problem hiding this comment.
This is the main logic to fix the src_paths resolution.
|
I just ran into this problem. I have a mono repo with multiple python projects in sub folders that each have an .isort.cfg. When running isort from the root on a file in directory one, everything works as expected, same on a file in directory two. But when I run on a file from directory one and two at the same time (when using pre-commit), only one of the files is formatted correctly. I thought |
There was a problem hiding this comment.
Pull request overview
This PR fixes --resolve-all-configs behavior so src_paths resolves relative to the directory containing the config file selected for a given file (instead of the process CWD), addressing misclassification of first-party vs third-party imports (Fixes #2045). It also refactors config discovery to improve performance when scanning large trees.
Changes:
- Update
find_all_configsto scan viaos.scandir(with default-skip pruning) and populateconfig_data["directory"]from the config file’s directory when unset. - Extend unit tests for
find_all_configsto validate expectedsrc_pathsresolution and.venvskipping. - Update documentation to clarify
src_pathsresolution semantics under--resolve-all-configs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
isort/settings.py |
Refactors config discovery and injects directory into config data for correct src_paths resolution under --resolve-all-configs. |
tests/unit/test_settings.py |
Updates find_all_configs tests to validate src_paths resolution and skip-directory behavior. |
docs/configuration/options.md |
Clarifies --resolve-all-configs behavior for src_paths resolution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
This needs a rebase unfortunately. Since I can't use Copilot on another repository I can't use it to fix the PR. Unfortunately I'll close it for now. Sorry! We should have gotten to this earlier, but maintenance was a bit low in previous years. |
|
@DanielNoord I can go ahead and rebase this if you re-open. Or alternatively, I can just open a new PR. |
When using `--resolve-all-configs`, there is unexpected behavior in that `src_paths` ends up resolving relative to the project root, which defaults to the current working directory. This results in first-party modules being marked as third-party modules in the default case. Under the previous implementation, one possible workaround would be to specify the relative path to config directory (e.g. `relative/path/to/configdir/src`). However, assuming that the most common use of `--resolve-all-configs` is to support multiple sub-projects in the same repository/overall directory, this workaround would now require each sub-project to understand where it lives in the filesystem. This change proposes a fix that sets `directory` on the `config_data` to be the directory containing the used configuration file if not already set. Downstream, this directory is then used to resolve the absolute paths specified by `src_paths`. Fixes PyCQA#2045
Avoid recursing into default skip files like .venv. Also avoid doing an iteration per config file type. Lastly, use scandir to improve file system walk performance. The previous implementation would walk the entire tree, and iterate over each config source type to test if the file exists. Instead, walk over existing files and do a fast filter to remove candidates.
- Preserve CONFIG_SOURCES precedence when a directory contains multiple config files. os.scandir returns entries in an arbitrary order, so the scandir-based rewrite could pick a lower-precedence config (e.g. setup.cfg over .isort.cfg); restore the deterministic behavior of the previous os.walk implementation. - Do not recurse into symlinked directories in _scanwalk_files, matching os.walk's followlinks=False default, to avoid infinite loops on cyclic symlinks. - Skip directories/entries that raise OSError (permissions, broken links) instead of crashing. - Add regression tests for config-source precedence, symlink-loop safety, and ignoring config files without a valid isort section.
|
@DanielNoord rebased and also addressed Copilot's feedback + fixed a edge case where config priority wasn't being respected. PerformanceBenchmarked the current implementation vs the previous
Most of the benefit comes from pruning |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2142 +/- ##
==========================================
- Coverage 99.39% 99.24% -0.16%
==========================================
Files 41 41
Lines 3156 3182 +26
Branches 682 688 +6
==========================================
+ Hits 3137 3158 +21
- Misses 11 15 +4
- Partials 8 9 +1 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (2)
isort/settings.py:792
- This hard-coded pruning drops configs for explicitly passed files. Explicit CLI paths are still processed unless
--filter-filesis set (isort/main.py:303-308), soisort .venv/foo.py --resolve-all-configsnow sortsfoo.pywith the default/ancestor config instead of.venv/pyproject.toml; the previousos.walkimplementation found that config. Pruning needs to be conditional on the actual file-filtering behavior/targets rather than unconditionalDEFAULT_SKIP.
for potential_config_file in _scanwalk_files(
path, exclude_fn=lambda entry: entry.name in DEFAULT_SKIP
):
isort/settings.py:804
- Because
os.scandirorder is arbitrary, this can parse a lower-priority config before the higher-priority file is encountered. If that lower-priority file is malformed, isort emits a warning (and pays the parse cost) even though a valid.isort.cfgshould make it irrelevant; the previousCONFIG_SOURCESloop never opened lower-priority files after finding a valid higher-priority config. Group candidates per directory and try them inCONFIG_SOURCESorder, stopping at the first valid config.
try:
config_data = _get_config_data(
potential_config_file.path, CONFIG_SECTIONS[potential_config_file.name]
)
|
@sudowork I think the review from Copilot correctly hints to the fact that the speed optimization might better fit a separate PR as there is a lot of "gotcha's". Can we split this into a PR that does the fix and a PR that does the speed improvement? |
When using
--resolve-all-configs, there is unexpected behavior in thatsrc_pathsends up resolving relative to the project root, which defaults to the current working directory. This results in first-party modules being marked as third-party modules in the default case.Under the previous implementation, one possible workaround would be to specify the relative path to config directory (e.g.
relative/path/to/configdir/src). However, assuming that the most common use of--resolve-all-configsis to support multiple sub-projects in the same repository/overall directory, this workaround would now require each sub-project to understand where it lives in the filesystem.This change proposes a fix that sets
directoryon theconfig_datato be the directory containing the used configuration file if not already set. Downstream, this directory is then used to resolve the absolute paths specified bysrc_paths.This change also introduces performance improvements to
find_all_configsby pruning as we walk the filesystem and other smaller performance enhancements.Fixes #2045