[Bug] Import-records fix when importing from older InvenTree+Postgres version

[bbzylstra]

Fixes #10780 where exports from older versions of InvenTree+Postgres cause the import-records task to fail when importing user profiles.

User profiles are created with a 1-to-1 mapping to user accounts via a signal during account creation. When the import-records task is run, user accounts are imported first into the database, so by the time the import-records tool begins importing user profiles there will already be a set of (signal-created) user profiles in the db.

The 1-to-1 constraint will fail if the primary key of the user profile does not line up with the primary key of the user account (for example, the dump has user profile pk: 1 set to user: admin, but in the db user profile pk: 3 is set to admin, when import-records imports user profile pk:1, the 1-to-1 constraint is violated since 2 user profiles (pk:1 and pk: 3) share the admin user).

The proposed fix is to reorder the primary keys of the user profiles to match that of the user accounts (so if admin is pk: 3, the user profile corresponding to admin will have its pk set to 3). How user profiles are created must have changed in a recent version of InvenTree as these seem to enforce this pairing in the db exports.

I have another version that fixes this by turning off the automatic creation of user profiles during import-records as discussed in the bug report, but it requires turning off the other user/group validators as well. I can PR that version if reordering the user profile primary keys (non-destructively in a temp-file) is an issue.

[netlify]

✅ Deploy Preview for inventree-web-pui-preview canceled.

Name	Link
🔨 Latest commit	7f160d1
🔍 Latest deploy log	https://app.netlify.com/projects/inventree-web-pui-preview/deploys/69be68e90dd6d200096ce39c

[Copilot]

Pull Request Overview

This PR fixes a bug where importing data from older InvenTree+Postgres versions fails due to primary key misalignment between user profiles and user accounts. The fix ensures user profile primary keys match their corresponding user account primary keys during the import process.

Key changes:

• Added logic to track user primary keys during data preprocessing
• Implemented pk reordering for user profiles to align with their associated user accounts
• Updated comment to reflect the dual purpose of the user data processing section

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bbzylstra]

Comments look good, I tested them with the broken export and everything is still working as intended. If you think the primary key reordering is okay then I think it's ready (I have another version that turns off user profile creation + profile checks during data import). If you want to do further testing I can provide my export.json (with PII stripped) that was originally failing the import-records. Or you can generate one yourself by changing the order of primary keys in the user.profile sections of the inventree test data.

[SchrodingersGat]

@matmair I'd like your input on this - would you prefer the approach implemented here by @bbzylstra or to disable user profile creation entirely on data import?

[matmair]

Not sure I like this approach, introducing any more custom behavior in the importer is dangerous. I would prefer to handle it in the signal and maybe the api (UserProfileDetail) to create the profile if it is missing

[bbzylstra]

@matmair @SchrodingersGat I reverted tasks.py and updated users/models.py to disable the two validators that cause the upload to fail, and tested that it is working. I am holding off on handling what to do if user profiles are missing, perhaps the importer should verify that the db is in a valid state (1-to-1 pairing between user accounts and profiles) after the upload is finished?

Conversely, maybe the importer should verify there is a 1-to-1 pairing between user profiles and accounts before importing the data, and if not shut down with a warning to the user. An export in this state would imply there was corruption/failure during export.

[bbzylstra]

Another option is to just wrap instance.profile.save() inside of create_or_update_user_profile with an error handler for User.profile.RelatedObjectDoesNotExist like this:

@receiver(post_save, sender=User)
def create_or_update_user_profile(sender, instance, created, **kwargs):
    """Create or update user profile when user is saved."""
    # Disable profile creation if importing data from file
    if isImportingData():
        return

    if created:
        UserProfile.objects.create(user=instance)

    try:
        instance.profile.save()
    except User.profile.RelatedObjectDoesNotExist:
        UserProfile.objects.create(user=instance)

Tested that this works when a corrupted export does not include user profiles for all imported accounts. The user profile will be created on next sign-in from that user.

[SchrodingersGat]

@bbzylstra the current simplified approach sits much better with me. Is this a sufficient approach for the original issue?

[codecov]

Codecov Report

❌ Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.23%. Comparing base (3a38163) to head (7f160d1).
⚠️ Report is 1 commits behind head on master.

❌ Your project status has failed because the head coverage (91.44%) is below the target coverage (92.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10862      +/-   ##
==========================================
- Coverage   89.24%   89.23%   -0.01%     
==========================================
  Files         962      962              
  Lines       49706    49710       +4     
==========================================
+ Hits        44359    44361       +2     
- Misses       5347     5349       +2     

Flag	Coverage Δ
backend	89.23% <50.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Backend Apps	89.03% <50.00%> (-0.01%)	⬇️
Backend General	91.44% <ø> (ø)
Frontend	∅ <ø> (∅)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bbzylstra]

@bbzylstra the current simplified approach sits much better with me. Is this a sufficient approach for the original issue?

@SchrodingersGat It fixes the issue, the problem now is figuring out what to do if there are missing user profiles during import. The validation checks would normally catch these but they are turned off here. I tested an import with a missing user profile and it prevents that user from signing into their account (other accounts are unaffected). I gave a few suggestions on how to fix this, either doing some validation in the importer or error handling in the create_or_update_user_profile function.

[SchrodingersGat]

@bbzylstra thanks for confirming. What about a combination of all these ideas:

1. Add a hook in tasks.py which runs after an import, that calls the internal "update_user_profiles" function
2. The "update_user_profiles" function includes new error handling as outlined in your code suggestion above
3. The "update_user_profiles" function is prevented from running during import, to prevent conflicts

[SchrodingersGat]

@bbzylstra any thoughts on the suggested combined approach?

[SchrodingersGat]

Hey @bbzylstra it would be great to get this merged into the next release - are you able to spend any time on it?

[SchrodingersGat]

The changes provided here are worth adding on their own merit. @bbzylstra if you get a chance to implement the other ones, please do that in a separate PR.