Switch to patchmatch-cython from pypatchmatch due to default availability on more platforms

[aleyan]

Summary

pypatchmatch has a dependency on OpenCV, which is tricky to install and/or compile on many platforms. This replaces it for another patchmatch implementation with cython wheels prebuilt for most common platforms.

Related Issues / Discussions

Closes #8781

QA Instructions

Saw "patchmatch" option appear in canvas Compositing/Infill. Generates infill.

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[iwr-redmond]

The Dockerfiles for CUDA and ROCm will also need to be updated.

[aleyan]

The Dockerfiles for CUDA and ROCm will also need to be updated.

Removed building of patchmatch in Dockerfiles. Kept the installing of libopencv-dev, but it is not clear to me if opencv bindings are needed anymore.

[iwr-redmond]

OpenCV is still recommended for patchmatch-cython and is required for invisible-watermark.

See also: launcher#81

[aleyan]

OpenCV is still recommended for patchmatch-cython and is required for invisible-watermark.

See also: launcher#81

Ok, based on that, I will keep the opencv bindings in the Dockerfiles.

Are any other changes required?

[dunkeroni]

Seems to be working, about 3.1x faster than the pypatchmatch version, but I get significantly worse results with it. Outputs are choppy and frequently show horizontal black lines 1 pixel wide. Also, results are no longer deterministic.

[dunkeroni]

Is there a way to make patchmatch-cython work as well as the pypathmatch implementation that we have?

[iwr-redmond]

Looking at the PR code and the patchmatch-cython readme, it appears that cv2 needs to be added. Perhaps this explains the quality reduction?

[aleyan]

@dunkeroni
This is worse than I experienced locally. Is the demo image just patchmatch without any model going over it?

On determinism, patchmatch-cython can take a seed value. I will hard code that to 0, but we may consider passing in the seed from the workflow.

[aleyan]

I don't know how to test old patchmatch implementation because I could never get it to build (hence the motivation to get the cython one in).

I have made the following changes:

1. Set the seed on patchmatch to 0, so that the output is deterministic.
2. Increased the PATCH_SIZE from 3 to 5. This seems to have decreased the amount of small pixel noise.
3. Changed the alpha threshold from 255 to 128, which seemed to have gotten rid of some 0,0,0-black lines and patch noise in the patchmatch output.

@dunkeroni Can you please test it out again?

@iwr-redmond I investigated this, patchmatch-cython will use opencv automatically, if cv2 is available, which it is.

[dunkeroni]

I'm still seeing the same issues on the update:

What sort of system are you attempting to build it on? Some potentially old but maybe helpful documentation for common systems: https://invoke-ai.github.io/InvokeAI/installation/patchmatch/

[iwr-redmond]

I believe @Teriks is still on vacation but tagging for feedback on patchmatch-cython.

[aleyan]

Thank you @dunkeroni for additional testing. Looking at that image, I think there maybe at least two issues with quality. The first one is overall quality, which I want to attribute to patch size, seed, or some other variable that wasn't held constant between patchmatch implementations. The second is the 1 pixel horizontal black lines. I think that has got to be a bug in patchmatch-cython, because the surface area of the PR is too small to introduce off-by-one or scaling bug that explain it.

I am a bit out of my depth in this. Probably the best course of action is to wait until Teriks responds on the upstream issue @iwr-redmond created in patchmatch-cython.

[lstein]

I'm going to push this out to 6.14.x given the uncertainties.