Skip to content

use system props when creating the RestTransport#539

Open
fer-marino wants to merge 11 commits intoweaviate:mainfrom
fer-marino:feature/system-props
Open

use system props when creating the RestTransport#539
fer-marino wants to merge 11 commits intoweaviate:mainfrom
fer-marino:feature/system-props

Conversation

@fer-marino
Copy link

this is a simple pull request that enable the usage of system properties when creating a RestTransport client. This is particularly useful when you need to configure the HTTP client for edge cases, eg. using an http proxy:

-Dhttp.proxyHost=10.250.0.1 -Dhttp.proxyPort=33128

Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@weaviate-git-bot
Copy link

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

@bevzzz
Copy link
Collaborator

bevzzz commented Feb 23, 2026

Nice addition, thank you @fer-marino

Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Warning SAST high 1   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca
🛡️ The following SAST misconfigurations have been detected
NAME FILE
high Prevent Server-Side Request Forgery (SSRF) via User Input in URLs ...ltGrpcTransport.java View in code

Note: The scan should have failed if no policies were configured in warn-only mode.

@fer-marino
Copy link
Author

oh sorry. I've added a lot more stuff. Using an http proxy proved much more difficult than I thought, so here is the implementation. I've also added an extra method for the OIDC authentication to use resourceOwnerPassword providing also a client secret id (needed if anon login is disabled)

@bevzzz
Copy link
Collaborator

bevzzz commented Mar 6, 2026

@fer-marino it's been a busy week, but I'm going to get back to this PR first thing Monday. Appreciate you taking time to contribute!

Using an http proxy proved much more difficult than I thought

Could you please elaborate on the difficulties you ran into with configuring http proxy via system properties? I guess one limitation of that approach would be that it doesn't allow using a different proxy per client. Was there something else you encountered?

Copy link
Collaborator

@bevzzz bevzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest focusing this PR on adding proxy support and introduce ROPC authentication in a separate PR.

@fer-marino
Copy link
Author

hi, I've implemented most of your suggestions.

About my difficulties in implementing the proxy, is the different http libraries used by the different interfaces. My initial solution worked only for the rest transport, leaving out gRPC and OIDC authentication.

@bevzzz
Copy link
Collaborator

bevzzz commented Mar 9, 2026

Thanks! Could you please rebase your branch on the latest main -- I've just merged a PR that should skip OIDC Client Credentials test if the secret is not available in the CI environment (which it isn't for outside contributions). All tests should be passing again then.

Copy link
Collaborator

@bevzzz bevzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is shaping up really well. Here are a couple more things I noticed.

Is there any simple test we could write for the HTTP / gRPC proxy configs? We're already using MockServer for some tests, can we extend the test suite to also verify request are being proxied correctly? e.g. using MockServer's proxying utility

return () -> grant; // Reuse cached authorization grant
}

static Flow resourceOwnerPassword(String clientId, String clientSecret, String username, String password) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to resourceOwnerPasswordCredentials

@bevzzz
Copy link
Collaborator

bevzzz commented Mar 9, 2026

My initial solution worked only for the rest transport, leaving out gRPC and OIDC authentication.

Makes sense, yes. Thanks a lot for taking these two aspects into consideration.

fer-marino and others added 7 commits March 9, 2026 14:47
Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
….java

Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
…usTokenProvider.java

Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
…java

Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
…usTokenProvider.java

Co-authored-by: dyma solovei <53943884+bevzzz@users.noreply.github.com>
@fer-marino
Copy link
Author

added some more test units, including one for the proxy. I've also modified the pom, as in my setup the lombok annotation processor was not getting invoked during unit tests execution.

Comment on lines +20 to +23
this(ep == null ? "NULL" : ep.method(req),
ep == null ? "NULL" : ep.requestUrl(req),
ep == null ? "NULL" : ep.body(req),
ep == null ? null : ep.queryParameters(req));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would endpoint be null? The endpoint describes how to execute the request.

}

private List<Request<?>> requests = new ArrayList<>();
private Object nextResponse = null;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like nextResponse is only ever set to null. If so, the condition on line 52 if (nextResponse != null) {...} is never going to fire. What do we need this field for?

@bevzzz
Copy link
Collaborator

bevzzz commented Mar 9, 2026

added some more test units, including one for the proxy

Thank you 👍

@fer-marino
Copy link
Author

pushed another commit with your suggestions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants