HBASE-30211 Exclude dnsjava InetAddressResolver SPI from shaded jars

[junegunn]

Jira

https://issues.apache.org/jira/browse/HBASE-30211

Description

hbase-shaded-client ships a META-INF/services/java.net.spi.InetAddressResolverProvider naming a class that no classloader can load. On Java 18+ the JVM reads that declaration on the first name lookup, so every InetAddress call in the process throws, not just HBase calls.

java.util.ServiceConfigurationError: java.net.spi.InetAddressResolverProvider:
    Provider org.apache.hadoop.hbase.shaded.org.xbill.DNS.spi.DnsjavaInetAddressResolverProvider not found
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
	at java.base/java.net.InetAddress.loadResolver(InetAddress.java:508)

Three things combine:

1. dnsjava 3.6.1 is a multi-release jar. Its service file sits at the root, but DnsjavaInetAddressResolverProvider ships only under META-INF/versions/18/.
2. Shade relocates org.xbill. It rewrites the versioned class's bytecode to the relocated name but leaves its jar entry path alone, so the name and the path disagree. ServicesResourceTransformer then writes the merged service file naming the relocated class.
3. The shaded jar's manifest has no Multi-Release: true, so everything under META-INF/versions/** is invisible to the JVM regardless.

JEP 418 introduced the SPI in Java 18, which is why 11 and 17 are unaffected.

Affected versions

dnsjava 3.6.0 added the provider, and Hadoop 3.4.1 is the first Hadoop release to depend on dnsjava 3.6.1. Any HBase artifact built against Hadoop 3.4.1 or later carries it. Verified by inspecting the published jars on Maven Central:

artifact	state
2.5.11-hadoop3 through 2.5.15-hadoop3	affected
2.6.2-hadoop3 through 2.6.6-hadoop3	affected
3.0.0-beta-2, 3.0.0	affected
2.5.10-hadoop3 and earlier, 3.0.0-beta-1 and earlier	clean
plain Hadoop 2 builds of 2.5.x and 2.6.x, and all 2.4.x	clean, older dnsjava

hbase-shaded-testing-util is affected on the same builds. hbase-shaded-client-byo-hadoop and hbase-shaded-mapreduce bundle no dnsjava, which is why switching to byo-hadoop is the downstream workaround.

Reproduction

Any JDK 18+, no cluster needed, with an affected hbase-shaded-client on the classpath:

System.out.println(java.net.InetAddress.getByName("localhost"));

A script that inspects a given jar and runs the lookup, with captured output on Java 11 and Java 21 against both a published jar and a patched build: https://gist.github.com/junegunn/5e51baf73f32ecbfcd4a9116b7df775a

A minimal project with only dnsjava 3.6.1 and maven-shade-plugin 3.6.0 reproduces it byte for byte, so nothing here is HBase specific. Shade 3.6.2, the latest release, produces the same broken output, so a plugin bump is not an alternative.

Fix

Extend the existing dnsjava:dnsjava filter in hbase-shaded/pom.xml to drop the service file. The filter lives in the parent pluginManagement, so one edit covers every shaded artifact.

The versioned entries under META-INF/versions/18/ are deliberately left in place. Removing the declaration is what fixes the failure, verified by rebuilding with only that exclusion. Those class entries carry relocated bytecode at an unrelocated path, so they cannot be loaded under either name, and excluding them would only drop a few KB of dead weight. The provider is opt-in behind the org.dnsjava.spi.enable system property, so removing it costs nothing and all 315 relocated dnsjava classes stay.

Marking the uber jar Multi-Release: true instead does not work, since shade still leaves the versioned entry path unrelocated, and a relocated shaded client should not be installing a JVM wide DNS resolver anyway.

The commit also adds a check to ensure-jars-have-correct-contents.sh in both invariants modules, failing the build when a shaded jar declares this SPI. Static rather than runtime, because precommit and nightly build on JDK 8, 11 and 17, where the SPI is never consulted and no runtime test can see the breakage. A general check, rejecting any service provider that is not loadable from its own jar, is left as future work.

Unrelated but worth flagging: the surrounding content check in that script is a no-op on master and branch-3, from an allowed_expr assignment bug in HBASE-29226, so it prints grep: empty (sub)expression. Filing separately.

Test result

Rebuilt hbase-shaded-client and hbase-shaded-testing-util: the service file is gone, all 315 relocated dnsjava classes are retained, and InetAddress.getByName succeeds on Java 21.

mvn verify passes on both invariants modules. The new check exits 1 on an affected published jar and 0 on the rebuilt artifacts.

Context

Depended on by HBASE-29546.

[Copilot]

Pull request overview

This PR addresses a Java 18+ runtime failure caused by a shaded InetAddressResolverProvider service entry that points to a provider class that is not loadable from the shaded jar, causing InetAddress DNS lookups to throw ServiceConfigurationError. The fix is applied at shading time and reinforced via invariants checks.

Changes:

• Excludes META-INF/services/java.net.spi.InetAddressResolverProvider from shaded output for dnsjava:dnsjava.
• Adds an invariants check to fail the build if a shaded artifact declares the InetAddressResolverProvider service.
• Applies the invariants check in both shaded invariants modules (with- and without-Hadoop).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
hbase-shaded/pom.xml	Extends the dnsjava shade filter to exclude the problematic InetAddressResolverProvider service file.
hbase-shaded/hbase-shaded-with-hadoop-check-invariants/src/test/resources/ensure-jars-have-correct-contents.sh	Adds a jar-contents invariant rejecting shaded jars that declare the InetAddressResolverProvider service.
hbase-shaded/hbase-shaded-check-invariants/src/test/resources/ensure-jars-have-correct-contents.sh	Adds the same jar-contents invariant in the non-Hadoop invariants module.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[liuxiaocs7]

Thanks! @junegunn

[junegunn]

Thanks @liuxiaocs7 for taking a look!