Skip to content

Add shared postgres client package#111

Merged
reyortiz3 merged 7 commits into
mainfrom
add-postgres-package
May 19, 2026
Merged

Add shared postgres client package#111
reyortiz3 merged 7 commits into
mainfrom
add-postgres-package

Conversation

@reyortiz3
Copy link
Copy Markdown
Contributor

Summary

Introduces a postgres/ package mirroring redis/: a validated Config, a NewPool factory that wraps pgxpool, and a dynamic-authentication dispatcher with AWS RDS IAM as the first backend.

This is the consolidation point for PostgreSQL connection plumbing currently duplicated in toolhive-registry-server (internal/app/storage/database_factory.go + internal/app/storage/auth/). Additional consumers planned. Schema concerns — migrations, sqlc bindings, application-specific type codecs — stay in each caller.

Public surface

  • postgres.Config — pool tuning, app + migration user, DynamicAuth block. Implements slog.LogValuer to redact credentials.
  • postgres.NewPool(ctx, *Config, ...Option) — auto-installs a BeforeConnect dynamic-auth hook when DynamicAuth is set.
  • Options: WithAfterConnect (for custom type codecs), WithBeforeConnect (escape hatch), WithLogger.
  • postgres.NewAuthToken / postgres.NewDynamicAuthFunc — token / hook constructors for callers that need them outside NewPool (e.g. golang-migrate's one-shot connection).
  • postgres.StartPoolStatsLogger — opt-in periodic pool-stats emitter.
  • postgres.DynamicAuthAWSRDSIAMRegion: "detect" discovers region via IMDS; static regions are used as-is.

Design notes

  • Single package, not a sub-package per backend. Adding Vault/GCP-IAM later means a new *backend*.go file and a new dispatcher branch — no API churn at the call sites.
  • No mutual-exclusion of Password and DynamicAuth at the library boundary. Some callers want a local-dev static fallback. Mutual exclusion belongs in the consumer's config-validation layer (per registry-server's .claude/rules/secrets.md §4).
  • Password is a plain string. Secret-file reads, env-var overrides, and pgpass fallback all stay in the consumer's config loader; this package never opens a file.
  • pgxpool.NewWithConfig is lazy. NewPool returns without verifying connectivity; dial errors surface on first acquire, matching pgxpool's design.

Test plan

  • task lint — 0 issues
  • task test — all packages pass under -race
  • task license-check — clean
  • Coverage 82.5% (above the 70% bar in CLAUDE.md). Uncovered code is the AWS SDK call sites that require real credentials to exercise.

🤖 Generated with Claude Code

rdimitrov
rdimitrov previously approved these changes May 19, 2026
Copy link
Copy Markdown
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Batched review of the postgres/ package. Six findings — two HIGH (DSN injection / pool tuning naming), two MEDIUM (auth-hook composition / SSL default), two LOW (dead nil-check / stability marker). None blocking, but recommend addressing the HIGH and MEDIUM before tagging a release. Test coverage at 82.5% and lint clean.

Comment thread postgres/config.go
Comment thread postgres/config.go Outdated
Comment thread postgres/pool.go
Comment thread postgres/config.go
Comment thread postgres/config.go Outdated
Comment thread postgres/doc.go
reyortiz3 added a commit that referenced this pull request May 19, 2026
The previous NewPool path formatted a libpq URL with fmt.Sprintf and
passed it to pgxpool.ParseConfig. url.UserPassword only escapes the
userinfo segment, so a Host containing "@" shifted authority parsing
(url.Parse uses the last "@" as the userinfo delimiter, so
"good.rds@evil.example.com" resolved to "evil.example.com") and a
Database containing "?" introduced a second query section that could
silently override sslmode.

Switch NewPool to a hybrid approach: bootstrap pgxpool.Config from a
minimal "postgres://localhost?sslmode=X" DSN so pgx still translates the
SSL mode into a *tls.Config, then assign Host/Port/User/Password/Database
structurally onto the resulting ConnConfig. Caller-controlled fields
never enter url.Parse this way.

Add defense in depth in Config.Validate by rejecting Host containing
@/?/# or whitespace and Database containing ?/#/ or whitespace. Tighten
the Port check to require 1..65535 so the uint16 conversion in
buildPoolConfig has a real invariant behind it. The connection-string
methods stay public for callers that hand a DSN to a migration tool;
they are now safe by virtue of Validate rejecting the dangerous shapes
at the boundary.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
reyortiz3 added a commit that referenced this pull request May 19, 2026
In database/sql, MaxIdleConns is a ceiling on connections sitting idle.
In pgxpool, MinConns is a floor — the pool actively keeps that many
connections open even when the application is quiet. Mapping
MaxIdleConns to pgxpool.MinConns meant a developer setting
MaxIdleConns: 50 was opening 50 permanent connections, the opposite of
what the field name implied.

Rename the field to MinConns and add a doc comment that points out the
database/sql contrast for readers carrying that mental model. Since
this package is still Alpha (per Stability Guarantees in CLAUDE.md), the
break is cheap now and disappears later.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
reyortiz3 added a commit that referenced this pull request May 19, 2026
After the early return on AWSRDSIAM == nil, the subsequent
AWSRDSIAM != nil check is always true. staticcheck flags this as SA4031.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
reyortiz3 added a commit that referenced this pull request May 19, 2026
Declare stability via a top-of-file `// Stability: Alpha` marker on
doc.go, mirroring the convention documented in CLAUDE.md. Add a row to
the Available Packages table in README.md and the Current Packages
table in CLAUDE.md so the package is discoverable through the same
indexes consumers already use.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@reyortiz3
Copy link
Copy Markdown
Contributor Author

Pushed 4 commits addressing the HIGH and LOW findings. The two MEDIUMs are still open — counter-proposals below.

Addressed

Finding Commit Notes
[HIGH] DSN injection via Host/Database d6c19e7 Verified both attacks empirically — url.Parse on good.rds@evil:5432 resolves to evil:5432, and a ? in Database produces a second query section that pgx resolves in caller-hostile order. Went with a hybrid fix: structural override of Host/Port/User/Password/Database onto a pgxpool.Config bootstrapped from a minimal sslmode=X DSN (so pgx still translates SSL mode → *tls.Config), plus character-class validation in Validate() so the ConnectionString methods are also safe for migration-tool callers. Also tightened the Port check to 1..65535 to give the uint16 conversion a real invariant.
[HIGH] MaxIdleConnsMinConns ba45e27 Renamed plus a doc comment that calls out the database/sql contrast for readers carrying that mental model.
[LOW] Dead nil-check de3e617 Collapsed as suggested.
[LOW] Stability marker + tables 13ca610 Added // Stability: Alpha, plus rows in README.md and CLAUDE.md. Agree redis/ is missing from both — happy to handle as a separate follow-up.

Open — pushing back on both MEDIUMs

[MEDIUM] WithBeforeConnect silently replaces dynamic-auth hook. Agree the silent override is a footgun, but compose-by-default has its own footgun: both hooks write to conn.Password, and which one wins depends on order. The "want to layer metrics on top of auth" case is rare; so is "replace the auth hook entirely." I'd rather error loudly when both cfg.DynamicAuth != nil and WithBeforeConnect are set — force the caller to pick. Callers who really mean to replace the auth hook leave cfg.DynamicAuth = nil and pass their own. No silent failure, no surprising composition order. WDYT?

[MEDIUM] DefaultSSLMode = "require" doesn't verify cert. Agree the comment overstates protection. Less convinced about flipping the default to verify-full:

  • verify-full requires a working CA chain. Cloud RDS works (Amazon publishes a bundle), but self-hosted Postgres with a self-signed cert — common in dev/staging — breaks on first run.
  • registry-server today uses require. A default flip would silently change behavior on bump unless we also pin SSLMode: "require" at the extraction step.

Proposal: keep require as the library default (least surprise on extraction), drop "safe production default" from the doc, and add a paragraph in doc.go recommending verify-full for production cloud Postgres with CA bundles. Security guidance in the docs, not in a default that breaks dev environments. WDYT?

🤖 Generated with Claude Code

reyortiz3 and others added 5 commits May 19, 2026 09:14
Introduce a postgres/ package that wraps github.com/jackc/pgx/v5/pgxpool
with a Config type, a NewPool factory, and a dynamic-authentication
dispatcher. The package is the consolidation point for PostgreSQL
connection plumbing currently duplicated across consumers
(toolhive-registry-server today, with additional consumers planned).
Schema management - migrations, sqlc bindings, application-specific type
codecs - is intentionally left to each caller.

The public surface mirrors the redis/ package: a single validated Config
including pool tuning knobs and a DynamicAuth block, plus a NewPool
constructor that auto-installs a BeforeConnect hook when DynamicAuth is
configured. WithAfterConnect lets callers register custom pgx type codecs
(for example, application-defined enum array codecs) without leaking
those schema concerns into this package. NewAuthToken is exported for
short-lived connections - migrations, in particular - where a pool hook
is not available.

AWS RDS IAM is the first dynamic-auth backend. Region "detect" triggers
IMDS-based discovery. The auth dispatcher is structured so additional
backends (Vault, GCP IAM) can be added without changing the call sites
in NewPool or NewAuthToken.

Config implements slog.LogValuer to redact password fields and reports
only presence-indicators (has_password, has_migration_password,
dynamic_auth) for credentials.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous NewPool path formatted a libpq URL with fmt.Sprintf and
passed it to pgxpool.ParseConfig. url.UserPassword only escapes the
userinfo segment, so a Host containing "@" shifted authority parsing
(url.Parse uses the last "@" as the userinfo delimiter, so
"good.rds@evil.example.com" resolved to "evil.example.com") and a
Database containing "?" introduced a second query section that could
silently override sslmode.

Switch NewPool to a hybrid approach: bootstrap pgxpool.Config from a
minimal "postgres://localhost?sslmode=X" DSN so pgx still translates the
SSL mode into a *tls.Config, then assign Host/Port/User/Password/Database
structurally onto the resulting ConnConfig. Caller-controlled fields
never enter url.Parse this way.

Add defense in depth in Config.Validate by rejecting Host containing
@/?/# or whitespace and Database containing ?/#/ or whitespace. Tighten
the Port check to require 1..65535 so the uint16 conversion in
buildPoolConfig has a real invariant behind it. The connection-string
methods stay public for callers that hand a DSN to a migration tool;
they are now safe by virtue of Validate rejecting the dangerous shapes
at the boundary.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In database/sql, MaxIdleConns is a ceiling on connections sitting idle.
In pgxpool, MinConns is a floor — the pool actively keeps that many
connections open even when the application is quiet. Mapping
MaxIdleConns to pgxpool.MinConns meant a developer setting
MaxIdleConns: 50 was opening 50 permanent connections, the opposite of
what the field name implied.

Rename the field to MinConns and add a doc comment that points out the
database/sql contrast for readers carrying that mental model. Since
this package is still Alpha (per Stability Guarantees in CLAUDE.md), the
break is cheap now and disappears later.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the early return on AWSRDSIAM == nil, the subsequent
AWSRDSIAM != nil check is always true. staticcheck flags this as SA4031.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Declare stability via a top-of-file `// Stability: Alpha` marker on
doc.go, mirroring the convention documented in CLAUDE.md. Add a row to
the Available Packages table in README.md and the Current Packages
table in CLAUDE.md so the package is discoverable through the same
indexes consumers already use.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@reyortiz3 reyortiz3 force-pushed the add-postgres-package branch from 13ca610 to 677615b Compare May 19, 2026 13:14
reyortiz3 and others added 2 commits May 19, 2026 09:25
Previously NewPool silently let WithBeforeConnect win when cfg.DynamicAuth
was also set — a documented footgun whose failure mode is production
tokens expiring ~15 minutes after deploy. Compose-by-default was
considered but rejected: both hooks write to conn.Password, so
composition order is itself a surprise.

Reject the combination at construction time instead. Callers that
genuinely want to layer logic on top of dynamic auth can call
NewDynamicAuthFunc, compose the two hooks themselves, and pass the
composition via WithBeforeConnect with cfg.DynamicAuth left nil. The
ambiguity goes away and the failure mode is loud at startup rather than
silent in production.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The existing comment overstated the protection of "require" by calling
it "the safe production default" — sslmode=require encrypts the
connection but does not verify the server certificate against a CA, so
it does not defend against an active attacker who can intercept TCP.
Flipping the default to "verify-full" was considered and rejected: it
would silently break self-signed and dev environments on first run, and
registry-server today relies on the "require" default.

Drop the "safe production default" wording on DefaultSSLMode and add a
TLS and SSL Mode section to the package overview that explains the
tradeoff and points production deployments at "verify-full" with a CA
bundle. Also update the Hooks section to describe the
WithBeforeConnect/DynamicAuth mutual-exclusion that was introduced in
the previous commit.

Addresses code review feedback on #111.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@reyortiz3
Copy link
Copy Markdown
Contributor Author

Pushed both MEDIUM fixes — but didn't take the reviewer's suggested shape on either; took the alternatives from my earlier reply instead.

Finding Commit Approach
[MEDIUM] WithBeforeConnect silently replaces dynamic-auth c9327c4 Reject the combination at construction time rather than compose. NewPool returns an error when cfg.DynamicAuth != nil AND WithBeforeConnect is also set. Callers that genuinely want both compose the hooks themselves via NewDynamicAuthFunc + custom hook, then pass the composition with cfg.DynamicAuth = nil. No ambiguous default order; no silent token expiry.
[MEDIUM] DefaultSSLMode = "require" overstates protection e2caa83 Kept the default value at "require" (flipping to verify-full would silently break self-signed / dev environments on first run). Dropped the "safe production default" wording from the const comment, and added a TLS and SSL Mode section in doc.go that explains the tradeoff and points production cloud-Postgres deployments at verify-full with a CA bundle.

Happy to revisit either if the reasoning doesn't land — open to a compose-by-default WithBeforeConnect if you'd rather not error, and to bumping the default to verify-full if breaking dev environments is acceptable as part of the security stance.

🤖 Generated with Claude Code

@reyortiz3 reyortiz3 merged commit 5cdb061 into main May 19, 2026
5 checks passed
@reyortiz3 reyortiz3 deleted the add-postgres-package branch May 19, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants