couch replication auth plugin for IBM IAM with refresh by rnewson · Pull Request #6069 · apache/couchdb

rnewson · 2026-07-17T12:25:23Z

Overview

Teach the CouchDB replicator how to acquire an access token from an apikey from IBM's IAM authentication service.

Note that an access token has an expiration (currently one hour but subject to change).

The plugin fetches a new token 5 minutes before the currently held one expires.

Testing recommendations

will be covered by tests

Related Issues or Pull Requests

N/A

Checklist

This is my own work, I did not use AI, LLM's or similar technology
Code is written and works correctly
Changes are covered by tests
Any new configurable parameters are documented in rel/overlay/etc/default.ini
Documentation changes were made in the src/docs folder
Documentation changes were backported (separated PR) to affected branches

nickva

This is a good start. But I think we can simplify it by using the same pattern as in https://github.com/apache/couchdb/blob/main/src/couch_replicator/src/couch_replicator_auth_session.erl with a gen_server, also use a first type synchronous token fetch in init.

For ibrowse requests don't use the default lb pool, replicator has it's own connection pool so we'd be using that instead (see https://github.com/apache/couchdb/blob/main/src/couch_replicator/src/couch_replicator_auth_session.erl).

Also wondering how easy it would be to have this not just IBM-specific. We have JWT token auth for our server side. Maybe this can be adapted somehow to work in a more generic way so it could work with Azure, AWS, other IAM-like auth systems. But, perhaps configuring it then would be tricky and keeping the plugins separate just to be able to configure the token url, then maybe separate plugins is just easier.

For config and naming maybe we should use some indication this for a replicator auth plugin not some generic IBM integration (thinking of "ibm_iam" section).

nickva · 2026-07-17T15:07:30Z

+        ignore ->
+            ignore;
+        {ok, APIKey} ->
+            Acquirer = spawn_link(?MODULE, token_acquirer, [APIKey]),


It might be better to do a first time synchronous token fetch on init. So when the job starts we'd know right away it has an invalid creds

I had that earlier but it entailed duplication of the 'get token' ibrowse code (one synchronous, the other not). so I simplified to one version.

That's why I think we can just use a similar structure as the session plugin and always use the synchronous version.

rnewson · 2026-07-17T19:56:33Z

I do want to think of a more generic thing, and we should maybe make it possible for an auth plugin to have application like semantics (so they can start a supervision tree etc). in that version things become simpler. the auth plugin would launch a gen_server that makes a public ETS table. each replication job would insert the apikey when it starts and then fetch the token in update_headers as needed. the gen_server could refresh all the tokens slightly in advance of their expiration, handle retries on timeout or 500's, etc.

nickva · 2026-07-20T22:09:22Z

I tried adding a supervisor-level API for replicator auth plugins #6070

As discussed in the comments of the new IAM plugin in [1], it would be nice to have a per-plugin supervisor-level context for each plugin. So, for example, they can create some kind of a ETS cache table for their tokens. This is what we implement here. The API is simple: * `sup_initialize() -> Ctx` * `sup_cleanup(Ctx) -> ok` The APIs are optional to implement. The name pattern mirror the regular plugin API names: `initialize(...) -> {ok, ..., Ctx}` and `cleanup(Ctx) -> ok` [1] #6069

nickva

Looks very nice. Using gun is great and like the shared ets and using sigils.

One thing that might be possible to simplify not handling all the gun async messages in the gen_server, it's easy to miss some and get into a blocked or stuck state. I think we could still avoid blocking the gen_server with a gun_await/await_body inline if we spawn a separate process to handle the connection and just return the result/error in a single DOWN message (kind of like how we have opener processes for couch_server / indexes).

IBM refresh time is an hour so even for 100 different keys it's only ~30 second intervals and most likely users will just use a few keys only.

So maybe something like

{_, Ref} = spawn_monitor(fun() -> 
  exit(try fetch(URIMap, APIKey, Timeout) catch T:E -> {error, {T,E}} end)
end)

fetch(...)->  
   gun:post(...)
   case gun:await(...) ->
      {response, nofin, 200} ->
           {ok, Body} = gun:await_body(...)
           decode(Body);
       ...

  end.

nickva · 2026-08-14T17:50:12Z

+    {noreply, State};
+handle_info({gun_down, GunPid, _Protocol, closed, []}, #state{gun_pid = GunPid} = State) ->
+    {noreply, State};
+handle_info({gun_down, GunPid, _Protocol, Reason, KilledStreams}, #state{gun_pid = GunPid} = State) ->


Gun docs indicated there are also gun_error messages send and we don't have clauses for them

nickva · 2026-08-14T17:51:14Z

+handle_info(token_url_change, State) ->
+    case token_uri_map() of
+        {ok, TokenURIMap} ->
+            {noreply, State#state{token_uri_map = TokenURIMap}};


Wonder if we should clear connections or reset anything with the old url here

nickva · 2026-08-14T17:54:02Z

+    end,
+    {noreply, State};
+handle_info(
+    {gun_response, GunPid, GunStreamRef, nofin, StatusCode, _Headers},


Is there a gun_response with fin to handle? Got headers but no body, maybe some redirect or 500 case?

hm, yes, perhaps. will add a clause in case (an http 204, say. IAM don't say they send it, but we should handle every possibility)

nickva · 2026-08-14T17:55:24Z

+    {continue, APIKeyMAC}.
+
+cleanup(_APIKeyMAC) ->
+    ok.


If we don't do cleanup does it mean we'd end up refreshing a once used API key forever even if there are no more jobs using it?

... yes. the problem is I've no way to know when no replication job refers to a particular apikey. 'register' gets called multiple times per replication, which is why I ripped out my refcounting (which did ref-decrementing in cleanup and deleted entries when that hit 0).

perhaps I can add a 'last used' timestamp in the private table and just not renew an entry that hasn't been used for a while

nickva · 2026-08-14T17:57:25Z

+                            cancel_timer(Entry#private_entry.refresh_ref),
+                            cancel_timer(Entry#private_entry.expires_ref),
+                            RefreshRef = erlang:send_after(
+                                max(?MIN_REFRESH_MS, ExpiresInMs - ?EARLY_REFRESH_MS),


In some places dealing with periodic execution (replication jobs starts) we usually add a bit of jitter to avoid a thundering herd problem wonder if we should add a bit of that here as well.

I figure they'd be staggered already but can certainly add some jitter here

rnewson force-pushed the ibm-iam-auth branch 3 times, most recently from 49d39f0 to de37af9 Compare July 17, 2026 13:46

nickva reviewed Jul 17, 2026

View reviewed changes

rnewson force-pushed the ibm-iam-auth branch from de37af9 to 9316f4d Compare July 20, 2026 13:29

nickva mentioned this pull request Jul 20, 2026

Supervisor-level replicator auth plugin callbacks #6070

Merged

rnewson force-pushed the ibm-iam-auth branch 2 times, most recently from 5952a7a to ed0a99e Compare July 22, 2026 18:34

rnewson force-pushed the ibm-iam-auth branch 11 times, most recently from d904aae to bbda84a Compare July 28, 2026 11:24

rnewson force-pushed the ibm-iam-auth branch 2 times, most recently from e81e46a to 19183b7 Compare July 31, 2026 12:37

rnewson force-pushed the ibm-iam-auth branch 4 times, most recently from 4dab79e to 9ce7723 Compare August 10, 2026 14:34

rnewson force-pushed the ibm-iam-auth branch 7 times, most recently from a34d85b to c188e34 Compare August 13, 2026 09:32

rnewson marked this pull request as ready for review August 13, 2026 10:51

rnewson force-pushed the ibm-iam-auth branch from c188e34 to c924c28 Compare August 13, 2026 10:52

couch replication auth plugin for IBM IAM with refresh

074dd1b

rnewson force-pushed the ibm-iam-auth branch from c924c28 to 074dd1b Compare August 13, 2026 11:00

rnewson requested a review from nickva August 13, 2026 11:47

nickva reviewed Aug 14, 2026

View reviewed changes

address PR comments

64e66cc

Uh oh!

Conversation

rnewson commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Testing recommendations

Related Issues or Pull Requests

Checklist

Uh oh!

nickva left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rnewson commented Jul 17, 2026

Uh oh!

nickva commented Jul 20, 2026

Uh oh!

nickva left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rnewson commented Jul 17, 2026 •

edited

Loading

nickva left a comment •

edited

Loading