Skip to content

feat: deploy through a one-hop SSH jump host - #106

Merged
vishr merged 1 commit into
mainfrom
feat/ssh-jump-host
Aug 23, 2026
Merged

feat: deploy through a one-hop SSH jump host#106
vishr merged 1 commit into
mainfrom
feat/ssh-jump-host

Conversation

@vishr

@vishr vishr commented Aug 23, 2026

Copy link
Copy Markdown
Member

Closes #104.

A deployment target reachable only through a bastion had no route: ob dials and verifies on its own rather than shelling out to ssh, so a ProxyJump in ~/.ssh/config never applied to it.

An environment may now name the jump host it is reached through, beside server and in the same two forms:

environments:
  production:
    server: root@10.20.0.10
    jump: deploy@bastion.example.com

Nothing is deployed to the jump host — it forwards one TCP connection and runs no commands. Onebox stays one application on one host.

Security

  • Both hops are verified against known_hosts and authenticated separately. A trusted bastion never vouches for the target, and the jump's key is rejected before the forward request names the private address.
  • The SSH agent is never forwarded. It may sign for either hop; its socket is never exposed to the bastion.
  • Exactly one hop, structurally: a route holds an address, not another route. No ProxyCommand, no ~/.ssh/config parsing.

Failure staging

Errors name the hop and the stage, so the six failure modes are distinguishable:

jump ssh deploy@bastion:2222: dial tcp: lookup bastion: no such host
jump ssh deploy@bastion:2222: host key: ...
jump ssh deploy@bastion:2222: authenticate: ...
target ssh root@10.20.0.10:22: not reachable from the jump host: ...
target ssh root@10.20.0.10:22: host key: ...
target ssh root@10.20.0.10:22: authenticate: ...

A transport failure names no stage at all — a timeout is not a rejected key.

Three details that are load-bearing

All consequences of the second hop travelling inside the first:

  • An SSH channel rejects SetDeadline outright, so the second hop's handshake is bounded by a timer. Releasing the channel is itself only a message sent through the bastion, so when the bastion is what stopped answering, the timer must also drop the bastion's own TCP connection — otherwise the handshake waits forever, and an interactive context carries no deadline to rescue it.
  • Shutdown has the same shape: every graceful close writes through the bastion, so closing is bounded and falls back to dropping that connection. Without it, Ctrl-C during a transfer through a wedged bastion never returns.
  • The jump host's raw connection is therefore kept, not just its client.

Plan binding

Plans and approvals bind the whole route (root@10.20.0.10 via deploy@bastion.example.com:2222) rather than the target alone, so swapping the bastion invalidates a confirmation that never mentioned it. No new field and no approval-schema bump: one function is the single authority for that string, consumed by plan time, execution binding, and both re-check sites.

A route with no jump renders exactly what Destination() rendered before, so direct deployments are byte-identical in configuration, behaviour, error text, and sealed artifacts.

Local hooks

local: true hooks run on the operator's machine, which has no tunnel, so they receive OB_SSH_JUMP alongside OB_SERVER — empty when direct:

ssh ${OB_SSH_JUMP:+-J "$OB_SSH_JUMP"} "$OB_SERVER" -p "$OB_SSH_PORT" 'uptime'

Two adjacent fixes this uncovered

  • Regression fix: the scalar server: user@host:port left the port inside the hostname. It round-tripped through Destination(), so it looked correct everywhere except at the moment something dialled it — the old code happened to survive it by reparsing that string. Both scalars now expand through the shared address grammar.
  • Jump validation checks the authored components rather than a recomposed string, which had made an IPv6 bastion impossible to express in any spelling while letting a host smuggle in a port or user that only failed at dial.

Behaviour change worth flagging

A bracketed IPv6 scalar with no port (server: "root@[2001:db8::1]") now renders unbracketed in the binding string, matching what the object form has always produced. Executable plans expire in 15 minutes, so the window in which a sealed artifact could disagree is small.

Testing

New in-process SSH server harness (the repo had none) covering: connection through a jump, non-default ports, IPv6 on both hops, unknown jump key failing before the target is named, jump auth failure, unreachable target reported as reachability not auth, unknown target key through a trusted jump, target auth failure, close accounting on success and every partial-failure path, cancellation before and during the handshake, and a wedging TCP proxy for the two hang cases above.

Every new test was mutation-checked: breaking the code it guards makes it fail.

  • just check — pass
  • just ci — pass
  • go test -race ./... — pass
  • go test -count=3 ./internal/transport/ — no flakes
  • OB_E2E=1 go test ./e2e/ with Docker — pass

Docs

New guide guides/deploy-through-a-jump-host, plus known_hosts enrollment for both hops, the failure-stage table, IPv6 forms, and the local-hook caveat. Schema and field reference regenerated; sidebar, shorthand table, first-deploy prerequisites, and the README host-access row updated.

🤖 Generated with Claude Code

A deployment target reachable only through a bastion previously had no
route: ob dials and verifies on its own rather than shelling out to ssh,
so a ProxyJump in ~/.ssh/config never applied to it.

An environment may now name the jump host it is reached through, beside
the server and in the same two forms:

    environments:
      production:
        server: root@10.20.0.10
        jump: deploy@bastion.example.com

Both hops are verified against known_hosts and authenticated separately;
a trusted bastion never vouches for the target. The agent may sign for
either hop but its socket is never forwarded. Exactly one hop is
possible by construction — a route holds an address, not another route.

Failures name the hop and the stage they happened in, so a bastion that
cannot be reached, one that refuses the key, and a target the bastion
cannot forward to are three different messages rather than one. A
transport failure names no stage at all: a timeout is not a rejected
key, and reporting it as one sends the operator to check credentials
that are fine.

Three details are load-bearing, all of them consequences of the second
hop travelling inside the first:

  - An SSH channel rejects SetDeadline outright, so the second hop's
    handshake is bounded by a timer instead. Releasing the channel is
    itself only a message sent through the bastion, so when the bastion
    is what stopped answering the timer must also drop the bastion's own
    TCP connection — otherwise the handshake waits forever, and an
    interactive context carries no deadline to rescue it.
  - Shutdown has the same shape: every graceful close writes through the
    bastion, so closing is bounded and falls back to dropping that
    connection. Without it, Ctrl-C during a transfer through a wedged
    bastion never returns.
  - The jump host's connection is therefore kept, not just its client.

The timer and the handshake settle under one lock rather than by
signalling after the fact, so a handshake that completes as the timer
fires cannot return a client whose connection is already being torn
down.

Plans and approvals bind the whole route rather than the target alone,
so swapping the bastion invalidates a confirmation that never mentioned
it. A route with no jump renders exactly what Destination() rendered
before, leaving direct deployments byte-identical in configuration,
behaviour, error text, and sealed artifacts.

Local hooks run on the operator's machine, which has no tunnel, so they
receive OB_SSH_JUMP alongside OB_SERVER — empty when direct.

Routing an address structurally rather than as a rendered string exposed
that nothing had been holding these fields to the dialling grammar,
because every connection used to re-parse that string on its way out:

  - The scalar `server: user@host:port` left the port inside the
    hostname. It round-tripped through Destination(), so it looked
    correct everywhere except at the moment something dialled it. Both
    scalars now expand through the shared grammar.
  - A bracketed IPv6 `server.host` survived for the same reason and
    would now be dialled as [[2001:db8::1]]:22. Brackets belong to the
    scalar spelling, so they are stripped from the object form of both
    fields.
  - Server and jump addresses are validated as authored, rather than
    recomposed into a string and parsed — which could not tell a bad
    host from a bad user, made an IPv6 bastion impossible to express,
    and let a host smuggle in a port that only failed at dial.

One shape changes as a result: a bracketed IPv6 scalar with no port now
renders unbracketed, matching what the object form has always produced.

Closes #104

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vishr
vishr force-pushed the feat/ssh-jump-host branch from 5b45d3f to 3f6b2b5 Compare August 23, 2026 15:19
@vishr

vishr commented Aug 23, 2026

Copy link
Copy Markdown
Member Author

Pushed fixes for three review findings (force-push, same single commit — now 3f6b2b5).

1. server.host was no longer validated or normalised anywhere (medium). Every connection used to re-parse the rendered destination on its way out, which was quietly doing this work. Routing structurally removed that, so a bracketed IPv6 server: {host: "[2001:db8::1]"} would have been dialled as [[2001:db8::1]]:22, and {host: "example.com:2222"} reached DNS instead of being refused.

Brackets are now stripped from the object form of both server and jump (they belong to the scalar spelling, where they mark the address off from the port), and both addresses are validated as authored via ValidHost/ValidUser. Confirmed against the built binary:

$ ob preflight   # server: {host: "[2001:db8::1]", user: root}
✗ ob: cannot reach root@2001:db8::1: ssh root@2001:db8::1:22: dial tcp [2001:db8::1]:22: i/o timeout

$ ob preflight   # server: {host: "example.com:2222"}
✗ ob: server host "example.com:2222" must be a DNS name, an IPv4 address, or an unbracketed IPv6 address; write the port as `port` and the user as `user`
  at: environments.production.server.host

A knock-on: {host: "[2001:db8::1]"} for jump is now normalised rather than refused, so that case moved from the rejection table to the accepted-forms test. The loader could not take the net import this needed — TestGenerationCannotReachATarget forbids it, correctly — so the check goes through the grammar package.

2. The abandoned guard did not close the race it documented (low). Correct: the watcher signalled after closing, so a handshake completing just before the timer fired could take the default branch and return a client whose connection was being torn down — and for the target hop hardClose had already dropped jumpTCP, killing the whole route. The watcher and the handshake now settle under one lock: whichever gets there first wins, and the loser does nothing.

3. A context deadline returned bare ctx.Err() (low). It dropped the hop prefix the new guide tells operators to read, in exactly the case a wedged bastion makes likeliest. Now wrapped with the stage; errors.Is(err, context.DeadlineExceeded) still holds. New test asserts both.

Each fix was mutation-checked — reverting it makes its test fail.

Re-verified: just check 0, go test -race ./... 0, go test -race ./internal/transport/ -count=2 clean, OB_E2E=1 go test ./e2e/ pass (285s).

The one behaviour change already flagged in the description stands: a bracketed IPv6 scalar with no port renders unbracketed in the binding string, matching the object form.

@vishr
vishr merged commit f7503ad into main Aug 23, 2026
5 checks passed
@vishr
vishr deleted the feat/ssh-jump-host branch August 23, 2026 18:37
vishr added a commit that referenced this pull request Aug 24, 2026
A deployment target reachable only through a bastion previously had no
route: ob dials and verifies on its own rather than shelling out to ssh,
so a ProxyJump in ~/.ssh/config never applied to it.

An environment may now name the jump host it is reached through, beside
the server and in the same two forms:

    environments:
      production:
        server: root@10.20.0.10
        jump: deploy@bastion.example.com

Both hops are verified against known_hosts and authenticated separately;
a trusted bastion never vouches for the target. The agent may sign for
either hop but its socket is never forwarded. Exactly one hop is
possible by construction — a route holds an address, not another route.

Failures name the hop and the stage they happened in, so a bastion that
cannot be reached, one that refuses the key, and a target the bastion
cannot forward to are three different messages rather than one. A
transport failure names no stage at all: a timeout is not a rejected
key, and reporting it as one sends the operator to check credentials
that are fine.

Three details are load-bearing, all of them consequences of the second
hop travelling inside the first:

  - An SSH channel rejects SetDeadline outright, so the second hop's
    handshake is bounded by a timer instead. Releasing the channel is
    itself only a message sent through the bastion, so when the bastion
    is what stopped answering the timer must also drop the bastion's own
    TCP connection — otherwise the handshake waits forever, and an
    interactive context carries no deadline to rescue it.
  - Shutdown has the same shape: every graceful close writes through the
    bastion, so closing is bounded and falls back to dropping that
    connection. Without it, Ctrl-C during a transfer through a wedged
    bastion never returns.
  - The jump host's connection is therefore kept, not just its client.

The timer and the handshake settle under one lock rather than by
signalling after the fact, so a handshake that completes as the timer
fires cannot return a client whose connection is already being torn
down.

Plans and approvals bind the whole route rather than the target alone,
so swapping the bastion invalidates a confirmation that never mentioned
it. A route with no jump renders exactly what Destination() rendered
before, leaving direct deployments byte-identical in configuration,
behaviour, error text, and sealed artifacts.

Local hooks run on the operator's machine, which has no tunnel, so they
receive OB_SSH_JUMP alongside OB_SERVER — empty when direct.

Routing an address structurally rather than as a rendered string exposed
that nothing had been holding these fields to the dialling grammar,
because every connection used to re-parse that string on its way out:

  - The scalar `server: user@host:port` left the port inside the
    hostname. It round-tripped through Destination(), so it looked
    correct everywhere except at the moment something dialled it. Both
    scalars now expand through the shared grammar.
  - A bracketed IPv6 `server.host` survived for the same reason and
    would now be dialled as [[2001:db8::1]]:22. Brackets belong to the
    scalar spelling, so they are stripped from the object form of both
    fields.
  - Server and jump addresses are validated as authored, rather than
    recomposed into a string and parsed — which could not tell a bad
    host from a bad user, made an IPv6 bastion impossible to express,
    and let a host smuggle in a port that only failed at dial.

One shape changes as a result: a bracketed IPv6 scalar with no port now
renders unbracketed, matching what the object form has always produced.

Closes #104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support one-hop SSH jump hosts for private deployment targets

1 participant