Expose fetch observation metadata

[andrew]

Adds opt-in FetchObserved and FetchObservedWithHeaders methods that retain requested and final URLs, response timing and status, declared size and media type, and an allow-list of response headers.

The observed body calculates SHA-256 and SHA-512 while streaming. Byte counts and digests are only set after EOF, which leaves incomplete reads unverified. The circuit breaker wrapper supports the same API.

Closes #57

[Copilot]

Pull request overview

Adds an opt-in “observed fetch” API to the fetch package so callers can retain successful response metadata (requested/final URL, status, timing, allow-listed headers, declared size/media type) and verify the downloaded content via streaming SHA-256/SHA-512 digests that are only finalized at EOF (so partial reads remain distinguishable). This extends the same API through the circuit-breaker wrapper and documents usage.

Changes:

• Introduces FetchObservation, ObservedArtifact, and an observedBody wrapper that counts bytes and computes SHA-256/SHA-512 only on EOF.
• Adds FetchObserved / FetchObservedWithHeaders to Fetcher and wires observation through the shared fetch path.
• Extends CircuitBreakerFetcher and adds documentation/tests for the new observed fetch behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.md	Documents how to call FetchObserved and what metadata/headers are retained.
fetch/observation.go	Adds observation types, allow-listed header copier, and streaming digest/body wrapper.
fetch/observation_test.go	Adds tests for redirects, allow-listed headers, EOF-complete semantics, and incomplete reads.
fetch/fetcher.go	Implements observed fetch methods and plumbs observation through request execution.
fetch/circuit_breaker.go	Adds circuit-breaker wrappers for observed fetch methods.
fetch/circuit_breaker_test.go	Adds a success test for FetchObserved via the circuit breaker.

Suppressed comments (1)

fetch/observation_test.go:54

• ResponseTime can be legitimately 0 for very fast local httptest responses (timer resolution / scheduling), which can make this assertion flaky. Consider allowing zero and only rejecting negative durations.

	if observation.ResponseTime <= 0 {
		t.Errorf("ResponseTime = %v, want a positive duration", observation.ResponseTime)
	}

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.