fix(bridge): make native TransformerBridge state_dict()/load_state_dict() true inverses - #1598
Open
LightWork666 wants to merge 1 commit into
Open
Conversation
…ct() true inverses state_dict() emits TL-renamed keys, but load_state_dict() only matched raw native names, so a round trip silently loaded nothing and strict=True was silently downgraded to strict=False. Adds the inverse key mapping (including aliased parameters reachable via multiple attribute paths, e.g. GPT-2's split q/k/v views into c_attn) and proper missing/unexpected key accounting that raises under strict=True. Fixes TransformerLensOrg#1587
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1587.
The bug
On a native
TransformerBridge,state_dict()returns TL-renamed keys (embed.weight,blocks.0.attn.q.weight, ...), butload_state_dict()only knew how to match raw native parameter names. Sobridge.load_state_dict(bridge.state_dict())silently did nothing — no error, no warning, params just stayed whatever they were before. Worse,strict=Truewas silently downgraded tostrict=Falsewhenever the key counts didn't line up, so there was no way to even notice the round trip had failed.The fix
load_state_dictnow builds the inverse of the TL-key renaming thatstate_dict()applies (_tl_key_to_actual_keys), so it can map TL-format keys back to the underlying native parameter paths before handing them to the wrapped model's ownload_state_dict. One wrinkle: some bridge components expose the same underlying parameter through more than one attribute path — e.g. GPT-2'sq/k/vare views into the wrapped module's combinedc_attnweight, reachable both through a block-level shortcut and through the nested_original_componentchain. Writing to only one of those paths leaves the model's actual forward pass untouched even thoughstate_dict()looks fine, so the mapping keeps every alias for a given TL key, not just the first one found.The silent
strict=True → Falsedowngrade is gone. Missing/unexpected keys are now computed properly (scoped to what the TL state dict actually needs) and raise a realRuntimeErrorunderstrict=True, matching howHookedTransformeralready behaves.The raw-key loading path used by
tracr(make_tracr_transformer_bridge_state_dict) still works — raw native keys are matched directly before falling through to the TL-key path.Testing
tests/unit/model_bridge/test_state_dict_round_trip.py: round-trip actually overwrites params (not a no-op),strict=Trueraises on both missing and unexpected keys,strict=Falsedoesn't raise on a partial dict, the tracr raw-key path still loads, and a real GPT-2 case checking forward-pass logits match after a zero-and-reload cycle (this last one is what caught the aliasing issue above — a naive key-rename fix passes the round-trip-key-equality check but still produces different logits, because it never touches the aliased storage).uv run mypy .— clean.tests/unit/model_bridge/suite (207 files) in isolated batches; everything passes except a handful of pre-existinggenerate()/KV-cache crashes already tracked as an upstream PyTorch/HF bug on Apple Silicon intests/QUARANTINES.md— confirmed viagit stashthat those reproduce identically on unmodifieddev-4.x, unrelated to this change.Note on #1595
I noticed after finishing this that #1595 is already open for the same issue, taking a related but different approach (it detects aliasing by comparing live tensor identity rather than by TL-key-name collisions, which is arguably a more principled check). I'm submitting this anyway since it was already done and transparency seemed better than not mentioning it. Happy to have the maintainers pick whichever they prefer, or close this if #1595 is the better fix.