[Question] Pythia models do not obtain act. name ```blocks.0.hook_resid_mid``` in cache

[NekoRin-Stellar]

Question

As described in title, I tested Pythia models (2.8b and 6.9b) with test demo function test_cache_hook_names() and they all resulted in assertion failure. I did not find cache name blocks.0.hook_resid_mid. I'm wondering if it is a bug or there is some alternative cache name for this.

[kapedalex]

You can use code below to check available hooks

 from transformer_lens import HookedTransformer

model = HookedTransformer.from_pretrained("eleutherai/pythia-70m")
print(model.hook_dict.keys())

Pythia models with parallel attention do not have mid layers, they simply do not exist.