Coming from TransformerLens¶

This page is for people who read activations with HookedTransformer.run_with_cache, name sites with hook-point strings, or patch with run_with_hooks. The examples were written against TransformerLens 4.x (the version line in landscape.md) and the TorchLens version on this branch. Checked 2026-11-03.

TorchLens labels come from eager PyTorch execution, not from TransformerLens hook-point names, so the migration is a capture plus site discovery, or a name lookup through torchlens.mechinterp.resolve_alias.

The same task in both¶

The task: read the residual stream after the first transformer block of a tiny model.

In TransformerLens¶

# migration-test: tool=transformer_lens expected=[1, 4, 16]
import torch
from transformer_lens import HookedTransformer, HookedTransformerConfig

torch.manual_seed(0)
cfg = HookedTransformerConfig(
    n_layers=1, d_model=16, n_ctx=8, d_head=8, n_heads=2, d_vocab=64, act_fn="relu"
)
model = HookedTransformer(cfg)
tokens = torch.tensor([[1, 2, 3, 4]])
_, cache = model.run_with_cache(tokens)
RESULT = list(cache["blocks.0.hook_resid_post"].shape)

In TorchLens¶

# migration-test: tool=torchlens expected=[1, 4, 16]
import torch
import transformers

import torchlens as tl

torch.manual_seed(0)
config = transformers.GPT2Config(n_layer=2, n_embd=16, n_head=2, vocab_size=64, n_positions=16)
model = transformers.GPT2LMHeadModel(config).eval()
tokens = torch.tensor([[1, 2, 3, 4]])

trace = tl.trace(model, tokens, save=tl.module("transformer.h.0"))
RESULT = list(trace["transformer.h.0"].out.shape)
trace.cleanup()

Translation table¶

Every torchlens.mechinterp and facet name above is experimental and may change in any release: the workflows are stable, the exact names may be ratified differently.

What TorchLens adds¶

• A trace of the stock model. tl.trace(model, x) records every executed tensor operation of any PyTorch model, including a stock Hugging Face checkpoint, with no model-family list and no re-implementation of the architecture.
• Replay verification: tl.validate checks the recording against the model's own output, so a mismatch between what ran and what was recorded is reported rather than silent.
• Source lines per operation, loop and pass identity (attn_1_1:2 is the second call), and containment per module.
• Hook-name resolution on the real model: torchlens.mechinterp.resolve_alias(trace, name) reports whether a name has a counterpart (real, reconstructed_read_only, needs_capture, structurally_absent, ambiguous) and gives the remedy when it does not.
• Interventions as declared, replayable specs (tl.when, trace.fork().do(...)), and .tlspec persistence of the recipe.

What TransformerLens does that TorchLens does not¶

• TransformerLens is more ergonomic when your whole workflow lives inside its supported model families and hook-point vocabulary: everything is pre-named, and its teaching corpus (ARENA notebooks, tutorials) speaks those names natively.
• TransformerLens 4.x adds vLLM and Inspect drivers, the backward lens, sparse probing and SVD circuit tools, and makes TransformerBridge its main model class. TorchLens has none of these drivers; for serving-scale capture see landscape.md.
• Where an earlier version of this page said TorchLens lacked a transformer component vocabulary and turnkey patching helpers: torchlens.mechinterp now ships the ActivationCache-style analysis kit, name-alias resolution and turnkey patch grids, so that concession is withdrawn. The remaining gap is the curated pre-named vocabulary, not the analysis verbs.

Using both¶

TransformerLens feeds TorchLens¶

There is no bridge module for TransformerLens. Two routes work. Trace a HookedTransformer directly (it captures fully and is the subject TorchLens's own oracle tests compare against), or reload the stock Hugging Face model fresh and trace that, which keeps working as HookedTransformer is retired. TorchLens refuses to trace a transformer_lens.TransformerBridge (3.x or 4.x); see Semantic differences.

TorchLens feeds TransformerLens¶

There is no bridge. trace[label].out is a plain torch.Tensor, so a tensor or a patching value captured in TorchLens can be returned from a TransformerLens hook function by hand. torchlens.mechinterp.resolve_alias goes the other way for names: it accepts TransformerLens spellings and returns the native TorchLens coordinates.

Semantic differences¶

• A TransformerBridge cannot be traced. TransformerLens 4.0 (September 2026) makes TransformerBridge its main model class and starts deprecating HookedTransformer. TorchLens refuses to capture a transformer_lens.TransformerBridge with a typed CompatibilityError naming the remedy. The bridge redirects attribute assignment to the HF modules it wraps, so TorchLens's forward instrumentation can never land on it; without the refusal, capture died partway through with an internal AttributeError. Trace a pristine model instead.
• The bridge and boot machinery (boot_transformers, bridge construction) mutates the HF model it wraps in place. A model instance that has already passed through the bridge is no longer the pristine model you loaded; reload it fresh before tracing rather than reusing it.
• TransformerLens hook names still work on the traced Hugging Face model, and an unknown name is reported rather than guessed:

# migration-test: tool=torchlens expected=('real', 'transformer.h.1.attn', 'structurally_absent')
import torch
import transformers

import torchlens as tl
import torchlens.mechinterp as mi

torch.manual_seed(0)
config = transformers.GPT2Config(n_layer=2, n_embd=32, n_head=2, vocab_size=64, n_positions=16)
config._attn_implementation = "eager"
model = transformers.GPT2LMHeadModel(config).eval()
tokens = torch.randint(0, 64, (2, 8))

trace = tl.trace(model, tokens, save="all")
RESULT = (
    mi.resolve_alias(trace, "blocks.0.attn.hook_z").status,
    mi.resolve_alias(trace, ("pattern", 1)).module_address,
    mi.resolve_alias(trace, "nonsense.hook_foo").status,
)
trace.cleanup()