Layer Scope: How I used $20 of compute to make a new way to look at LLMs

August 07, 2026

Inspired by Anthropic’s newest paper on LLM interpretability, an excellent blog post series by David Noel Ng, and other research I’m currently working on, I’ve created a cool new way to read the state in the middle of LLM networks! The best part is that it’s a wonderfully simple approach:

1. Choose a transformer block and token in an LLM you want to inspect.
2. Run the last 4 layers of the LLM.

With just that I’m able to replicate almost all of Anthropic’s reading examples! I call it the layer scope. If you want to play around with some examples go here.

The rest of the article is here to explain how I arrived at the idea for this probe. I also want to show that AI research is something that’s a lot more approachable than people think. This next part is written more for SWEs who know some things about LLMs and is not intended to be a serious research paper.

A Quick Refresher on high-level LLM architecture

LLMs operate by multiplying your input by an embedding matrix to get a high-dimensional vector representing your token. This vector is known as the hidden state. The hidden state is then fed through a set of transformer blocks, also referred to as layers^[1]1. It’s annoying that the word “layer” also overlaps with MLP/feed-forward layers. Whenever I mention layers in this post, just think “transformer block”.. These layers take the hidden state and add some Δh to it. Finally, at the end you multiply by an unembedding matrix to get the final output distribution for the next token.^[2]2. Huh, you could write a book on all the stuff I just glossed over… Go check out Welch Labs if you haven’t!

An open challenge in AI/ML is figuring out how to interpret the hidden state as it flows through the model.^[3]3. citation needed It turns out that lists of 1000s of numbers make for poor reading material! So let’s go through some of the ways researchers have done this in the past.

Logit Lens

One of the earliest and simplest ways that researchers tried to probe the hidden state between layers was to throw the hidden state directly into the unembedding matrix. This approach became known as the logit lens.

While cheap, simple, and easy to understand, the logit lens falls apart the farther back in the layers you go. Probing the last few layers works alright, but the hidden state seems to be structured fundamentally differently in the earlier layers. This causes the logit lens to start outputting random garbage.

If only there were a way to move from those middle layer states to something that was intelligible to the unembedding matrix… Hey look, a paper by Anthropic about something called the J lens!

What is the J Lens?

The J Lens is short for Jacobian lens. So what’s a Jacobian?

A Jacobian is a matrix of all first-order partial derivatives of a vector-valued function.

In plainer English, it’s a list of derivatives for each value of a vector that you pass into a function. This gives you an idea of how small tweaks to the input vector will change the output of the function.

So what is Anthropic doing here?

(A) Jℓ is computed by backpropagating from the final-layer residual stream to hℓ and averaging the resulting Jacobians over token positions and over a corpus of prompts.

One way to think of this is that we are taking a derivative of the model’s output for the next few tokens with respect to each part of the hidden state vector at a specific layer. This derivative is taken for several of the next tokens and averaged together to get some idea of how things affect future tokens.

The resulting Jacobian is really useful for this specific token in this specific text, but would likely fall apart if we tried to use it elsewhere. If we calculated the Jacobian on a cookie recipe, it might be useful for a cake recipe since they are semantically similar. Try it on a sci-fi story and you might get garbage since the story lives in another part of the hidden state’s vector space not well represented by cooking.

The way the J lens tries to get around this is by calculating the Jacobian for around 1000 different prompts from a large variety of texts (cooking, poetry, math, etc.), and then averaging them together. The hope^[4]4. This is a big if. ML research has a long history of things not generalizing, and you could do a research project looking into where this breaks down! is that this results in a more general Jacobian that will tell you the general direction things are going based on the hidden state at a particular layer.

Overall this is expensive! You’re basically doing multiple backprop steps for each of these Jacobians. Specifically that ends up with roughly:

$\frac{4\text{backprop steps per Jacobian}\ast 1000\text{jacobians}\ast 32\text{layers}\ast 4\text{models}}{2}=\sim 256,000\text{backprops}$^[5]5. This is divided by two because you only backprop the layers
ahead of the layer that you are on. This averages out to half the number of full backprops.

This took about four and a half hours on the H100 I was renting (about $9 of compute). The nice thing is that once you’ve averaged the Jacobians you can keep reusing that final matrix for almost free.

Using the J lens

To get a reading you take the hidden state and multiply it by the Jacobian, normalize it, and pass the result through the unembedding matrix. An easy way to think about this is that the matrix multiplication gives you a direction, and normalization gets the vector length back to something normal for text. Finally, the result is read out by the unembedding matrix to get a distribution of tokens… something human readable!

Writing with the J lens is done using those derivatives! You can steer the model away from certain outputs by modifying the hidden state to lower their strength in the J lens.

But why Jacobians?

Once we have this Jacobian we get two key things:

• A way to make the hidden state more understandable. We can multiply the hidden state by the Jacobian to see what output tokens are highly affected by the current state.
• The derivatives tell us how small changes to the hidden state affect future output. This gives a nice way to guide future behavior of the model with those small changes.

This gives a wonderfully elegant way of both reading and writing to the hidden state!

With that, let’s move on to the background for the layer scope.

Neural Anatomy Explanation

My decision to stick with the last few layers is based on a pattern that I’ve seen pop up a lot in the literature that I’ve been reading, and the other research I’m currently doing^[6]6. hopefully I’ll be able to share that here some day!. For the purpose of this blog post I’m going to give a brief version based on David Noel Ng’s wonderful blog series (Please give those a read if you haven’t!).

A lot of research has observed that different transformer blocks seem to handle different things as a model scales up. This differentiation seems to be an emergent property of scale that shows up around the 10B parameter mark. The incredibly brief summary is that the first few transformer blocks seem to encode the hidden state of the tokens into some type of universal “thought” space. This vector space seems to make queries in different languages but with similar content land near each other in the hidden state. One interpretation might be that the first few layers are an extension of the embedding matrix used at the beginning of the model.^[7]7. As with many things in this blog post, things are more complicated than they seem here. Some papers suggest the early-mid layers also store a lot of factual knowledge.

Now that everything is in this more universal latent space, the middle layers go to town. They seem to operate on these hidden states and subtly perform computation on them. Messing around with these layers gives some interesting results. Repeating some layers might make the model better at math or reasoning. Dropping some layers here might destroy the model’s ability to handle a logic problem, while leaving the ability to do poetry.

The last few layers seem to encode the output token back into the original language of the query… Or that’s at least one theory about what’s going on.

Putting it all together

Let’s put all of these ideas together and create a hypothesis:

1. Decoding the internal state of an LLM is really hard. Current techniques range from cheap but limited, to better results with questionable generality.
2. LLMs have an overarching structure to them.

3. That structure is roughly:

   1. The early layers encode into idea space.
   2. The middle layers make a bunch of tweaks to things in that idea space, doing some computation on the hidden state.
   3. The last few layers decode from idea space back into language.

It was at this point that I asked:

Can we reuse the existing structure of the LLM to decode the middle layers?

And then:

It can’t be as simple as stapling on the last few layers of the model? Surely something will break!

With this framing it’s a fairly small leap to think that LLMs already have a tool inside of them to introspect on this hidden state. The last few layers do this for every token! So let’s create a scope that just runs these final layers on the hidden state!

This probe has a few really nice things about it:

1. It’s way less compute than calculating 1000s of Jacobians.^[8]8. To be fair, the J lens is a single reusable matrix multiplication once you’ve got it. So it’ll be cheaper in the long run.
2. It basically costs 0 VRAM! The weights for those layers are already loaded into the GPU, so we can reuse them. The only thing we need to keep track of is a single row in the KV cache!
3. I can test this without needing the resources of a frontier lab behind me! :D

The main drawback here is that this approach does not give us the ability to write things like the J lens can. That being said, we still get a lot, and there might be other ways of getting something similar (calculating a Jacobian on just 4 layers is pretty cheap!).

The layer scope!

And so I spent the weekend bringing this thing into reality, and testing it against the examples in the Anthropic paper. Let’s start out with something simple. Here’s one of the paper’s examples of seeing things in the middle layers:

In this example they detect the token “orange” in the middle layers of Sonnet. I don’t have access to Anthropic’s weights^[9]9. let me know if y’all want to share!, so I’ve done my analysis using open-source Qwen and Gemma models. I also don’t have the 1000 prompt Jacobian tuning they mention in the paper, so my J lens is not quite the same. So I decided to replicate the prompt and found “apples”, “juicy”, “citrus” and “orange” floating around with my scope!

Cool!

Observations of the layer scope

After messing around with both for a while I’ve come to some conclusions about these introspection tools:

• Both the layer scope and the J lens are able to pick up on the reading examples shown in the Anthropic paper.
• In the “think about citrus” example, the lenses only seem to show citrus on very predictable tokens. This also seems to only crop up in a small number of the tokens for both the J lens and my layer scope. For many of these tokens, the model is focused on repeating “The old painting hung crookedly on the wall.”

Overall the J lens seems to be more consistent at picking things up across the examples, but it’s not perfect. For example: the layer scope really struggles with the math example, but does better than the J lens on the color of mars example. Why, I have no idea!^[10]10. Get used to thinking/saying this a lot if you start doing this type of research.

Tuning the number of end layers to use is a bit of an art. If too few layers are used, the output starts degrading and becomes less interpretable. My guess is that you get random garbage out because it’s asking too much of too few layers. It breaks down in the same way as the logit lens. Add too many layers and it starts looking like a normal LLM again. You lose the ability to see those middle states, and the model starts to just predict the next token. In the think of citrus example, you start losing the orange traces.

Pros and Limitations

Layer scope is cool but I don’t think this is a replacement for the J lens. There are several pretty big downsides here.

• First of all, layer scope has no way of altering the future tokens of the model; it’s read-only. Since we don’t get any derivatives out, we can’t nudge the hidden state.
• The J lens looks forward at future tokens when creating its Jacobian. The layer scope at best looks backward via attention. After looking at a lot of these traces I’m not sure if this is strictly a downside, it’s just a difference.
• Just like the J lens, this method does not give good visibility into the early layers of a model.

The main upside here is that layer scope is really cheap to try out. Since we’re reusing the parameters of existing layers this is VRAM neutral. At most you’re creating an extra row in the KV cache. Each readout only runs 4 of the model’s 30–64 layers, so it costs roughly 6–13% of a full forward pass.

The bigger difference is the setup cost. Before the J lens can read anything it needs its Jacobian estimated for the model, and on my hardware that took between 14 minutes (gemma-4-26B-A4B) and 2¼ hours (Qwen3.5-27B) per model. For comparison, a complete layer scope sweep of a prompt — every layer, every token, at four different values of k — took between 6 and 55 seconds on those same models. That makes the one-time J lens setup 140–220× the cost of an entire sweep^[11]11. Estimation used 32 contexts of 64 tokens over all layers; sweeps cover k ∈ {0, 1, 2, 4}.:

To be fair to the J lens, that’s a one-time cost per model: once the Jacobian exists, applying it is cheap, so it amortizes if you’re probing many prompts on one model.

Speculation and Opinions

After doing this research I’ve accumulated a laundry list of opinions and speculations that don’t really fit in anywhere else in this article. I don’t have any formal proof or math confirming these, so it’s mostly a gut feel thing. I think they might be useful to others so I’ll dump them in this list.

1. My feelings about the J lens

While testing this out, I’ve come to find that the J lens is very inconsistent. I kinda alluded to this earlier when I mentioned that the citrus example seems to only come up on very predictable tokens. I’m not sure if this is a problem with the lens, or if this is an emergent property of the models^[12]12. I’d bet on the emergent property here. Models doing extra compute on predictable tokens would be a cool find!. It seems to make sense that since these tokens don’t have much else going on, there might be space for orange-related vectors (this is a research project in and of itself if anyone wants to look at it!).

Overall when looking through Anthropic’s data and doing my own experiments I think there is a lot of cherry picking going on in that paper (or at least their results aren’t quite as strong as the paper states on the surface). That being said, this is a cool paper with good work.

There are a lot of research opportunities here:

• I have no idea if this breaks down at super long contexts. It would be cool to check if a J lens trained with few tokens still works at token 500,000.
• I’d bet the J lens roughly works across languages based on the neuroanatomy stuff. I think the output would be semantically correct, but it would be translated into the language of the tokens used to create the Jacobian. That being said it might also pick up on whatever determines the output language if you create the Jacobian with text that spans languages.
• What’s the optimal number of Jacobians to average together?
• Is there a good way to select example tokens? I’d bet trying to span a lot of middle layer hidden state space might be a good start.

2. Could you use the J lens to gate/control Fable?

Short answer: yes. Slightly longer answer: I’m not sure if it can handle the full variety of queries.

Anthropic currently uses a two-stage classifier to scan for malicious inputs. The first classifier is a set of observed activations inside the model known to be correlated with cybersecurity, bioweapons, or any other sensitive areas. Once that triggers, the text is run past a fine-tuned LLM that does a more thorough but expensive check.

If running another model is considered expensive, then doing a single matrix multiplication would be cheap. I could see the J lens being used for the cheap check IF it generalizes well.

Controlling a model kinda lands in a similar place. Cheap enough to use in production, but only if things generalize. My first guess would be to look at how consistent those middle layer representations are, and how consistent the J space ablations are across different token streams. There’s some cool research here if I had the time to dig deeper into J lens steering across contexts.

3. AI research is more accessible than I thought!

I always thought LLM research was something that could only be done by fancy people with PhDs at big labs. It feels cool that I, a regular old SWE, can mess around with this stuff! This was the main reason I felt inspired to write up this article. I recommend anyone wondering about this stuff to indulge that curiosity!

Start reading papers, messing around with models, and talking with people you know. You can learn a lot in a short amount of time.

Thanks

I’d like to say thanks to a few people for helping me along, inspiring me to get into LLMs, and helping me write up this article.

• My PhD friend Sam. He’s entertained my ideas and answered so many of my questions! Without you I wouldn’t have gotten into all of this work.
• David Noel Ng. Your posts were very influential for me. They both informed me and showed me that normal engineers can mess around with LLM research.
• Welch Labs. Your AI book really helped get my head around machine learning and the architecture of LLMs. Keep on going with your work to make this more accessible.

@article{warfield2026layerscope,
   title   = {Layer Scope: How I used \$20 of compute to make a new way to look at LLMs},
   author  = {Warfield, Liam},
   year    = {2026},
   month   = {August},
   url     = {/layer-scope/}
}