What if neurons didn’t need to be explored?

[luolearning]

Hi — I saw you're exploring neuron-level behavior and internal activations.

I’ve been experimenting with a small plug-in that makes activations directly locatable, so relevant neurons don’t need to be found through exploration or scanning.

Curious if this connects to what you're building.

https://github.com/luolearning/luoshu_kit

[multicody10]

Hi — I saw you're exploring neuron-level behavior and internal activations.

I’ve been experimenting with a small plug-in that makes activations directly locatable, so relevant neurons don’t need to be found through exploration or scanning.

Curious if this connects to what you're building.

https://github.com/luolearning/luoshu_kit

Thanks for sharing. I actually didn't learn about SAEs until after I made that whole white paper on Neuron Labeling and began programming this Neuron Explorer.

Your structured addressing layer of coordinate based direct access and intervention on residual streams is a strong complement. I imagine you could get great results injecting the bridge at TokenSensor hooks or indexing discovered SAE features by Luoshu coordinates. The GPT-2 proof of concept in V0.3 looks promising.

[luolearning]

Thanks! Those are great suggestions.

To be honest, I came into this from error localization and structured computation, not from the SAE community, so I'm still learning these tools.

The TokenSensor hook idea makes a lot of sense. And indexing SAE features by Luoshu coordinates is something I hadn't considered before.

My GPT-2 tests so far are still quite early, but these sound like exactly the kinds of directions worth exploring next.