To understand sparse auto encoder, They are basically same [[autoencoders]] but instead of making the neural network learn using all of its neurons, it penalizes the hidden layers to use only a small set of neurons. The reason so that the knowledge is confined to that part of the network. All the other neurons are basically turned off.

LLM neurons are [[polysemantic]]. It would help to disentangle the model’s neuron activation for a concept.

Inner Working

We take in a layer from LLM. The layer will have activations. Lets say

• that activation has some 900 dimensions
• we expand it by 4.
• in the new newrons, we fix most of the neurons to be zero.
• We force it to disentangle concepts to flow through the limited neuron groups.
• So that in future, we exacty know which concept is being triggerd from activation.
• It also has a step to reconstruct the expanded neurons back to original dimension.