Thought experiment - enclosing an LLM in a box

Now we observe that LLMs can self-distill. They can improve entirely from thie Suppose we put an LLM in an isolated container and had it answer 100 hard questions. Let’s run this over N trials so we can measure the distribution of correct answers.

Now if this LLM has no access to new data, no annotations, no feedback mechanism when it takes an action, we would expect the distribution of correct answers to remain fixed.

It turns out, thanks to research from Apple, that this model can actually improve. And the way it can do so is, as the Apple researchers put it, is “embarassingly effective” Under this logic, you wouldn’t expect a model to improve in you put it in an isolated container

Simple Self Distillation

Let’s come back to that set of 100 hard questions.

  • take our LLM and crank up the temperature so that get a more diverse set of answers
  • Then fine tune the LLM on those Q-A pairs that were just produced
  • Run the fine tuned model in inference mode ask the set of questions. However decode at at a lower temperature

With that simple procedure, researchers a 15.3% gain improvement on their test set of coding questions across hard questions.

But why does this work

Why does this matter

There are all sorts delightfully strange behaviors of LLMs

In undergrad, I studied physics, because I quite enjoy knowing how things work. About half way through, I picked up another obsession for trying to understand why people behave the way they do. These interests turned into one of the oddest combinations of double majors, between physics and psychology.

People are strange. It’s truly hard to understand predict human behavior, because we operate in a symphony of stimuli because our social contexts, neurobiology, our upbringings, our genetic dispositions, etc. But it’s quite fun to study and understand the odd behaviors.

I appreciate how information is encoded in a neural network. There is a reasonable order to encoding words of similiar semantic meaning in a vector space over an LLMs training, that’s reasonable.

Then there’s a set of LLM behaviors that we haven’t even discovered yet. Like that LLMs can improve on their own without new data or support in any way. LLMs will continue to occupy more of our lives. Let’s continue to experiment, tinker, and uncover all the delightful odd LLM phenomena.