English Español Français Deutsch
← Back to the list
Foundational figures Foundational figures Directly upstream of ai100's methodology

Omer Levy

Whether a language model can infer the task from examples alone — without being told what to do — and what that ability reveals about how it represents instructions.

Levy's "Instruction Induction" paper (2022, senior author) reversed the standard instruction-following experiment: instead of giving the model an instruction and observing whether it complies, give it a few input-output examples and ask the model to articulate what instruction those examples imply. The findings were both encouraging and disquieting — modern LLMs can induce reasonable instructions from a handful of examples on many tasks, which suggests their internal representation of "task" is genuinely abstract, but they sometimes induce instructions that subtly differ from the human-intended one in ways that affect downstream behavior. For ai100, this matters because most engine queries don't include an explicit instruction — the engine is implicitly inferring the task from context, and the inference can go differently than expected.

Worth following when
you want to understand the implicit-instruction-inference step that happens before an LLM produces an answer, especially when the user query is underspecified.
Topics
instruction induction from examples; what implicit-instruction-inference reveals about model representations; the LIMA / Less Is More for Alignment line of work.
Key works
Instruction Induction (2022, senior author); LIMA: Less Is More for Alignment (2023, co-author); foundational work on neural word embeddings (with Goldberg, 2014).