I saw a paper presented at the Economic Science Association meeting that relates to my project on fine-tuning with Josh Foster.
Aligning AI Agents with Economic Preferences in Strategic Environments
Abstract: As large language models (LLMs) increasingly act as autonomous agents in markets and organizations, their behavior in strategic environments becomes economically consequential. We document that off-the-shelf LLM agents exhibit systematic deviations from payoff-sensitive behavior in canonical economic games, including excessive cooperation and limited responsiveness to incentives. We introduce a supervised fine-tuning approach that aligns agent behavior…
LLMs agents get us thinking about humans in a different way (e.g. consciousness and language). We find that fine-tuning alters LLM “behavior.” Prompting matters, too, since everything matters. But fine-tuning is probably more reliable as a way to ensure that a model will act the way you want it to in a wide variety of contexts.
An analogy comes to mind. Prompting is like yelling at your kid to get up so they are not late for school (usually works, but you have to do it every morning). Fine-tuning would be like teaching them to set their own alarm.
