Fine-Tuning LLMs with Ruby

Technologies for building LLM applications, such as RAG and MCP, are gradually gaining traction in the Ruby ecosystem. But what about fine-tuning, which adapts the LLM itself to specific requirements?

While RAG provides a model with relevant external context at inference time, fine-tuning adapts the model's parameters to shape its behavior for a particular task. Fine-tuning is typically done in Python, with its mature machine learning ecosystem, but it can also be used from Ruby.

This talk introduces the fundamentals of fine-tuning and shows how to work with it using Ruby, focusing on LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique. While the underlying machine learning backend is implemented in Python, Ruby code through PyCall provides a practical way to understand how fine-tuning works. The talk also explores how fine-tuning is used with local LLMs and shares the current state of the Ruby ecosystem around AI.