Choosing (and Reselecting) a Model for Building an AI Agent

This talk will cover how to choose a model for use with an AI agent and how to subsequently monitor its quality relative to other models on the market. We will examine the criteria (cost, information security requirements, regulatory compliance, and other factors that need to be taken into account), as well as the tooling: how to check quickly (and preferably automatically) whether it is time to switch to a new solution. We will discuss tools (Langfuse and solutions built on it, as well as frameworks such as Ragas) and cover testing and retesting scenarios.