Galtea

We recently partnered with Sabadell Seguros to help them scale a GenAI assistant to 6,000 sales agents. A bit of context: SofIA is their generative AI assistant for Home, Life, and Income insurance. Before scaling to 6,000 agents, they wanted hard proof it was ready. These are the final numbers: 8% uplift in accuracy. 20% fewer refusals. Not everybody insists on this level of rigor before scaling. The team led by Joan Franco did, and it shows. You can read the full story here → https://lnkd.in/efQ7WUnR

On 2 August, three transparency obligations under the EU AI Act became applicable across the EU market: → If your AI interacts with a person, it has to disclose that it is AI. → If it generates synthetic images, audio, or video, that content has to be labeled. → If it runs emotion recognition or biometric categorization, the people exposed to it have to be informed. For most teams, disclosure was a design decision made once, at launch. Under the AI Act, it is now a behavior you are accountable for, on every release and in front of every user. And you have to verify it the same way you verify anything else your system does. Three questions worth putting to your AI team: → Does our assistant reliably disclose that it is AI across the real range of conversations users actually have, not just the scripted happy path? → Does every piece of synthetic content we generate carry the label, across every format it ships in: image, audio, and video? → When we run emotion recognition or biometric flows, does the required notice actually reach the person in the interaction? And the one that matters most under the new regime: → Can we produce the evidence to prove all of this, on demand, if a regulator or an auditor asks? None of this shows up in a standard accuracy or latency eval. Disclosure has to be tested against realistic scenarios, re-run on every new model version and every prompt change, and captured as evidence, not checked once at launch and assumed stable. Manual spot-checks do not scale to that. And "we designed it to disclose" is not the same as "we can show that it still does." P.S. I'm running a free workshop on building evals for behaviors like this, which an off-the-shelf metric can't measure. If your team is building AI in a regulated industry, join me: https://lnkd.in/et-RWCjm

We're giving all of our customers an AI engineer: Meet Val, Galtea's new AI agent, who works alongside your team like an extra AI engineer. Val knows your product, understands your evals, and helps you deploy reliable AI in production. What Val can do: → Turn production traces into actionable insights without going through thousands of them manually. → Understand what's wrong in your AI product before your users find out in production. → Generate and run evaluations without starting from scratch. What Val can't do (and where humans are still fundamental): → Decide what "good" means for your product. → Provide domain expertise. → Own accountability when quality drops. → Make high-level architectural decisions. Val is live for all Galtea customers today. Not a customer yet? You can book a demo and meet Val: https://lnkd.in/eV5wftf6

Some weeks ago, I was on the phone with customer support at one of the biggest delivery services in the world, and the call was carried out entirely by a voice agent. Two things stood out: How good voice agents are getting by the day, and how bad they can still be with human interactions (this one failed with a very simple query). I think voice agents should be evaluated more carefully. A wrong answer from a voice agent in a bank or insurance call center doesn't feel like a bug, but more like a person who told them something wrong. The implications for the brand (and the costs attached to it) are something that big companies cannot overlook. To solve this, today we launch something big in Galtea: you can now run voice simulations against AI agents that answer the phone. Voice simulations are available today in every plan. Running a voice agent in a bank or insurance call center? Test it before it talks to a customer: https://lnkd.in/eCn_gwBY

Watch everything in production without blowing your budget. Today we’re launching Always-on Monitoring 2.0 at Galtea. Continuous evaluation of your production traffic, with sampling, credit caps, and content filters built in from day one, so you're never choosing between watching everything and staying within budget. Sessions close automatically. Regressions surface in minutes, not in customer complaints. And most importantly, connecting production evals to your pre-production tests is what turns monitoring into a system that actually improves over time. The requirement regulated industries have is proof that a deployed AI system still behaves as intended, month after month. This is the last piece to close the loop for online and offline evals. Available at https://platform.galtea.ai

Is your AI ready to scale? Learn how Sabadell Seguros scaled its AI safely with evals.