Castellum.AI

⚡ The importance of speed – does your sanctions system alert you to new OFAC designations before the official press release? OFAC (along with the UK and Australia) just issued new sanctions targeting Russian ransomware operations, with OFAC including crypto addresses in the designations. We instantly ingested, cleaned and structured crypto addresses and provided a monitoring alert for designations targeting a new crypto wallet, all before OFAC published its own notification. This is also an interesting case of coordination between allied countries. OFAC’s designations were coordinated with Australia and the UK to target “bulletproof hosting infrastructure” (new to me) that enables cyber criminals to evade detection by cybersecurity tools, operate with anonymity and evade existing sanctions. I’d encourage everyone to check their screening system. If these names or crypto address aren’t flagging in your sanctions screening system, there’s a problem. 18dLDAWi8LmrHbEq3QzDJb9SLxCf4uimXB HYPERCORE LTD MEDIA LAND, LLC Interested in learning more? DMs are open

One of the most important things to understand about AI agent performance is how errors compound. An agent reaches a decision through a chain of tasks. An error early in that chain can affect everything that follows. For an alert review agent, the series of steps could look like: ➡️ Data ingestion - capturing all the inputs needed to make a decision ➡️ Data normalization - parsing what each piece of data actually means, what is a “name” vs an “address” ➡️ Evidence gathering - retrieving the information needed to evaluate the case ➡️ Comparison - evaluating the normalized data against the relevant criteria ➡️ Decisioning - using comparison output to reach a disposition aligned with the organization’s risk tolerance Every step must be correct. In this five-step process, even if each step is 95% accurate, the probability of reaching the correct final disposition is about 77%. To assess whether an agent will perform reliably in production, teams need to know: - How it performs across a representative sample of alerts - The well each step performs and where failures occur - How the steps affect one another - Whether QA and escalation controls catch those errors before the decision is finalized This is what makes AI agents tricky to monitor in production. Strong pre-deployment results do not guarantee that performance will remain consistent over time. We go deeper into this in our new guide, including five places where model risk can enter the workflow. I’ll share the link in the comments.

“AI agent” has become a catch-all term. More than 70 companies market some type of “AI agents for AML/KYC.” When we looked closer, 80% of them aren’t selling AI agents at all. Some offer chatbots or AI that summarize cases. Many are AI wrappers plugged into existing software. If you need agents for alert review, you’re faced with dozens of solutions to sort through. Most market maps don’t make this easier. They put screening, transaction monitoring, fraud, case management and onboarding vendors on the same page. You can see how large (and overwhelming) the market is, but not which products are built to solve the problem you actually have. So we built one focused on agentic screening and alert resolution. The map compares vendors on two dimensions: agentic screening capabilities and governance and reliability. We assessed each vendor across eight criteria, such as: - Does the agent follow an institution’s policies and risk thresholds? - Can the institution test and validate its performance and trace its decisions? - How much of the workflow can it handle, from initial alert review through deeper investigation and enhanced due diligence? - We also looked at the risk data and screening technology behind the agent's decisions: who controls them, how current the data is and where third parties are involved. Castellum.AI was evaluated using the same criteria and placed in the Trusted quadrant. But no single placement tells the whole story. A vendor may be strong on sanctions alerts and offer less for adverse media or ongoing customer screening. The more useful question is which vendor can do the work your institution needs done. Yes, usual caveats apply. This map is based on the publicly available details on each vendor's agent. Use the map to narrow your shortlist and ask better questions. Link to the market map in the comments.

Brussels, we have a problem. The EU's consolidated sanctions list went dark yesterday 🌑 At 11:35 am on Thursday, October 1st, we had a problem. The Financial Sanctions Files (IYKYK) went dark when Europe webgate, the server that hosts EU sanctions, went down. Our monitors flagged the failure and searched for a workaround. The root cause? Permissions errors redirected the once open data feed to a login wall… Our team (hat tip to Dan Luchi) notified the EU the same afternoon. Of course all this happens after hours in Europe. An automated response received back, but no resolution. At 6:40 am the feed was restored and our system automatically processed the latest update. A gap closed before most would even have noticed. This isn’t the first time the EU’s sanctions program disappears, and it’s part of a broader pattern of discrepancies. EU designations take effect when they're published in the Official Journal. The consolidated list catches up days later, sometimes weeks later. Between outages and lag, the consolidated list can't be your only source. Make sure your sanctions data partner: 1. Updates in real time, with monitoring on the sources themselves. Official sources go down too. Without monitoring on the pipeline, a stale file looks the same as a current one, and screening keeps running against it. 2. Knows the difference between the Official Journal and the consolidated list, and monitors the Journal in real time. Miss a designation announced in the Journal and you're looking at potential false negatives and violations. 3. Flags data issues directly with the issuing authority. When a source breaks, someone has to tell the people who run it. That's how it gets fixed for every institution screening against it. In a world of real-time payments, a list that's hours or days behind is a live exposure gap. Learn more: https://lnkd.in/e7GECxuD

It all started with a frustrated bank CEO telling me, “I can’t tell compliance vendors apart anymore. They all make the same claims. I just want to know which AI tools will actually deliver.” Today, we answer that question with the FinCrime Agent Benchmark, the only benchmark that tests how AI, including Claude Fable 5.1 and GPT 6 Astra, perform on real financial crime screening and adjudication work. Most AI benchmarks score models performance on generic tasks. But AML/KYC does not run on one set of rules. Every institution has their own SOPs, risk appetite and screening thresholds. To make a real comparison, you need the same cases, the same procedures and the same scoring criteria. That had never been done for AML/KYC. FCA Bench asks the questions compliance teams actually need answered. Can the AI apply an institution’s policies, identify risk and reach the right disposition consistently? We evaluated nine LLMs on their own and with Castellum.AI's agentic harness. The results speak for themselves. ➡️ Unharnessed LLMs missed up to 72% of true positives, meaning risk is either never surfaced or wrongly cleared. ➡️ Without a harness, even the latest models failed at basic compliance work, where regulators expect minimum accuracy in the high nineties. GPT 6 Astra achieved 71.1% accuracy standalone, but 99.3% with Castellum.AI’s harness. Claude Fable 5.1 standalone had 64.3% accuracy, but 99.5% with Castellum.AI’s harness. ➡️ Standalone model accuracy was as low as 30.8% (Meta Muse Spark 1.2 — worse than a coin flip). ➡️ Standalone LLMs achieved only 45.2% consistency when tested on the same case three times. Imagine explaining to your auditors and regulators that your screening system is as good as a coin flip… Castellum.AI’s harness raised LLM consistency to 97.2%. A model going from 71% to 99% accuracy isn’t just a huge leap, it's the difference between unacceptable and acceptable to regulators. It also makes one thing very clear. You cannot prompt your way to reliable AML/KYC decisions. LLMs need guardrails, infrastructure and specialized data to be accurate, consistent and defensible. The full research is linked in the comments Read the full benchmark: https://lnkd.in/g5Rq6ey7

What could your compliance team accomplish with 83% less time spent on alert review? Institutions using Arbiter AI agents are already finding out. Join us October 6 for a live product demo, exclusively for Bridger® and Firco™ teams, showing Arbiter AI agents adjudicating alerts inside both systems.