Ethyca

Dario Amodei just published the most serious essay yet on frontier AI risk, focused squarely on model alignment. Since *governance* is the deterministic guarantee that a model cannot cross a boundary at all, whereas alignment only asks how the model will behave when presented with a boundary, I think it's a missed opportunity. I agree completely with Amodei’s diagnosis that capability is now outrunning our ability to govern it. But he frames the debate purely around the power of the models, and not about the controls we should implement to govern what they do once they’re connected to data. Every proposed answer is about the models themselves. Make them more aligned, make them more interpretable, slow their development so safety can catch up. All of that is worth doing and none of it answers the issue that gets bigger every time we connect one of these systems to systems and data in the real world, namely who decides what an AI agent is actually allowed to do once it's running whether on your desktop, in your cloud or on the open internet? That decision isn't something a frontier lab can make for you. Anthropic can shape how Claude behaves in general but it can’t decide whether a particular agent, acting for a particular employee in your company, may touch a particular customer record for a particular purpose under your contracts, your consent obligations, and your regulator's rules. That authority belongs to individuals and institutions, and right now most institutions have no way to enforce it at the speed this innovation is moving. I've written about why I think this is the missing half of the AI safety conversation, and why it's the one part any institution can start building today without waiting for Washington, Beijing, or the labs to agree on anything. Alignment governs how a model behaves, but governance decides what it's authorized to do. We need both, and we've spent almost no time on that vital second line of defense. I’ll link to the essay in the comments.

We spoke with a company that built an AI model to assess customers’ creditworthiness, and inadvertently violated the Equal Credit Opportunity Act because the model accessed unauthorized racial data. The model’s training data included financial history pulled from a massive dataset. Unintentionally, because it was run through an automatic feature-selection step, that training data brought in racial and ethnic information too. It was never the company’s intention to make race a criteria in a creditworthiness model. The data was just there. When it was combined together and served to the model, it turned into something they would never have knowingly allowed…because it’s illegal. This is a specific example of a core theme large enterprise governance teams come to us with right now. It arises without malicious intent. It happens because of the combination of 1) huge appetite for AI-assisted insights, 2) vast datasets, and 3) incomplete understanding of what’s in them and how it’s allowed to be used. Traditional human-in-the-loop governance reviews have very little chance of catching this in an AI model unless everyone thinks of every question that has to be asked, which could take months, and never makes a mistake. Here's how this problem gets handled end-to-end by Ethyca’s Astralis platform and its purpose based access control (PBAC). First, Astralis establishes the facts about what the data actually is, and all of the obligations (legal, contractual, internal policy) that apply to it. Then it makes a decision. It infers that what's really being built is more than than the plain financial-risk model the request was described as, but profiling that leans, directly or indirectly, on race. It then holds up this inference against the company's policy, which forbids it outright due to legal obligations (like ECOA). Then it transforms the data so the model can proceed without the sensitive attributes. It logs the risk and the decision, then notifies the person who needs to know. At machine speed, data governance needs an end-to-end process built directly into infrastructure and code. Having the right policy in place is meaningless if it can’t be evaluated and actioned at the speed your technology is moving. Astralis works because all three phases of governance ( facts, decision, action) happen together as part of the same request, at runtime. If you want to check it out for yourself, feel free: https://lnkd.in/ePKiSRYN

It’s quite cool and a bit frightening to watch Claude react when our platform tells it “no, you can’t use that data you want.” Safe to say Claude doesn’t take our directive lying down. It immediately starts scheming about how to potentially override the denial of request for data access by rephrasing its request. Cheeky Claude. Reading the LLM’s internal monologue when dealing with Purpose Based Access Control constraints is fun if you’re a geek like me, but it also shows how strong your technical foundation needs to be for doing AI governance at speed and scale. With Ethyca’s Fides ontology, company data policies live at row and column level in the warehouse. PBAC is then able to evaluate, decide, and enforce a decision on the data in that row or column at agent or model runtime. So in this case, once Claude was prompted and its gears started spinning, it went to try to pull data out of the warehouse and ran straight into Ethyca’s PBAC waggling its finger and saying “don’t touch”.Pretty cool! On the other hand, if you have a written document somewhere talking about how this or that database should or shouldn’t be used for different kinds of activities, well good luck. Claude is going to eat your data policy for lunch.

Today we’re releasing Astralis, an end-to-end platform for runtime data governance in the AI era. It’s built on our open-source ontology Fides, and it’s the end goal of six years’ work to make enterprise technology systems more trustworthy. In 2019, weeks after starting this company, we wrote an internal document with a vision of the future: the way enterprise did data governance - people filling in workflow tools to produce point-in-time reports for regulators - would collapse under the scale of the software it was meant to govern. The collapse is happening now, and AI is what caused it. The old compliance model assumed a person could stand between the rules and the data. AI removes the person, and acts at a speed and volume no team can review. So, governance does the only thing it can: say "no." It blocks access, stalls the AI initiative, and protects the company by preventing it from using its own data. In doing so, it blocks innovation. Meanwhile the business is asking, reasonably, why millions invested into AI initiatives aren't producing the expected outcomes. This can't be fixed by more people watching machines. The fix is machines watching machines, applying the company's actual policy at the moment data is used, everywhere. That's Astralis in one sentence. Its open-source foundation, Fides, already runs inside the New York Times, Condé Nast and Ramp. Here are things Astralis lets you do: - Impact assessments in real time: These prove to regulators your new data processing activity took consideration of user preferences and varied local regulations in advance. They’re an essential piece of governance frameworks, and they can take literally months for a large enterprise initiative. With Astralis, the same assessment can be completed in minutes, in the flow of work. The time and cost savings are staggering. - Purpose Based Access Control (PBAC): This ensures data is always available to business users and technologies, provided the intended purpose of use matches businesses policies and regulatory context. Enforcement runs in real time at every consumption point across the enterprise. You get the same policy across pipelines, agents, MCP servers, SQL clients, and users. I'm very grateful to the Ethyca team and our pilot partners who have led us to this moment. This is the one we've been building toward. If you want to check it out for yourself, feel free: https://lnkd.in/ePKiSRYN

For 50 years, engineers built systems to efficiently collect data. Now we need them to efficiently manage trust processing data at AI scale. That’s something most legacy systems can't do. Legacy systems were designed to gather, process, and retain data efficiently, not to enforce dynamic privacy preferences or support responsible data usage in AI at enterprise scale. This is much more than a minor inconvenience. It's a fundamental challenge for all organizations, made even more existential by the AI innovation wave. This is also the challenge we’ve built Fides to help enterprises overcome. Consider what happens when modern AI governance requirements crash into legacy architecture: • Hard-coded data models that can't adapt to evolving AI ethics policies • Tight coupling between systems that makes selective data usage impossible • Embedded business logic that assumes permanent, unrestricted data access • Missing audit trails for AI model data lineage and usage • Limited APIs that don't support real-time consent enforcement • Undocumented data flows that create AI governance blind spots • Complex dependencies that make changes risky Building trusted data infrastructure from scratch, designing AI governance controls from the ground up, is relatively straightforward. But which enterprise has the luxury of rebuilding everything from scratch before deploying AI? Instead, they must retrofit trust into systems designed for a world where data was collected without question and used without constraint. This isn't about replacing everything. It's about building the trusted data layer that bridges legacy systems and infrastructure that’s suitable for AI innovation and ready for hyper-scale data governance. Because the real challenge isn't the technology, it's enabling data-driven innovation while evolving systems to meet trust requirements that scale with your data ambitions. Is your organization's data infrastructure built for AI-scale trust, or still optimized for unlimited collection? What's blocking your teams from using sensitive data to create value, ethically and at speed with personal data? I'd love to hear your experiences, either in the comments below or by DM.

Purpose based access control. PBAC. We think it’s the only way to do data governance at runtime properly for agents and MCPs, and that we’re the only ones who can pull it off. So we bet the product roadmap on it. “Why are you so excited about it Cillian?” I’m an excitable person in general, but in this case it’s fully warranted. PBAC is a new paradigm made for governing data in this moment, where AI has exploded data siloes across the organization, and complex regulatory frameworks pile on top of the business such that context for any two data processing requests are rarely the same. Other access control frameworks primarily control *who* can access the data. PBAC controls *what* purpose the data can be accessed for. You can’t get close to meeting this moment with a blunt instrument like RBAC (roles based access), which was de facto standard for a long time. Attributes based access control is better, but it can’t account for intended use of data, which is the whole focus of the regulations that these businesses are trying to follow. If you’re using either of those frameworks, you’ll end up with far less data reaching your AI model than could be used, because caution plus blunt instrument equals blanket restrictions on data processing. In other words, the impact of your AI initiative gets blunted by access controls that lack nuance and can’t understand purpose. By contrast, PBAC ensures that data is always available to business users and technologies provided the intended purpose of use matches the businesses policies and regulatory context. Sounds great. How? It’s the combination of Ethyca’s established Fides tooling and our new Large Language Regulatory Model, which, for every data usage request, infers purpose of use, evaluates risk within a global regulatory context, and renders a decision on whether the data in question is allowed to be used for the stated purpose. That decision is then enforced at runtime every time an agent, MCP, or model wants to pull data for processing. The reaction we’ve got from showing sophisticated data governance buyers what PBAC can do is unlike anything I’ve seen in my time building Ethyca. It feels like a perfect confluence, years of deeply technical R&D coming to fruition at a time when the market is crying out with need for a solution. PBAC is letting our partners bring meaningful, trustworthy guardrails to their AI and in so doing, it empowers them to get more out of their AI, by leveraging far more of their data than they had been able to previously. We’d love to show you how it works. My DMs are open.