NexusOne

Every quarter you stay on legacy Cloudera, Hortonworks, or MapR, you’re paying twice: once for the infrastructure that can’t scale, and again in AI projects that never ship. A Top 5 U.S. bank was running three separate Hadoop platforms and still couldn’t get multi-tenancy, modern security, or container-native architecture. Every vendor promised an “upgrade” requiring 12+ months. They chose a different path and modernized 30 production AI apps in 5 weeks—saving $30M+ in license costs alone. This 2-page breakdown shows exactly how.

In June 2024, Databricks paid a reported $1 billion to $2 billion for Tabular, a 40-person company whose product was built on an open specification anyone can download for free. That price tag tells us a lot about the table format war. Three formats had launched to replace Hive: Hudi, built for Uber's streaming upserts; Delta Lake, built to sell Databricks; and Iceberg, built at Netflix as a neutral spec with no engine attached. Delta arguably had the deeper Spark integration and the bigger installed base. Iceberg won anyway, because it was the only format whose success required nothing from its creator, and enterprises picking a ten-year home for their data priced that in. So what does a vendor buy when the software is free? The people who steer the spec, and position in the layer above it. The same week the Tabular deal was announced, Snowflake and Databricks both open-sourced their catalogs. The format officially went neutral, so the fight moved to the catalog. We wrote up the full history: the three origin stories, the adoption wave that ended the war, and what standardizing on Iceberg means for the catalog layer. Link in the comments. #ApacheIceberg #DataEngineering #DataArchitecture #OpenSource #DataStrategy

The least glamorous component in the data stack is the one everyone is fighting over. For fifteen years, nobody fought over the catalog. It was the Hive Metastore, a Thrift service Facebook built in 2008, and every engine bolted itself to it because nothing else existed. Then Iceberg's REST catalog spec broke that monopoly in 2022, and within one week in June 2024 both Snowflake and Databricks moved to occupy the vacated ground: Polaris announced on a Tuesday, the Tabular acquisition on a Wednesday, Unity Catalog open-sourced six days later. The reason is mechanical. The catalog is the service every engine must ask which tables exist and whether it may touch them, and increasingly it hands out the storage credentials too. Whoever operates it decides which engines reach your data and on what terms. The format war taught vendors the format could not be owned, so the fight moved to the layer that still can be: identity integration, policy enforcement, and which engines get first-class treatment. We wrote a decoder: what a catalog decides, what each vendor's version of "open" includes and withholds, and why an estate with mainframes and three clouds needs a catalog that federates rather than one that annexes: https://lnkd.in/gkp3QGfv #DataEngineering #ApacheIceberg #DataGovernance #OpenSource #DataArchitecture

Your CEO is asking about AI. Your board wants a timeline. And you’re still explaining why the data isn’t ready. If you’re running Cloudera, Hortonworks, or MapR, this is the conversation you’re stuck in. Legacy Hadoop wasn’t built for ML pipelines, agentic AI, or hybrid cloud—and no upgrade path fixes that. This 2-page guide shows how a Top 5 U.S. bank eliminated $30M+ in Cloudera costs and modernized 30 AI apps in under 5 weeks—without a rip-and-replace. See how it works.

Four times in the last decade, a project at the center of enterprise infrastructure changed its license, and the community forked rather than follow. Presto's creators left Facebook and their fork, Trino, is where the community moved; the trademark stayed behind and mattered little. Elastic went SSPL in 2021; OpenSearch passed a billion downloads and Elastic re-added AGPL in 2024. HashiCorp went BUSL in 2023; OpenTofu shipped features Terraform lacks, and HashiCorp sold to IBM for $6.4 billion. Redis changed licenses in March 2024; Valkey forked eight days later, became the default on AWS and Google managed caches, and Redis re-added AGPL within fourteen months. The vendors had real grievances, since hyperscalers were monetizing their work without contributing back. But every fork resolved in the community's favor, and the actual casualties were customers of the license-changers whose stacks were welded to one vendor's distribution. When Aiven needed out of Redis, it moved 15,000 servers to Valkey in about three months, because its architecture treated the cache as swappable. The license was never the protection. Composability is the only license insurance that pays out. The story, with the license mechanics and what foundation governance structurally prevents (plus our prediction for how AI accelerates open-source innovation sharing): https://lnkd.in/g7U9ihVF #OpenSource #DataEngineering #Trino #DataArchitecture #DevOps

On October 19, Satya Nadella wrote that "we need to separate the supply of intelligence from the authority over it." The same day, Microsoft launched Guardian Agents. It was the last of five agent-control moves in four weeks. Okta rolled out an Agent Gateway at Oktane on September 22. Workday shipped its Agent Harness on October 13. On October 15, ServiceNow announced AI Passport, a signed credential for every agent, and Proofpoint agreed to pay $6.3 billion for Varonis, citing the data agents can reach. With Palo Alto's $25 billion CyberArk deal and CrowdStrike's $740 million for SGNL, security incumbents have committed $32 billion in fifteen months to owning who an agent is. But each control has a different sphere of control. Okta's sits in the directory, Workday's in Workday, ServiceNow's in its CMDB, Microsoft's in its tenant. When an agent's task runs through Trino, a Spark job on object storage and a mainframe extract, the decision about what each query returns still happens engine by engine, outside all five. We mapped every deal and launch, what each buyer got that it could not build, and the three questions to ask before you sign for the next agent control. Link in the comments. #AIAgents #DataGovernance #IdentitySecurity #DataArchitecture #AIGovernance