Daily Archives: July 9, 2026
The 12 Data Architecture Patterns Every Data Engineer Should Master in 2026
Most engineers argue about tools.
… But production data platforms aren’t won by tools — they’re won by architecture.
… This article breaks down the 12 data architecture patterns every data engineer should understand in 2026, when to use them, their trade-offs, and how modern systems combine multiple patterns together. — Read More
How to build robust data pipelines with AI
Writing a data pipeline with AI has never been easier. You type a prompt, wait a minute, and something that runs shows up. The pipeline is green. The number it returns is… wrong.
Indeed, a pipeline that runs and a pipeline that is correct are two different things, and AI is very good at the first one. Two things work against you.
First, AI is non-deterministic. Ask three times, get two different implementations, and the green checkmark won’t tell you which one is correct.
Second, often, AI can’t see your data and metadata. So it guesses: your schema, what counts as a duplicate, what the units are. Those guesses are silent bugs. Again, no error will be seen in the pipeline, just plausible wrong numbers. — Read More
Where AI Agents Belong in Data Engineering: The Correctness Layer
With ever-changing models, new and better ones coming out every few months, it’s great if we don’t have to rely on them too heavily. The better your tooling, the less dependent you become on any single model. That’s also why the deterministic harness matters: a correctness layer that lets you reproduce outputs and trace lineage regardless of which model you’re running underneath. This is especially true during maintenance or extending the project, where verification is the real job.
The danger isn’t only a crash or an error message, but a wrong number that didn’t break. It might be a clean query, but it introduces duplicated rows.
In this article, we go through the three levels of AI agents in data engineering, how to structure projects so the AI delivers its best outcomes, and how dedicated agents with a deterministic core help us build higher-quality pipelines — ones we can actually trust. And we look at a practical example of how it works with a blast radius analysis. — Read More
The Ultimate Guide for AI in 2026
Recently, a client asked me the following: If I had to explain the AI industry in one go, what would you give me? This had to be explained in a high-level, very intuitive way for anyone who’s not quite as deep into AI as I am to follow.
In particular, answer the following:
1. What needs to be known across the entire value chain?
2. In this very dynamic industry, what is constant?
3. What bets is the industry making
4. What does the future hold?
In this piece, we’re doing just that. It’s my longest article ever, but I’ve genuinely never packed more information and insights into a single piece. Ever. It was a hustle (I honestly don’t know when I decided it was a good idea to put so much effort into something that can be purchased for $20/month, but I guess I really dislike doing things halfway). — Read More
Generative AI as a transformational logic for cognitive neuroscience
Cognitive neuroscience faces a paradox: neural data are abundant, yet conceptual synthesis has stalled because dominant contrast-based approaches show where activity differs but not how cognitive operations relate or transform. Here, we propose a generative-transformational logic grounded in AI and neural geometry, treating cognition as lawful mappings among neural states. Generative models can learn latent transformations linking states across tasks, contexts, and individuals. Because transformation success is testable, this framework enables counterfactual simulation and connects data-driven modeling with theory-driven inference. It moves cognitive neuroscience from mapping correlates toward algorithmic explanations of how the brain generates and reorganizes cognition over time. — Read More
