I usually scroll past “strategic partnership” announcements without pausing my music. You know the type: two massive tech giants shake hands, issue a press release full of buzzwords, and then… nothing changes for the engineers actually writing code. When Databricks announced that $100M deal to bake OpenAI models directly into their platform back in late 2025, I assumed it was just more executive posturing.

Actually, I should clarify — I was wrong.

I’ve spent the last three weeks ripping out my custom LangChain glue code and replacing it with the native Databricks implementation. It’s messy in spots, sure, but for the first time in a long time, I feel like I’m not fighting the infrastructure.

The “Bring the Model to the Data” Thing Actually Works

Here’s the headache we’ve all been dealing with since 2023: You have terabytes of sensitive customer data sitting in Delta Lake. You want to use GPT-4 or the new GPT-5 models to reason over it. But your CISO threatens to fire anyone who pipes that data out to a public API endpoint without a mountain of paperwork.

So we built these fragile RAG pipelines. We moved data. We scrubbed PII. We prayed.

The integration Databricks rolled out changes the calculus. By wrapping the OpenAI models inside the Unity Catalog, the governance isn’t an afterthought—it’s the wrapper. I tested this on a Databricks Runtime 16.1 ML cluster last Tuesday. I set up a permissions model where the AI agent could only access rows in our silver_sales table where the region_id matched the querying user’s AD group.

It just worked. The model didn’t hallucinate access it didn’t have. It didn’t throw a permission error. It just returned the subset of data it was allowed to see. That logic used to take me 400 lines of Python and a custom middleware to enforce. Now it’s a SQL grant.

Hands-on: The ai_query Experience

If you haven’t used the ai_query SQL function yet, it’s basically the magic wand we were promised.

I ran a benchmark comparing my old external API approach against the native integration.

• Old Way: Python UDF calling OpenAI API. Serialization overhead. Network latency. Average query time for a batch of 50 summaries: 14.2 seconds.
• New Way: Native ai_query inside a Delta Live Tables pipeline. Average time: 3.8 seconds.

That’s not a typo. By keeping the execution closer to the data plane and optimizing the batching under the hood, the latency drop is massive.

However, it’s not perfect. I ran into a weird edge case yesterday. I was trying to pass a massive context window (about 110k tokens) into a prompt using the new GPT-5-turbo endpoint they exposed. The job failed silently. No error message in the driver logs, just a generic timeout. I wasted three hours debugging network rules before I realized the default timeout for the SQL function is set too low for that volume of tokens.

Pro tip: If you’re doing heavy context work, manually override the timeout_seconds parameter in your session config. Set it to at least 600 if you don’t want to pull your hair