I usually scroll past “strategic partnership” announcements without pausing my music. You know the type: two massive tech giants shake hands, issue a press release full of buzzwords, and then… nothing changes for the engineers actually writing code. When Databricks announced that $100M deal to bake OpenAI models directly into their platform back in late 2025, I assumed it was just more executive posturing.

Actually, I should clarify — I was wrong.

I’ve spent the last three weeks ripping out my custom LangChain glue code and replacing it with the native Databricks implementation. It’s messy in spots, sure, but for the first time in a long time, I feel like I’m not fighting the infrastructure.

The “Bring the Model to the Data” Thing Actually Works

Here’s the headache we’ve all been dealing with since 2023: You have terabytes of sensitive customer data sitting in Delta Lake. You want to use GPT-4 or the new GPT-5 models to reason over it. But your CISO threatens to fire anyone who pipes that data out to a public API endpoint without a mountain of paperwork.

So we built these fragile RAG pipelines. We moved data. We scrubbed PII. We prayed.

The integration Databricks rolled out changes the calculus. By wrapping the OpenAI models inside the Unity Catalog, the governance isn’t an afterthought—it’s the wrapper. I tested this on a Databricks Runtime 16.1 ML cluster last Tuesday. I set up a permissions model where the AI agent could only access rows in our silver_sales table where the region_id matched the querying user’s AD group.

It just worked. The model didn’t hallucinate access it didn’t have. It didn’t throw a permission error. It just returned the subset of data it was allowed to see. That logic used to take me 400 lines of Python and a custom middleware to enforce. Now it’s a SQL grant.

Hands-on: The ai_query Experience

If you haven’t used the ai_query SQL function yet, it’s basically the magic wand we were promised.

I ran a benchmark comparing my old external API approach against the native integration.

• Old Way: Python UDF calling OpenAI API. Serialization overhead. Network latency. Average query time for a batch of 50 summaries: 14.2 seconds.
• New Way: Native ai_query inside a Delta Live Tables pipeline. Average time: 3.8 seconds.

That’s not a typo. By keeping the execution closer to the data plane and optimizing the batching under the hood, the latency drop is massive.

However, it’s not perfect. I ran into a weird edge case yesterday. I was trying to pass a massive context window (about 110k tokens) into a prompt using the new GPT-5-turbo endpoint they exposed. The job failed silently. No error message in the driver logs, just a generic timeout. I wasted three hours debugging network rules before I realized the default timeout for the SQL function is set too low for that volume of tokens.

Pro tip: If you’re doing heavy context work, manually override the timeout_seconds parameter in your session config. Set it to at least 600 if you don’t want to pull your hair

Questions readers ask

How does the Databricks OpenAI integration handle data governance through Unity Catalog?

The Databricks OpenAI integration wraps OpenAI models inside Unity Catalog, making governance the wrapper rather than an afterthought. In testing on a Databricks Runtime 16.1 ML cluster, a permissions model restricted the AI agent to rows in a silver_sales table matching the user’s AD group region_id. The model returned only authorized data without hallucinating access or throwing permission errors, replacing roughly 400 lines of custom Python middleware with a simple SQL grant.

How much faster is ai_query compared to calling the OpenAI API from a Python UDF?

In a direct benchmark, the native ai_query SQL function inside a Delta Live Tables pipeline averaged 3.8 seconds for a batch of 50 summaries, compared to 14.2 seconds using a Python UDF calling the external OpenAI API. The speedup comes from eliminating serialization overhead and network latency by keeping execution closer to the data plane, along with optimized batching handled automatically under the hood.

Why does ai_query fail silently with large context windows on GPT-5-turbo?

When passing very large prompts—around 110k tokens—into the GPT-5-turbo endpoint, the ai_query job can fail silently with only a generic timeout and no error message in the driver logs. The root cause is that the default timeout for the SQL function is set too low for that token volume. The fix is to manually override the timeout_seconds parameter in your session config, setting it to at least 600 seconds.

Why were RAG pipelines needed before the Databricks OpenAI partnership?

Before the integration, teams had terabytes of sensitive customer data in Delta Lake but couldn’t pipe it to public API endpoints like GPT-4 without triggering CISO objections and extensive paperwork. Engineers built fragile RAG pipelines that moved data externally, scrubbed PII, and hoped nothing leaked. The $100M Databricks-OpenAI deal announced in late 2025 embedded the models directly inside the platform, eliminating the need to move sensitive data out to reason over it.