Databricks is building the connectors we used to hand-build into the platform. Our early experience was not great, but the effort since is clear, so we are revisiting them and will report back.
The direction is right
More and more of the connectors we used to build ourselves are arriving built in to Databricks. Lakeflow Connect, its managed ingestion feature, now lists around 70 SaaS connectors on Azure Databricks, alongside database, file and streaming connectors. The pace is striking: since May, more than 50 new connectors have been announced in Beta, and five more have reached general availability since June: Dynamics 365, SharePoint, Google Drive, NetSuite and MySQL change data capture.
It matters because every connector that becomes native is one less custom notebook and pipeline to build, schedule, monitor and fix. Fewer moving parts means simpler set-ups, and ingestion you can run and watch in the same place as the rest of your data.
On Azure Databricks the same pattern holds. Dynamics 365 and SharePoint are generally available, while Microsoft Entra ID, Microsoft Graph logs and Azure DevOps are still in Beta.
It is close to home for us. Several of the systems in our Retail insight transformed set-up now have managed connectors: NetSuite (generally available), Shopify (Beta) and Salesforce. Today the only native pipeline we use there is the one for Google Analytics, and the rest we bring in with APIs and JDBC connectors, so each connector that matures is a chance to simplify what we built.
Our experience so far
We did not have a great experience early on. Today we use Azure Data Factory to bring in Salesforce data. In Data Factory we run a simple query for the last few days of changes, and we do not use formula fields. Databricks has a Salesforce connector. We tried it a good while ago, and for us it was slower.
Here is what the current documentation says, which may explain it:
- By default the connector ingests every column in a table, formula fields included.
- It re-reads those formula fields in full on each run, because Salesforce does not update a record's change timestamp when a formula's result changes.
- You can leave columns out, but only by editing the pipeline's configuration in code. The setup screen lets you choose tables, not columns, which is a gap for a connector meant to simplify things.
- A Beta option handles formula fields incrementally, but it works only on a new pipeline and returns NULL for formulas that use some functions.
So we were probably carrying a cost for fields we do not use. The connector has evolved since, and these details are from its current documentation.
This is not only a Databricks quirk:
- Salesforce works out formula results when a record is read, so asking for records changed since last time can miss a formula that changed.
- At least one major vendor does not sync formula fields by default, and recommends recalculating them downstream instead.
- One published comparison of the Databricks connector and Data Factory reported similar run times but a higher cost per run for Databricks, so it is worth setting a budget policy before a trial.
That is one experience with one connector, and a lot has changed since. We can see how much effort Databricks has put in, and it is now time for us to revisit it and explore all the possibilities. We are hoping for a better experience this time.
What we are trying next
We have been a little slow out of the gate. Some of these connectors have been out for a while: Dynamics 365 has been generally available since June, and NetSuite, SharePoint and Google Drive since August. But with this many arriving, we cannot ignore them any longer. Here is what we are trying:
- Salesforce. We tried the connector a good while ago and it has evolved since. We will try it again, this time leaving out the formula fields we do not use.
- NetSuite. We use SuiteAnalytics Connect for NetSuite today. The managed NetSuite connector has been generally available since 4 August, so it is on our list. Both read through the same licensed interface, so what we will look for is what the connector adds on top: incremental loading, history tracking and new and dropped columns handled for us, and less custom code to run.
- PostgreSQL change data capture. We are testing the new connector, which is in Public Preview, on behalf of a client who already uses Databricks and is looking to change their underlying database technology.
- Salesforce Marketing Cloud. The connector has been in Beta since June. We integrate Marketing Cloud through Salesforce Data 360 today, so it will be interesting to see how a direct connector compares. Databricks' own earlier guidance was to route Marketing Cloud through Salesforce Data Cloud, which is close to our current route.
We will report back once we have done the work, rather than judge any of them on paper.
What we already know about specific connectors
- Salesforce. Formula fields are re-read in full on every run by default, and hard deletes at the source only reach Databricks after a full refresh of the destination table.
- NetSuite. It needs a SuiteAnalytics JDBC drivers licence and the NetSuite2.com data source, uses token-based authentication, and does not ingest the DeletedRecords table. The documentation does not say how deleted records are handled, so that is one of the first things we will test.
- PostgreSQL. Changes are captured from a primary server only, sign-in is by username and password, and the gateway has to run continuously. Deleting a pipeline does not remove its replication slot, so you have to drop it yourself. Your partitioning strategy matters too: partitioned tables are supported, with each partition treated as a separate table, and adding or removing a partition needs a full refresh of the table.
Things to keep in mind
- Beta is not for production. Beta connectors have no SLA and no formal support, and can change or be withdrawn without notice. Public Preview and general availability are supported.
- Access is not always self-service. Many Beta connectors have to be switched on by a workspace admin in the Previews page, and some connectors labelled Public Preview ask you to contact your Databricks account team.
- Deletes need planning. For several SaaS connectors, deleting a table at the source does not delete it in Databricks, so check how each connector handles deletes.
- Statuses change month to month, and the documentation does not always agree with itself, so check the release notes for the connector you want before planning around it.
Our verdict: don't rush in, but we are excited
The direction is clearly right, and the more of this that becomes native, the less custom plumbing there is to look after. We are excited about the potential. But if you have solid, working pipelines, do your homework and test properly before switching them over. Some of the quirks are in the documentation, which does not always agree with itself, and no doubt we will find more as we start trialling.
Where a connector is generally available, it deserves a proper trial against what you run today. Where it is in Beta, we treat it as one to watch, and we will not be planning production work around it yet.
Sources
- Lakeflow Connect connectors on Azure Databricks, Microsoft Learn
- SaaS connectors list, Microsoft Learn
- Azure Databricks release notes, August 2026 and October 2026, Microsoft Learn
- Azure Databricks release types, Microsoft Learn
- Salesforce formula fields, Microsoft Learn
- PostgreSQL connector and NetSuite connector, Microsoft Learn
