Readiness check

Private Azure Databricks with Palantir Foundry

A lakehouse with no public IP addresses. Cluster subnets are injected into your network, the workspace and control plane are reached over Private Link, Unity Catalog governs access centrally, and Palantir Foundry connects through a private, authenticated pipeline.

ImplementationThe architecture from our healthcare network engagementAligned to Databricks Well-Architected guidance

Book an assessment Take the readiness check
Fee
Set after an architecture assessment
Built for
HIPAA workloads
Integration
Palantir Foundry
Public lakehouse IPs
None

The private data perimeter

Cluster runtimes talk across delegated private subnets. Metadata, credentials, and telemetry flow through private endpoints into Unity Catalog and Azure Key Vault.

graph TB subgraph DataVNet["Azure data network"] subgraph Delegated["Delegated Databricks subnets"] HostSub["Host subnet
driver and worker communication"] ContainerSub["Container subnet
cluster node isolation"] end subgraph PESubnet["Private endpoint subnet"] PE_DFS["ADLS Gen2 private endpoint
privatelink.dfs.core.windows.net"] PE_DB["Databricks control plane private endpoint
privatelink.azuredatabricks.net"] PE_KV["Key Vault private endpoint
privatelink.vaultcore.azure.net"] end end subgraph Metastore["Unity Catalog"] UC["Central metastore
row and column masking
table and column lineage"] end subgraph Integration["Integration"] Foundry["Palantir Foundry
mutual TLS, private ingress"] end ContainerSub --> PE_DFS ContainerSub --> PE_DB ContainerSub --> PE_KV PE_DB --> UC PE_DFS --> UC Foundry -->|"Authenticated pipeline connector"| PE_DFS
Every service the clusters use is reached through a private endpoint with its own private DNS zone.

Built for HIPAA workloads

Storage account firewalls reject all public internet traffic, and clusters run with secure cluster connectivity (no public IP). That removes the public network paths data could otherwise leave by.

What it covers

The code, configuration, and runbooks, delivered into your repository.

  • Private lakehouse infrastructure

    The Databricks workspace, network peering, subnets, network security groups, and private endpoint DNS.

  • Unity Catalog configuration

    Metastore setup, storage credentials, external data access controls, and a catalog and schema isolation model.

  • Foundry integration runbooks

    Pipeline integration guides, authentication scripts, and troubleshooting steps for synchronization with Palantir Foundry.

  • Cluster policies and security baseline

    Cluster policy definitions that block non-approved instance types, enforce idle auto-termination, and disable public internet breakout.

Design controls

Mapped to the security, governance, performance, and cost areas of the Databricks Well-Architected Framework.

  • Governance and lineage

    • One metastore across all workspaces
    • Row filters and column masks
    • Table and column lineage tracking
    • Audit logs streamed to Log Analytics

  • Security and network isolation

    • VNet injection with no public IP mode
    • Private Link for the workspace UI and the control plane
    • Cluster policies that require single-user or isolated access modes
    • Customer-managed keys for managed storage

  • Palantir Foundry connector

    • Automated, authenticated synchronization with Foundry
    • Service principal authentication with narrowly scoped permissions
    • Delta Lake bronze, silver, and gold layers
    • Schema validation on ingestion

Questions

How is the fee set?

After a free 30-minute architecture assessment. We look at your Azure environment, your data sources, and how you use Foundry, then quote the implementation.

Is this the architecture from your healthcare case study?

Yes. It is the pattern we built for a healthcare provider network: a private Databricks lakehouse with Unity Catalog and private connectivity to Palantir Foundry, run under HIPAA. A reference call is available on request.

How does this differ from the Azure AI Lakehouse Module?

Both use the same private network and Unity Catalog pattern. The AI Lakehouse Module is a fixed-fee package with private Azure OpenAI connectivity. This implementation adds the Palantir Foundry integration.

Will this make us HIPAA compliant?

No single build does that. HIPAA compliance also depends on your policies, agreements, and administrative safeguards. This implementation covers the network isolation, encryption, access control, and audit logging for the lakehouse itself.

Scope your private lakehouse and Foundry integration.

The fee is set after a free 30-minute architecture assessment of your environment.

Book an assessment Take the readiness check