The reality for most organizations is that data lives everywhere. As engineering teams expand and adopt specialized platforms for their workloads, enterprise data naturally disperses across multiple clouds and operational systems. When analytical and AI initiatives require context from across the entire organization,these teams traditionally face a difficult trade-off: build complex extraction pipelines to copy data into a central repository or accept fragmented reporting.

Traditional architectures address this fragmentation with scheduled extraction pipelines that copy remote datasets into a central staging warehouse. This approach creates synchronization lag, adds recurring storage costs, and triggers variable network egress fees. When schemas change upstream, pipelines break, and teams spend hours reconciling metrics across out-of-sync dashboards.

BigQuery and Google Cloud’s borderless Lakehouse offer a streamlined way to solve this challenge with a foundation based on the open Apache Iceberg REST catalog specification. Rather than moving data to the compute engine, you can bring Google Cloud compute and generative AI directly to your data, querying it in place across Databricks, Snowflake, and AWS Glue.

In this post, we walk through how to configure Databricks Unity Catalog for federation into BigQuery (AWS Glue and Snowflake set up in appendix), run a single query joining data across all three external catalogs, and use conversational analytics in BigQuery to explore the consolidated data with natural language.

This solution uses Lakehouse runtime catalog to synchronize metadata from other cloud providers, allowing you to query data with BigQuery or external open source engines through the Apache Iceberg REST catalog endpoint, all without migrating the data. Because Databricks Unity Catalog, Snowflake Horizon Catalog, and AWS Glue Data Catalog support the open Apache Iceberg REST catalog specification, you can apply a consistent connection pattern across all three platforms.

When you run a federated query, Lakehouse runtime catalog separates metadata synchronization from storage access:

Metadata synchronization: Lakehouse runtime catalog periodically syncs schemas, snapshot pointers, and table definitions from the remote catalog based on a refresh schedule you define.

Direct storage access: During query runtime, a compatible engine reads Parquet data files directly from remote object storage, such as Amazon S3, using temporary credentials issued by the remote platform.