Google Cloud announced a borderless Lakehouse architecture for BigQuery that uses open Apache Iceberg REST catalogs to query data across Databricks, Snowflake, and AWS Glue in place.
In-Place Cross-Cloud Querying
Google Cloud introduced a borderless Lakehouse architecture for BigQuery designed to eliminate complex data extraction pipelines. According to Google Cloud, the architecture uses the open Apache Iceberg REST catalog specification to let teams query data in place across Databricks, Snowflake, and AWS Glue without migrating data.
The system separates metadata synchronization from storage access. A Lakehouse runtime catalog periodically synchronizes schemas, snapshot pointers, and table definitions, while compatible engines read Parquet data files directly from remote object storage during query execution.
Conversational Analytics and Configuration
Google Cloud stated that federated Iceberg tables behave like native tables within BigQuery, allowing downstream analytical features to inherit access. Users can point conversational analytics at these federated schemas to explore consolidated cross-cloud data using natural language prompts.
The architecture requires explicit configuration across platforms, including creating federated catalogs via the gcloud command-line tool, setting refresh intervals, and granting permissions within Databricks, Snowflake, and AWS environments.
Key facts and where they come from
- BigQuery and Google Cloud's borderless Lakehouse utilize the open Apache Iceberg REST catalog specification for cross-cloud querying.
BigQuery and Google Cloud’s borderless Lakehouse offer a streamlined way to solve this challenge with a foundation based on the open Apache Iceberg REST catalog specification.
- The architecture separates metadata synchronization from storage access to enable in-place queries across external catalogs.
This solution uses Lakehouse runtime catalog to synchronize metadata from other cloud providers, allowing you to query data with BigQuery or external open source engines through the Apache Iceberg REST catalog endpoint
- Federated Iceberg tables in BigQuery support conversational analytics for natural language data exploration.
use conversational analytics in BigQuery to explore the consolidated data with natural language.
