Are you transitioning from a legacy ecosystem or managing multiple data ecosystems with different technology stacks? You might find yourself dealing with a data lake created using a File-Based CDC solution. In such cases, optimising workloads on BigQuery can be key!
In this video, I’ll demonstrate a simple, low-maintenance solution for efficiently getting your data into BigQuery and maximising the benefits of native tables. I’ll also explain why federated tables might not be the best option compared to native BigQuery tables.
Hope you find it insightful! #BigQueryStorageTransfer #BigQuery #DataLake #TechSolutions #CloudComputing #DataOptimisation #PracticalGCP
01:33 - What is a File Based CDC solution
02:43 - Why BigQuery is relevant here
04:13 - Federated query vs native query
06:33 - The low maintenance BigQuery Transfer Service solution
09:34 - Demo
18:13 - Summary
Deck: docs.google.com/presentation/d/10St8MdzanWrP_30U-RPeZGUUQmX4gXeVc2-dHnOvvOE
Code: github.com/rocketechgroup/parquet_to_bigquery.git
Негізгі бет Ғылым және технология Use BigQuery Transfer Service to Optimise a File Based CDC
Пікірлер: 6