partition and clustering – BigQuery
Vishal Jadhav Limitations maximum 8000 partitions possible on a single table and upto 4 columns can be used for clustering Partition can be done on Date, DATETIME, TIMESTAMP OR INTERGER RANGE COLUMN.…
Data Engineering
Time travel – BigQuery
what is time travel in bigquery? Using big-query’s time travel concept we can access any modified(updated or delete) data. we can access this data till last 7 days. This concept…
Difference Between WHERE and HAVING Clause in sql
HAVING Clause is used to filter the records from the groups based on the given condition. HAVING clause alwaysed used after
What is a federated query in BigQuery?Wha
Federated queries are used to query the data on external sources like Cloud Storage, BigTable, CloudSql, Spreadsheet in google Drive. Federated queries or have queries that access data that isn’t…
BigTable – Tall tables vs narro Tables
Bigtable is a NoSQL database service provided by Google Cloud Platform (GCP), designed to handle massive-scale, high-throughput workloads. While it’s commonly associated with storing and processing large amounts of data,…
Ultimate Guide to BigQuery Slots: Boosting Performance and Managing Costs Efficiently
BigQuery slots are a fundamental aspect of Google Cloud’s BigQuery service, which is a fully managed, serverless, highly scalable, and cost-effective multi-cloud data warehouse. Slots represent the computational power required…
GCP Dataflow – Apache beam
Most important concept In the context of data stream processing and windowing, these terms—tumbling windows, hopping windows, session windows, and global windows—describe different ways to group and analyze continuous streams…
Comprehensive Guide to Windowing Functions in Data Stream Processing: Tumbling, Hopping, Sliding, Session, and Global Windows Explained
windowing in stream processing
BigQuery – Anonymous dataset
BigQuery writes all query results to a table. The table is either explicitly identified by the user (a destination table), or it is a temporary, cached results table. Temporary, cached…
problem with to many partition in database or storage systems
Having too many partitions in a system, especially in a data storage context like a database or a distributed file system, can lead to several issues.Here’s a breakdown of the…