Compute Costs

Masthead analyzes BigQuery compute resource consumption and provides recommendations based on the pipelines and resource usage patterns. These recommendations help leverage the most cost-effective BigQuery compute models.

Saving options on Compute Cost Insights page

Alternative Compute Models

BigQuery offers different compute models that impact access to various compute features and directly impact the costs. Masthead analyzes usage to determine the optimal model among On-demand, Standard and Enterprise editions.

Alternative Compute Model for a Project

Masthead analyzes and aggregates all compute resources used within a project to recommend the most cost-effective compute pricing model. We consider overall project usage patterns and potential savings by switching between On-Demand and Editions Standard models.

Technology workload optimization

Various technologies integrated with BigQuery have different attributes of the workload they are running: ETL, BI, reverse ETL, ML, etc. We analyze the workloads of all the popular technologies to identify the best compute usage resources for each of them.

Pipeline-specific Compute Cost Optimizations

Masthead provides a granular view of compute costs, grouped by pipeline. This allows to optimize costs at the pipeline level by considering moving them under BigQuery Standard Editions model based on slot-consumption. Filter by labels to analyze specific workloads.

Recommendation

Review pipeline-specific recommendations on Compute Costs page and consider adjusting pipeline compute configurations.

Alternative model plan for a pipeline

Read more about combining different compute billing models.

Pipelines with dead-end downstream

Masthead identifies and tags tables with no downstream consumption as 'Dead-End Tables'. This allows you to then identify running upstream pipelines that only produce these tables. Review and reclaim resources by switching off such pipelines.

Recommendation

  1. Check recommendations for pipelines with dead-end tables on Costs Insights page:

    Pipelines with dead-end tables

  2. Verify the tables have no consumers downstream on the Dictionary and Lineage pages.

  3. Adjust or disable upstream pipelines to reduce unnecessary compute spend.

Last updated