Costs Recommendations

Masthead's Cost Recommendations identify actionable optimizations to improve your cloud spend and ROI for your data products.

The costs are estimates based on past usage. The costs saving estimates are intended to help to compare and prioritize cost saving recommendations. Credits or discounts aren't included neither in costs nor savings calculated.

Compute Costs Recommendations

Masthead analyzes your BigQuery compute resource consumption and provides recommendations based on your pipelines and resource usage patterns. These recommendations help you leverage the most cost-effective BigQuery compute billing models (e.g. on-demand or capacity pay-as-you-go).

Compute Billing Models: BigQuery offers different compute billing models that impact costs. Masthead analyzes usage to determine the optimal model for your needs. More details in Google Cloud documentation.

Masthead sees switching between On-Demand and BigQuery Editions Standard (pay-as-you-go) as the most common and biggest cost optimization opportunity among our customers. We currently focus on this type of analysis and recommendation in the product. Please contact us if you would like to hear our advice on further compute models alternatives and opportunities.

Check recommendations in your dashboard:

Costs recommendations on Dashboard page

Alternative Pricing for a Project

Masthead analyzes and aggregates all compute resources used within your project to recommend the most cost-effective compute pricing model. We consider overall project usage patterns and potential savings by switching between On-Demand and Editions Standard models.

Check your overall compute usage and see your project level recommendation on Costs Insights page.

Pipeline-Specific Compute Cost Optimization

Masthead provides a granular view of your compute costs, grouped by pipeline. This allows you to optimize costs at the pipeline level by considering moving them under BigQuery Standard Editions model based on slot-consumption. You can filter by labels to analyze specific workloads.

Recommendation: Review pipeline-specific recommendations on the Costs Insights page and consider adjusting pipeline compute configurations.

Read more about combining different compute billing models.

Check pipeline cost recommendations on Costs Insights page:

Alternative model plan for pipelines

Optimize Compute Costs for Dead-end Tables

Masthead identifies tables with no downstream consumption, tagging them as "Dead-End Tables". This allows you to analyze related pipelines and reclaim resources.

Recommendation:

1. Review compute usage associated with dead-end tables.

2. Verify the tables have no consumers downstream

3. Adjust or disable related pipelines to reduce unnecessary compute spend.

Check recommendations for pipelines with dead-end tables on Costs Insights page:

Pipelines with dead-end tables

Storage Costs Recommendations

BigQuery provides different storage billing options (Logical and Physical) that impact costs based on schema, values, and write patterns. Masthead analyzes resources metadata and usage logs to estimate costs for each billing model.

Alternative Pricing Model for a Dataset

Masthead collects and analyzes BigQuery resource metadata and usage logs to estimate the cost for each of the billing models and shows you the recommendations to save your storage cost. These recommendations involve switching between BigQuery storage models to optimize for either logical or physical bytes, depending on your data characteristics.

Recommendation:

1. Review storage billing recommendations on the Storage Insights page.

2. Update dataset storage configurations for cost savings.

These configuration changes have no impact on pipeline performance.

Read more about switching between storage billing models.

Check the dataset storage billing model recommendations on Storage Insights page:

Dataset storage billing model recommendations