Integrate using Google Cloud console

Open the project to integrate with in the Google Cloud console

Select a project where you would like Masthead to monitor you BigQuery resources. Use this project to create all the following cloud resources.

Create Pub/Sub resources

1. Go to Pub/Sub. Click Create Topic.

2. Set Topic ID as masthead-topic. Remove checkbox for Add a default subscription.

1. Navigate into the newly created. Click Create Subscription

   
2. Set Subscription ID as masthead-agent-subscription

3. Select Cloud Pub/Sub topic as masthead-topic, then scroll down a little ...

4. Set Acknowledgement deadline as 60 seconds

   
5. Click Create and we are all done with Pub/Sub topic and Subscription.

Create Logs Router

1. Go to Logging -> Log router

2. Click Create sink at the of the page and fill in the required fields:

   

   1. Sink name: masthead-agent-sink

   2. Sink destination: Cloud Pub/Sub topic

   3. Select a newly created Pub/Sub topic: masthead-topic

1. Choose logs to include in the sink: copy the filter text below and paste it into the field.

protoPayload.methodName="google.cloud.bigquery.storage.v1.BigQueryWrite.AppendRows" OR "google.cloud.bigquery.v2.JobService.InsertJob" 
OR "google.cloud.bigquery.v2.TableService.InsertTable" 
OR "google.cloud.bigquery.v2.JobService.Query" 
resource.type ="bigquery_table" OR resource.type ="bigquery_dataset" OR resource.type ="bigquery_project"

1. Click Create Sink

Grant roles to Masthead Service Account

1. Go to IAM & Admin -> IAM

2. Click on Grant access

3. In the New principals field enter [email protected]

4. Click on Select a role and enter BigQuery Metadata Viewer. Select BigQuery Metadata Viewer role.

5. Repeat for BigQuery Resource Viewer and Pub/Sub Subscriber .

6. Click Save

Grant Masthead Service Account to get insights quickly using retrospective metadata

Masthead can provide you with insights within a few hours after integration by parsing retrospective logs and creating a data model around them.

This step enables Masthead to look up recently produced events in the Google Cloud logs that correspond to the Cloud Logs sink filter.

1. Go to IAM & Admin -> IAM

2. Click on Grant access

3. In the New principals field enter [email protected]

4. Click on Select a role and enter Private Logs Viewer.

5. Select Private Logs Viewer role.

6. Click Save

Continue integration on the UI

Go back to Deployment page, click Verify custom deployment and enter the project ID.

Click Check permissions and connect.

Configuration Completed

Masthead installation will create your workspace and open your dashboard once setup is completed.

Workspace is ready!

You'll receive an email notification with the link to the workspace. We'll notify you once all the metadata is collected and the insights are ready.

All done! Thank you for completing installation!