Skip to content

Latest commit

 

History

History
88 lines (65 loc) · 3.92 KB

README.md

File metadata and controls

88 lines (65 loc) · 3.92 KB
Raw

Data Playground

This blueprint creates a minimum viable architecture for a data experimentation project with the needed APIs enabled, VPC and Firewall set in place, BigQuery dataset, GCS bucket and an AI notebook to get started.

This is the high level diagram:

High-level diagram

Managed resources and services

This sample creates several distinct groups of resources:

  • project
  • networking
  • Vertex AI Workbench notebook configured with a private IP and using a dedicated Service Account
  • One GCS bucket
  • One BigQuery dataset

Virtual Private Cloud (VPC) design

As is often the case in real-world configurations, this blueprint accepts as input an existing Shared-VPC via the network_config variable. Make sure that 'container.googleapis.com', 'notebooks.googleapis.com' and 'servicenetworking.googleapis.com' are enabled in the VPC host project.

If the network_config variable is not provided, one VPC will be created in each project that supports network resources (load, transformation and orchestration).

Deploy your environment

We assume the identity running the following steps has the following role:

  • resourcemanager.projectCreator in case a new project will be created.
  • owner on the project in case you use an existing project.

Run Terraform init:

terraform init

Configure the Terraform variable in your terraform.tfvars file. You need to specify at least the following variables:

prefix = "prefix"
project_id      = "data-001"

You can run now:

terraform apply

You can now connect to the Vertex AI notbook to perform your data analysis.

Variables

name description type required default
prefix Prefix used for resource names. string ✓
project_id Project id, references existing project if project_create is null. string ✓
deletion_protection Prevent Terraform from destroying data storage resources (storage buckets, GKE clusters, CloudSQL instances) in this blueprint. When this field is set in Terraform state, a terraform destroy or terraform apply that would delete data storage resources will fail. bool false
location The location where resources will be deployed. string "EU"
network_config Shared VPC network configurations to use. If null networks will be created in projects with preconfigured values. object({…}) null
project_create Provide values if project creation is needed, uses existing project if null. Parent format: folders/folder_id or organizations/org_id. object({…}) null
region The region where resources will be deployed. string "europe-west1"

Outputs

name description sensitive
bucket GCS Bucket URL.
dataset GCS Bucket URL.
notebook Vertex AI notebook details.
project Project id.
vpc VPC Network.

Test

module "test" {
  source     = "./fabric/blueprints/data-solutions/data-playground"
  project_id = "sampleproject"
  prefix     = "tst"
  project_create = {
    billing_account_id = "123456-123456-123456",
    parent             = "folders/467898377"
  }
}
# tftest modules=8 resources=66