Skip to content

Commit

Permalink
Deprecate cloud-bench
Browse files Browse the repository at this point in the history
  • Loading branch information
nkraemer-sysdig committed Nov 5, 2024
1 parent bd56fe9 commit 43ebbfd
Show file tree
Hide file tree
Showing 49 changed files with 28 additions and 1,047 deletions.
3 changes: 0 additions & 3 deletions CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,4 +1 @@
* @sysdiglabs/cloud-native

# compliance
/modules/services/cloud-bench/ @haresh-suresh @nkraemer-sysdig @sysdiglabs/cloud-native
1 change: 0 additions & 1 deletion CONTRIBUTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
- Use **conventional commits** | https://www.conventionalcommits.org/en/v1.0.0
- Current suggested **scopes** to be used within feat(scope), fix(scope), ...
- threat
- bench
- scan
- docs
- tests
Expand Down
86 changes: 1 addition & 85 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,11 @@
Terraform module that deploys the [**Sysdig Secure for Cloud** stack in **Google Cloud**](https://docs.sysdig.com/en/docs/installation/sysdig-secure-for-cloud/deploy-sysdig-secure-for-cloud-on-gcp/).
<br/>

Provides unified threat-detection, compliance, forensics and analysis through these major components:
Provides unified threat-detection, forensics and analysis through these major components:


* **[Threat Detection](https://docs.sysdig.com/en/docs/sysdig-secure/insights/)**: Tracks abnormal and suspicious activities in your cloud environment based on Falco language. Managed through `cloud-connector` module. <br/>

* **[Compliance](https://docs.sysdig.com/en/docs/sysdig-secure/posture/compliance/compliance-unified-/)**: Enables the evaluation of standard compliance frameworks. Requires both modules `cloud-connector` and `cloud-bench`. <br/>

* **[Image Scanning](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/)**: Automatically scans all container images pushed to the registry (GCR) and the images that run on the GCP workload (currently CloudRun). Managed through `cloud-connector`. <br/>Disabled by Default, can be enabled through `deploy_scanning` input variable parameters.<br/>

For other Cloud providers check: [AWS](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud), [Azure](https://github.com/sysdiglabs/terraform-azurerm-secure-for-cloud)
Expand Down Expand Up @@ -84,14 +82,6 @@ Besides, the following GCP **APIs must be enabled** ([how do I check it?](#q-how
* [Cloud Build API](https://console.cloud.google.com/marketplace/product/google/cloudbuild.googleapis.com)
* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)

##### Cloud Benchmarks
* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)
* [IAM Service Account Credentials API](https://console.cloud.google.com/marketplace/product/google/iamcredentials.googleapis.com)
* [Cloud Resource Manager API](https://console.cloud.google.com/marketplace/product/google/cloudresourcemanager.googleapis.com)
* [Security Token Service API](https://console.cloud.google.com/marketplace/product/google/sts.googleapis.com)
* [Cloud Asset API](https://console.cloud.google.com/marketplace/product/google/cloudasset.googleapis.com)


<br/>

## Confirm the Services are Working
Expand Down Expand Up @@ -155,29 +145,6 @@ output "me" {
}
```
### Q: In organizaitonal setup, Compliance trust-relationship is not being deployed on our projects
As for 2023 April, organizations with projects under organizational unit folders, is supported with the
[organizational compliance example](./examples/organization-org_compliance)
<br/>S: If you want to target specific projects, you can still use the `benchmark_project_ids` parameter so you can define
the projects where compliance role is to be deployed explicitly.
<br/>You can use the [fetch-gcp-rojects.sh](./resources/fetch-gcp-projects.sh) utility to list organization member projects
<br/>Let us know if this workaround won't be enough, and we will work on implementing a solution.
### Q: Compliance is not working. How can I check everything is properly setup
A: On your GCP infrastructure, per-project where Comliance has been setup, check following points<br/>
1. there is a Workload Identity Pool and associated Workload Identity Pool Provider configured, which must have an ID of `sysdigcloud` (display name doesn't matter)
2. the pool should have a connected service account with the name `sfcsysdigcloudbench`, with the email `[email protected]`
3. this serviceaccount should allow access to the following format `principalset: principalSet://iam.googleapis.com/projects/<PROJECTID>/locations/global/workloadIdentityPools/sysdigcloud/attribute.aws_role/arn:aws:sts::***:assumed-role/***`
4. the serviceaccount should have the `viewer role` on the target project, as well as a custom role containing the "storage.buckets.getIamPolicy", "bigquery.tables.list", "cloudasset.assets.listIamPolicy" and "cloudasset.assets.listResource" permissions
5. the pool provider should allow access to Sysdig's trusted identity, retrieved through
```
$ curl https://<SYSDIG_SECURE_URL>/api/cloud/v2/gcp/trustedIdentity \
--header 'Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>'
```
### Q: Getting "Error creating Service: googleapi: got HTTP response code 404" "The requested URL /serving.knative.dev/v1/namespaces/***/services was not found on this server"
```
Expand All @@ -193,57 +160,6 @@ A: This error is given by the Terraform GCP provider when an invalid region is u
### Q: Error because it cannot resolve the address below, "https://-run.googleapis.com/apis/serving.knative.dev"
A: GCP region was not provided in the provider block
### Q: Why do we need `google-beta` provider?
A: Some resources we use, such as the [`google_iam_workload_identity_pool_provider`](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/iam_workload_identity_pool_provider) are only available in the beta version.<br/>
### Q: Getting "Error creating WorkloadIdentityPool: googleapi: Error 409: Requested entity already exists"<br/>
A: Currently Sysdig Backend does not support dynamic WorkloadPool and it's name is fixed to `sysdigcloud`.
<br/>Moreover, Google, only performs a soft-deletion of this resource.
https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#delete-pool
> You can undelete a pool for up to 30 days after deletion. After 30 days, deletion is permanent. Until a pool is permanently deleted, you cannot reuse its name when creating a new workload identity pool.<br/>
<br/>S: For the moment, federation workload identity pool+provider have fixed name.
Therea are several options here
- For single-account, in case you want to reuse it, you can make use of the `reuse_workload_identity_pool` attribute available in some
examples.
- For organizational setups, you can make use of a single workload-identity for all the organization, with the [/organization-org_compliance](./examples/organization-org_compliance)
- Alternatively, you can reactivate and import it, into your terraform state manually.
```bash
# re-activate pool and provider
$ gcloud iam workload-identity-pools undelete sysdigcloud --location=global
$ gcloud iam workload-identity-pools providers undelete sysdigcloud --workload-identity-pool="sysdigcloud" --location=global
# import to terraform state
# for this you have to adapt the import resource to your specific usage
# ex.: for single-project, input your project-id
$ terraform import 'module.secure-for-cloud_example_single-project.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool.pool' <PROJECT_ID>/sysdigcloud
$ terraform import 'module.secure-for-cloud_example_single-project.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool_provider.pool_provider' <PROJECT_ID>/sysdigcloud/sysdigcloud
# ex.: for organization example you should change its reference too, per project
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool.pool' <PROJECT_ID>/sysdigcloud
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool_provider.pool_provider' <PROJECT_ID>/sysdigcloud/sysdigcloud
```

The import resource to use, is the one pointed out in your terraform plan/apply error messsage
```
-- for
Error: Error creating WorkloadIdentityPool: googleapi: Error 409: Requested entity already exists
with module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool,
on .... in resource "google_iam_workload_identity_pool" "pool":
resource "google_iam_workload_identity_pool" "pool" {
-- use
' module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool' as your import resource
-- such as
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool' 'org-child-project-1/sysdigcloud'
```

Note: if you're using terragrunt, run `terragrunt import`

### Q: Getting "Error creating Topic: googleapi: Error 409: Resource already exists in the project (resource=gcr)"
```text
│ Error: Error creating Topic: googleapi: Error 409: Resource already exists in the project (resource=gcr).
Expand Down
45 changes: 5 additions & 40 deletions examples/organization-org_compliance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,12 @@ This example deploys Secure for Cloud into a GCP organizational account.
## Prerequisites

1. Configure [Terraform **GCP** Provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs)
2. Run the script at `resources/fetch-gcp-projects.sh <organization_ID>`. copy the output and provide it as input in the module
as benchmark_project_ids. e.g benchmark_project_ids = ["id1","id2"]. This script provides list of
all projects under folders and subfolders under an organization. If you don't provide this list
by default only those projects are selected which are directly under the org.
3. To find your organization id please visit https://cloud.google.com/resource-manager/reference/rest/v1/projects/getAncestry
4. Following **roles** are required in your GCP organization/project credentials
2. To find your organization id please visit https://cloud.google.com/resource-manager/reference/rest/v1/projects/getAncestry
3. Following **roles** are required in your GCP organization/project credentials
* _Owner_
* _Organization Admin_
* _Organization ID_
5. Besides, the following GCP **APIs must be enabled** to deploy resources correctly for:
4. Besides, the following GCP **APIs must be enabled** to deploy resources correctly for:

### Cloud Connector

Expand All @@ -41,13 +37,6 @@ This example deploys Secure for Cloud into a GCP organizational account.
* [Cloud Build API](https://console.cloud.google.com/marketplace/product/google/cloudbuild.googleapis.com)
* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)

### Cloud Benchmarks

* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)
* [IAM Service Account Credentials API](https://console.cloud.google.com/marketplace/product/google/iamcredentials.googleapis.com)
* [Cloud Resource Manager API](https://console.cloud.google.com/marketplace/product/google/cloudresourcemanager.googleapis.com)
* [Security Token Service API](https://console.cloud.google.com/marketplace/product/google/sts.googleapis.com)


## Usage

Expand Down Expand Up @@ -83,21 +72,7 @@ provider "google" {
region = "<REGION_ID>; ex. us-central1"
}
provider "google" {
alias = "multiproject"
region = "<REGION_ID>; ex. us-central1"
}
provider "google-beta" {
alias = "multiproject"
region = "<REGION_ID>; ex. us-central1"
}
module "secure-for-cloud_example_organization" {
providers = {
google.multiproject = google.multiproject
google-beta.multiproject = google-beta.multiproject
}
source = "sysdiglabs/secure-for-cloud/google//examples/organization-org_compliance"
organization_domain = "<ORG_DOMAIN>"
Expand All @@ -113,21 +88,19 @@ module "secure-for-cloud_example_organization" {
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 4.21.0, < 5.0.0 |
| <a name="requirement_google-beta"></a> [google-beta](#requirement\_google-beta) | >= 4.21.0, < 5.0.0 |
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.46 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_google"></a> [google](#provider\_google) | >= 4.21.0, < 5.0.0 |
| <a name="provider_sysdig"></a> [sysdig](#provider\_sysdig) | >= 0.5.46 |
| <a name="provider_google"></a> [google](#provider\_google) | 4.58.0 |
| <a name="provider_sysdig"></a> [sysdig](#provider\_sysdig) | 0.7.4 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloud_bench_workload_identity"></a> [cloud\_bench\_workload\_identity](#module\_cloud\_bench\_workload\_identity) | ../../modules/services/cloud-bench-workload-identity | n/a |
| <a name="module_cloud_build_permission"></a> [cloud\_build\_permission](#module\_cloud\_build\_permission) | ../../modules/infrastructure/cloud_build_permission | n/a |
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
| <a name="module_connector_organization_sink"></a> [connector\_organization\_sink](#module\_connector\_organization\_sink) | ../../modules/infrastructure/organization_sink | n/a |
Expand All @@ -151,9 +124,6 @@ module "secure-for-cloud_example_organization" {
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_organization_domain"></a> [organization\_domain](#input\_organization\_domain) | Organization domain. e.g. sysdig.com | `string` | n/a | yes |
| <a name="input_benchmark_project_ids"></a> [benchmark\_project\_ids](#input\_benchmark\_project\_ids) | Google cloud project IDs to run Benchmarks on. It will create a trust-relationship on each, to allow Sysdig usage. If empty, all organization projects will be defaulted. | `list(string)` | `[]` | no |
| <a name="input_benchmark_role_name"></a> [benchmark\_role\_name](#input\_benchmark\_role\_name) | The name of the Service Account that will be created. | `string` | `"sysdigcloudbench"` | no |
| <a name="input_deploy_benchmark"></a> [deploy\_benchmark](#input\_deploy\_benchmark) | whether benchmark module is to be deployed | `bool` | `true` | no |
| <a name="input_deploy_scanning"></a> [deploy\_scanning](#input\_deploy\_scanning) | true/false whether scanning module is to be deployed | `bool` | `false` | no |
| <a name="input_max_instances"></a> [max\_instances](#input\_max\_instances) | Max number of instances for the workloads | `number` | `1` | no |
| <a name="input_name"></a> [name](#input\_name) | Name to be assigned to all child resources. A suffix may be added internally when required. Use default value unless you need to install multiple instances | `string` | `"sfc"` | no |
Expand All @@ -179,7 +149,6 @@ Apache 2 Licensed. See LICENSE for full details.
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 4.21.0 |
| <a name="requirement_google-beta"></a> [google-beta](#requirement\_google-beta) | >= 4.21.0 |
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.46 |

## Providers
Expand All @@ -193,7 +162,6 @@ Apache 2 Licensed. See LICENSE for full details.

| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloud_bench_workload_identity"></a> [cloud\_bench\_workload\_identity](#module\_cloud\_bench\_workload\_identity) | ../../modules/services/cloud-bench-workload-identity | n/a |
| <a name="module_cloud_build_permission"></a> [cloud\_build\_permission](#module\_cloud\_build\_permission) | ../../modules/infrastructure/cloud_build_permission | n/a |
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
| <a name="module_connector_organization_sink"></a> [connector\_organization\_sink](#module\_connector\_organization\_sink) | ../../modules/infrastructure/organization_sink | n/a |
Expand All @@ -216,9 +184,6 @@ Apache 2 Licensed. See LICENSE for full details.

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_benchmark_project_ids"></a> [benchmark\_project\_ids](#input\_benchmark\_project\_ids) | Google cloud project IDs to run Benchmarks on. It will create a trust-relationship on each, to allow Sysdig usage. If empty, all organization projects will be defaulted. | `list(string)` | `[]` | no |
| <a name="input_benchmark_role_name"></a> [benchmark\_role\_name](#input\_benchmark\_role\_name) | The name of the Service Account that will be created. | `string` | `"sysdigcloudbench"` | no |
| <a name="input_deploy_benchmark"></a> [deploy\_benchmark](#input\_deploy\_benchmark) | whether benchmark module is to be deployed | `bool` | `true` | no |
| <a name="input_deploy_scanning"></a> [deploy\_scanning](#input\_deploy\_scanning) | true/false whether scanning module is to be deployed | `bool` | `false` | no |
| <a name="input_max_instances"></a> [max\_instances](#input\_max\_instances) | Max number of instances for the workloads | `number` | `1` | no |
| <a name="input_name"></a> [name](#input\_name) | Name to be assigned to all child resources. A suffix may be added internally when required. Use default value unless you need to install multiple instances | `string` | `"sfc"` | no |
Expand Down
19 changes: 0 additions & 19 deletions examples/organization-org_compliance/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -112,22 +112,3 @@ module "pubsub_http_subscription" {
push_to_cloudrun = true
deploy_scanning = var.deploy_scanning
}


#--------------------
# benchmark
#--------------------
module "cloud_bench_workload_identity" {
providers = {
google = google.multiproject
google-beta = google-beta.multiproject
}

count = var.deploy_benchmark ? 1 : 0
source = "../../modules/services/cloud-bench-workload-identity"

organization_domain = var.organization_domain
role_name = var.benchmark_role_name
project_ids = var.benchmark_project_ids
project_id = data.google_client_config.current.project
}
21 changes: 0 additions & 21 deletions examples/organization-org_compliance/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,6 @@ variable "repository_project_ids" {
description = "Projects were a `gcr`-named topic will be to subscribe to its repository events. If empty, all organization projects will be defaulted."
}

#
# benchmark
#
variable "deploy_benchmark" {
type = bool
description = "whether benchmark module is to be deployed"
default = true
}

variable "benchmark_project_ids" {
default = []
type = list(string)
description = "Google cloud project IDs to run Benchmarks on. It will create a trust-relationship on each, to allow Sysdig usage. If empty, all organization projects will be defaulted."
}

variable "benchmark_role_name" {
type = string
description = "The name of the Service Account that will be created."
default = "sysdigcloudbench"
}

#
# general
#
Expand Down
10 changes: 2 additions & 8 deletions examples/organization-org_compliance/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,8 @@ terraform {

required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.21.0, < 5.0.0"
configuration_aliases = [google.multiproject]
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.21.0, < 5.0.0"
configuration_aliases = [google-beta.multiproject]
source = "hashicorp/google"
version = ">= 4.21.0, < 5.0.0"
}
sysdig = {
source = "sysdiglabs/sysdig"
Expand Down
Loading

0 comments on commit 43ebbfd

Please sign in to comment.