Skip to content

Commit

Permalink
Merge branch 'master' into semgrep-dependencies
Browse files Browse the repository at this point in the history
  • Loading branch information
hanzo committed Oct 29, 2024
2 parents 678a3aa + 5fcbe83 commit db7b784
Show file tree
Hide file tree
Showing 41 changed files with 82 additions and 84 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ If you are changing a node or relationship:
- [ ] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md).

If you are implementing a new intel module:
- [ ] Use the NodeSchema [data model](https://lyft.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).
- [ ] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2019 Lyft, Inc.
Copyright 2024 The Linux Foundation

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
4 changes: 0 additions & 4 deletions NOTICE

This file was deleted.

67 changes: 35 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,30 @@ Cartography aims to enable a broad set of exploration and automation scenarios.

Service owners can generate asset reports, Red Teamers can discover attack paths, and Blue Teamers can identify areas for security improvement. All can benefit from using the graph for manual exploration through a web frontend interface, or in an automated fashion by calling the APIs.

Cartography is not the only [security](https://github.com/dowjones/hammer) [graph](https://github.com/BloodHoundAD/BloodHound) [tool](https://github.com/Netflix/security_monkey) [out](https://github.com/vysecurity/ANGRYPUPPY) [there](https://github.com/duo-labs/cloudmapper), but it differentiates itself by being fully-featured yet generic and [extensible](https://lyft.github.io/cartography/dev/writing-analysis-jobs.html) enough to help make anyone better understand their risk exposure, regardless of what platforms they use. Rather than being focused on one core scenario or attack vector like the other linked tools, Cartography focuses on flexibility and exploration.
Cartography is not the only [security](https://github.com/dowjones/hammer) [graph](https://github.com/BloodHoundAD/BloodHound) [tool](https://github.com/Netflix/security_monkey) [out](https://github.com/vysecurity/ANGRYPUPPY) [there](https://github.com/duo-labs/cloudmapper), but it differentiates itself by being fully-featured yet generic and [extensible](https://cartography-cncf.github.io/cartography/dev/writing-analysis-jobs.html) enough to help make anyone better understand their risk exposure, regardless of what platforms they use. Rather than being focused on one core scenario or attack vector like the other linked tools, Cartography focuses on flexibility and exploration.

You can learn more about the story behind Cartography in our [presentation at BSidesSF 2019](https://www.youtube.com/watch?v=ZukUmZSKSek).


## Supported platforms

- [Amazon Web Services](https://lyft.github.io/cartography/modules/aws/index.html) - API Gateway, Config, EC2, ECS, ECR, Elasticsearch, Elastic Kubernetes Service (EKS), DynamoDB, IAM, Inspector, KMS, Lambda, RDS, Redshift, Route53, S3, Secrets Manager, Security Hub, SQS, SSM, STS, Tags
- [Google Cloud Platform](https://lyft.github.io/cartography/modules/gcp/index.html) - Cloud Resource Manager, Compute, DNS, Storage, Google Kubernetes Engine
- [Google GSuite](https://lyft.github.io/cartography/modules/gsuite/index.html) - users, groups
- [Amazon Web Services](https://cartography-cncf.github.io/cartography/modules/aws/index.html) - API Gateway, Config, EC2, ECS, ECR, Elasticsearch, Elastic Kubernetes Service (EKS), DynamoDB, IAM, Inspector, KMS, Lambda, RDS, Redshift, Route53, S3, Secrets Manager, Security Hub, SQS, SSM, STS, Tags
- [Google Cloud Platform](https://cartography-cncf.github.io/cartography/modules/gcp/index.html) - Cloud Resource Manager, Compute, DNS, Storage, Google Kubernetes Engine
- [Google GSuite](https://cartography-cncf.github.io/cartography/modules/gsuite/index.html) - users, groups
- [Oracle Cloud Infrastructure](docs/setup/config/oci.md) - IAM
- [Okta](https://lyft.github.io/cartography/modules/okta/index.html) - users, groups, organizations, roles, applications, factors, trusted origins, reply URIs
- [Github](https://lyft.github.io/cartography/modules/github/index.html) - repos, branches, users, teams
- [DigitalOcean](https://lyft.github.io/cartography/modules/digitalocean/index.html)
- [Microsoft Azure](https://lyft.github.io/cartography/modules/azure/index.html) - CosmosDB, SQL, Storage, Virtual Machine
- [Kubernetes](https://lyft.github.io/cartography/modules/kubernetes/index.html) - Cluster, Namespace, Service, Pod, Container
- [PagerDuty](https://lyft.github.io/cartography/modules/pagerduty/index.html) - Users, teams, services, schedules, escalation policies, integrations, vendors
- [Crowdstrike Falcon](https://lyft.github.io/cartography/modules/crowdstrike/index.html) - Hosts, Spotlight vulnerabilities, CVEs
- [NIST CVE](https://lyft.github.io/cartography/modules/cve/index.html) - Common Vulnerabilities and Exposures (CVE) data from NIST database
- [Lastpass](https://lyft.github.io/cartography/modules/lastpass/index.html) - users
- [BigFix](https://lyft.github.io/cartography/modules/bigfix/index.html) - Computers
- [Duo](https://lyft.github.io/cartography/modules/duo/index.html) - Users, Groups, Endpoints
- [Kandji](https://lyft.github.io/cartography/modules/kandji/index.html) - Devices
- [SnipeIT](https://lyft.github.io/cartography/modules/snipeit/index.html) - Users, Assets
- [Okta](https://cartography-cncf.github.io/cartography/modules/okta/index.html) - users, groups, organizations, roles, applications, factors, trusted origins, reply URIs
- [GitHub](https://cartography-cncf.github.io/cartography/modules/github/index.html) - repos, branches, users, teams
- [DigitalOcean](https://cartography-cncf.github.io/cartography/modules/digitalocean/index.html)
- [Microsoft Azure](https://cartography-cncf.github.io/cartography/modules/azure/index.html) - CosmosDB, SQL, Storage, Virtual Machine
- [Kubernetes](https://cartography-cncf.github.io/cartography/modules/kubernetes/index.html) - Cluster, Namespace, Service, Pod, Container
- [PagerDuty](https://cartography-cncf.github.io/cartography/modules/pagerduty/index.html) - Users, teams, services, schedules, escalation policies, integrations, vendors
- [Crowdstrike Falcon](https://cartography-cncf.github.io/cartography/modules/crowdstrike/index.html) - Hosts, Spotlight vulnerabilities, CVEs
- [NIST CVE](https://cartography-cncf.github.io/cartography/modules/cve/index.html) - Common Vulnerabilities and Exposures (CVE) data from NIST database
- [Lastpass](https://cartography-cncf.github.io/cartography/modules/lastpass/index.html) - users
- [BigFix](https://cartography-cncf.github.io/cartography/modules/bigfix/index.html) - Computers
- [Duo](https://cartography-cncf.github.io/cartography/modules/duo/index.html) - Users, Groups, Endpoints
- [Kandji](https://cartography-cncf.github.io/cartography/modules/kandji/index.html) - Devices
- [SnipeIT](https://cartography-cncf.github.io/cartography/modules/snipeit/index.html) - Users, Assets


## Philosophy
Expand All @@ -54,28 +54,28 @@ Here are some points that can help you decide if adopting Cartography is a good
- Cartography is not designed for very fast updates. Cartography writes to the database in a batches (not streamed).
- Cartography is also limited by how most upstream sources only provide APIs to retrieve assets in a batched manner.
- By itself, Cartography does not capture data changes over time.
- Although we do include a [drift detection](https://lyft.github.io/cartography/usage/drift-detect.html) feature.
- Although we do include a [drift detection](https://cartography-cncf.github.io/cartography/usage/drift-detect.html) feature.
- It's also possible to implement other processes in your Cartography installation to make this happen.


## Install and configure

### Trying out Cartography on a test machine
Start [here](https://lyft.github.io/cartography/install.html) to set up a test graph and get data into it.
Start [here](https://cartography-cncf.github.io/cartography/install.html) to set up a test graph and get data into it.

### Setting up Cartography in production
When you are ready to try it in production, read [here](https://lyft.github.io/cartography/ops.html) for recommendations on getting cartography spun up in your environment.
When you are ready to try it in production, read [here](https://cartography-cncf.github.io/cartography/ops.html) for recommendations on getting cartography spun up in your environment.

## Usage

### Querying the database directly

![poweruser.png](docs/root/images/poweruser.png)

Now that data is in the graph, you can quickly start with our [querying tutorial](https://lyft.github.io/cartography/usage/tutorial.html). Our [data schema](https://lyft.github.io/cartography/usage/schema.html) is a helpful reference when you get stuck.
Now that data is in the graph, you can quickly start with our [querying tutorial](https://cartography-cncf.github.io/cartography/usage/tutorial.html). Our [data schema](https://cartography-cncf.github.io/cartography/usage/schema.html) is a helpful reference when you get stuck.

### Building applications around Cartography
Directly querying Neo4j is already very useful as a sort of "swiss army knife" for security data problems, but you can also build applications and data pipelines around Cartography. View this doc on [applications](https://lyft.github.io/cartography/usage/applications.html).
Directly querying Neo4j is already very useful as a sort of "swiss army knife" for security data problems, but you can also build applications and data pipelines around Cartography. View this doc on [applications](https://cartography-cncf.github.io/cartography/usage/applications.html).


## Community
Expand All @@ -86,34 +86,37 @@ Directly querying Neo4j is already very useful as a sort of "swiss army knife" f
- Recorded videos are posted [here](https://www.youtube.com/playlist?list=PLMga2YJvAGzidUWJB_fnG7EHI4wsDDsE1).
- Our current project roadmap is [here](https://github.com/orgs/lyft/projects/26/views/1).

## License

This project is licensed under the [Apache 2.0 License](LICENSE).

## Contributing
Thank you for considering contributing to Cartography!

### Code of conduct
Legal stuff: This project is governed by [Lyft's code of conduct](https://github.com/lyft/code-of-conduct).
All contributors and participants agree to abide by its terms.
All contributors and participants of this project must follow the [CNCF code of conduct](https://github.com/cncf/foundation/blob/main/code-of-conduct.md).

### Bug reports and feature requests and discussions
Submit a GitHub issue to report a bug or request a new feature. If we decide that the issue needs more discussion - usually because the scope is too large or we need to make careful decision - we will convert the issue to a [GitHub Discussion](https://github.com/lyft/cartography/discussions).

### Developing Cartography

Get started with our [developer documentation](https://lyft.github.io/cartography/dev/developer-guide.html). Please feel free to submit your own PRs to update documentation if you've found a better way to explain something.

#### Sign the Contributor License Agreement (CLA)

We require a CLA for code contributions, so before we can accept a pull request
we need to have a signed CLA. Please [visit our CLA service](https://oss.lyft.com/cla)
and follow the instructions to sign the CLA.
Get started with our [developer documentation](https://cartography-cncf.github.io/cartography/dev/developer-guide.html). Please feel free to submit your own PRs to update documentation if you've found a better way to explain something.

## Who uses Cartography?

1. [Lyft](https://www.lyft.com)
1. [Thought Machine](https://thoughtmachine.net/)
1. [MessageBird](https://messagebird.com)
1. [Cloudanix](https://www.cloudanix.com/)
1. [ZeusCloud](https://www.zeuscloud.io/)
1. [Corelight](https://www.corelight.com/)
1. {Your company here} :-)

If your organization uses Cartography, please file a PR and update this list. Say hi on Slack too!

---

Cartography is a [Cloud Native Computing Foundation](https://www.cncf.io/) sandbox project.<br>
<div style="background-color: white; display: inline-block; padding: 10px;">
<img src="docs/root/images/cncf-color.png" alt="CNCF Logo" width="200">
</div>
2 changes: 1 addition & 1 deletion cartography/driftdetect/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def _build_parser(self):
'graph database and reports the deviations.'
),
epilog='For more documentation please visit: '
'https://lyft.github.io/cartography/usage/drift-detect.html',
'https://cartography-cncf.github.io/cartography/usage/drift-detect.html',
)
parser.add_argument(
'-v',
Expand Down
2 changes: 1 addition & 1 deletion docs/containers/testing-with-docker.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
This document has been moved [here](https://lyft.github.io/cartography/dev/testing-with-docker.html)
This document has been moved [here](https://cartography-cncf.github.io/cartography/dev/testing-with-docker.html)
2 changes: 1 addition & 1 deletion docs/dev/developer-guide.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
This document has been moved [here](https://lyft.github.io/cartography/dev/developer-guide.html)
This document has been moved [here](https://cartography-cncf.github.io/cartography/dev/developer-guide.html)
2 changes: 1 addition & 1 deletion docs/dev/writing-analysis-jobs.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
This document has been moved [here](https://lyft.github.io/cartography/dev/writing-analysis-jobs.html)
This document has been moved [here](https://cartography-cncf.github.io/cartography/dev/writing-analysis-jobs.html)
2 changes: 1 addition & 1 deletion docs/dev/writing-intel-modules.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
This document has been moved [here](https://lyft.github.io/cartography/dev/writing-intel-modules.html)
This document has been moved [here](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html)
Binary file added docs/root/images/cncf-color.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 7 additions & 7 deletions docs/root/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This is the quickest way to get started (assuming docker does what it's supposed

1. **Configure and run Cartography.**

In this example we will run Cartography on [AWS](https://lyft.github.io/cartography/modules/aws/config.html) with a profile called "1234_testprofile" and default region set to "us-east-1".
In this example we will run Cartography on [AWS](https://cartography-cncf.github.io/cartography/modules/aws/config.html) with a profile called "1234_testprofile" and default region set to "us-east-1".

```bash
docker-compose run \
Expand All @@ -40,7 +40,7 @@ This is the quickest way to get started (assuming docker does what it's supposed
**Notes:**
- You can view a full list of Cartography's CLI arguments by running `docker-compose run cartography --help`.
- Also see the configuration section of [each relevant intel module](https://lyft.github.io/cartography/modules) to set up each data source. This generally involves specifying environment variables to cartography, or making a config/credential file on the host available to the container.
- Also see the configuration section of [each relevant intel module](https://cartography-cncf.github.io/cartography/modules) to set up each data source. This generally involves specifying environment variables to cartography, or making a config/credential file on the host available to the container.
- You can pass in environment variables to the cartography container using the docker-compose format like this: `-e VARIABLE1 -e VARIABLE2=value2`.
- You can make files available to the cartography container by editing the volumes in the docker-compose.yml file. See docker-compose documentation on how to do that.
Expand Down Expand Up @@ -114,7 +114,7 @@ Read on to see [other things you can do with Cartography](#things-to-do-next).
1. **Configure and run Cartography.**
See the configuration section of [each relevant intel module](https://lyft.github.io/cartography/modules) to set up each data source. In this example we will use [AWS](https://lyft.github.io/cartography/modules/aws/config.html).
See the configuration section of [each relevant intel module](https://cartography-cncf.github.io/cartography/modules) to set up each data source. In this example we will use [AWS](https://cartography-cncf.github.io/cartography/modules/aws/config.html).
This command runs cartography on an AWS profile called "1234_testprofile" on region us-east-1. We also expose the host machine's ~/.aws directory to ~/var/cartography/.aws in the container so that AWS configs work.

Expand Down Expand Up @@ -189,7 +189,7 @@ Do this if you prefer to install and manage all the dependencies yourself. Carto

1. **Configure your data sources.**

See the configuration section of [each relevant intel module](https://lyft.github.io/cartography/modules) for more details. In this example we will use [AWS](https://lyft.github.io/cartography/modules/aws/config.html).
See the configuration section of [each relevant intel module](https://cartography-cncf.github.io/cartography/modules) for more details. In this example we will use [AWS](https://cartography-cncf.github.io/cartography/modules/aws/config.html).

1. **Run cartography.**

Expand Down Expand Up @@ -229,8 +229,8 @@ Do this if you prefer to install and manage all the dependencies yourself. Carto
## Things to do next
Here's some ideas to get the most out of Cartography:
- [Set up other data providers](https://lyft.github.io/cartography/modules)
- [Set up other data providers](https://cartography-cncf.github.io/cartography/modules)
- View our [Operations Guide](ops.html) for tips on running Cartography in production
- Read our [usage instructions](https://lyft.github.io/cartography/usage/tutorial.html) and [schema](https://lyft.github.io/cartography/usage/schema.html) to learn how to query the graph
- Think of [applications](https://lyft.github.io/cartography/usage/applications.html) to build around it
- Read our [usage instructions](https://cartography-cncf.github.io/cartography/usage/tutorial.html) and [schema](https://cartography-cncf.github.io/cartography/usage/schema.html) to learn how to query the graph
- Think of [applications](https://cartography-cncf.github.io/cartography/usage/applications.html) to build around it
- Consider [writing your own Cartography custom modules](dev/writing-intel-modules.md)
2 changes: 1 addition & 1 deletion docs/root/modules/aws/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ In a nutshell, Cartography uses the [boto3](https://github.com/boto/boto3) libra

### Multiple AWS Account Setup

There are many ways to allow Cartography to pull from more than one AWS account. We can't cover all of them, but we _can_ show you the way we have things set up at Lyft. In this scenario we will assume that you are going to run Cartography on an EC2 instance.
There are many ways to allow Cartography to pull from more than one AWS account. We can't cover all of them, but here's one way that works at Lyft. In this scenario we will assume that you are going to run Cartography on an EC2 instance.

1. Pick one of your AWS accounts to be the "**Hub**" account. This Hub account will pull data from all of your other accounts - we'll call those "**Spoke**" accounts.

Expand Down
4 changes: 2 additions & 2 deletions docs/root/ops.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@ how that process works.

Each sync run has an `update_tag` associated with it,
which is the [Unix timestamp of when the sync started](https://github.com/lyft/cartography/blob/8d60311a10156cd8aa16de7e1fe3e109cc3eca0f/cartography/sync.py#L131-L134).
See our [docs for more details](https://lyft.github.io/cartography/dev/writing-intel-modules.html#handling-cartographys-update_tag).
See our [docs for more details](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#handling-cartographys-update_tag).

### Cleanup jobs

Each node and relationship created or updated during the sync will have their `lastupdated` field set to the
`update_tag`. At the end of a sync run, nodes and relationships with out-of-date `lastupdated` fields are considered
stale and will be deleted via a [cleanup job](https://lyft.github.io/cartography/dev/writing-intel-modules.html#cleanup).
stale and will be deleted via a [cleanup job](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#cleanup).

### Sync frequency

Expand Down
Loading

0 comments on commit db7b784

Please sign in to comment.