Create a new post for creating a new database integration with data prepper #3338
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…repper Signed-off-by: Taylor Gray <[email protected]>
graytaylor0
requested review from
elfisher,
AMoo-Miki,
nknize,
krisfreedain,
peterzhuamazon,
CEHENKLE,
dtaivpp,
kolchfa-aws,
nateynateynate and
natebower
as code owners
dinujoh
approved these changes
Oct 22, 2024
|
@vagimeli will perform doc review prior to editorial. Thanks! |
vagimeli
reviewed
Oct 30, 2024
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
vagimeli
reviewed
Oct 31, 2024
There was a problem hiding this comment.
@pajuric @graytaylor0 I've completed the doc review. This blog required significant rewriting to bring it to OpenSearch writing standards. Once my rewrites and suggestions are addressed, please tag me to re-review and ensure it's ready for editorial review. Thank you.
| - technical-post | ||
| twittercard: | ||
| description: " Data Prepper contains support for migrating databases like MongoDB and DynamoDB to OpenSearch. This infrastructure is extendable to other databases, and involves creating a new Data Prepper source plugin." | ||
| redirect_from: "/blog/technical-post/2024/09/How to Create a New Database Integration with Data Prepper/" |
There was a problem hiding this comment.
Update this link once you accept title edits
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
kolchfa-aws
reviewed
Nov 1, 2024
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
natebower
reviewed
Nov 1, 2024
There was a problem hiding this comment.
@graytaylor0 @pajuric Editorial review complete. Please see my comments and changes and let me know if you have any questions. Thanks!
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
Signed-off-by: Taylor Gray <[email protected]>
|
@natebower @vagimeli Thanks for the review and improvements on the post! I have committed the recommended changes. Please let me know if there are any I've missed and I will address them. |
…a-prepper.markdown Signed-off-by: Nathan Bower <[email protected]>
natebower
reviewed
Nov 4, 2024
There was a problem hiding this comment.
@graytaylor0 LGTM with these final changes. Thanks!
Cc: @pajuric
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Show resolved
Hide resolved
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
…a-prepper.markdown Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Taylor Gray <[email protected]>
|
Thanks @natebower! I have committed all of your suggestions. |
Signed-off-by: Taylor Gray <[email protected]>
natebower
reviewed
Nov 5, 2024
_posts/2024-09-24-creating-a-new-database-integration-with-data-prepper.markdown
Outdated
Show resolved
Hide resolved
…a-prepper.markdown Signed-off-by: Nathan Bower <[email protected]>
natebower
reviewed
Nov 5, 2024
There was a problem hiding this comment.
@graytaylor0 Thanks for all of your work on this! LGTM
Cc: @pajuric
pajuric
reviewed
Nov 5, 2024
| authors: | ||
| - tylgry | ||
| - dinujoh | ||
| date: 2022-09-21 10:00:00 -0500 |
pajuric
reviewed
Nov 5, 2024
| date: 2022-09-21 10:00:00 -0500 | ||
| categories: | ||
| - technical-post | ||
| twittercard: |
There was a problem hiding this comment.
Please add the following meta:
meta_keywords: Data Prepper, database inintegration, OpenSearch data collector, data pipeline, mongodb
meta_description: Use this step-by-step guide to set up and test a Data Prepper pipeline. Create a new database integration that streams data from MongoDB to OpenSearch using source coordination.
|
@graytaylor0 - Just added meta and asked for the date update. If you can make those changes, we can push this live. |
Signed-off-by: Taylor Gray <[email protected]>
|
@krisfreedain @nateynateynate - This blog is ready for primetime! Let's publish! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This post is meant to inform potential contributors to Data Prepper on how to create a new source plugin, and how to use the Source Coordination Infrastructure of Data Prepper.
It also walks through how to set up a MongoDB Data Prepper pipeline locally
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.