Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added explicit storage type selection for MongoDB Event Store #151

Merged
merged 9 commits into from
Dec 28, 2024

Conversation

oskardudycz
Copy link
Collaborator

@oskardudycz oskardudycz commented Dec 17, 2024

Exposed strongly typed storage options for MongoDB Event Store. Now they can be explicitly selected and mutually exclusive:

  • COLLECTION_PER_STREAM_TYPE - default option, where each stream type has a dedicated storage. As we're keeping inline projections in the same document as the stream, then that gives a better option for custom indexing and better storage usage.
  • SINGLE_COLLECTION can be either custom or default to emt:streams. This can be a fair choice if you don't have a large number of streams or you don't need customisation per stream type.
  • CUSTOM - you can define your own collection resolution based on the stream type. It might be that you need to route databases per tenant or handle some other more advanced stuff, so this is the choice for you.

Added the possibility of providing custom collection and database resolution.

Moved storage-related code to a dedicated file, as it got a bit too crowded in the MongoDBEventStore implementation.

Fixes #142

Exposed strongly typed storage options for MongoDB Event Store.

Added possibility to provide custom collection and database resolution.

Moved storage related code to a dedicated file, as it got a bit too crowdy in the MongoDBEventStore implementation.
@oskardudycz
Copy link
Collaborator Author

@alex-laycalvert, I need to add tests for the collection resolution-related code, but could you check if this works for you?

I was going back and forth on whether we should make the storage setting explicit or not. On the one hand, it's an important decision; on the other, many new users probably won't know which one is better, so we'll still need to document it. So, at this point, I decided to make the default choice implicit to collection per stream type to encourage people to use the default.

@oskardudycz
Copy link
Collaborator Author

oskardudycz commented Dec 17, 2024

It sounds like I was a bit sloppy and the commit cc43517 got into #150 instead of here. I’ll reverted it on main to keep a full diff here.

@oskardudycz oskardudycz force-pushed the mongodb_storage_options branch 2 times, most recently from 45bc34d to af0f285 Compare December 18, 2024 11:09
@oskardudycz oskardudycz marked this pull request as ready for review December 18, 2024 11:10
@oskardudycz
Copy link
Collaborator Author

@alex-laycalvert, I added unit and e2e tests for storage resolution, it's fully ready for code review 👍

@oskardudycz oskardudycz force-pushed the mongodb_storage_options branch from af0f285 to 021db39 Compare December 18, 2024 11:13
We have a dedicated set of e2e tests for projections. No need to keep the noise in the other places.
MongoClient,
} from 'mongodb';

export const getDummyCollection = <TSchema extends Document = Document>(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COMMENT: Intention behind this dummy collection is to be used in the structure tests, not to act as an in-memory collection.

Added also comments for Mongo dummy mocks
@oskardudycz oskardudycz force-pushed the mongodb_storage_options branch from aaf7242 to dacbef5 Compare December 18, 2024 11:38
@oskardudycz oskardudycz force-pushed the mongodb_storage_options branch from 38cda85 to 63a684e Compare December 18, 2024 12:08
@oskardudycz oskardudycz force-pushed the mongodb_storage_options branch from 63a684e to c991d64 Compare December 18, 2024 12:10
Copy link
Collaborator

@alex-laycalvert alex-laycalvert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it!

@alex-laycalvert
Copy link
Collaborator

I think this should be enough for a while as far as how far the store options can be customized.

@oskardudycz
Copy link
Collaborator Author

@alex-laycalvert, thanks for checking it out!

I think this should be enough for a while as far as how far the store options can be customized.

Yes, I think that we now have good foundations of the MongoDB storage layout 👍

I'll release it next week, and also write an article about it 🙂

@oskardudycz oskardudycz merged commit 4d5eaf2 into main Dec 28, 2024
3 checks passed
@oskardudycz oskardudycz deleted the mongodb_storage_options branch December 28, 2024 16:27
@oskardudycz oskardudycz added this to the 0.23.0 milestone Dec 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an explicit switch for MongoDB collection storage
2 participants