Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/overlay middleware #409

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

SwitchTV-BenBettridge
Copy link

@SwitchTV-BenBettridge SwitchTV-BenBettridge commented Feb 25, 2022

This adds a new Middleware that implements writes using a local BlobStore and passes reads through to an upstream BlobStore using some simple logic:

  • Read requests are returned from the local BlobStore if the key exists, otherwise from the upstream bucket.
  • Requests that list container contents are merged between the local and upstream BlobStore
  • Writes are written to the local BlobStore
  • Delete's are implemented by creating a file in the local BlobStore which indicates the key should be removed from results.

Notes:

  • I am not a Java Developer (at least, not in the most recent decade) and this code might not follow all, or many, best practices. Particularly any features used that weren't a part of Java 7.
  • This is not production-ready. It was designed for development and testing purposes only. A non-exhaustive list of things that would be needed to be Production Ready are:
    • Use a more robust mechanism to generate "Not Found" errors other than trying to load a hopefully non-existent file.
    • The local BlobStore might need a mechanism to limit the amount of space it can consume, and how to handle the case where that space is exhausted.

This addresses a common use-case where developers need to test or replicate issues against production storage without actually writing anything back.

@gaul gaul requested review from timuralp and gaul March 26, 2022 13:07
@gaul
Copy link
Owner

gaul commented Jan 19, 2023

This seems like a good feature -- sorry I haven't had time to look at it. Could you squash the commits and resolve the conflicts before I start reviewing it?

@bbetter173
Copy link

@gaul - Good to go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants