Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Version #6

Merged
merged 7 commits into from
Jan 29, 2025
Merged

Initial Version #6

merged 7 commits into from
Jan 29, 2025

Conversation

rgaudin
Copy link
Member

@rgaudin rgaudin commented Jan 25, 2025

  • Basic functionnality present
  • Works both standalone and all-in-one Docker image

@benoit74 that's basically a transfer of what was on kci and running on kathrin.
Please raise your questions and I'll document whatever needs to be.

@rgaudin rgaudin requested a review from benoit74 January 25, 2025 20:00
@rgaudin rgaudin self-assigned this Jan 25, 2025
@codecov-commenter
Copy link

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Copy link

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good work obviously.

Of course I have found lot's of things to comment, but nothing very big.

I think the program overall misses few safeguards:

  • avoid silly (small) settings of sleep period (e.g. never smaller than 5 mins) to avoid flooding our servers
  • avoid removing too many ZIMs at once => this is a bit blurry, but the scenario I have in mind is that should we publish an empty catalog for any reason (or should the program believe the catalog is empty for any reason), it would be a pity / concern to cause all ZIMs of all peers to be deleted within few hours ; maybe even deleting too many ZIMs at once is an indicator of an issue ; that been said, since we keep the ZIMs for --keep days, the impact is obviously not immediate ... should we simply have an alerting system? a summary file one can watch with details about files which are going to be removed in 1 week, 2 weeks, 4 weeks, 8 weeks, 12 weeks?

.github/workflows/DockerCD.yaml Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
src/kiwixseeder/utils.py Outdated Show resolved Hide resolved
src/kiwixseeder/entrypoint.py Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
src/kiwixseeder/context.py Outdated Show resolved Hide resolved
seeder-start-restart.sh Outdated Show resolved Hide resolved
- Basic functionnality present
- Works both standalone and all-in-one Docker image
- added Title and Description filters.
- Removed nodet flavour choice (now gone)
- All matches now case insensitive
- Default sleep interval to 1d for docker script
@rgaudin
Copy link
Member Author

rgaudin commented Jan 29, 2025

Thank you @benoit74 ; I've made all the necessary edits ; and rebased. I opened #8 regarding mass removal as I have no straight forward solution

@rgaudin rgaudin merged commit 6e1d131 into main Jan 29, 2025
4 checks passed
@rgaudin rgaudin deleted the initial_version branch January 29, 2025 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants