Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logging: resolve confusion with 'crawl done' not being written to log… #375

Merged
merged 2 commits into from
Sep 13, 2023

Conversation

ikreymer
Copy link
Member

…, because the log is itself stored in the WACZ: (fixes #365)

  • keep log file open until end, even if its being written to WACZ
  • add logging of 'crawling done' when crawling is done (writing to WACZ or not)
  • add debug logging of 'end of log file' to indicate log file is being added to WACZ and nothing else will be added there in the WACZ.

…, because the log is itself stored in the WACZ: (fixes #365)

- keep log file open until end, even if its being written to WACZ
- add logging of 'crawling done' when crawling is done (writing to WACZ or not)
- add debug logging of 'end of log file' to indicate log file is being added to WACZ and nothing else will be added there in the WACZ.
@Chickensoupwithrice
Copy link
Contributor

Chickensoupwithrice commented Sep 13, 2023

Wouldn't we still want to close the log once the log has been written to the WACZ?

On second thought it doesn't matter too much since the crawler will spin down anyway? Still think it would be "good practice"?

@ikreymer
Copy link
Member Author

Wouldn't we still want to close the log once the log has been written to the WACZ?

On second thought it doesn't matter too much since the crawler will spin down anyway? Still think it would be "good practice"?

Yea, the process and container will exit soon after, I suppose could add it back in for successful exit.

@ikreymer ikreymer merged commit 283fa00 into main Sep 13, 2023
2 checks passed
@ikreymer ikreymer deleted the keep-log-open branch September 13, 2023 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Last Line of Logs - Docker Logs versus written Crawl Log
3 participants