Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix LevelDB panic when shutting down the node #210

Open
EnchanterIO opened this issue Feb 11, 2020 · 2 comments
Open

Fix LevelDB panic when shutting down the node #210

EnchanterIO opened this issue Feb 11, 2020 · 2 comments
Labels
bug Something isn't working

Comments

@EnchanterIO
Copy link
Contributor

EnchanterIO commented Feb 11, 2020

How to reproduce

  • Boot up a new no-synced node
  • Before node is fully sync force a shut down (CTRL+C)

Screen Shot 2020-02-11 at 13 28 50

Steps to resolve:

  1. Shutting down the node should close all goroutines gracefully
@EnchanterIO EnchanterIO added the bug Something isn't working label Feb 11, 2020
@ggarri
Copy link
Contributor

ggarri commented Mar 17, 2020

Current status
After the first research over this issue I could narrow the issue to the scenario where a node being shut down triggers tendermint dbs to close down and it appears to happen that the tendermint stateDb is closed and Tendermint node try to persist a block right after that happens. This is a racing condition and it does not happen on every shut down.

Next steps
Try to validate the potential solution of stopping Tendermint ABCI reactor before DBs are closed

@EnchanterIO
Copy link
Contributor Author

Seems like the 1s timeout should be enough for persisting the changes to disk. If it fails we increase it or implement a proper solution. I am bumping the version and will test it for few days on my node. If everything goes fine we merge it by the end of the month 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants