LangChain's BM25 retriever is undermaintained

[antonpibm]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

In my usecase I want to have a local storage with documents which I want to search with BM25, also I need the ability to add documents to the storage
The current LC BM25 retriever implementation relies on rank_bm25, which is not maintained - last commit was 2 years ago, Issues and PRs are not handled. The package itself is relatively thin in the first place and missing key features required by me such as persistence.

Is the the go-to local BM25 implementation in LangChain, other than the Elastic based version, or is there a better implementation available?
If that's the go-to, is there a room for changing the dependency to a more mature and better maintained dependency?

Motivation

In my usecase I want to have a local storage with updates capabilities for documents which I want to search with BM25

Proposal (If applicable)

Change the dependency to a more mature and better maintained dependency

[antonpibm]

Hi team, any thoughts about this?

[jankovicsandras]

Hi,
I'm not from LangChain, but this might help somewhat: https://github.com/jankovicsandras/bm25opt

[mspronesti]

Hi, I opened a PR supporting an alternative and persistable implementation based on bm25s #28123.