Fix unused FstSegmenter warning when not using khmer compiler features

[timvisee]

Pull Request

Related issue

Fixes #260
Depends on #259

What does this PR do?

• Fixes compilation warnings without thai or khmer features
• Puts related module behind conditional feature attribute

PR checklist

Please check if your PR fulfills the following requirements:

• Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
• Have you read the contributing guidelines?
• Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

[irevoire]

Great, thanks

I didn't know qdrant was using charabia; that's awesome! I love your work

May I ask in which part of your pipeline it found its place? 👀

[irevoire]

bors merge

[meili-bors]

Build succeeded:

• tests

[timvisee]

I didn't know qdrant was using charabia; that's awesome! I love your work

😄

May I ask in which part of your pipeline it found its place? 👀

Of course. We use it to tokenize our full text payload/metadata indexes. Saves us a lot of hassle building our own tokenizer. Thank you for your work!

In code: https://github.com/qdrant/qdrant/blob/dev/lib/segment/src/index/field_index/full_text_index/tokenizers.rs

Again, thanks for handling this PR quickly.