https://arxiv.org/abs/2406.05074
-
08.19.2024: We release a CellViT-Hibou model, which is a hybrid model combining the CellViT and Hibou architectures. This model comes under CC BY-NC-SA 4.0 license. Check the
segmentation_example.ipynb
notebook for an example of how to use the model. The weights can be downloaded from Hugging Face 🤗. CellViT-Hibou is a model trained on PanNuke dataset for panoptic cell segmentation. It can segment and classify cells and tissues. For more information visit the original CellViT repository here. Huge thanks to the authors of CellViT for their amazing work! -
08.09.2024: We are excited to announce the release of Hibou-L under the Apache 2.0 license. You can find Hibou-L on Hugging Face 🤗 here.
This repository contains the code to run the Hibou models locally. For inquiries about accessing Hibou-L on CellDX, please contact us at [email protected].
The easiest way to use the Hibou models is through the HuggingFace repository. Run the following code to get started:
from transformers import AutoImageProcessor, AutoModel
processor = AutoImageProcessor.from_pretrained("histai/hibou-b", trust_remote_code=True)
model = AutoModel.from_pretrained("histai/hibou-b", trust_remote_code=True)
OR
from transformers import AutoImageProcessor, AutoModel
processor = AutoImageProcessor.from_pretrained("histai/hibou-L", trust_remote_code=True)
model = AutoModel.from_pretrained("histai/hibou-L", trust_remote_code=True)
We use a customized implementation of the DINOv2 architecture from the transformers library to add support for registers, which requires the trust_remote_code=True
flag.
If you prefer to use the model without the transformers library, follow these steps:
-
Install the requirements and the package:
git clone https://github.com/HistAI/hibou.git cd hibou pip install -r requirements.txt && pip install -e .
-
Download the model weights:
-
Load the model with the following code:
from hibou import build_model model = build_model("weights-path")
For more information, refer to the example.ipynb notebook.
Table: Linear probing benchmarks reporting top-1 accuracy.
*Metrics for Virchow and RudolfV are derived from the respective papers.
Dataset | Phikon | Kaiko-B8 | Virchow* | RudolfV* | Prov-GigaPath | H-optimus-0 | Hibou-B | Hibou-L |
---|---|---|---|---|---|---|---|---|
CRC-100K | 0.917 | 0.949 | 0.968* | 0.973* | 0.968 | 0.970 | 0.955 | 0.966 |
PCAM | 0.916 | 0.919 | 0.933* | 0.944* | 0.947 | 0.942 | 0.946 | 0.953 |
MHIST | 0.791 | 0.832 | 0.834* | 0.821* | 0.839 | 0.861 | 0.812 | 0.858 |
MSI-CRC | 0.750 | 0.786 | - | 0.755* | 0.771 | 0.767 | 0.779 | 0.793 |
MSI-STAD | 0.760 | 0.814 | - | 0.788* | 0.784 | 0.797 | 0.797 | 0.829 |
TIL-DET | 0.944 | 0.945 | - | 0.943* | 0.939 | 0.948 | 0.942 | 0.942 |
AVG (1-3) | 0.875 | 0.900 | 0.912 | 0.913 | 0.918 | 0.924 | 0.904 | 0.926 |
AVG (1-6) | 0.846 | 0.874 | - | 0.871 | 0.875 | 0.881 | 0.872 | 0.890 |
This repository is licensed under the Apache License, Version 2.0. See the LICENSE file for the full license text.
We would like to thank the authors of the DINOv2 repository, upon which this project is built. The original repository can be found here.
Feel free to reach out at [email protected] if you have any questions or need further assistance!
If you use our work, please cite:
@misc{nechaev2024hibou,
title={Hibou: A Family of Foundational Vision Transformers for Pathology},
author={Dmitry Nechaev and Alexey Pchelnikov and Ekaterina Ivanova},
year={2024},
eprint={2406.05074},
archivePrefix={arXiv},
primaryClass={eess.IV}
}