Skip to content

Commit

Permalink
Nicer doc
Browse files Browse the repository at this point in the history
  • Loading branch information
colon3ltocard committed Dec 6, 2024
1 parent 934a9e7 commit 734b185
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,15 +63,29 @@ class HalfUNet(ModelABC, nn.Module):

Currently we support the following neural network architectures:


## Convolutional Neural Networks

| Model | Research Paper | Input Shape | ONNX exportable ? | Notes | Use-Cases at MF |
| :---: | :---: | :---: | :---: | :---: | :---: |
| [DeepLabV3Plus](mfai/torch/models/deeplabv3.py#L1) | [arxiv link](https://arxiv.org/abs/1802.02611) | (Batch, features, Height, Width) | Yes | As a very large receptive field versus U-Net, Half-Unet, ... | Front Detection, Nowcasting |
| [HalfUNet](mfai/torch/models/half_unet.py#L1) | [researchgate link](https://www.researchgate.net/publication/361186968_Half-UNet_A_Simplified_U-Net_Architecture_for_Medical_Image_Segmentation) | (Batch, features, Height, Width) | Yes | In prod/oper on [Espresso](https://www.mdpi.com/2674-0494/2/4/25) V2 with 128 filters and standard conv blocks instead of ghost | Satellite channels to rain estimation |
| [UNet](mfai/torch/models/unet.py#L1) | [arxiv link](https://arxiv.org/pdf/1505.04597.pdf) | (Batch, features, Height, Width) | Yes | Vanilla U-Net | Radar image cleaning |
| [CustomUnet](mfai/torch/models/unet.py#L1) | [arxiv link](https://arxiv.org/pdf/1505.04597.pdf) | (Batch, features, Height, Width) | Yes | U-Net like architecture with a variety of resnet encoder choices | Radar image cleaning


## Vision Transformers

| Model | Research Paper | Input Shape | ONNX exportable ? | Notes | Use-Cases at MF |
| :---: | :---: | :---: | :---: | :---: | :---: |
| [Segformer](mfai/torch/models/segformer.py#L1) | [arxiv link](https://arxiv.org/abs/2105.15203) | (Batch, features, Height, Width) | Yes | On par with u-net like on Deepsyg (MF internal), added an upsampling stage. Adapted from [Lucidrains' github](https://github.com/lucidrains/segformer-pytorch) | Segmentation tasks |
| [SwinUNETR](mfai/torch/models/swinunetr.py#L1) | [arxiv link](https://arxiv.org/abs/2201.01266) | (Batch, features, Height, Width) | No | 2D Swin Unet transformer (Pangu and archweather uses customised 3D versions of Swin Transformers). Plugged in from [MONAI](https://github.com/Project-MONAI/MONAI/). The decoders use Bilinear2D + Conv2d instead of Conv2dTranspose to remove artefacts/checkerboard effects | Segmentation tasks |
| [UNETRPP](mfai/torch/models/unetrpp.py#L1) | [arxiv link](https://arxiv.org/abs/2212.04497) | (Batch, features, Height, Width) or (Batch, features, Height, Width, Depth) | Yes | Vision transformer with a reduced GFLOPS footprint adapted from [author's github](https://github.com/Amshaker/unetr_plus_plus). Modified to work both with 2d and 3d inputs. The decoders use Bilinear2D + Conv2d instead of Conv2dTranspose to remove artefacts/checkerboard effects | Front Detection, LAM Weather Forecasting |

## Graph Neural Networks

| Model | Research Paper | Input Shape | ONNX exportable ? | Notes | Use-Cases at MF |
| :---: | :---: | :---: | :---: | :---: | :---: |
| [hilam, graphlam](mfai/torch/models/nlam/__init__.py) | [arxiv link](https://arxiv.org/abs/2309.17370) | (Batch, graph_node_id, features) | Imported and adapted from [Joel's github](https://github.com/joeloskarsson/neural-lam) |

# SegmentationLightningModule
Expand Down

0 comments on commit 734b185

Please sign in to comment.