[Experimental] Packaging a minimal and flatten algorithm from downstr…

…eam repos (#229)
open-mmlab · Oct 31, 2023 · c9a5ea6 · c9a5ea6
1 parent 5d54183
commit c9a5ea6
Show file tree

Hide file tree

Showing 16 changed files with 2,666 additions and 16 deletions.
diff --git a/mim/_internal/export/README.md b/mim/_internal/export/README.md
@@ -0,0 +1,99 @@
+# Export (experimental)
+
+`mim export` is a new feature in `mim`, which can export a minimum trainable and testable model package through the config file.
+
+The minimal model compactly combines `model, datasets, engine` and other components. There is no need to search for each module separately based on the config file. Users who do not install from source can also get the model file at a glance.
+
+In addition, lengthy inheritance relationships, such as `CondInstBboxHead -> FCOSHead -> AnchorFreeHead -> BaseDenseHead -> BaseModule` in `mmdetection`, will be directly flattened into `CondInstBboxHead -> BaseModule`. There is no need to open and compare multiple model files when all functions inherited from the parent class are clearly visible.
+
+### Instructions Usage
+
+```bash
+mim export config_path
+
+# config_path has the following two optional types:
+
+# 1. config of downstream repo: Complete the call through repo::model_name/xxx.py. For example:
+mim export mmdet::configs/mask_rcnn/mask-rcnn_r101_fpn_1x_coco.py
+
+# 2. config in a certain folder, for example:
+mim export config_dir/mask-rcnn_r101_fpn_1x_coco.py
+```
+
+### Minimum model package directory structure
+
+```
+minimun_package(Named as pack_from_{repo}_20231212_121212)
+|- pack
+| |- configs # Configuration folder
+| | |- model_name
+| | |- xxx.py # Configuration file
+| |
+| |- models # model folder
+| | |- model_file.py
+| | |- ...
+| |
+| |- data # data folder
+| |
+| |- demo # demo folder
+| |
+| |-datasets #Dataset class definition
+| | |- transforms
+| |
+| |- registry.py # Registrar
+|
+|- tools
+| |- train.py # training
+| |- test.py # test
+|
+```
+
+### limit
+
+Currently, `mim export` only supports some config files of `mmpose`, `mmdetection`, `mmagic`, and `mmsegmentation` Besides, there are some constraints on the downstream repo.
+
+#### For downstream repos
+
+1. It is best to name the config without special symbols, otherwise it cannot be parsed through `mmengine.hub.get_config()`, such as:
+
+   - gn+ws/faster-rcnn_r101_fpn_gn-ws-all_1x_coco.py
+   - legacy_1.x/cascade-mask-rcnn_r50_fpn_1x_coco_v1.py
+
+2. For `mmsegmentation`, before using `mim.export` for config in `mmseg`, you should firstly modify it like `mmseg/registry/registry.py -> mmseg/registry.py`, without a directory to wrap `registry.py`
+
+3. It is recommended that the downstream Registry name inherited from mmengine should not be changed. For example, mmagic renamed `EVALUATOR` to `EVALUATORS`
+
+   ```python
+   from mmengine.registry import EVALUATOR as MMENGINE_EVALUATOR
+
+   # Evaluators to define the evaluation process.
+   EVALUATORS = Registry(
+       'evaluator',
+       parent=MMENGINE_EVALUATOR,
+       locations=['mmagic.evaluation'],
+   )
+   ```
+
+4. In addition, if you add a register that is not in mmengine, such as `DIFFUSION_SCHEDULERS` in mmagic, you need to add a key-value pair in `REGISTRY_TYPE` in `mim/_internal/export/common.py` for registering `torch `Module to `DIFFUSION_SCHEDULERS`
+
+   ```python
+   # "mmagic/mmagic/registry.py"
+   # modules for diffusion models that support adding noise and denoising
+   DIFFUSION_SCHEDULERS = Registry(
+       'diffusion scheduler',
+       locations=['mmagic.models.diffusion_schedulers'],
+   )
+
+   # "mim/utils/mmpack/common.py"
+   REGISTRY_TYPE = {
+       ...
+       'diffusion scheduler': 'DIFFUSION_SCHEDULERS',
+       ...
+   }
+   ```
+
+#### Need to be improve
+
+1. Currently, the inheritance relationship expansion of double parent classes is not supported. Improvements will be made in the future depending on the needs.
+2. Here comes ERROR when the origin config contains other files' `path` that can not be found in the current path. You can avoid export errors by manually modifying the `path` in the origin config.
+3. When isinstance() is used, if the parent class is just a class in the inheritance chain, the judgment may be False after expansion, because the original inheritance relationship will not be retained.
diff --git a/mim/_internal/export/README_zh-CN.md b/mim/_internal/export/README_zh-CN.md
@@ -0,0 +1,100 @@
+# Export (experimental)
+
+`mim export` 是 `mim` 里面一个新的功能，可以实现通过 config 文件，就能够导出一个最小可训练测试的模型包。
+
+最小模型将 `model、datasets、engine`等组件紧凑地组合在一起，不用再根据 config 文件单独寻找每一个模块，对于非源码安装的用户也能够获取到一目了然的模型文件。
+
+此外，对于模型中冗长的继承关系，如 `mmdetection` 中的 `CondInstBboxHead -> FCOSHead -> AnchorFreeHead -> BaseDenseHead -> BaseModule`，将会被直接展平为 `CondInstBboxHead -> BaseModule`，即无需再在多个模型文件之间跳转比较了，所有继承父类的函数一览无遗。
+
+### 使用方法
+
+```bash
+mim export config_path
+
+# config_path 有以下两种可选类型：
+# 下游 repo 的 config：通过 repo::configs/xxx.py 来完成调用。例如：
+mim export mmdet::configs/mask_rcnn/mask-rcnn_r101_fpn_1x_coco.py
+
+# 在某文件夹下的 config, 例如：
+mim export config_dir/mask-rcnn_r101_fpn_1x_coco.py
+```
+
+### 最小模型包目录结构
+
+```
+minimun_package(Named as pack_from_{repo}_20231212_121212)
+|- pack
+|   |- configs # 配置文件夹
+|   |  |- model_name
+|   |      |- xxx.py # 配置文件
+|   |
+|   |- models # 模型文件夹
+|   |  |- model_file.py
+|   |  |- ...
+|   |
+|   |- data # 数据文件夹
+|   |
+|   |- demo # demo文件夹
+|   |
+|   |- datasets # 数据集类定义
+|   |  |- transforms
+|   |
+|   |- registry.py # 注册器
+|
+|- tools
+|  |- train.py # 训练
+|  |- test.py # 测试
+|
+```
+
+### 限制
+
+`mim export` 目前只支持 `mmpose`、`mmdetection`、`mmagic` 和 `mmsegmentation` 的部分 config 配置文件，并且对下游算法库有一些约束。
+
+#### 针对下游库
+
+1. config 命名最好**不要有特殊符号**，否则无法通过 `mmengine.hub.get_config()` 进行解析，如：
+
+   - gn+ws/faster-rcnn_r101_fpn_gn-ws-all_1x_coco.py
+   - legacy_1.x/cascade-mask-rcnn_r50_fpn_1x_coco_v1.py
+
+2. 针对 `mmsegmentation`, 在使用 `mim.export` 导出 `mmseg` 的 config 之前, 首先需要去掉对于 `registry.py` 的外层文件夹封装， 即修改 `mmseg/registry/registry.py -> mmseg/registry.py`。
+
+3. 建议下游继承于 mmengine 的 Registry 名字不要改动，如 mmagic 中就将 `EVALUATOR` 重新命名为了 `EVALUATORS`
+
+   ```python
+   from mmengine.registry import EVALUATOR as MMENGINE_EVALUATOR
+
+   # Evaluators to define the evaluation process.
+   EVALUATORS = Registry(
+       'evaluator',
+       parent=MMENGINE_EVALUATOR,
+       locations=['mmagic.evaluation'],
+   )
+   ```
+
+4. 另外，如果添加了 mmengine 中没有的注册器，如 mmagic 中的 `DIFFUSION_SCHEDULERS`，需要在 `mim/_internal/export/common.py` 的 `REGISTRY_TYPE` 中添加键值对，用于注册 `torch` 模块到 `DIFFUSION_SCHEDULERS`
+
+   ```python
+   # "mmagic/mmagic/registry.py"
+   # modules for diffusion models that support adding noise and denoising
+   DIFFUSION_SCHEDULERS = Registry(
+       'diffusion scheduler',
+       locations=['mmagic.models.diffusion_schedulers'],
+   )
+
+   # "mim/utils/mmpack/common.py"
+   REGISTRY_TYPE = {
+       ...
+       'diffusion scheduler': 'DIFFUSION_SCHEDULERS',
+       ...
+   }
+   ```
+
+#### 对于 `mim.export` 功能需要改进的地方
+
+1. 目前还不支持双父类的继承关系展开，后续看需求进行改进
+
+2. 对于用到 `isinstance()` 时，如果父类只是继承链中某个类，可能展开后判断就会为 False，因为并不会保留原有的继承关系
+
+3. 当 config 文件中含有当前文件夹没法被访问到的`数据集路径`，导出可能会失败。目前的临时解决方法是：将原来的 config 文件保存到当前文件夹下，然后需要用户手动修改`数据集路径`为当前路径下的可访问路径。如：`data/ADEChallengeData2016/ -> your_data_dir/ADEChallengeData2016/`
diff --git a/mim/_internal/export/__init__.py b/mim/_internal/export/__init__.py
@@ -0,0 +1 @@
+# Copyright (c) OpenMMLab. All rights reserved.
diff --git a/mim/_internal/export/common.py b/mim/_internal/export/common.py
@@ -0,0 +1,75 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+
+# initialization template for __init__.py
+_init_str = """
+import os
+import os.path as osp
+
+all_files = os.listdir(osp.dirname(__file__))
+
+for file in all_files:
+    if (file.endswith('.py') and file != '__init__.py') or '.' not in file:
+        exec(f'from .{osp.splitext(file)[0]} import *')
+"""
+
+# import pack path for tools
+_import_pack_str = """
+import os.path as osp
+import sys
+sys.path.append(osp.dirname(osp.dirname(__file__)))
+import pack
+
+"""
+
+OBJECTS_TO_BE_PATCHED = {
+    'MODELS': [
+        'BACKBONES',
+        'NECKS',
+        'HEADS',
+        'LOSSES',
+        'SEGMENTORS',
+        'build_backbone',
+        'build_neck',
+        'build_head',
+        'build_loss',
+        'build_segmentor',
+        'CLASSIFIERS',
+        'RETRIEVER',
+        'build_classifier',
+        'build_retriever',
+        'POSE_ESTIMATORS',
+        'build_pose_estimator',
+        'build_posenet',
+    ],
+    'TASK_UTILS': [
+        'PIXEL_SAMPLERS',
+        'build_pixel_sampler',
+    ]
+}
+
+REGISTRY_TYPES = {
+    'runner': 'RUNNERS',
+    'runner constructor': 'RUNNER_CONSTRUCTORS',
+    'hook': 'HOOKS',
+    'strategy': 'STRATEGIES',
+    'dataset': 'DATASETS',
+    'data sampler': 'DATA_SAMPLERS',
+    'transform': 'TRANSFORMS',
+    'model': 'MODELS',
+    'model wrapper': 'MODEL_WRAPPERS',
+    'weight initializer': 'WEIGHT_INITIALIZERS',
+    'optimizer': 'OPTIMIZERS',
+    'optimizer wrapper': 'OPTIM_WRAPPERS',
+    'optimizer wrapper constructor': 'OPTIM_WRAPPER_CONSTRUCTORS',
+    'parameter scheduler': 'PARAM_SCHEDULERS',
+    'param scheduler': 'PARAM_SCHEDULERS',
+    'metric': 'METRICS',
+    'evaluator': 'EVALUATOR',  # TODO EVALUATORS in mmagic
+    'task utils': 'TASK_UTILS',
+    'loop': 'LOOPS',
+    'visualizer': 'VISUALIZERS',
+    'vis_backend': 'VISBACKENDS',
+    'log processor': 'LOG_PROCESSORS',
+    'inferencer': 'INFERENCERS',
+    'function': 'FUNCTIONS',
+}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		# Copyright (c) OpenMMLab. All rights reserved.