Table Recognition: Training model is transferred to the inference model, the prediction effect is inconsistent? #14505

vuthehuy1997 · 2024-11-27T03:27:06Z

vuthehuy1997
Nov 27, 2024

🔎 Search before asking

I have searched the PaddleOCR Docs and found no similar bug report.
I have searched the PaddleOCR Issues and found no similar bug report.
I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

When I finished training the SLANet recognition table model, I exported it to a format for inference, but the results were wrong. i only edit SLANet.yml to SLANet_finetune.yml like tutorial on github document

Model training command (finetune from model ch_ppstructure_mobile_v2.0_SLANet_train)

python3 tools/train.py -c configs/table/SLANet_finetune.yml\
 -o Global.pretrained_model=./weights/whl/table/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy \
 Global.use_amp=True Global.scale_loss=1024.0 Global.use_dynamic_loss_scaling=True

export command:

python3 tools/export_model.py -c configs/table/SLANet_finetune.yml -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/SLANet_ch/best_accuracy Global.save_inference_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune

If i run the model after training with the command:

python3 tools/infer_table.py -c configs/table/SLANet_finetune.yml -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/SLANet_ch/best_accuracy Global.infer_img=ppstructure/docs/table/table .jpg

Then it gives correct results

result: ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="2"', '>', '</td>', '<td', ' rowspan="2"', '>', '</ td>', '<td', ' colspan="2"', '>', '</td>', '<td', ' rowspan="2"', '>', '</td> ', '<td', ' rowspan="2"', '>', '</td>', '<td', ' rowspan="2"', '>', '</td>', '<td', ' rowspan= "2"', '>', '</td>', '<td', ' rowspan="2"', '>', '</td>', '<td', ' rowspan="2 "', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', ..., '</table>' , '</body>', '</html>'], [[0.7451880574226379, 1.5897825956344604, 122.04891204833984, 1.5919700860977173, 122.1083755493164, 34.912353515625, 0.7465599775314331, 34.94774627685547], ...

If running model was export:

cd ppstructure/
python table/predict_structure.py\
 --table_model_dir=../../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
 --table_char_dict_path=../ppocr/utils/dict/table_structure_dict_ch.txt \
 --image_dir=docs/table/table.jpg \
 --output=../output/table

Then the result will be wrong

['<html>', '<body>', '<table>', '</tbody>', '</tbody>', '</tbody>', '</tbody>', '</ tbody>', '</thead>', '</tbody>', '</tbody>', '</thead>', '</thead>', '</tbody>', '</thead >', ..., '<tr>', '</table>', '</body>', '</html>'], [[2.9795612022098794e-07, 2.2673059163480502e-07, 8.608171120361163e-11, 9.731257932799053e-07, 8.894006642279351e-10, 3.4046809105348075e-07, 3.1248458043364735e-08, 17.488468170166016, 9.80240444370395e-10, 8.043410382185812e-09, ...

Does anyone know what the error is when exporting?

🏃‍♂️ Environment (运行环境)

Ubuntu
paddlepaddle_gpu=3.0.0b0
NVIDIA A40
cuda 12.3

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

Global:
  use_gpu: True
  epoch_num: 400
  log_smooth_window: 20
  print_batch_step: 20
  save_model_dir: ./output/SLANet_ch
  save_epoch_step: 400
  # evaluation is run every 331 iterations after the 0th iteration
  eval_batch_step: [0, 331]
  cal_metric_during_train: True
  pretrained_model: ./weights/whl/table/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy.pdparams
  checkpoints: 
  save_inference_dir: ./output/SLANet_ch/infer
  use_visualdl: False
  infer_img: ppstructure/docs/table/table.jpg
  # for data or label process
  character_dict_path: ppocr/utils/dict/table_structure_dict_ch.txt
  character_type: en
  max_text_length: &max_text_length 500
  box_format: &box_format xyxyxyxy # 'xywh', 'xyxy', 'xyxyxyxy'
  infer_mode: False
  use_sync_bn: True
  save_res_path: output/infer
    
Optimizer:
  name: Adam
  beta1: 0.9
  beta2: 0.999
  clip_norm: 5.0
  # lr:
  #   learning_rate: 0.001
  # regularizer:
  #   name: 'L2'
  #   factor: 0.00000
  lr:
    name: Cosine
    learning_rate: 0.00017 #
    warmup_epoch: 0
  regularizer:
    name: 'L2'
    factor: 0

Architecture:
  model_type: table
  algorithm: SLANet
  Backbone:
    name: PPLCNet
    scale: 1.0
    pretrained: True
    use_ssld: True
  Neck:
    name: CSPPAN
    out_channels: 96
  Head:
    name: SLAHead
    hidden_size: 256
    max_text_length: *max_text_length
    loc_reg_num: &loc_reg_num 8

Loss:
  name: SLALoss
  structure_weight: 1.0
  loc_weight: 2.0
  loc_loss: smooth_l1

PostProcess:
  name: TableLabelDecode
  merge_no_span_structure: &merge_no_span_structure True

Metric:
  name: TableMetric
  main_indicator: acc
  compute_bbox_metric: False
  loc_reg_num: *loc_reg_num
  box_format: *box_format
  del_thead_tbody: True

Train:
  dataset:
    name: PubTabDataSet
    data_dir: ./datasets/train/table/table_envi/
    label_file_list: [./datasets/train/table/table_envi/train.jsonl]
    transforms:
      - DecodeImage:
          img_mode: BGR
          channel_first: False
      - TableLabelEncode:
          learn_empty_box: False
          merge_no_span_structure: *merge_no_span_structure
          replace_empty_cell_token: False
          loc_reg_num: *loc_reg_num
          max_text_length: *max_text_length
      - TableBoxEncode:
          in_box_format: *box_format
          out_box_format: *box_format
      - ResizeTableImage:
          max_len: 488
      - NormalizeImage:
          scale: 1./255.
          mean: [0.485, 0.456, 0.406]
          std: [0.229, 0.224, 0.225]
          order: 'hwc'
      - PaddingTableImage:
          size: [488, 488]
      - ToCHWImage:
      - KeepKeys:
          keep_keys: ['image', 'structure', 'bboxes', 'bbox_masks', 'length', 'shape']
  loader:
    shuffle: True
    batch_size_per_card: 16
    drop_last: True
    num_workers: 1

Eval:
  dataset:
    name: PubTabDataSet
    data_dir: ./datasets/train/table/table_envi/
    label_file_list: [./datasets/train/table/table_envi/val.jsonl]
    transforms:
      - DecodeImage:
          img_mode: BGR
          channel_first: False
      - TableLabelEncode:
          learn_empty_box: False
          merge_no_span_structure: *merge_no_span_structure
          replace_empty_cell_token: False
          loc_reg_num: *loc_reg_num
          max_text_length: *max_text_length
      - TableBoxEncode:
          in_box_format: *box_format
          out_box_format: *box_format
      - ResizeTableImage:
          max_len: 488
      - NormalizeImage:
          scale: 1./255.
          mean: [0.485, 0.456, 0.406]
          std: [0.229, 0.224, 0.225]
          order: 'hwc'
      - PaddingTableImage:
          size: [488, 488]
      - ToCHWImage:
      - KeepKeys:
          keep_keys: ['image', 'structure', 'bboxes', 'bbox_masks', 'length', 'shape']
  loader:
    shuffle: False
    drop_last: False
    batch_size_per_card: 16
    num_workers: 1

GreatV · 2024-11-27T03:36:34Z

GreatV
Nov 27, 2024
Maintainer

The issue you are experiencing, where the inference results using the exported model are inconsistent or incorrect compared to the results from training or direct inference with the best_accuracy checkpoint, is a common problem when exporting models for static inference. Below are potential causes and solutions based on the provided details and similar issues encountered in PaddleOCR:

Potential Causes

Inconsistent Preprocessing Between Training and Inference
- The preprocessing steps (e.g., image normalization, resizing, padding) during training or inference might differ from those applied in predict_structure.py. If the exported inference model uses preprocessing steps different from those used during training, this could lead to mismatched results.
Postprocessing Differences
- The way the predictions are decoded (e.g., parsing table structure) might vary between infer_table.py and predict_structure.py. This can cause discrepancies in the output.
Character Dictionary Mismatch
- During inference, the wrong character dictionary might be used, leading to incorrect decoding of the table structure.
Export Command Configuration
- The export process might not fully replicate the configuration used during training, such as missing parameters like character_dict_path, max_text_length, or merge_no_span_structure.
Dynamic vs. Static Graph Differences
- In PaddleOCR, the training model (best_accuracy) uses a dynamic computational graph, while the exported model uses a static graph. Some components (e.g., batch normalization or variable shapes) may behave differently in static graphs compared to dynamic ones.
Error in the Exported Model
- There might be a bug in the export_model.py implementation or the configuration passed to it, causing the exported model to incorrectly represent the trained model.

Steps to Debug and Resolve

1. Verify Preprocessing Consistency

Compare the preprocessing steps in SLANet_finetune.yml with those in predict_structure.py. Key preprocessing components to check:
- DecodeImage
- NormalizeImage (mean, stddev)
- ResizeTableImage
- PaddingTableImage
Ensure that the same transformations are applied in both training and inference.

2. Ensure Postprocessing Consistency

Check the PostProcess block in your configuration file and ensure that parameters like merge_no_span_structure are consistent with how predict_structure.py decodes the output.
Modify predict_structure.py to align it with infer_table.py's decoding logic if needed.

3. Confirm Character Dictionary

Verify that the correct character dictionary (table_structure_dict_ch.txt) is used. Ensure that the same dictionary is referenced during training, inference, and export.

4. Pass All Necessary Parameters During Export

Ensure that all critical parameters from SLANet_finetune.yml are passed during export. For example:

python3 tools/export_model.py -c configs/table/SLANet_finetune.yml \
  -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/SLANet_ch/best_accuracy \
     Global.save_inference_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
     Global.character_dict_path=ppocr/utils/dict/table_structure_dict_ch.txt \
     Global.max_text_length=500 \
     PostProcess.merge_no_span_structure=True

5. Run Direct Inference on Exported Model

Test if the exported model is functional by using tools/infer_table.py directly on the exported model:

python3 tools/infer_table.py -c configs/table/SLANet_finetune.yml \
  -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
    Global.infer_img=ppstructure/docs/table/table.jpg

If the result is correct, the issue lies specifically in predict_structure.py.

6. Validate the Exported Model

Use the paddle_infer API to check whether the exported model is working as expected:

paddle_infer --model_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
  --image_dir=ppstructure/docs/table/table.jpg

If the results are incorrect, the issue is likely during the export process.

7. Check for Known Issues

Similar problems have been reported in PaddleOCR's GitHub repository. For example:
- Issue #14064
- Issue #14088
These issues often involve differences in preprocessing or bugs in the export script.

8. Debug Static Graph Behavior

Some differences might arise due to the transition from a dynamic to a static graph. Try retraining with the Global.infer_mode=True option to force the model into inference mode during training:
```
Global:
  infer_mode: True
```

9. Test with a Pretrained Model

As a sanity check, export the pretrained model (ch_ppstructure_mobile_v2.0_SLANet_train) without fine-tuning to ensure the export process works correctly:

python3 tools/export_model.py -c configs/table/SLANet.yml \
  -o Global.pretrained_model=./weights/whl/table/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy \
     Global.save_inference_dir=./output/pretrained_inference

Recommended Workflow

Preprocessing/Postprocessing Check: Ensure preprocessing and postprocessing are consistent across training, inference, and export.
Parameter Alignment: Pass all required parameters during both training and export.
Export Validation: Test the exported model directly using tools/infer_table.py.
Static Graph Debugging: Use infer_mode=True during training to debug differences in dynamic vs. static graph behavior.

If none of these solutions resolve the issue, you might need to open a detailed issue on the PaddleOCR GitHub repository with a focus on the export process. Be sure to include:

The exact commands you ran.
Configuration files used.
Input/output examples from both infer_table.py and predict_structure.py.

Response generated by 🤖 feifei-bot | chatgpt-4o-latest

0 replies

GreatV · 2024-11-27T03:57:33Z

GreatV
Nov 27, 2024
Maintainer

@vuthehuy1997 you may need try paddleocr 2.8.1.

0 replies

vuthehuy1997 · 2024-11-28T08:18:32Z

vuthehuy1997
Nov 28, 2024
Author

The issue you are experiencing, where the inference results using the exported model are inconsistent or incorrect compared to the results from training or direct inference with the best_accuracy checkpoint, is a common problem when exporting models for static inference. Below are potential causes and solutions based on the provided details and similar issues encountered in PaddleOCR:

Potential Causes

1. **Inconsistent Preprocessing Between Training and Inference**
   
   * The preprocessing steps (e.g., image normalization, resizing, padding) during training or inference might differ from those applied in `predict_structure.py`. If the exported inference model uses preprocessing steps different from those used during training, this could lead to mismatched results.

2. **Postprocessing Differences**
   
   * The way the predictions are decoded (e.g., parsing table structure) might vary between `infer_table.py` and `predict_structure.py`. This can cause discrepancies in the output.

3. **Character Dictionary Mismatch**
   
   * During inference, the wrong character dictionary might be used, leading to incorrect decoding of the table structure.

4. **Export Command Configuration**
   
   * The export process might not fully replicate the configuration used during training, such as missing parameters like `character_dict_path`, `max_text_length`, or `merge_no_span_structure`.

5. **Dynamic vs. Static Graph Differences**
   
   * In PaddleOCR, the training model (`best_accuracy`) uses a dynamic computational graph, while the exported model uses a static graph. Some components (e.g., batch normalization or variable shapes) may behave differently in static graphs compared to dynamic ones.

6. **Error in the Exported Model**
   
   * There might be a bug in the `export_model.py` implementation or the configuration passed to it, causing the exported model to incorrectly represent the trained model.

Steps to Debug and Resolve

1. Verify Preprocessing Consistency

* Compare the preprocessing steps in `SLANet_finetune.yml` with those in `predict_structure.py`. Key preprocessing components to check:
  
  * `DecodeImage`
  * `NormalizeImage` (mean, stddev)
  * `ResizeTableImage`
  * `PaddingTableImage`

* Ensure that the same transformations are applied in both training and inference.

2. Ensure Postprocessing Consistency

* Check the `PostProcess` block in your configuration file and ensure that parameters like `merge_no_span_structure` are consistent with how `predict_structure.py` decodes the output.

* Modify `predict_structure.py` to align it with `infer_table.py`'s decoding logic if needed.

3. Confirm Character Dictionary

* Verify that the correct character dictionary (`table_structure_dict_ch.txt`) is used. Ensure that the same dictionary is referenced during training, inference, and export.

4. Pass All Necessary Parameters During Export

* Ensure that all critical parameters from `SLANet_finetune.yml` are passed during export. For example:
  ```shell
  python3 tools/export_model.py -c configs/table/SLANet_finetune.yml \
    -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/SLANet_ch/best_accuracy \
       Global.save_inference_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
       Global.character_dict_path=ppocr/utils/dict/table_structure_dict_ch.txt \
       Global.max_text_length=500 \
       PostProcess.merge_no_span_structure=True
  ```

5. Run Direct Inference on Exported Model

* Test if the exported model is functional by using `tools/infer_table.py` directly on the exported model:
  ```shell
  python3 tools/infer_table.py -c configs/table/SLANet_finetune.yml \
    -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
      Global.infer_img=ppstructure/docs/table/table.jpg
  ```

* If the result is correct, the issue lies specifically in `predict_structure.py`.

6. Validate the Exported Model

* Use the `paddle_infer` API to check whether the exported model is working as expected:
  ```shell
  paddle_infer --model_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
    --image_dir=ppstructure/docs/table/table.jpg
  ```

* If the results are incorrect, the issue is likely during the export process.

7. Check for Known Issues

* Similar problems have been reported in PaddleOCR's GitHub repository. For example:
  
  * [Issue #14064](https://github.com/PaddlePaddle/PaddleOCR/issues/14064)
  * [Issue #14088](https://github.com/PaddlePaddle/PaddleOCR/discussions/14088)

* These issues often involve differences in preprocessing or bugs in the export script.

8. Debug Static Graph Behavior

* Some differences might arise due to the transition from a dynamic to a static graph. Try retraining with the `Global.infer_mode=True` option to force the model into inference mode during training:
  ```yaml
  Global:
    infer_mode: True
  ```

9. Test with a Pretrained Model

* As a sanity check, export the pretrained model (`ch_ppstructure_mobile_v2.0_SLANet_train`) without fine-tuning to ensure the export process works correctly:
  ```shell
  python3 tools/export_model.py -c configs/table/SLANet.yml \
    -o Global.pretrained_model=./weights/whl/table/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy \
       Global.save_inference_dir=./output/pretrained_inference
  ```

Recommended Workflow

1. **Preprocessing/Postprocessing Check**: Ensure preprocessing and postprocessing are consistent across training, inference, and export.

2. **Parameter Alignment**: Pass all required parameters during both training and export.

3. **Export Validation**: Test the exported model directly using `tools/infer_table.py`.

4. **Static Graph Debugging**: Use `infer_mode=True` during training to debug differences in dynamic vs. static graph behavior.

If none of these solutions resolve the issue, you might need to open a detailed issue on the PaddleOCR GitHub repository with a focus on the export process. Be sure to include:

* The exact commands you ran.

* Configuration files used.

* Input/output examples from both `infer_table.py` and `predict_structure.py`.

Response generated by 🤖 feifei-bot | chatgpt-4o-latest

Thanks for your support

But not work
About
Steps to Debug and Resolve

Verify Preprocessing Consistency
Ensure Postprocessing Consistency
Confirm Character Dictionary
Pass All Necessary Parameters During Export

I use config from github is configs/table/SLANet_ch.yml -> change data and add pretrained according to finetune instructions -> configs/table/SLANet_finetune.yml

Run Direct Inference on Exported Model

python3 tools/infer_table.py -c configs/table/SLANet_finetune.yml
-o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune
Global.infer_img=ppstructure/docs/table/table.jpg

Get results
AssertionError: The ../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune_infer/.pdparams does not exists!

tools/infer_table.py is used to run the infer model which does not support running mdel train

Validate the Exported Model

Use the paddle_infer API to check whether the exported model is working as expected:

paddle_infer --model_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune
--image_dir=ppstructure/docs/table/table.jpg

I do not use

Check for Known Issues

I can not get problem from it

Debug Static Graph Behavior

Some differences might arise due to the transition from a dynamic to a static graph. Try retraining with the Global.infer_mode=True option to force the model into inference mode during training:

Global:
infer_mode: True
-> Tested but the result is still not correct

Test with a Pretrained Model
python3 tools/export_model.py -c configs/table/SLANet_ch.yml -o
Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy
Global.save_inference_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_train_infer

python3 tools/infer_table.py -c configs/table/SLANet_ch.yml -o
Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy Global.infer_img=ppstructure/docs/table/table.jpg

cd ppstructure/
python table/predict_structure.py
--table_model_dir=../../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_train_infer
--table_char_dict_path=../ppocr/utils/dict/table_structure_dict_ch.txt
--image_dir=docs/table/table.jpg
--output=../output/

-> The result is still wrong

paddleocr 2.8.1 : I also tried to install it but it didn't work..

I noticed that the predicted output of the exported model is very small, and many,

the model train predicts the box to shape (8,10)
then the exported model outputs to (50, 60),

The result returns 50 instead of 8 coordinates xyxyxyxy

0 replies

liu-jiaxuan · 2025-01-06T04:01:31Z

liu-jiaxuan
Jan 6, 2025

please try the latest paddleocr branch.

0 replies

GreatV · 2025-01-07T02:05:50Z

GreatV
Jan 7, 2025
Maintainer

The issue you are experiencing, where the SLANet recognition table model produces correct results during training but incorrect results after exporting for inference, likely stems from differences in the configurations, preprocessing, or postprocessing steps between the training/finetuning and the inference stages. Below are the key areas to investigate and potential solutions.

1. Preprocessing and Postprocessing Mismatch

The preprocessing and postprocessing steps used during training/finetuning and inference must be consistent. Any mismatch in these steps can lead to incorrect results.

Steps to Check:

Ensure that the preprocessing steps for the inference script (table/predict_structure.py) match those in the training configuration (SLANet_finetune.yml). Specifically, check:
- Image resizing and normalization (e.g., ResizeTableImage, NormalizeImage)
- Padding settings (e.g., PaddingTableImage)
- Character dictionary paths (e.g., table_char_dict_path)

Fix:

Compare the preprocessing pipeline in predict_structure.py with the transforms section in your Train and Eval configurations. Update predict_structure.py if there are inconsistencies.

2. Export Configuration

The export command you used references the finetuned model, but the exported inference model may not include all necessary configuration details (e.g., postprocessing parameters, dictionary paths).

Steps to Check:

Ensure that the Global.character_dict_path in SLANet_finetune.yml is correctly passed during export. This is critical for decoding the predictions.
Verify that the Global.save_inference_dir contains the correct model files (inference.pdmodel, inference.pdiparams, and inference.pdiparams.info).

Fix:

Update the export command to explicitly include all necessary parameters:

python3 tools/export_model.py \
  -c configs/table/SLANet_finetune.yml \
  -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/SLANet_ch/best_accuracy \
     Global.character_dict_path=ppocr/utils/dict/table_structure_dict_ch.txt \
     Global.save_inference_dir=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune

3. Postprocessing Parameters

The PostProcess section in your SLANet_finetune.yml specifies merge_no_span_structure: True. This parameter might not be correctly applied during inference, leading to incorrect HTML structure predictions.

Steps to Check:

Ensure that the inference script (predict_structure.py) applies the same postprocessing logic as defined in the PostProcess configuration.

Fix:

Modify predict_structure.py to include the merge_no_span_structure parameter when decoding predictions.

4. Dynamic vs Static Shapes

If the model was trained with dynamic shapes (e.g., variable input sizes), but the inference model expects static shapes, this can lead to inconsistencies.

Steps to Check:

Verify the input shape compatibility between the training and inference stages.

Fix:

Ensure that the input image dimensions (e.g., [488, 488] as per PaddingTableImage) are consistent during both training and inference.

5. AMP (Automatic Mixed Precision)

You have enabled Global.use_amp=True during training. If AMP is not properly handled during inference, it could lead to numerical inconsistencies.

Steps to Check:

Verify whether AMP is supported and correctly implemented during inference.

Fix:

If AMP is causing issues, try disabling it during training and re-export the model:

python3 tools/train.py -c configs/table/SLANet_finetune.yml \
  -o Global.pretrained_model=./weights/whl/table/ch_ppstructure_mobile_v2.0_SLANet_train/best_accuracy \
     Global.use_amp=False

6. Model Architecture and Export Logic

There may be subtle issues in how the model is exported. For example, some layers or operations might behave differently during inference due to bugs in the export logic.

Steps to Check:

Test the exported model directly using the tools/infer_table.py script instead of predict_structure.py.

Fix:

Run this command to test the exported model:

python3 tools/infer_table.py \
  -c configs/table/SLANet_finetune.yml \
  -o Global.pretrained_model=../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
     Global.infer_img=ppstructure/docs/table/table.jpg

If the results are correct, the issue lies within predict_structure.py. Otherwise, recheck the export process.

7. Character Dictionary

The table_char_dict_path must match the dictionary used during training. If the wrong dictionary is used during inference, it can lead to incorrect predictions.

Steps to Check:

Verify that the dictionary paths are consistent (ppocr/utils/dict/table_structure_dict_ch.txt).

Fix:

Explicitly pass the correct dictionary path during inference:

python table/predict_structure.py \
  --table_model_dir=../../translate-pdf-scan/src/weights/table_rec/ch_ppstructure_mobile_v2.0_SLANet_finetune \
  --table_char_dict_path=../ppocr/utils/dict/table_structure_dict_ch.txt \
  --image_dir=docs/table/table.jpg \
  --output=../output/table

8. Debugging and Logs

Enable debugging logs to identify where the issue arises.

Steps to Check:

Add print statements in predict_structure.py to log intermediate results.

Fix:

Compare the outputs of tools/infer_table.py and predict_structure.py to pinpoint discrepancies.

Summary of Recommendations:

Verify preprocessing and postprocessing consistency.
Ensure correct export configurations, including the character dictionary path.
Check for dynamic vs static shape mismatches.
Disable AMP if it causes numerical issues.
Debug the inference script (predict_structure.py) for postprocessing flaws.
Test the exported model directly using tools/infer_table.py.

By systematically addressing these potential issues, you should be able to resolve the inconsistencies between training and inference results.

Response generated by feifei-bot | chatgpt-4o-latest

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Table Recognition: Training model is transferred to the inference model, the prediction effect is inconsistent? #14505

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Potential Causes

Steps to Debug and Resolve

1. Verify Preprocessing Consistency

2. Ensure Postprocessing Consistency

3. Confirm Character Dictionary

4. Pass All Necessary Parameters During Export

5. Run Direct Inference on Exported Model

6. Validate the Exported Model

7. Check for Known Issues

8. Debug Static Graph Behavior

9. Test with a Pretrained Model

Recommended Workflow

{{title}}

{{title}}

Select a reply

Table Recognition: Training model is transferred to the inference model, the prediction effect is inconsistent? #14505

vuthehuy1997 Nov 27, 2024

🔎 Search before asking

🐛 Bug (问题描述)

🏃‍♂️ Environment (运行环境)

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

Replies: 5 comments

GreatV Nov 27, 2024 Maintainer

Potential Causes

Steps to Debug and Resolve

1. Verify Preprocessing Consistency

2. Ensure Postprocessing Consistency

3. Confirm Character Dictionary

4. Pass All Necessary Parameters During Export

5. Run Direct Inference on Exported Model

6. Validate the Exported Model

7. Check for Known Issues

8. Debug Static Graph Behavior

9. Test with a Pretrained Model

Recommended Workflow

GreatV Nov 27, 2024 Maintainer

vuthehuy1997 Nov 28, 2024 Author

Potential Causes

Steps to Debug and Resolve

1. Verify Preprocessing Consistency

2. Ensure Postprocessing Consistency

3. Confirm Character Dictionary

4. Pass All Necessary Parameters During Export

5. Run Direct Inference on Exported Model

6. Validate the Exported Model

7. Check for Known Issues

8. Debug Static Graph Behavior

9. Test with a Pretrained Model

Recommended Workflow

Thanks for your support

liu-jiaxuan Jan 6, 2025

GreatV Jan 7, 2025 Maintainer

1. Preprocessing and Postprocessing Mismatch

Steps to Check:

Fix:

2. Export Configuration

Steps to Check:

Fix:

3. Postprocessing Parameters

Steps to Check:

Fix:

4. Dynamic vs Static Shapes

Steps to Check:

Fix:

5. AMP (Automatic Mixed Precision)

Steps to Check:

Fix:

6. Model Architecture and Export Logic

Steps to Check:

Fix:

7. Character Dictionary

Steps to Check:

Fix:

8. Debugging and Logs

Steps to Check:

Fix:

Summary of Recommendations:

vuthehuy1997
Nov 27, 2024

GreatV
Nov 27, 2024
Maintainer

GreatV
Nov 27, 2024
Maintainer

vuthehuy1997
Nov 28, 2024
Author

liu-jiaxuan
Jan 6, 2025

GreatV
Jan 7, 2025
Maintainer