SVTR - Fine-Tuning of en_PP-OCRv4_rec Model

[VishyAnand28]

Dear Paddle Community,

I have some queries to ask regarding the model performance. I humbly request fot guidance regarding this.
Context: I am fine-tuning text recognition with around 7200 as training and around 800 as eval data. Please check both my train log and config file for your reference.

Queries:

1. Why does the Train iteration only contain 108 iters? Even when I change batch_size_per_card to 1, it still shows 108. When I change batch_size_per_card to 1 for eval dataset, I am able to see value of 800. Does 108 for training have to do with Multiscaledataset?
2. The best epoch is 430 as per log file and corresponding accuracy was 1.0000 (clearly overfitting), although, this metric is displayed at end:
   best metric, acc: 0.6927083243136937, is_float16: False, norm_edit_dis: 0.909473259516582, fps: 419.28201347491864, best_epoch: 430
   what is this discrepancy about?
3. Finally, my model seems to have a worse case of overfitting. I saw to not implement GTC strategy as per fine-tuning document but what does GTC correspond to?
4. How to avoid overfitting or to improve my config file?

I apologize for so many queries but I massively appreciate the effort, documentation put by multiple authors of paddle community. Thanks in advance!
I humbly request @GreatV @WenmuZhou @LDOUBLEV @MissPenguin @tink2123 @UserWangZz and others ........ for guidance.
train_mod.log

Config file in comments!

[VishyAnand28]

Config File:

Global:
debug: false
use_gpu: true
epoch_num: 500
log_smooth_window: 20
print_batch_step: 10
save_model_dir: C:/Users/I011786/Pictures/01_Paddle/04_Paddle_Models/04_recognizer_pretrained/01_SVTR_Default/v1_train
save_epoch_step: 10
eval_batch_step:

• 0
• 2000
  cal_metric_during_train: true
  pretrained_model: C:/Users/I011786/Pictures/01_Paddle/04_Paddle_Models/04_recognizer_pretrained/01_SVTR_Default/en_PP-OCRv4_rec_train/best_accuracy.pdparams
  checkpoints: null
  save_inference_dir: null
  use_visualdl: false
  infer_img: null
  character_dict_path: C:/Users/I011786/Pictures/01_Paddle/04_Paddle_Models/04_recognizer_pretrained/01_SVTR_Default/en_dict.txt
  max_text_length: 25
  infer_mode: false
  use_space_char: true
  distributed: true
  save_res_path: null
  Optimizer:
  name: Adam
  beta1: 0.9
  beta2: 0.999
  lr:
  name: Cosine
  learning_rate: 0.0005
  warmup_epoch: 5
  regularizer:
  name: L2
  factor: 3.0e-05
  Architecture:
  model_type: rec
  algorithm: SVTR_LCNet
  Transform: null
  Backbone:
  name: PPLCNetV3
  scale: 0.95
  Head:
  name: MultiHead
  head_list:
  • CTCHead:
    Neck:
    name: svtr
    dims: 120
    depth: 2
    hidden_dims: 120
    kernel_size:
    - 1
    - 3
    use_guide: true
    Head:
    fc_decay: 1.0e-05
  • NRTRHead:
    nrtr_dim: 384
    max_text_length: 25
    Loss:
    name: MultiLoss
    loss_config_list:
• CTCLoss: null
• NRTRLoss: null
  PostProcess:
  name: CTCLabelDecode
  Metric:
  name: RecMetric
  main_indicator: acc
  ignore_space: false
  Train:
  dataset:
  name: MultiScaleDataSet
  ds_width: false
  data_dir: C:/Users/I011786/Pictures/01_Paddle
  ext_op_transform_idx: 1
  label_file_list: C:/Users/I011786/Pictures/01_Paddle/02_paddle_img_rec/paddle_rec_prep_train.txt
  transforms:
  • DecodeImage:
    img_mode: BGR
    channel_first: false
  • RecConAug:
    prob: 0.5
    ext_data_num: 2
    image_shape:
    • 48
    • 320
    • 3
      max_text_length: 25
  • RecAug: null
  • MultiLabelEncode:
    gtc_encode: NRTRLabelEncode
  • KeepKeys:
    keep_keys:
    • image
    • label_ctc
    • label_gtc
    • length
    • valid_ratio
      sampler:
      name: MultiScaleSampler
      scales:
  • • 320
    • 32
  • • 320
    • 48
  • • 320
    • 64
      first_bs: 96
      fix_bs: false
      divided_factor:
  • 8
  • 16
    is_training: true
    loader:
    shuffle: true
    batch_size_per_card: 96
    drop_last: true
    num_workers: 8
    Eval:
    dataset:
    name: SimpleDataSet
    data_dir: C:/Users/I011786/Pictures/01_Paddle
    label_file_list: C:/Users/I011786/Pictures/01_Paddle/02_paddle_img_rec/paddle_rec_prep_eval.txt
    transforms:
  • DecodeImage:
    img_mode: BGR
    channel_first: false
  • MultiLabelEncode:
    gtc_encode: NRTRLabelEncode
  • RecResizeImg:
    image_shape:
    • 3
    • 48
    • 320
  • KeepKeys:
    keep_keys:
    • image
    • label_ctc
    • label_gtc
    • length
    • valid_ratio
      loader:
      shuffle: false
      drop_last: false
      batch_size_per_card: 128
      num_workers: 4
      profiler_options: null

[GreatV]

Addressing Your Queries on Fine-Tuning the en_PP-OCRv4_rec Model:

Here’s a detailed response to your questions based on the context and the shared discussions:

---

1. Why does the Train iteration only contain 108 iters?

This behavior is primarily related to how the dataset and batch size interact with the MultiScaleDataset configuration in PaddleOCR. Here's a breakdown:

• Number of Iterations (Iters): The number of training iterations (iters) is calculated as:
  [
  \text{iters} = \frac{\text{size of training dataset}}{\text{batch size per card} \times \text{number of GPUs}}
  ]

  If your training dataset is around 7200 samples and you are using batch_size_per_card = 64 (default), the number of iterations would be:
  [
  \text{iters} = \frac{7200}{64} \approx 112.5
  ]
  PaddleOCR likely rounds this down slightly.

• Impact of MultiScaleDataset: The MultiScaleDataset in PaddleOCR may further adjust the effective number of samples per epoch based on how it augments or preprocesses the data. This could lead to the observed discrepancy in iterations.

• Why Changing batch_size_per_card Affects Evaluation but Not Training: This discrepancy likely arises because the evaluation dataset (800 samples) is smaller and unaffected by MultiScaleDataset. When you set batch_size_per_card = 1, evaluation iterations match the dataset size. In contrast, the training dataset undergoes additional processing, leading to the fixed 108 iters.

---

2. Why is there a discrepancy between the epoch accuracy and the final metric?

• Epoch Accuracy (1.0000): The accuracy shown during the epoch (1.0000) is for the training dataset. This clearly indicates overfitting, as the model has memorized the training data perfectly.

• Final Metric (acc = 0.6927): The final metric is computed on the evaluation dataset and reflects the model's generalization. The significant drop from 1.0000 to 0.6927 indicates that the model is unable to perform well on unseen data, further confirming overfitting.

• Overfitting Indicator: Overfitting is expected when the model performs exceptionally well on the training set but poorly on the evaluation set. This implies that the model is not learning general features but is instead memorizing the training data.

---

3. What does GTC correspond to?

• GTC in PaddleOCR: GTC refers to Guided Training of Characters, a data augmentation or training strategy that focuses on improving recognition accuracy by guiding the model to focus on key character features. It is typically used for large-scale training or pretraining.

• Why Not Use GTC for Fine-Tuning? For fine-tuning tasks, GTC is usually disabled because:

  • Fine-tuning generally involves smaller datasets where the model needs to adapt to specific features of the new data.
  • GTC may introduce unnecessary complexity and slow down convergence during fine-tuning.

---

4. How to avoid overfitting and improve the config file?

To address overfitting and improve your model's performance, consider the following steps:

1. Data Augmentation:

   • Use robust data augmentation techniques (e.g., rotation, cropping, adding noise) to artificially increase the diversity of your training dataset.
   • Ensure that MultiScaleDataset is configured correctly to generate a variety of training samples.

2. Regularization:

   • Add dropout layers (if supported by the model) to reduce overfitting.
   • Experiment with weight decay (L2 regularization) by setting a nonzero value in your optimizer configuration.

3. Learning Rate Adjustment:

   • Use a smaller learning rate for fine-tuning. For instance, try reducing the learning rate by a factor of 10 compared to the pretraining rate.

4. Early Stopping:

   • Use an early stopping strategy to terminate training when the validation loss stops improving, thereby preventing the model from overfitting.

5. Batch Size and Iterations:

   • Experiment with smaller batch sizes to allow the model to generalize better. Smaller batch sizes can make the optimization process noisier, which sometimes improves generalization.

6. Pretrained Weights:

   • Ensure that the pretrained weights you are using are compatible with your model architecture. The warnings like backbone.conv1.conv.weight not in model suggest a potential mismatch between the pretrained weights and the model configuration.

7. Evaluation Metrics:

   • Monitor both accuracy (acc) and normalized edit distance (norm_edit_dis) to get a more comprehensive view of the model's performance. A low norm_edit_dis indicates better recognition quality.

8. Increase Dataset Size:

   • If possible, increase the size of your training dataset. Fine-tuning on a small dataset often leads to overfitting, as the model cannot learn diverse features.

---

Additional Resources:

• Fine-Tuning Documentation: Refer to the official PaddleOCR fine-tuning guide: Fine-Tuning in PaddleOCR.
• Discussion on Model Configurations: Check out this discussion for insights into improving model configurations: Fine-Tuning Discussion.

---

By addressing these points, you should be able to mitigate overfitting and improve your model's generalization. Feel free to share your updated config file or logs for further assistance.

---

Response generated by feifei-bot | chatgpt-4o-latest

[l03126]

Oh understand Vishy. Can I know how did you split the dataset quantitity? I did 70:15:15. Try reducing your eval_batch_step by alot as well. Helped me improve the acc. I have 2100 images and has a eval step at 50

[VishyAnand28]

I have 8000 images so far. I have 7200 as training and 800 as eval dataset contributing to 90-10 split. Would you recommend me to reduce this further?
Test dataset is not included in this list and it is separate.

[VishyAnand28]

@l03126 since I am using MultiScaleDataSet, the batch size has no impact on training dataset. No matter what batch size I give, it is showing 108 iters. This was my first question that I asked above. Please check my config and training log files, you will know. It would also be the similar case I suppose to u.

[l03126]

I reckon maybe it may help. Maybe try 80:20 or even 70:30. Reducing the batch eval may also help

[VishyAnand28]

okay thanks will try it.