(InvalidArgument) Broadcast dimension mismatch

[C01DER]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

I have searched the existing bugs, but they have been fixed. But I still hit this kind of issue.

  File "/opt/mineru_venv/lib/python3.10/site-packages/paddleocr/tools/infer/predict_rec.py", line 616, in __call__
    self.predictor.run()
    │    │         └ <instancemethod run at 0x7fdefe2e68c0>
    │    └ <paddle.base.libpaddle.PaddleInferPredictor object at 0x7fdcfd7ce070>
    └ <tools.infer.predict_rec.TextRecognizer object at 0x7fdcfd9a7cd0>

ValueError: (InvalidArgument) Broadcast dimension mismatch. Operands could not be broadcast together with the shape of X = [6, 96, 3, 20] and the shape of Y = [6, 96, 4, 20]. Received [3] in X is not equal to [4] in Y at i:2.
  [Hint: Expected x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1 == true, but received x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1:0 != true:1.] (at ../paddle/phi/kernels/funcs/common_shape.h:84)
  [operator < elementwise_add > error]

🏃‍♂️ Environment (运行环境)

Use MinerU to reproduce the problem.

# Use the official Ubuntu base image
FROM ubuntu:22.04

# Set environment variables to non-interactive to avoid prompts during installation
ENV DEBIAN_FRONTEND=noninteractive

# Update the package list and install necessary packages
RUN apt-get update && \
    apt-get install -y \
        software-properties-common && \
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update && \
    apt-get install -y \
        python3.10 \
        python3.10-venv \
        python3.10-distutils \
        python3-pip \
        wget \
        git \
        libgl1 \
        libglib2.0-0 \
        && rm -rf /var/lib/apt/lists/*

# Set Python 3.10 as the default python3
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1

# Create a virtual environment for MinerU
RUN python3 -m venv /opt/mineru_venv

# Activate the virtual environment and install necessary Python packages
RUN /bin/bash -c "source /opt/mineru_venv/bin/activate && \
    pip3 install --upgrade pip && \
    wget https://gitee.com/myhloli/MinerU/raw/master/requirements-docker.txt && \
    pip3 install -r requirements-docker.txt --extra-index-url https://wheels.myhloli.com -i https://mirrors.aliyun.com/pypi/simple && \
    pip3 install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/"

# Copy the configuration file template and install magic-pdf latest
RUN /bin/bash -c "wget https://gitee.com/myhloli/MinerU/raw/master/magic-pdf.template.json && \
    cp magic-pdf.template.json /root/magic-pdf.json && \
    source /opt/mineru_venv/bin/activate && \
    pip3 install -U magic-pdf"

# Download models and update the configuration file
RUN /bin/bash -c "pip3 install modelscope && \
    wget https://gitee.com/myhloli/MinerU/raw/master/scripts/download_models.py && \
    python3 download_models.py && \
    sed -i 's|cpu|cuda|g' /root/magic-pdf.json"

# Set the entry point to activate the virtual environment and run the command line tool
ENTRYPOINT ["/bin/bash", "-c", "source /opt/mineru_venv/bin/activate && exec \"$@\"", "--"]

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

test-1.pdf
Use this file be able to reproduce the issue.

[jingsongliujing]

This error is caused by the fact that the format converted by magic-pdf does not match the input format of paddleocr. Please check whether the format output by magic-pdf is in line with the input format of paddleocr.

[GreatV]

The error (InvalidArgument) Broadcast dimension mismatch suggests a shape mismatch issue during a tensor operation in the PaddleOCR pipeline. Here's a detailed explanation of the problem and steps to resolve it:

---

📝 Problem Analysis

1. Shape Mismatch: The error (InvalidArgument) Broadcast dimension mismatch indicates that two tensors involved in an operation (e.g., elementwise_add) have incompatible dimensions. Specifically:

   • The shapes of the tensors are [6, 96, 3, 20] (tensor X) and [6, 96, 4, 20] (tensor Y).
   • The mismatch occurs at dimension i:2, where tensor X has size 3 and tensor Y has size 4.

2. SAR Model and rec_image_shape: The issue may arise because the --rec_image_shape parameter is incorrectly set.

   • In the command:

     python3 tools/infer/predict_rec.py --rec_image_shape="3, 100, 100,320"

     The specified rec_image_shape="3, 100, 100, 320" is invalid. The SAR model typically expects a format closer to "3, 48, 320" or "3, 32, 320".

3. Transpose Issue: The secondary error shows a problem with the transpose2 operator:

   ValueError: (InvalidArgument) shape should have the same dim with perm, but received shape size is:4, perm size is:3.
   

   This is likely caused by incorrect input dimensions being passed to the model due to the wrong rec_image_shape.

---

🔧 Steps to Fix

1. Correct the rec_image_shape Parameter

• The correct rec_image_shape depends on the OCR model being used:
  • For PP-OCRv3: Use --rec_image_shape="3, 48, 320".
  • For PP-OCRv2 or older models: Use --rec_image_shape="3, 32, 320".
• Update your command accordingly:

  python3 tools/infer/predict_rec.py --image_dir="./predict/0.jpg" \
  --rec_model_dir="./inference/rec/sar" \
  --rec_image_shape="3, 48, 320" \
  --rec_char_dict_path="./ppocr/utils/vi_dict.txt"

2. Check the Model Directory

• Ensure that the --rec_model_dir points to the correct SAR model directory. The SAR model should be compatible with the rec_image_shape you specify.

3. Verify Input Image and Preprocessing

• Ensure that ./predict/0.jpg is a valid image and doesn't have unusual dimensions.
• If needed, preprocess the image to fit the expected input size of the SAR model.

4. Update PaddleOCR and PaddlePaddle

• Ensure you are using compatible versions of PaddleOCR and PaddlePaddle:
  • PaddlePaddle: Version 2.6.1 (or a version supported by your SAR model).
  • PaddleOCR: Check the latest release and update if necessary.

  pip install --upgrade paddleocr paddlepaddle-gpu

5. Debugging Tips

• Add debugging prints to verify tensor shapes in predict_rec.py before the error occurs. For example, log shapes before calling self.predictor.run():

  print("Input tensor shape:", input_tensor.shape)

6. Consider Known Issues

• Issue Link: The linked issue (#13874) discusses a similar transpose error. The root cause was incorrect configuration or preprocessing. Ensure your setup aligns with the SAR model's requirements.

---

🛠️ Final Command Example

Here’s an example of a corrected command:

python3 tools/infer/predict_rec.py --image_dir="./predict/0.jpg" \
--rec_model_dir="./inference/rec/sar" \
--rec_image_shape="3, 48, 320" \
--rec_char_dict_path="./ppocr/utils/vi_dict.txt"

---

📚 Additional Notes

1. If the issue persists:

   • Test with a different model (e.g., PP-OCRv2 or PP-OCRv3) to confirm if the SAR model configuration is causing the error.
   • Use a sample image provided in the PaddleOCR repository to rule out issues with your custom image.

2. If you need further assistance, consider sharing the following:

   • Logs for debugging.
   • The SAR model configuration and training details.
   • A minimal reproducible example.

By addressing the rec_image_shape and ensuring compatibility with the SAR model, you should be able to resolve the error.

---

Response generated by feifei-bot | chatgpt-4o-latest