Improve orientation predictor results for long words/lines

While trying to get EasyOCR and PaddleOCR text detection models to work,
a problem popped-up with the OnnxTR orientation predictor model. It
works fine for words, and since OnnxTR text detection step outputs
words, it works fine.

But in EasyOCR and PaddleOCR the output is lines, which by their nature
can be very long. And when long lines get squeezed to fit a 256x256
square with a ratio-preserving resize, too much information is lost in
one of the dimensions, which causes the output to be pretty random.

As a workaround, now we will truncate input images, when the ratio is
too extreme. On the surface, this shouldn't cause any issues, as even
if you truncate a perfectly fine word, it would still look like text,
so there will still be enough information to work with.

The truncation is done symmetrically on both sides, which is pretty
important. This way we have a higher chance to catch text and not the
margins/padding. Doing it only on one side gives worse results.

Somewhat surprisingly, based on the tests that required updating, it
also improved results there with the docTR model, which is also a
positive sign.

Author: Eswcvlad

pdfocr-onnxtr/src/main/java/com/itextpdf/pdfocr/onnxtr/orientation/OnnxOrientationPredictor.java

@@ -39,6 +39,12 @@ This file is part of the iText (R) project.
 public class OnnxOrientationPredictor
         extends AbstractOnnxPredictor<BufferedImage, TextOrientation>
         implements IOrientationPredictor {
+    /**
+     * If an input image ratio exceeds this value, then the image will be
+     * truncated.
+     */
+    private static final double IMAGE_RATIO_LIMIT = 4;
+
     /**
      * Configuration properties of the predictor.
      */
@@ -94,8 +100,29 @@ public OnnxOrientationPredictorProperties getProperties() {
      */
     @Override
     protected FloatBufferMdArray toInputBuffer(List<BufferedImage> batch) {
-        // Just your regular BCHW input
-        return BufferedImageUtil.toBchwInput(batch, properties.getInputProperties());
+        /*
+         * This orientation predictor was initially made based on the OnnxTR
+         * one. There the output of the text detection model is words, which
+         * are pretty narrow. So when they were resized to fit a square input
+         * buffer, there were no issues.
+         *
+         * But in the other tools, like EasyOCR and PaddleOCR, the text
+         * detection step outputs lines. And since they could be really wide,
+         * when they are resized for the square input buffer, there will be,
+         * like, 1 or 2 pixels in one of the dimensions, which is not enough
+         * for the orientation model to work.
+         *
+         * To counteract that we will just truncate the images. This shouldn't
+         * make the output worse, as you don't need the whole line or word to
+         * figure out the orientation. On the other hand, not having one of the
+         * dimensions getting degraded to nothing is much more useful.
+         */
+        final List<BufferedImage> truncatedBatch = new ArrayList<>(batch.size());
+        for (int i = 0; i < batch.size(); ++i) {
+            truncatedBatch.add(BufferedImageUtil.truncateToRatio(batch.get(i), IMAGE_RATIO_LIMIT));
+        }
+        // After that it is just a regular BCHW input conversion
+        return BufferedImageUtil.toBchwInput(truncatedBatch, properties.getInputProperties());
     }
 
     /**

pdfocr-onnxtr/src/main/java/com/itextpdf/pdfocr/onnxtr/util/BufferedImageUtil.java

@@ -289,6 +289,47 @@ private static BufferedImage resize(BufferedImage image, int width, int height,
         return result;
     }
 
+    /**
+     * Truncates the input image, so that neither width/height, nor
+     * height/width ratios exceed the limit.
+     *
+     * <p>
+     * If width/height ratio exceeds the limit, the image will be truncated
+     * on left and right equally.
+     *
+     * <p>
+     * If height/width ratio exceeds the limit, the image will be truncated
+     * on top and bottom equally.
+     *
+     * @param image      input image to truncate
+     * @param ratioLimit target ratio limit
+     *
+     * @return the truncated image
+     */
+    public static BufferedImage truncateToRatio(BufferedImage image, double ratioLimit) {
+        final int width = image.getWidth();
+        final int height = image.getHeight();
+
+        // If w/h ratio is too big, truncating by width
+        final double imageRatio = (double) width / height;
+        if (imageRatio > ratioLimit) {
+            final int newWidth = Math.max(1, (int) (ratioLimit * height));
+            final int newX = (width - newWidth) / 2;
+            return image.getSubimage(newX, 0, newWidth, height);
+        }
+
+        // If h/w ratio is too big, truncating by height
+        final double imageRatioInv = 1. / imageRatio;
+        if (imageRatioInv > ratioLimit) {
+            final int newHeight = Math.max(1, (int) (ratioLimit * width));
+            final int newY = (height - newHeight) / 2;
+            return image.getSubimage(0, newY, width, newHeight);
+        }
+
+        // Otherwise leaving as-is
+        return image;
+    }
+
     private static void putRgbImageWithNormalization(
             FloatBuffer outputBuffer,
             BufferedImage image,

pdfocr-onnxtr/src/main/java/com/itextpdf/pdfocr/onnxtr/util/Dimensions2D.java

@@ -0,0 +1,65 @@
+/*
+    This file is part of the iText (R) project.
+    Copyright (c) 1998-2025 Apryse Group NV
+    Authors: Apryse Software.
+
+    This program is offered under a commercial and under the AGPL license.
+    For commercial licensing, contact us at https://itextpdf.com/sales.  For AGPL licensing, see below.
+
+    AGPL licensing:
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+package com.itextpdf.pdfocr.onnxtr.util;
+
+import java.util.Objects;
+
+/**
+ * A basic 2-element tuple with a width and a height.
+ */
+public class Dimensions2D {
+    private final int width;
+    private final int height;
+
+    public Dimensions2D(int width, int height) {
+        this.width = width;
+        this.height = height;
+    }
+
+    public int getWidth() {
+        return width;
+    }
+
+    public int getHeight() {
+        return height;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (o == null || this.getClass() != o.getClass()) {
+            return false;
+        }
+        final Dimensions2D that = (Dimensions2D) o;
+        return width == that.width && height == that.height;
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(width, height);
+    }
+
+    @Override
+    public String toString() {
+        return width + "x" + height;
+    }
+}

pdfocr-onnxtr/src/test/java/com/itextpdf/pdfocr/onnxtr/orientation/OnnxOrientationPredictorTest.java

@@ -0,0 +1,78 @@
+/*
+    This file is part of the iText (R) project.
+    Copyright (c) 1998-2025 Apryse Group NV
+    Authors: Apryse Software.
+
+    This program is offered under a commercial and under the AGPL license.
+    For commercial licensing, contact us at https://itextpdf.com/sales.  For AGPL licensing, see below.
+
+    AGPL licensing:
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+package com.itextpdf.pdfocr.onnxtr.orientation;
+
+import com.itextpdf.pdfocr.TextOrientation;
+import com.itextpdf.test.ExtendedITextTest;
+
+import java.awt.image.BufferedImage;
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import javax.imageio.ImageIO;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+@Tag("IntegrationTest")
+public class OnnxOrientationPredictorTest extends ExtendedITextTest {
+    private static final String TEST_DIRECTORY =
+            "./src/test/resources/com/itextpdf/pdfocr/onnxtr/orientation/OnnxOrientationPredictorTest/";
+    private static final String TARGET_DIRECTORY =
+            "./target/test/resources/com/itextpdf/pdfocr/onnxtr/orientation/OnnxOrientationPredictorTest/";
+    private static final String MOBILENETV3 =
+            "./src/test/resources/com/itextpdf/pdfocr/models/mobilenet_v3_small_crop_orientation-5620cf7e.onnx";
+    private static IOrientationPredictor PREDICTOR;
+
+    @BeforeAll
+    public static void beforeClass() {
+        createOrClearDestinationFolder(TARGET_DIRECTORY);
+        PREDICTOR = OnnxOrientationPredictor.mobileNetV3(MOBILENETV3);
+    }
+
+    @AfterAll
+    public static void afterClass() throws Exception {
+        PREDICTOR.close();
+    }
+
+    public static Iterable<Object[]> predictWithLongLinesParams() {
+        return Arrays.asList(new Object[][] {
+                {TextOrientation.HORIZONTAL, "line_0.png"},
+                {TextOrientation.HORIZONTAL_ROTATED_90, "line_90.png"},
+                {TextOrientation.HORIZONTAL_ROTATED_180, "line_180.png"},
+                {TextOrientation.HORIZONTAL_ROTATED_270, "line_270.png"},
+        });
+    }
+
+    @ParameterizedTest(name = "predictWithLongLines: {1}")
+    @MethodSource("predictWithLongLinesParams")
+    public void predictWithLongLines(TextOrientation expectedResult, String inputFileName) throws IOException {
+        final BufferedImage inputImage = ImageIO.read(new File(TEST_DIRECTORY + inputFileName));
+        final TextOrientation actualResult = PREDICTOR.predict(Collections.singleton(inputImage)).next();
+        Assertions.assertEquals(expectedResult, actualResult);
+    }
+}

pdfocr-onnxtr/src/test/java/com/itextpdf/pdfocr/onnxtr/util/BufferedImageUtilTest.java

@@ -34,6 +34,8 @@ This file is part of the iText (R) project.
 import org.junit.jupiter.api.Assertions;
 import org.junit.jupiter.api.Tag;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
 
 @Tag("UnitTest")
 public class BufferedImageUtilTest extends ExtendedITextTest {
@@ -66,6 +68,23 @@ public void toBchwInputRgbBasicTest() {
         toBchwInputBasicTest(expectedShape, expectedData, images, props);
     }
 
+    public static Iterable<Object[]> truncateToRatioTestParams() {
+        return Arrays.asList(new Object[][] {
+                {new Dimensions2D(100, 20), new Dimensions2D(100, 20), 8.},
+                {new Dimensions2D(160, 20), new Dimensions2D(1000, 20), 8.},
+                {new Dimensions2D(100, 800), new Dimensions2D(100, 2000), 8.},
+        });
+    }
+
+    @ParameterizedTest(name = "truncateToRatioTest: {1}")
+    @MethodSource("truncateToRatioTestParams")
+    public void truncateToRatioTest(Dimensions2D expectedSize, Dimensions2D inputSize, double ratioLimit) {
+        final BufferedImage img = newBlankInputImage(inputSize);
+        final BufferedImage truncated = BufferedImageUtil.truncateToRatio(img, ratioLimit);
+        Assertions.assertEquals(expectedSize.getWidth(), truncated.getWidth());
+        Assertions.assertEquals(expectedSize.getHeight(), truncated.getHeight());
+    }
+
     private static void toBchwInputBasicTest(
             long[] expectedShape,
             float[] expectedData,
@@ -79,10 +98,14 @@ private static void toBchwInputBasicTest(
         Assertions.assertArrayEquals(expectedData, actualData, 1E-6F);
     }
 
+    private static BufferedImage newBlankInputImage(Dimensions2D dims) {
+        return new BufferedImage(dims.getWidth(), dims.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
+    }
+
     private static BufferedImage newRgbImage(int width, int height, int[] pixels) {
         final BufferedImage img = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
         final WritableRaster raster = img.getRaster();
         raster.setDataElements(0, 0, width, height, pixels);
         return img;
     }
-}
+}

pdfocr-onnxtr/src/test/resources/com/itextpdf/pdfocr/OcrPdfTest/cmp_textWithImages.pdf

@@ -33,15 +33,14 @@ X?
 a
 what is Lovew
 Lorem Ipsum simply dummy text of the printing typesetting
-industry. Lorem Ipsum has been the standard dummy
+industry. Lorem Ipsum has been the industry's standard dummy
 text ever since the 1500s, when an unknown printer took galley
 of type and scrambled it to make type specimen book. It has
 survived not only five centuries, but also the leap into electronic
 typesetting remaining essentially unchanged. It was popularised in
 the 1960s with the release of Letraset sheets containing Lorem
 Ipsum passages, and more recently with desktop publishing
 software like Aldus PageMaker including versions of Lorem Ipsum.
-sAusnpur
 pue s!
 jwMsdI
 a

pdfocr-onnxtr/src/test/resources/com/itextpdf/pdfocr/OnnxCreateTxtFileTest/cmp_createTxtFile.txt

@@ -6,27 +6,23 @@ cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non pr
 qui officia deserunt mollit anim id est laborum.
 Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium,
 totam rem aperiam. eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta
-sunt explicabo, Nemo enim ipsam quia voluptas sit aut odit aut fugit, sed quia
+sunt explicabo, Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia
 consequuntur magni dolores eos ratione voluptatem sequi nesciunt. Neque porro quisquam est,
 qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi
 tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem, Ut enim ad minima
-veniam, quis nostrum ullam corporis suscipit laboriosam, nisi ut aliquid ex ea
+veniam, quis nostrum exercitatationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea
 commodi consequatur? Quis autem vel eum iure reprehenderit qui ea voluptate velit esse quam
 nihil molestiae consequatur, vel illum qui dolorem eum fugiat volupt-as nulla pariatur?
-At vero eos et accusamus et justo odio dignissimos ducimus qui blanditiis voluptatum
+At vero eos et accusamus et justo odio dignissimos ducimus qui blanditiis praesentium voluptatum
 deleniti atque corrupti dolores et quas molestias excepturi sint occaecati cupiditate non
 provident, similique sunt in culpa qui officia deserunt mollitia animi, id est laborum et dolorum fuga.
 Et harum quidem rerum facilis est et expedita distinctio, Nam libero tempore, cum soluta nobis est
 eligendi optio cumque nihil impedit minus id maxime placeat facere possimus, omnis
 voluptas assumenda est, omnis dolor repellendus. Temporibus autem quibusdam et aut officiis
-debitis aut rerum saepe eveniet ut et voluptates repudiandae sint et molestiae non
+debitis aut rerum necessitatibus saepe eveniet ut et voluptates repudiandae sint et molestiae non
 recusandae. Itaque earum rerum hic tenetur a sapiente delectus, ut aut reiciendis voluptatibus
 maiores alias consequatur aut perferendis doloribus asperiores repellat.
-singessepou
 ponb onb
 sonb
-unguaseeid
 onb
-weuopepexe
-Inb
-ngeuadse wajeidnjoA
+Inb