📄 Awesome OCR multiple programing languages toolkits based on ONNX Runtime, OpenVINO, MNN, PaddlePaddle, TensorRT and PyTorch.
Optim python
SWHL3d94ffe01c9c
diff --git a/doc/FAQ.md b/doc/FAQ.mddeleted file mode 100644index fa5aab5..0000000--- a/doc/FAQ.md+++ /dev/null@@ -1,48 +0,0 @@-## 常见问题-#### Q14: Linux部署python的程序时,import cv2,会报`ImportError: ligGL.so.1: cannot open shared object file: No such file or directory`?-**A:**: [解决方法](https://stackoverflow.com/a/63978454/3335415-) 有两个(来自群友ddeef):- 1. 安装`opencv-python-headless`取代`opencv-python`;- 2. 运行`sudo apt-get install -y libgl1-mesa-dev`--#### Q13: 询问下,我编译出来的进程在win7下面通过cmd调用,发生了崩溃的情况?-**A:**: 不支持win7 (by @如果我有時光機)--#### Q12: 能不能搞个openmmlab类似的那个提取信息的?-**A:** 这个目前正在调研测试当中,如果mmocr中关键信息提取效果还可以,后期会考虑整合进来--#### Q11: RapidOCR和PaddleOCR是什么关系呢?-**A:** RapidOCR是将PaddleOCR的预训练模型转为onnx模型,不依赖paddle框架,方便各个平台部署--#### Q10: onnxruntime arm32 有人编译过吗? 我编译成功了,但是使用的时候libonnxruntime.so:-1: error: file not recognized: File format not recognized 应该是版本不匹配-**A:** 没遇到过。我是直接在当前平台编译的,我们用的是arm。估计是平台不兼容,建议在本身平台上编译。没遇到过问题。通常出在交叉编译方式下。--#### Q9: 请问一下c++ demo必须要vs2017及以上版本吗?-**A:** 最好用vs2019--#### Q8: 可以达到百度EasyEdge Free App的效果吗?-**A:** edge的模型应该没有开源。百度开源的模型里server det的识别效果可以达到,但是模型比较大。--#### Q7: 我用c++推理onnx貌似是cpu推理的,gpu没有反应?-**A:** 如果想用GPU的话,需要安装onnxruntime-gpu版,自己在onnxruntime的代码中添加EP (execution provider)。我们的定位是通用,只用cpu推理。--#### Q6: 您好,我想部署下咱们的ocr识别,有提供linux版本的ocr部署包吗?-**A:** linux版本的自己编译即可, 可以参考我们的action中的脚本;其实编译非常容易,安装个opencv后,在cmakelists.txt中修改一下onnxruntime的路径即可,具体参考这个: https://github.com/RapidOCR/RapidOCR/blob/main/.github/workflows/make-linux.yml--#### Q5: onnxruntime编译好的C++库,哪里可以下载到?-**A:** 从这里:https://github.com/RapidOCR/OnnxruntimeBuilder/releases/tag/1.7.0--#### Q4: 目前简单测试环境是 Win10 + Cygwin + gcc + 纯C编程,可以在C程序中直接接入简单OCR功能吗?-**A:** 直接使用API就行,API就是由c导出的--#### Q3: 模型下载地址--**A:** [提取码:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg)--#### Q2: onnxruntime 1.7 下出错:onnxruntime::SequentialExecutor::Execute] Non-zero status code returned while running ScatterND node. Name:'ScatterND@1' Status Message: updates--**A:** 由于模型只支持`onnxruntime=1.5.0`导致,请更新模型,下载地址见`Q3`--#### Q1: 边缘总有一行文字无法识别,怎么办?--**A:** 在 padding 参数中添加一个值 ,默认是0,你可以添加5或10, 甚至更大,直到能识别为止。注意不要添加过大,会浪费内存。\ No newline at end of filediff --git a/doc/README_en.md b/doc/README_en.mddeleted file mode 100644index 4035c92..0000000--- a/doc/README_en.md+++ /dev/null@@ -1,192 +0,0 @@--<div align="center">- <img src="./assets/RapidOCR_LOGO.png" width="45%" height="45%"/>-</div>--# RapidOCR--[简体中文](../README.md) | English--<p align="left">- <a href="https://colab.research.google.com/github/RapidAI/RapidOCR/blob/main/assets/RapidOCRDemo.ipynb" target="_blank"><img src="./assets/colab-badge.svg" alt="Open in Colab"></a>- <a href="./LICENSE"><img src="https://img.shields.io/badge/License-Apache%202-dfd.svg"></a>- <a href=""><img src="https://img.shields.io/badge/Python-3.6+-aff.svg"></a>- <a href=""><img src="https://img.shields.io/badge/OS-Linux%2C%20Win%2C%20Mac-pink.svg"></a>- <a href="https://github.com/RapidAI/RapidOCR/graphs/contributors"><img src="https://img.shields.io/github/contributors/RapidAI/RapidOCR?color=9ea"></a>- <a href="https://github.com/RapidAI/RapidOCR/stargazers"><img src="https://img.shields.io/github/stars/RapidAI/RapidOCR?color=ccf"></a>-</p>--<details>- <summary>Contents</summary>--- [RapidOCR](#rapidocr)- - [Introduction](#introduction)- - [Recently updates(more)](#recently-updatesmore)- - [🍿2022-05-15 update](#2022-05-15-update)- - [😀2022-05-12 upadte](#2022-05-12-upadte)- - [🎧2022-04-04 update](#2022-04-04-update)- - [FAQ](#faq)- - [SDK compilation status](#sdk-compilation-status)- - [Online demo](#online-demo)- - [Directory structure](#directory-structure)- - [Current Progress](#current-progress)- - [Model related](#model-related)- - [Download models( Google Drive|[Baidu NetDisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) )](#download-models-google-drivebaidu-netdisk-)- - [Model to onnx](#model-to-onnx)- - [Compared](#compared)- - [Text Det](#text-det)- - [Text Recognition](#text-recognition)- - [PaddleOCR-FAQ](#paddleocr-faq)- - [Original initiator and start-up author](#original-initiator-and-start-up-author)- - [Authorization](#authorization)- - [Contact us](#contact-us)- - [Demo](#demo)- - [Demonstration with C++/JVM](#demonstration-with-cjvm)- - [Demonstration with .Net](#demonstration-with-net)- - [Demonstratioin with multi_language](#demonstratioin-with-multi_language)-</details>---## Introduction-- Completely open source, free and support offline deployment of multi-platform and multi-language OCR SDK-- **Chinese Advertising**: Welcome to join our QQ group to download the model and test program, QQ group number: 887298230-- **Cause**: Baidu paddlepaddle engineering is not very good, in order to facilitate everyone to perform OCR reasoning on various terminals, we convert it to onnx format, use ``python/c++/java/swift/c#'' to change It is ported to various platforms.--- **Name Source**: Light, fast, economical and smart. OCR technology based on deep learning technology focuses on artificial intelligence advantages and small models, with speed as the mission and effect as the leading role.--- Based on Baidu's open source PaddleOCR model and training, anyone can use this inference library, or use Baidu's paddlepaddle framework for model optimization according to their own needs.--## Recently updates([more](./change_log_en.md))-#### 🍿2022-05-15 update-- Add the ONNX model converted from the PaddleOCR v3 rec model, just go to the network disk to download and replace it. ([Baidu Netdisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing ))-- Added a comparison table of the effects of each version of the text recognition model. For details, click [Comparison of the effects of various versions of ONNX models] (#Comparison of the effects of various versions of onnx models). The text recognition model of v3 is not as good as the previous one in terms of the indicators on the test set constructed by itself.--#### 😀2022-05-12 upadte-- Add the ONNX model converted from the PaddleOCR v3 det model, download it directly from the network disk, and replace it. ([Baidu Netdisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing ))-- Added a comparison table of text detection model effects of various versions. For details, click [Comparison of the effects of various versions of ONNX models] (#Comparison of the effects of various versions of onnx models). The text detection model of v3 is better than the previous v2 in terms of the indicators on the test set constructed by itself, and it is recommended to use it.--#### 🎧2022-04-04 update-- Add suport for OpenVINO under python-- Give the performance comparison table of OpenVINO and ONNXRuntime-- For details:[python/README](./python/README.md)--## [FAQ](./doc/FAQ.md)--## SDK compilation status-Since ubuntu users are all commercial users and have the ability to compile, pre-compiled packages are not provided for the time being, and they can be compiled by themselves.--| Platform | Compilation Status | Offer Status |-| --------------- | -------- | -------- |-| Windows x86/x64 | [](https://github.com/RapidOCR/RapidOCR/actions/workflows/windows-all-build.yaml) | [Download Link](https://github.com/RapidAI/RapidOCR/releases) |-| Linux x64 | [](https://github.com/RapidOCR/RapidOCR/actions/workflows/make-linux.yml) | Not available yet, compile by yourself |--## Online demo-- [Web demo](http://rapidocr.51pda.cn:9003/)-- The model combination used in the demo is: **ch_PP-OCRv2 det** + **mobile cls** + **mobile rec**-- Environment: `4 Core AMD EPYC 7K62 48-Core Processor `-- **Gif Demo**:- <div align="center">- <img src="./assets/demo.gif" width="100%" height="100%">- </div>--## Directory structure-<details>- <summary>click to expand</summary>-- RapidOCR- ├── android # Android project directory- ├── api4cpp # C language cross-platform interface library source code directory, directly compile with CMakelists.txt under the root- ├── assets # Some pictures for demonstration, not a test set- ├── commonlib # common library- ├── cpp # C++-based project folder- ├── datasets # Additional training datasets- ├── dotnet # .Net program directory- ├── FAQ.md # Some questions and answers- ├── images # Test pictures, two typical test pictures, one is a natural scene, the other is a long text- ├── include # The header file directory when compiling the c language interface library- ├── ios # Apple mobile phone platform project directory- ├── jvm # java-based project directory- ├── lib # Compilation library file directory, used to compile the C language interface library. Binary files are not uploaded by default- ├── ocrweb # Based on python and Flask web- ├── python # python reasoning code directory- ├── release #- └── tools # Some conversion scripts and the like--</details>---## Current Progress-- [x] C++ example (Windows/Linux/macOS): [demo](./cpp)-- [x] Jvm example (Java/Kotlin): [demo](./jvm)-- [x] .Net example (C#): [demo](./dotnet)-- [x] Android example: [demo](./android)-- [x] python example: [demo](./python)-- [ ] IOS example: waiting for someone to contribute code-- [ ] Rewrite the C++ reasoning code according to the python version to improve the reasoning effect, and add support for gif/tga/webp format pictures--## Model related-#### Download models( [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing)|[Baidu NetDisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) )--#### Model to onnx- - ⭐[PaddleOCRModelConverter](https://github.com/RapidAI/PaddleOCRModelConverter) by @[SWHL](https://github.com/SWHL)- - [Paddle2OnnxConvertor](https://github.com/RapidAI/Paddle2OnnxConvertor) by @[benjaminwan](https://github.com/benjaminwan)- - [Teach you to use ONNXRunTime to deploy PP-OCR](https://aistudio.baidu.com/aistudio/projectdetail/1479970?channelType=0&channel=0) by @Channingss--### Compared-#### Text Det-- test dataset: `Chinese and English (111, including cards, documents and natural images)`--| Model | infer_Speed(s/img) | precision | recall | hmean | Model Size |-| :---------------------------------: | :----------------: | :-------: | :----: | :----: | :------: |-| ch_ppocr_mobile_v2.0_det_infer.onnx | 0.4345742 | 0.7277 | 0.8413 | 0.7785 | 2.3M |-| ch_PP-OCRv2_det_infer.onnx | 0.5116553 | 0.7817 | 0.8472 | 0.8123 | 2.3M |-| ch_PP-OCRv3_det_infer.onnx | 0.5723512 | **0.7740** | **0.8837** | **0.8237** | 2.4M |--#### Text Recognition-- test dataset: `Chinese and English (168)`--| Model | infer_speed(s/img) | Score | Exact_Match | Char_Match | Model Size |-| :---------------------------------: | ------------------: | :-------: | :--------------: | :-------------: | :--: |-| ch_ppocr_mobile_v2.0_rec_infer.onnx | 0.0111 | **0.7287** | **0.5595** | 0.8979 | 4.3M |-| ch_PP-OCRv2_rec_infer.onnx | 0.0193 | 0.6955 | 0.4881 | **0.9029** | 8.0M |-| ch_PP-OCRv3_rec_infer.onnx | 0.0145 | 0.5537 | 0.3274 | 0.7800 | 11M |-| ch_PP-OCRv3_rec_train_student.onnx | 0.0157 | 0.5537 | 0.3274 | 0.7800 | 11M |-| ch_PP-OCRv3_rec_train_teacher.onnx | 0.0140 | 0.5381 | 0.3095 | 0.7667 | 11M |--## [PaddleOCR-FAQ](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.1/doc/doc_ch/FAQ.md)--## Original initiator and start-up author-<p align="left">- <a href="https://github.com/benjaminwan"><img src="https://avatars.githubusercontent.com/u/2362051?v=4" width=65 height=65></a>- <a href="https://github.com/znsoftm"><img src="https://avatars.githubusercontent.com/u/15354249?v=4" width=65 height=65></a>- <a href="https://github.com/SWHL"><img src="https://avatars.githubusercontent.com/u/28639377?v=4" width=65 height=65></a>-</p>--## Authorization-- The copyright of the OCR model belongs to Baidu, and the copyright of other engineering codes belongs to the owner of this warehouse.-- This software is licensed under LGPL. You are welcome to contribute code, submit an issue or even pr.--## Contact us-- You can contact us through QQ group: **887298230**-- If you can’t find the group number, please click here [**link**](https://jq.qq.com/?_wv=1027&k=P9b3olx6) to find the organization-- Scan the following QR code with QQ:-- <div align="center">- <img src="./assets/qq_team.bmp" width="25%" height="25%" align="center">- </div>--## Demo-#### Demonstration with C++/JVM-<div align="center">- <img src="./assets/demo_cpp.png" width="100%" height="100%">-</div>--#### Demonstration with .Net-<div align="center">- <img src="./assets/demo_cs.png" width="100%" height="100%">-</div>--#### Demonstratioin with multi_language-<div align="center">- <img src="./assets/demo_multi_language.png" width="80%" height="80%">-</div>diff --git a/doc/change_log.md b/doc/change_log.mddeleted file mode 100644index 04d2300..0000000--- a/doc/change_log.md+++ /dev/null@@ -1,78 +0,0 @@-#### 🍿2022-05-15 update-- 增加PaddleOCR v3 rec模型转换后的ONNX模型,直接去网盘下载替换即可。([百度网盘](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing))-- 增加文本识别模型各个版本效果对比表格,详情点击[各个版本ONNX模型效果对比](#各个版本onnx模型效果对比)。v3的文本识别模型从自己构建测试集上的指标来看不如之前的好。--#### 😀2022-05-12 upadte-- 增加PaddleOCR v3 det模型转换的ONNX模型,直接去网盘下载,替换即可。([百度网盘](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing))-- 增加各个版本文本检测模型效果对比表格,详情点击[各个版本ONNX模型效果对比](#各个版本onnx模型效果对比)。v3的文本检测模型从指标来看是好于之前的v2的,推荐使用。--#### 🎧2022-04-04 update-- 增加python下的基于OpenVINO推理引擎的支持-- 给出OpenVINO和ONNXRuntime的性能对比表格-- 详情参见:[python/README](./python/README.md)--#### 2022-02-24 update-- 优化python目录下的推理代码-- 添加调用不同语言模型的推理代码示例-- 详情参见:[python/onnxruntime_infer/README](./python/onnxruntime_infer/README.md)--#### 2021-12-18 update-- 添加[Google Colab Demo](https://colab.research.google.com/github/RapidAI/RapidOCR/blob/main/RapidOCRDemo.ipynb)--#### 2021-11-28 update-- 更新[ocrweb](http://rapidocr.51pda.cn:9003/)部分- - 添加显示各个阶段处理时间- - 更新说明文档- - 更换文本检测模型为`ch_PP-OCRv2_det_infer.onnx`,推理更快,更准--#### 2021-11-13 update-- 添加python版本中文本检测和识别可调节的超参数,主要有`box_thresh|unclip_ratio|text_score`,详情见[参数调节](python/README.md#相关调节参数)-- 将文本识别中字典位置以参数方式给出,便于灵活配置,详情见[keys_path](python/rapidOCR.sh)--#### 2021-10-27 update-- 添加使用onnxruntime-gpu版推理的代码(不过gpu版本的onnxruntime不太好用,按照[官方教程](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)配置,感觉没有调用起来GPU)-- 具体使用步骤参见: [onnxruntime-gpu推理配置](python/README.md)--#### 2021-09-13 update-- 添加基于`python`的whl文件,便于使用,详情参见`release/python_sdk`--#### 2021-09-11 update-- 添加PP-OCRv2新增模型onnx版本- - 使用方法推理代码不变,直接替换对应模型即可。-- 经过在自有测试集上评测:- - PP-OCRv2检测模型效果有大幅度提升,模型大小没变。- - PP-OCRv2识别模型效果无明显提升,模型大小增加了3.58M。-- 模型上传到[百度网盘 提取码:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg)--#### 2021-08-07 update-- 现在正在做的- - [ ] PP-Structure 表格结构和cell坐标预测 正在整理中--- 之前做的,未完成的,欢迎提PR- - [ ] 打Dokcer镜像- - [x] 尝试onnxruntime-gpu推理--#### 2021-07-17 update-- 完善README文档-- 增加**英文、数字识别**onnx模型,具体参见`python/en_number_ppocr_mobile_v2_rec`,用法同其他-- 整理一下[模型转onnx](#模型转onnx)--#### 2021-07-04 update-- 目前仓库下的python程序已经可以在树莓派4B上,成功运行,详细信息请进群,询问群主-- 更新整体结构图,添加树莓派的支持--#### 2021-06-20 update-- 优化ocrweb中识别结果显示,同时添加识别动图演示-- 更新`datasets`目录,添加一些常用数据库链接(搬运一下^-^)-- 更新[FAQ](./FAQ.md)--#### 2021-06-10 update-- 添加server版文本识别模型,详情见[提取码:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg)--#### 2021-06-08 update-- 整理仓库,统一模型下载路径-- 完善相关说明文档--#### 2021-03-24 update-- 新模型已经完全兼容ONNXRuntime 1.7 或更高版本。 特别感谢:@Channingss-- 新版onnxruntime比1.6.0 性能提升40%以上。diff --git a/doc/change_log_en.md b/doc/change_log_en.mddeleted file mode 100644index 2f26bdd..0000000--- a/doc/change_log_en.md+++ /dev/null@@ -1,75 +0,0 @@-#### 🍿2022-05-15 update-- Add the ONNX model converted from the PaddleOCR v3 rec model, just go to the network disk to download and replace it. ([Baidu Netdisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing ))-- Added a comparison table of the effects of each version of the text recognition model. For details, click [Comparison of the effects of various versions of ONNX models] (#Comparison of the effects of various versions of onnx models). The text recognition model of v3 is not as good as the previous one in terms of the indicators on the test set constructed by itself.--#### 😀2022-05-12 upadte-- Add the ONNX model converted from the PaddleOCR v3 det model, download it directly from the network disk, and replace it. ([Baidu Netdisk](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing ))-- Added a comparison table of text detection model effects of various versions. For details, click [Comparison of the effects of various versions of ONNX models] (#Comparison of the effects of various versions of onnx models). The text detection model of v3 is better than the previous v2 in terms of the indicators on the test set constructed by itself, and it is recommended to use it.--#### 2022-02-24 update-- Optimize the inference code of the python part.-- Add inference code examples that use the different language models.-- For details, see: [python/README](./python/README.md)--#### 2021-12-18 update-- Add [Google Colab Demo](https://colab.research.google.com/github/RapidAI/RapidOCR/blob/main/RapidOCRDemo.ipynb).-- Change the default det model of the `python/rapidOCR.sh`--#### 2021-11-28 update-- Update the [ocrweb](http://rapidocr.51pda.cn:9003/) part- - Add the display of the inference time of each stage.- - Add docs of the ocrweb.- - Change the det model(`ch_PP-OCRv2_det_infer.onnx`), faster and more accurate.--#### 2021-11-13 update-- Add adjustable super parameters for text detection and recognition in Python version, mainly `box_thresh|unclip_ratio|text_score`, see [parameter adjustment](python/README.md#相关调节参数) for details-- The dictionary position in text recognition is given in parameter mode to facilitate flexible configuration. See [keys_path](python/rapidOCR.sh) for details--#### 2021-10-27 update-- Add the code that uses the onnxruntime GPU version of infering follow the [official tutorial](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html) Configuration. (however, the onnxruntime GPU version is not stable to use)--- See: `python/README.md` for specific steps.--#### 2021-09-13 update-- Add a whl file based on `Python` for ease of use. See `release/python` for details.--#### 2021-09-11 update-- Add `PP-OCRv2` new model onnx version.-- The infering code of the method is unchanged, and the corresponding model can be directly replaced.-- After evaluation on its own test set:- - The effect of `PP-OCRv2` detection model has been greatly improved, and the model size has not changed.- - The effect of `PP-OCRv2` recognition model was not significantly improved, and the model size increased by 3.58M.--- Upload the model to [Baidu online disk extraction code: 30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg) or [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing)--#### 2021-08-07 update-- TODO:- - [ ] PP structure table structure and cell coordinate prediction are being sorted out.--- Previously done, unfinished, welcome to PR- - [ ] make dokcer image- - [x] try onnxruntime GPU reasoning--#### 2021-07-17 update-- Improve the README document-- Add **English, number recognition**onnx model, please refer to `python/en_number_ppocr_mobile_v2_rec` for details, the usage is the same as others-- Organize [Model to onnx](#model-related)--#### 2021-07-04 update-- The python program under the repository can be successfully run on the Raspberry Pi 4B. For more information, please enter the QQ group and ask the group owner-- Update the overall structure diagram and add support for Raspberry Pi--#### 2021-06-20 update-- Optimize the display of recognition results in ocrweb, and add recognition animations to demonstrate at the same time-- Update the `datasets` directory, add some commonly used database links--#### 2021-06-10 update-- Add server version text recognition model, see details [Extract code:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg)--#### 2021-06-08 update-- Organize the warehouse and unify the model download path-- Improve related documentation--#### 2021-03-24 update-- The new model is fully compatible with ONNXRuntime 1.7 or higher. Special thanks: @Channingss-- The performance of the new version of onnxruntime is improved by more than 40% compared to 1.6.0.\ No newline at end of filediff --git a/python/README.md b/python/README.mdindex 0dca742..916f169 100755--- a/python/README.md+++ b/python/README.md@@ -1,107 +1,78 @@-### 使用步骤-1. 下载`python/onnxruntime_infer`目录到本地-2. 下载相应模型和用于显示的字体文件- - 网盘下对应模型可直接替换([百度网盘](https://pan.baidu.com/s/1mkirNltJS481In4g81jP3w?pwd=zy37) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing))-- - 下载之后模型和相应字体文件放在`fonts`和`models`下,最终目录结构如下:- ```text- models- |-- ch_PP-OCRv2_det_infer.onnx- |-- ch_ppocr_mobile_v2.0_cls_infer.onnx- |-- ch_ppocr_mobile_v2.0_det_infer.onnx- |-- ch_ppocr_server_v2.0_det_infer.onnx- |-- ch_ppocr_server_v2.0_rec_infer.onnx- |-- en_number_mobile_v2.0_rec_infer.onnx- |-- korean_mobile_v2.0_rec_infer.onnx- `-- japan_rec_crnn.onnx-- fonts- |-- msyh.ttc- `-- korean.ttf- ```--1. 运行- - 接口调用方式运行- - 如果想要使用其他语言的识别模型,可以只对`rec_model_path`和`keys_path`作对应更改即可- - **!!!各个不同语言的推理代码区别只在于模型和字典文件!!!**- ```python- from rapid_ocr_api import TextSystem, visualize-- det_model_path = 'models/ch_ppocr_mobile_v2.0_det_infer.onnx'- cls_model_path = 'models/ch_ppocr_mobile_v2.0_cls_infer.onnx'-- # 中英文识别- rec_model_path = 'models/ch_ppocr_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/ppocr_keys_v1.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/ch_en_num.jpg'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)--- # 只有中英文和数字识别- rec_model_path = 'models/en_number_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/en_dict.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/en_num.png'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)+### Python版RapidOCR+<p>+ <a href=""><img src="https://img.shields.io/badge/Python-3.6+-aff.svg"></a>+ <a href=""><img src="https://img.shields.io/badge/OS-Linux%2C%20Win%2C%20Mac-pink.svg"></a>+</p>+- 所有常用的参数配置都在`config.yaml`下,一目了然,更加便捷+- 每个独立的模块下均有独立的`config.yaml`配置文件,可以单独使用+- `det`的`mobile`和`server`版,共用一个推理代码,直接更改配置文件中模型路径即可+- `rec`的`mobile`和`server`版本,共用一个推理代码,直接更改配置文件中模型路径即可- # 日语识别- rec_model_path = 'models/japan_rec_crnn.onnx'- keys_path = 'rec_dict/japan_dict.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/japan.png'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)-+### 使用步骤+1. 下载当前下的`rapidocr_onnxruntime`目录到本地+2. 下载链接下的`resources`目录(包含模型和显示的字体文件)+ - [百度网盘](https://pan.baidu.com/s/1PTcgXG2zEgQU6A_A3kGJ3Q?pwd=jhai) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing)+ - 最终目录如下:+ ```text+ .+ ├── README.md+ ├── config.yaml+ ├── test_demo.py+ ├── rapidocr_onnxruntime+ │ ├── __init__.py+ │ ├── ch_ppocr_v2_cls+ │ ├── ch_ppocr_v2_det+ │ ├── ch_ppocr_v2_rec+ │ ├── ch_ppocr_v3_rec+ │ └── rapid_ocr_api.py+ ├── requirements.txt+ ├── resources+ │ ├── fonts+ │ │ └── msyh.ttc+ │ ├── models+ │ │ ├── ch_PP-OCRv2_det_infer.onnx.onnx+ │ │ ├── ch_ppocr_mobile_v2.0_cls_infer.onnx+ │ │ └── ch_ppocr_mobile_v2.0_rec_infer.onnx+ │ └── rec_dict+ │ ├── en_dict.txt+ │ ├── japan_dict.txt+ │ ├── korean_dict.txt+ │ └── ppocr_keys_v1.txt+ └── test_images+ ├── det_images+ └── rec_images+ ```+3. 安装运行环境+ ```bash+ pip install -r requirements.txt -i https://pypi.douban.com/simple/+ ```+ - Note: 在Windows端,Shapely库可能自动安装会有问题,解决方案参见[Q15](../docs/FAQ.md#q15-装完环境之后运行python-mainpy之后报错oserror-winerror-126-找不到指定的模組)- # 韩语识别- rec_model_path = 'models/korean_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/korean_dict.txt'- font_path = 'fonts/korean.ttf'+4. 运行示例+ - 接口调用+ ```python+ from rapidocr_onnxruntime import TextSystem- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)+ config_path = 'config.yaml'+ text_sys = TextSystem(config_path)- image_path = r'test_images/det_images/korean_1.jpg'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res, font_path)+ image_path = r'test_images/det_images/ch_en_num.jpg'+ img = cv2.imread(image_path)+ dt_boxes, rec_res = text_sys(img)```-- - 命令行运行- ```shell- $ bash test_demo.sh+ - 直接运行`test_demo.py`+ ```bash+ python test_demp.py```-### 相关调节参数-|参数名称|作用|建议取值范围|默认值|代码位置|-|:---:|:---:|:---:|:---:|:---:|-|box_thresh|文本检测所得框是否保留的阈值,值越大,召回率越低|[0, 1]|0.5|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L55)|-|unclip_ratio|控制文本检测框的大小,值越大,检测框整体越大|[1.6, 2.0]|1.6|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L57)|-|text_score|文本识别结果置信度,值越大,把握越大|[0, 1]|0.5|[rapidOCR.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/rapidOCR.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L270)|++### [`config.yaml`](./config.yaml)中常用参数介绍+|参数名称|作用|建议取值范围|默认值|+|:---:|:---:|:---:|:---:|+|box_thresh|文本检测所得框是否保留的阈值,值越大,召回率越低|[0, 1]|0.5|+|unclip_ratio|控制文本检测框的大小,值越大,检测框整体越大|[1.6, 2.0]|1.6|+|text_score|文本识别结果置信度,值越大,把握越大|[0, 1]|0.5|### onnxruntime-gpu版推理配置@@ -141,85 +112,4 @@|推理方式|推理图像数目|耗费时间||:---:|:---:|:---:||CPU|46张图像|191 s|- |GPU|46张图像|52.38 s|-------------------------------------------------------------------------### 目录说明-- [onnxruntime_infer](./onnxruntime_infer): 基于ONNXRuntime推理引擎推理-- [openvino_infer](./openvino_infer): 基于OpenVINO推理引擎推理--### TODO-- [ ] 模型转INT8,正在尝试-- [x] OpenVINO推理方向分类模型有误,已经提了[issue](https://github.com/openvinotoolkit/openvino/issues/11501)→ 问题有已经解决,但是需要自己编译OpenVINO,参见[Answer](https://github.com/openvinotoolkit/openvino/issues/11501#issuecomment-1096366363)-- [x] 其他推理模型代码整理-- [x] 模型转换代码整理--### 关于OpenVINO-- OpenVINO可以直接推理IR、ONNX和PaddlePaddle模型,具体如下(图来源:[link](https://docs.openvino.ai/latest/openvino_docs_OV_UG_OV_Runtime_User_Guide.html#doxid-openvino-docs-o-v-u-g-o-v-runtime-user-guide)):-- <div align="center">- <img src="../assets/BASIC_FLOW_IE_C.svg">- </div>--- 和ONNXRuntime同时推理同一个ONNX模型,OpenVINO推理速度更快-- 但是从对比来看,OpenVINO占用内存更大,其原因是拿空间换的时间- - 当指定`input_shape`在一个区间范围时,推理时内存占用会减少一些- - 示例命令:- ```bash- mo --input_model models/ch_PP-OCRv2_det_infer.onnx \- --output_dir models/IR/static \- --input_shape "[1,3,960:1200,800]"- ```--### OpenVINO与ONNXRuntime性能对比-- 推理设备:`Windows 64位 Intel(R) Core(TM) i5-4210M CPU @ 2.60GHz 2.59 GHz`-- [测试图像宽高](https://drive.google.com/file/d/1iJcGvOVIdUlyOS52bBdvO8uzx8QORo5M/view?usp=sharing): `12119x810`--| 测试模型 | 推理框架 | 占用内存(3次平均) | 推理时间(3次平均) |-| ------------------------------------ | -------------------- | ----------------- | ----------------- |-| `ch_PP-OCRv2_det_infer.onnx` | `ONNXRuntime=1.10.0` | 0.8G | 5.354s |-| `ch_PP-OCRv2_det_infer.onnx` | `openvino=2022.1.0` | 3.225G | 2.53s |-| `ch_PP-OCRv2_det_infer.xml` FP32 动态图 | `openvino=2022.1.0` | 3.175G | 2.0455s |---### OpenVINO与ONNXRuntime推理代码写法对比-NOTE: 以`ch_ppocr_mobile_v2_det`中推理代码为例子--- ONNXRuntime-```python-import onnxruntime--# 声明-sess_opt = onnxruntime.SessionOptions()-sess_opt.log_severity_level = 4-sess_opt.enable_cpu_mem_arena = False-self.session = onnxruntime.InferenceSession(det_model_path, sess_opt)-self.input_name = self.session.get_inputs()[0].name-self.output_name = self.session.get_outputs()[0].name--# 推理-preds = self.session.run([self.output_name],- {self.input_name: img})-```--- OpenVINO-```python-from openvino.runtime import Core--# 初始化-ie = Core()-model_onnx = ie.read_model(det_model_path)-compile_model = ie.compile_model(model=model_onnx,- device_name='CPU')-self.vino_session = compile_model.create_infer_request()--# 推理-self.vino_session.infer(inputs=[img])-vino_preds = self.vino_session.get_output_tensor().data-```+ |GPU|46张图像|52.38 s|\ No newline at end of filediff --git a/python/rapidocr_onnxruntime/ch_ppocr_v2_cls/config.yaml b/python/rapidocr_onnxruntime/ch_ppocr_v2_cls/config.yamlindex 2c2182a..e6fa00e 100644--- a/python/rapidocr_onnxruntime/ch_ppocr_v2_cls/config.yaml+++ b/python/rapidocr_onnxruntime/ch_ppocr_v2_cls/config.yaml@@ -1,4 +1,4 @@-model_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\models\ch_ppocr_mobile_v2.0_cls_infer.onnx+model_path: resources/models/ch_ppocr_mobile_v2.0_cls_infer.onnxcls_image_shape: [3, 48, 192]cls_batch_num: 6diff --git a/python/rapidocr_onnxruntime/ch_ppocr_v2_det/config.yaml b/python/rapidocr_onnxruntime/ch_ppocr_v2_det/config.yamlindex cb7cfa8..0f08b09 100644--- a/python/rapidocr_onnxruntime/ch_ppocr_v2_det/config.yaml+++ b/python/rapidocr_onnxruntime/ch_ppocr_v2_det/config.yaml@@ -1,4 +1,4 @@-model_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\models\ch_ppocr_server_v2.0_det_infer.onnx+model_path: resources/models/ch_ppocr_server_v2.0_det_infer.onnxpre_process:DetResizeForTest:diff --git a/python/rapidocr_onnxruntime/ch_ppocr_v2_rec/config.yaml b/python/rapidocr_onnxruntime/ch_ppocr_v2_rec/config.yamlindex 2233f7a..3c3c9df 100644--- a/python/rapidocr_onnxruntime/ch_ppocr_v2_rec/config.yaml+++ b/python/rapidocr_onnxruntime/ch_ppocr_v2_rec/config.yaml@@ -1,5 +1,5 @@-model_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\models\ch_ppocr_server_v2.0_rec_infer.onnx+model_path: resources/models/ch_ppocr_server_v2.0_rec_infer.onnxrec_img_shape: [3, 32, 320]rec_batch_num: 6-keys_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\rec_dict\ppocr_keys_v1.txt+keys_path: resources/rec_dict/ppocr_keys_v1.txtdiff --git a/python/rapidocr_onnxruntime/ch_ppocr_v3_rec/config.yaml b/python/rapidocr_onnxruntime/ch_ppocr_v3_rec/config.yamlindex 43a1f04..4f903d6 100644--- a/python/rapidocr_onnxruntime/ch_ppocr_v3_rec/config.yaml+++ b/python/rapidocr_onnxruntime/ch_ppocr_v3_rec/config.yaml@@ -1,5 +1,5 @@-model_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\models\PPOCRv3\ch_PP-OCRv3_rec_infer.onnx+model_path: resources/models/PPOCRv3/ch_PP-OCRv3_rec_infer.onnxrec_img_shape: [3, 48, 320]rec_batch_num: 6-keys_path: E:\PythonProjects\RapidOCR\python\onnxruntime_infer\rec_dict\ppocr_keys_v1.txt+keys_path: resources/rec_dict/ppocr_keys_v1.txtdiff --git a/python/rapidocr_onnxruntime/test_demo.py b/python/rapidocr_onnxruntime/test_demo.pydeleted file mode 100644index 19b2a00..0000000--- a/python/rapidocr_onnxruntime/test_demo.py+++ /dev/null@@ -1,98 +0,0 @@-# -*- encoding: utf-8 -*--# @Author: SWHL-# @Contact: [email protected]-import math-import random-from pathlib import Path--import cv2-import numpy as np-from PIL import Image, ImageDraw, ImageFont--from rapid_ocr_api import TextSystem---def draw_ocr_box_txt(image, boxes, txts, font_path,- scores=None, text_score=0.5):- if not Path(font_path).exists():- raise FileNotFoundError(f'The {font_path} does not exists! \n'- f'Please download the file in the https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY')-- h, w = image.height, image.width- img_left = image.copy()- img_right = Image.new('RGB', (w, h), (255, 255, 255))-- random.seed(0)- draw_left = ImageDraw.Draw(img_left)- draw_right = ImageDraw.Draw(img_right)- for idx, (box, txt) in enumerate(zip(boxes, txts)):- if scores is not None and scores[idx] < text_score:- continue-- color = (random.randint(0, 255),- random.randint(0, 255),- random.randint(0, 255))- draw_left.polygon(box, fill=color)- draw_right.polygon([box[0][0], box[0][1],- box[1][0], box[1][1],- box[2][0], box[2][1],- box[3][0], box[3][1]],- outline=color)-- box_height = math.sqrt((box[0][0] - box[3][0])**2- + (box[0][1] - box[3][1])**2)-- box_width = math.sqrt((box[0][0] - box[1][0])**2- + (box[0][1] - box[1][1])**2)-- if box_height > 2 * box_width:- font_size = max(int(box_width * 0.9), 10)- font = ImageFont.truetype(font_path, font_size,- encoding="utf-8")- cur_y = box[0][1]- for c in txt:- char_size = font.getsize(c)- draw_right.text((box[0][0] + 3, cur_y), c,- fill=(0, 0, 0), font=font)- cur_y += char_size[1]- else:- font_size = max(int(box_height * 0.8), 10)- font = ImageFont.truetype(font_path, font_size, encoding="utf-8")- draw_right.text([box[0][0], box[0][1]], txt,- fill=(0, 0, 0), font=font)-- img_left = Image.blend(image, img_left, 0.5)- img_show = Image.new('RGB', (w * 2, h), (255, 255, 255))- img_show.paste(img_left, (0, 0, w, h))- img_show.paste(img_right, (w, 0, w * 2, h))- return np.array(img_show)---def visualize(image_path, boxes, rec_res, font_path="fonts/msyh.ttc"):- image = Image.open(image_path)- txts = [rec_res[i][0] for i in range(len(rec_res))]- scores = [rec_res[i][1] for i in range(len(rec_res))]-- draw_img = draw_ocr_box_txt(image, boxes,- txts, font_path,- scores,- text_score=0.5)-- draw_img_save = Path("./inference_results/")- if not draw_img_save.exists():- draw_img_save.mkdir(parents=True, exist_ok=True)-- image_save = str(draw_img_save / f'infer_{Path(image_path).name}')- cv2.imwrite(image_save, draw_img[:, :, ::-1])- print(f'The infer result has saved in {image_save}')---if __name__ == '__main__':- config_path = 'config.yaml'- text_sys = TextSystem(config_path)-- image_path = r'resources/test_images/det_images/ch_en_num.jpg'- img = cv2.imread(image_path)- dt_boxes, rec_res = text_sys(img)- visualize(image_path, dt_boxes, rec_res,- font_path='resources/fonts/msyh.ttc')diff --git a/python/resources/config.yaml b/python/resources/config.yamldeleted file mode 100644index 6a590c9..0000000--- a/python/resources/config.yaml+++ /dev/null@@ -1,48 +0,0 @@-Global:- text_score: 0.5- use_angle_cls: true- print_verbose: true--Det:- module_name: ch_ppocr_v2_det- class_name: TextDetector- model_path: resources/models/ch_ppocr_mobile_v2.0_det_infer.onnx-- pre_process:- DetResizeForTest:- limit_side_len: 736- limit_type: min- NormalizeImage:- std: [0.229, 0.224, 0.225]- mean: [0.485, 0.456, 0.406]- scale: 1./255.- order: hwc- ToCHWImage:- KeepKeys:- keep_keys: ['image', 'shape']-- post_process:- thresh: 0.3- box_thresh: 0.5- max_candidates: 1000- unclip_ratio: 1.6- use_dilation: true--Cls:- module_name: ch_ppocr_v2_cls- class_name: TextClassifier- model_path: resources/models/ch_ppocr_mobile_v2.0_cls_infer.onnx-- cls_image_shape: [3, 48, 192]- cls_batch_num: 6- cls_thresh: 0.9- label_list: ['0', '180']--Rec:- module_name: ch_ppocr_v2_rec- class_name: TextRecognizer- model_path: resources/models/ch_ppocr_mobile_v2.0_rec_infer.onnx-- rec_img_shape: [3, 32, 320]- rec_batch_num: 6- keys_path: resources/rec_dict/ppocr_keys_v1.txtdiff --git a/python/resources/test_images/det_images/ch_en_num.jpg b/python/resources/test_images/det_images/ch_en_num.jpgdeleted file mode 100644index d352416..0000000Binary files a/python/resources/test_images/det_images/ch_en_num.jpg and /dev/null differdiff --git a/python/resources/test_images/det_images/en_num.png b/python/resources/test_images/det_images/en_num.pngdeleted file mode 100644index 6ef7767..0000000Binary files a/python/resources/test_images/det_images/en_num.png and /dev/null differdiff --git a/python/resources/test_images/det_images/japan.png b/python/resources/test_images/det_images/japan.pngdeleted file mode 100644index 733c64e..0000000Binary files a/python/resources/test_images/det_images/japan.png and /dev/null differdiff --git a/python/resources/test_images/det_images/korean_1.jpg b/python/resources/test_images/det_images/korean_1.jpgdeleted file mode 100644index 4259c04..0000000Binary files a/python/resources/test_images/det_images/korean_1.jpg and /dev/null differdiff --git a/python/resources/test_images/rec_images/imgs_words_ch/1.png b/python/resources/test_images/rec_images/imgs_words_ch/1.pngdeleted file mode 100644index 76a517e..0000000Binary files a/python/resources/test_images/rec_images/imgs_words_ch/1.png and /dev/null differdiff --git a/python/resources/test_images/rec_images/imgs_words_ch/2.png b/python/resources/test_images/rec_images/imgs_words_ch/2.pngdeleted file mode 100644index f816fc4..0000000Binary files a/python/resources/test_images/rec_images/imgs_words_ch/2.png and /dev/null differdiff --git a/python/resources/test_images/rec_images/imgs_words_en/word_10.png b/python/resources/test_images/rec_images/imgs_words_en/word_10.pngdeleted file mode 100644index 07370f7..0000000Binary files a/python/resources/test_images/rec_images/imgs_words_en/word_10.png and /dev/null differdiff --git a/python/resources/test_images/rec_images/imgs_words_japan/1.jpg b/python/resources/test_images/rec_images/imgs_words_japan/1.jpgdeleted file mode 100644index 6848797..0000000Binary files a/python/resources/test_images/rec_images/imgs_words_japan/1.jpg and /dev/null differdiff --git a/python/resources/test_images/rec_images/imgs_words_korean/1.jpg b/python/resources/test_images/rec_images/imgs_words_korean/1.jpgdeleted file mode 100644index 48a8938..0000000Binary files a/python/resources/test_images/rec_images/imgs_words_korean/1.jpg and /dev/null differdiff --git a/python/test_demo.py b/python/test_demo.pyindex 71d5c01..be86d50 100644--- a/python/test_demo.py+++ b/python/test_demo.py@@ -1,3 +1,6 @@+# -*- encoding: utf-8 -*-+# @Author: SWHL+# @Contact: [email protected]import mathimport randomfrom pathlib import Path@@ -85,11 +88,12 @@ def visualize(image_path, boxes, rec_res, font_path="fonts/msyh.ttc"):if __name__ == '__main__':- config_path = 'resources/config.yaml'+ config_path = 'config.yaml'text_sys = TextSystem(config_path)- image_path = r'resources/test_images/det_images/ch_en_num.jpg'+ image_path = r'test_images/det_images/ch_en_num.jpg'img = cv2.imread(image_path)dt_boxes, rec_res = text_sys(img)+visualize(image_path, dt_boxes, rec_res,font_path='resources/fonts/msyh.ttc')\ No newline at end of file