📄 Awesome OCR multiple programing languages toolkits based on ONNX Runtime, OpenVINO, MNN, PaddlePaddle, TensorRT and PyTorch.
Add openvino support
SWHL932c4d186997
diff --git a/.gitignore b/.gitignoreindex fdac37d..5aefa29 100755--- a/.gitignore+++ b/.gitignore@@ -141,4 +141,14 @@ dmypy.json*.onnx*.ttf-*.ttc\ No newline at end of file+*.ttc++long1.jpg++*.bin+*.mapping+*.xml++*.pdiparams+*.pdiparams.info+*.pdmodel\ No newline at end of filediff --git a/README.md b/README.mdindex e8c9b9b..e4eaead 100755--- a/README.md+++ b/README.md@@ -21,9 +21,12 @@<summary>目录</summary>- [RapidOCR (捷智OCR)](#rapidocr-捷智ocr)+ - [TODO](#todo)+ - [Openvino](#openvino)- [简介](#简介)- [近期更新](#近期更新)- - [🔉2022-02-24 update](#2022-02-24-update)+ - [🎧2022-04-04 update](#2022-04-04-update)+ - [2022-02-24 update](#2022-02-24-update)- [2021-12-18 update](#2021-12-18-update)- [2021-11-28 update](#2021-11-28-update)- [之前更新记录](#之前更新记录)@@ -57,10 +60,10 @@</details>## TODO-- [ ] 测试openvino推理速度,给出详细设备上推理性能对比表格+- [x] 测试openvino推理速度,给出详细设备上推理性能对比表格- [ ] 基于openvino的推理部署方案-## Openvino+## Openvinohttps://blog.csdn.net/weixin_47567401/article/details/123525014@@ -74,10 +77,15 @@ https://blog.csdn.net/weixin_47567401/article/details/123525014- 基于百度的开源PaddleOCR 模型及训练,任何人可以使用本推理库,也可以根据自己的需求使用百度的paddlepaddle框架进行模型优化。## 近期更新-#### 🔉2022-02-24 update+#### 🎧2022-04-04 update+- 增加python下的基于OpenVINO推理引擎的支持+- 给出OpenVINO和ONNXRuntime的性能对比表格+- 详情参加[python/onnxruntime/README](./python/README.md)++#### 2022-02-24 update- 优化python目录下的推理代码- 添加调用不同语言模型的推理代码示例-- 详情参见:[python/README](./python/README.md)+- 详情参见:[python/README](./python/onnxruntime_infer/README.md)#### 2021-12-18 update- 添加[Google Colab Demo](https://colab.research.google.com/github/RapidAI/RapidOCR/blob/main/RapidOCRDemo.ipynb)diff --git a/python/README.md b/python/README.mdindex d317d9c..520f97b 100755--- a/python/README.md+++ b/python/README.md@@ -1,148 +1,60 @@-### 使用步骤-1. 下载相应模型和用于显示的字体文件- - [百度网盘 提取码:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing)+### 目录说明+- `onnxruntime_infer`: 基于ONNXRuntime推理引擎推理+- `openvino_infer`: 基于OpenVINO推理引擎推理++### TODO+- [ ] 模型转INT8,尚未找到转换代码+- [ ] 其他推理模型代码整理+- [ ] 模型转换代码整理++### 关于OpenVINO+- OpenVINO可以直接推理ONNX模型,可以不用转换直接使用之前ONNX模型+- OpenVINO推理速度更快,但是从对比来看,占用内存更多,目前正在排查原因++### OpenVINO与ONNXRuntime性能对比+- 推理设备:`Windows 64位 Intel(R) Core(TM) i5-4210M CPU @ 2.60GHz 2.59 GHz`+- 测试图像宽高: `12119x810`++| 测试模型 | 推理框架 | 占用内存(3次平均) | 推理时间(3次平均) |+| ------------------------------------ | -------------------- | ----------------- | ----------------- |+| `ch_PP-OCRv2_det_infer.onnx` | `ONNXRuntime=1.10.0` | 0.8G | 5.354s |+| `ch_PP-OCRv2_det_infer.onnx` | `openvino=2022.1.0` | 3.225G | 2.53s |+| `ch_PP-OCRv2_det_infer.xml` FP32 | `openvino=2022.1.0` | 3.175G | 2.0455s |+++### OpenVINO与ONNXRuntime推理代码写法对比+NOTE: 以`ch_ppocr_mobile_v2_det`中推理代码为例子++- ONNXRuntime+```python+import onnxruntime++# 声明+sess_opt = onnxruntime.SessionOptions()+sess_opt.log_severity_level = 4+sess_opt.enable_cpu_mem_arena = False+self.session = onnxruntime.InferenceSession(det_model_path, sess_opt)+self.input_name = self.session.get_inputs()[0].name+self.output_name = self.session.get_outputs()[0].name++# 推理+preds = self.session.run([self.output_name],+ {self.input_name: img})+```++- OpenVINO+```python+from openvino.runtime import Core++# 初始化+ie = Core()+model_onnx = ie.read_model(det_model_path)+compile_model = ie.compile_model(model=model_onnx,+ device_name='CPU')+self.vino_session = compile_model.create_infer_request()++# 推理+self.vino_session.infer(inputs=[img])+vino_preds = self.vino_session.get_output_tensor().data+```- - 下载之后模型和相应字体文件放在`fonts`和`models`下,最终目录结构如下:- ```text- models- |-- ch_PP-OCRv2_det_infer.onnx- |-- ch_ppocr_mobile_v2.0_cls_infer.onnx- |-- ch_ppocr_mobile_v2.0_det_infer.onnx- |-- ch_ppocr_server_v2.0_det_infer.onnx- |-- ch_ppocr_server_v2.0_rec_infer.onnx- |-- en_number_mobile_v2.0_rec_infer.onnx- |-- korean_mobile_v2.0_rec_infer.onnx- `-- japan_rec_crnn.onnx-- fonts- |-- msyh.ttc- `-- korean.ttf- ```--2. 运行- - 接口调用方式运行- - 如果想要使用其他语言的识别模型,可以只对`rec_model_path`和`keys_path`作对应更改即可- - **!!!各个不同语言的推理代码区别只在于模型和字典文件!!!**- ```python- from rapid_ocr_api import TextSystem, visualize-- det_model_path = 'models/ch_ppocr_mobile_v2.0_det_infer.onnx'- cls_model_path = 'models/ch_ppocr_mobile_v2.0_cls_infer.onnx'-- # 中英文识别- rec_model_path = 'models/ch_ppocr_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/ppocr_keys_v1.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/ch_en_num.jpg'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)--- # 只有中英文和数字识别- rec_model_path = 'models/en_number_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/en_dict.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/en_num.png'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)--- # 日语识别- rec_model_path = 'models/japan_rec_crnn.onnx'- keys_path = 'rec_dict/japan_dict.txt'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/japan.png'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res)--- # 韩语识别- rec_model_path = 'models/korean_mobile_v2.0_rec_infer.onnx'- keys_path = 'rec_dict/korean_dict.txt'- font_path = 'fonts/korean.ttf'-- text_sys = TextSystem(det_model_path,- rec_model_path,- use_angle_cls=True,- cls_model_path=cls_model_path,- keys_path=keys_path)-- image_path = r'test_images/det_images/korean_1.jpg'- dt_boxes, rec_res = text_sys(image_path)- visualize(image_path, dt_boxes, rec_res, font_path)- ```-- - 命令行运行- ```shell- $ bash test_demo.sh- ```--### 相关调节参数-|参数名称|作用|建议取值范围|默认值|代码位置|-|:---:|:---:|:---:|:---:|:---:|-|box_thresh|文本检测所得框是否保留的阈值,值越大,召回率越低|[0, 1]|0.5|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L55)|-|unclip_ratio|控制文本检测框的大小,值越大,检测框整体越大|[1.6, 2.0]|1.6|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L57)|-|text_score|文本识别结果置信度,值越大,把握越大|[0, 1]|0.5|[rapidOCR.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/rapidOCR.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L270)|---### onnxruntime-gpu版推理配置--1. ONNXRuntime-gpu需要严格按照与cuda、cudnn版本对应来安装,具体参考[文档](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), **这一步关乎后面是否可以成功调用GPU**-2. 推理代码中,加载onnx模型部分,用以下对应语言代码替换即可,详细参见:[官方教程](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)- - python版本- ```python- import onnxruntime as ort-- model_path = '<path to model>'-- providers = [- ('CUDAExecutionProvider', {- 'device_id': 0,- 'arena_extend_strategy': 'kNextPowerOfTwo',- 'gpu_mem_limit': 2 * 1024 * 1024 * 1024,- 'cudnn_conv_algo_search': 'EXHAUSTIVE',- 'do_copy_in_default_stream': True,- }),- 'CPUExecutionProvider',- ]-- session = ort.InferenceSession(model_path, providers=providers)- ```- - C/C++版本- ```c++- OrtSessionOptions* session_options = /* ... */;-- OrtCUDAProviderOptions options;- options.device_id = 0;- options.arena_extend_strategy = 0;- options.gpu_mem_limit = 2 * 1024 * 1024 * 1024;- options.cudnn_conv_algo_search = OrtCudnnConvAlgoSearch::EXHAUSTIVE;- options.do_copy_in_default_stream = 1;-- SessionOptionsAppendExecutionProvider_CUDA(session_options, &options);- ```-3. 推理时间粗略对比(跑完RapidOCR整个项目)-- |推理方式|推理图像数目|耗费时间|- |:---:|:---:|:---:|- |CPU|46张图像|191 s|- |GPU|46张图像|52.38 s|- ||||\ No newline at end of filediff --git a/python/onnxruntime_infer/README.md b/python/onnxruntime_infer/README.mdnew file mode 100644index 0000000..22b78cf--- /dev/null+++ b/python/onnxruntime_infer/README.md@@ -0,0 +1,148 @@+### 使用步骤+1. 下载相应模型和用于显示的字体文件+ - [百度网盘 提取码:30jv](https://pan.baidu.com/s/1qkqWK4wRdMjqGGbzR-FyWg) | [Google Drive](https://drive.google.com/drive/folders/1x_a9KpCo_1blxH1xFOfgKVkw1HYRVywY?usp=sharing)++ - 下载之后模型和相应字体文件放在`fonts`和`models`下,最终目录结构如下:+ ```text+ models+ |-- ch_PP-OCRv2_det_infer.onnx+ |-- ch_ppocr_mobile_v2.0_cls_infer.onnx+ |-- ch_ppocr_mobile_v2.0_det_infer.onnx+ |-- ch_ppocr_server_v2.0_det_infer.onnx+ |-- ch_ppocr_server_v2.0_rec_infer.onnx+ |-- en_number_mobile_v2.0_rec_infer.onnx+ |-- korean_mobile_v2.0_rec_infer.onnx+ `-- japan_rec_crnn.onnx++ fonts+ |-- msyh.ttc+ `-- korean.ttf+ ```++2. 运行+ - 接口调用方式运行+ - 如果想要使用其他语言的识别模型,可以只对`rec_model_path`和`keys_path`作对应更改即可+ - **!!!各个不同语言的推理代码区别只在于模型和字典文件!!!**+ ```python+ from rapid_ocr_api import TextSystem, visualize++ det_model_path = 'models/ch_ppocr_mobile_v2.0_det_infer.onnx'+ cls_model_path = 'models/ch_ppocr_mobile_v2.0_cls_infer.onnx'++ # 中英文识别+ rec_model_path = 'models/ch_ppocr_mobile_v2.0_rec_infer.onnx'+ keys_path = 'rec_dict/ppocr_keys_v1.txt'++ text_sys = TextSystem(det_model_path,+ rec_model_path,+ use_angle_cls=True,+ cls_model_path=cls_model_path,+ keys_path=keys_path)++ image_path = r'test_images/det_images/ch_en_num.jpg'+ dt_boxes, rec_res = text_sys(image_path)+ visualize(image_path, dt_boxes, rec_res)+++ # 只有中英文和数字识别+ rec_model_path = 'models/en_number_mobile_v2.0_rec_infer.onnx'+ keys_path = 'rec_dict/en_dict.txt'++ text_sys = TextSystem(det_model_path,+ rec_model_path,+ use_angle_cls=True,+ cls_model_path=cls_model_path,+ keys_path=keys_path)++ image_path = r'test_images/det_images/en_num.png'+ dt_boxes, rec_res = text_sys(image_path)+ visualize(image_path, dt_boxes, rec_res)+++ # 日语识别+ rec_model_path = 'models/japan_rec_crnn.onnx'+ keys_path = 'rec_dict/japan_dict.txt'++ text_sys = TextSystem(det_model_path,+ rec_model_path,+ use_angle_cls=True,+ cls_model_path=cls_model_path,+ keys_path=keys_path)++ image_path = r'test_images/det_images/japan.png'+ dt_boxes, rec_res = text_sys(image_path)+ visualize(image_path, dt_boxes, rec_res)+++ # 韩语识别+ rec_model_path = 'models/korean_mobile_v2.0_rec_infer.onnx'+ keys_path = 'rec_dict/korean_dict.txt'+ font_path = 'fonts/korean.ttf'++ text_sys = TextSystem(det_model_path,+ rec_model_path,+ use_angle_cls=True,+ cls_model_path=cls_model_path,+ keys_path=keys_path)++ image_path = r'test_images/det_images/korean_1.jpg'+ dt_boxes, rec_res = text_sys(image_path)+ visualize(image_path, dt_boxes, rec_res, font_path)+ ```++ - 命令行运行+ ```shell+ $ bash test_demo.sh+ ```++### 相关调节参数+|参数名称|作用|建议取值范围|默认值|代码位置|+|:---:|:---:|:---:|:---:|:---:|+|box_thresh|文本检测所得框是否保留的阈值,值越大,召回率越低|[0, 1]|0.5|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L55)|+|unclip_ratio|控制文本检测框的大小,值越大,检测框整体越大|[1.6, 2.0]|1.6|[text_detect.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/ch_ppocr_mobile_v2_det/text_detect.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L57)|+|text_score|文本识别结果置信度,值越大,把握越大|[0, 1]|0.5|[rapidOCR.py](https://github.com/RapidAI/RapidOCR/blob/6aa79aa390c9c9e8f41df0f0c35f3dca97e6dc93/python/rapidOCR.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L270)|+++### onnxruntime-gpu版推理配置++1. ONNXRuntime-gpu需要严格按照与cuda、cudnn版本对应来安装,具体参考[文档](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), **这一步关乎后面是否可以成功调用GPU**+2. 推理代码中,加载onnx模型部分,用以下对应语言代码替换即可,详细参见:[官方教程](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)+ - python版本+ ```python+ import onnxruntime as ort++ model_path = '<path to model>'++ providers = [+ ('CUDAExecutionProvider', {+ 'device_id': 0,+ 'arena_extend_strategy': 'kNextPowerOfTwo',+ 'gpu_mem_limit': 2 * 1024 * 1024 * 1024,+ 'cudnn_conv_algo_search': 'EXHAUSTIVE',+ 'do_copy_in_default_stream': True,+ }),+ 'CPUExecutionProvider',+ ]++ session = ort.InferenceSession(model_path, providers=providers)+ ```+ - C/C++版本+ ```c+++ OrtSessionOptions* session_options = /* ... */;++ OrtCUDAProviderOptions options;+ options.device_id = 0;+ options.arena_extend_strategy = 0;+ options.gpu_mem_limit = 2 * 1024 * 1024 * 1024;+ options.cudnn_conv_algo_search = OrtCudnnConvAlgoSearch::EXHAUSTIVE;+ options.do_copy_in_default_stream = 1;++ SessionOptionsAppendExecutionProvider_CUDA(session_options, &options);+ ```+3. 推理时间粗略对比(跑完RapidOCR整个项目)++ |推理方式|推理图像数目|耗费时间|+ |:---:|:---:|:---:|+ |CPU|46张图像|191 s|+ |GPU|46张图像|52.38 s|+ ||||\ No newline at end of filediff --git a/python/ch_ppocr_mobile_v2_cls/__init__.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/__init__.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_cls/__init__.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/__init__.pydiff --git a/python/ch_ppocr_mobile_v2_cls/run.sh b/python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/run.shsimilarity index 100%rename from python/ch_ppocr_mobile_v2_cls/run.shrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/run.shdiff --git a/python/ch_ppocr_mobile_v2_cls/text_cls.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/text_cls.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_cls/text_cls.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/text_cls.pydiff --git a/python/ch_ppocr_mobile_v2_cls/utils.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/utils.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_cls/utils.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_cls/utils.pydiff --git a/python/ch_ppocr_mobile_v2_det/__init__.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_det/__init__.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_det/__init__.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_det/__init__.pydiff --git a/python/ch_ppocr_mobile_v2_det/run.sh b/python/onnxruntime_infer/ch_ppocr_mobile_v2_det/run.shsimilarity index 100%rename from python/ch_ppocr_mobile_v2_det/run.shrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_det/run.shdiff --git a/python/ch_ppocr_mobile_v2_det/text_detect.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_det/text_detect.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_det/text_detect.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_det/text_detect.pydiff --git a/python/ch_ppocr_mobile_v2_det/utils.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_det/utils.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_det/utils.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_det/utils.pydiff --git a/python/ch_ppocr_mobile_v2_rec/__init__.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/__init__.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_rec/__init__.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/__init__.pydiff --git a/python/ch_ppocr_mobile_v2_rec/ppocr_keys_v1.txt b/python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/ppocr_keys_v1.txtold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_rec/ppocr_keys_v1.txtrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/ppocr_keys_v1.txtdiff --git a/python/ch_ppocr_mobile_v2_rec/run.sh b/python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/run.shsimilarity index 100%rename from python/ch_ppocr_mobile_v2_rec/run.shrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/run.shdiff --git a/python/ch_ppocr_mobile_v2_rec/text_recognize.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/text_recognize.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_rec/text_recognize.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/text_recognize.pydiff --git a/python/ch_ppocr_mobile_v2_rec/utils.py b/python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/utils.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_mobile_v2_rec/utils.pyrename to python/onnxruntime_infer/ch_ppocr_mobile_v2_rec/utils.pydiff --git a/python/ch_ppocr_server_v2_det/__init__.py b/python/onnxruntime_infer/ch_ppocr_server_v2_det/__init__.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_det/__init__.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_det/__init__.pydiff --git a/python/ch_ppocr_server_v2_det/run.sh b/python/onnxruntime_infer/ch_ppocr_server_v2_det/run.shsimilarity index 100%rename from python/ch_ppocr_server_v2_det/run.shrename to python/onnxruntime_infer/ch_ppocr_server_v2_det/run.shdiff --git a/python/ch_ppocr_server_v2_det/text_detect.py b/python/onnxruntime_infer/ch_ppocr_server_v2_det/text_detect.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_det/text_detect.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_det/text_detect.pydiff --git a/python/ch_ppocr_server_v2_det/utils.py b/python/onnxruntime_infer/ch_ppocr_server_v2_det/utils.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_det/utils.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_det/utils.pydiff --git a/python/ch_ppocr_server_v2_rec/__init__.py b/python/onnxruntime_infer/ch_ppocr_server_v2_rec/__init__.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_rec/__init__.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_rec/__init__.pydiff --git a/python/ch_ppocr_server_v2_rec/ppocr_keys_v1.txt b/python/onnxruntime_infer/ch_ppocr_server_v2_rec/ppocr_keys_v1.txtold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_rec/ppocr_keys_v1.txtrename to python/onnxruntime_infer/ch_ppocr_server_v2_rec/ppocr_keys_v1.txtdiff --git a/python/ch_ppocr_server_v2_rec/run.sh b/python/onnxruntime_infer/ch_ppocr_server_v2_rec/run.shsimilarity index 100%rename from python/ch_ppocr_server_v2_rec/run.shrename to python/onnxruntime_infer/ch_ppocr_server_v2_rec/run.shdiff --git a/python/ch_ppocr_server_v2_rec/text_recognize.py b/python/onnxruntime_infer/ch_ppocr_server_v2_rec/text_recognize.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_rec/text_recognize.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_rec/text_recognize.pydiff --git a/python/ch_ppocr_server_v2_rec/utils.py b/python/onnxruntime_infer/ch_ppocr_server_v2_rec/utils.pyold mode 100755new mode 100644similarity index 100%rename from python/ch_ppocr_server_v2_rec/utils.pyrename to python/onnxruntime_infer/ch_ppocr_server_v2_rec/utils.pydiff --git a/python/fonts/.gitkeep b/python/onnxruntime_infer/fonts/.gitkeepsimilarity index 100%rename from python/fonts/.gitkeeprename to python/onnxruntime_infer/fonts/.gitkeepdiff --git "a/python/models/\346\250\241\345\236\213\344\275\215\347\275\256.md" b/python/onnxruntime_infer/models/.gitkeepold mode 100755new mode 100644similarity index 100%rename from "python/models/\346\250\241\345\236\213\344\275\215\347\275\256.md"rename to python/onnxruntime_infer/models/.gitkeepdiff --git a/python/rapid_ocr_api.py b/python/onnxruntime_infer/rapid_ocr_api.pysimilarity index 100%rename from python/rapid_ocr_api.pyrename to python/onnxruntime_infer/rapid_ocr_api.pydiff --git a/python/rec_dict/en_dict.txt b/python/onnxruntime_infer/rec_dict/en_dict.txtsimilarity index 100%rename from python/rec_dict/en_dict.txtrename to python/onnxruntime_infer/rec_dict/en_dict.txtdiff --git a/python/rec_dict/japan_dict.txt b/python/onnxruntime_infer/rec_dict/japan_dict.txtsimilarity index 100%rename from python/rec_dict/japan_dict.txtrename to python/onnxruntime_infer/rec_dict/japan_dict.txtdiff --git a/python/rec_dict/korean_dict.txt b/python/onnxruntime_infer/rec_dict/korean_dict.txtsimilarity index 100%rename from python/rec_dict/korean_dict.txtrename to python/onnxruntime_infer/rec_dict/korean_dict.txtdiff --git a/python/rec_dict/ppocr_keys_v1.txt b/python/onnxruntime_infer/rec_dict/ppocr_keys_v1.txtsimilarity index 100%rename from python/rec_dict/ppocr_keys_v1.txtrename to python/onnxruntime_infer/rec_dict/ppocr_keys_v1.txtdiff --git a/python/requirements.txt b/python/onnxruntime_infer/requirements.txtold mode 100755new mode 100644similarity index 75%rename from python/requirements.txtrename to python/onnxruntime_infer/requirements.txtindex a2384eb..3bccfbb--- a/python/requirements.txt+++ b/python/onnxruntime_infer/requirements.txt@@ -4,4 +4,5 @@ onnxruntime==1.7.0opencv_python==4.5.1.48six==1.15.0numpy==1.19.2-Pillow==8.3.0\ No newline at end of file+Pillow==8.3.0+openvino==2022.1.0\ No newline at end of filediff --git a/python/test_demo.py b/python/onnxruntime_infer/test_demo.pysimilarity index 100%rename from python/test_demo.pyrename to python/onnxruntime_infer/test_demo.pydiff --git a/python/test_demo.sh b/python/onnxruntime_infer/test_demo.shsimilarity index 100%rename from python/test_demo.shrename to python/onnxruntime_infer/test_demo.shdiff --git a/python/test_images/det_images/ch_en_num.jpg b/python/onnxruntime_infer/test_images/det_images/ch_en_num.jpgsimilarity index 100%rename from python/test_images/det_images/ch_en_num.jpgrename to python/onnxruntime_infer/test_images/det_images/ch_en_num.jpgdiff --git a/python/test_images/det_images/en_num.png b/python/onnxruntime_infer/test_images/det_images/en_num.pngsimilarity index 100%rename from python/test_images/det_images/en_num.pngrename to python/onnxruntime_infer/test_images/det_images/en_num.pngdiff --git a/python/test_images/det_images/japan.png b/python/onnxruntime_infer/test_images/det_images/japan.pngsimilarity index 100%rename from python/test_images/det_images/japan.pngrename to python/onnxruntime_infer/test_images/det_images/japan.pngdiff --git a/python/test_images/det_images/korean_1.jpg b/python/onnxruntime_infer/test_images/det_images/korean_1.jpgsimilarity index 100%rename from python/test_images/det_images/korean_1.jpgrename to python/onnxruntime_infer/test_images/det_images/korean_1.jpgdiff --git a/python/test_images/rec_images/imgs_words_ch/1.png b/python/onnxruntime_infer/test_images/rec_images/imgs_words_ch/1.pngsimilarity index 100%rename from python/test_images/rec_images/imgs_words_ch/1.pngrename to python/onnxruntime_infer/test_images/rec_images/imgs_words_ch/1.pngdiff --git a/python/test_images/rec_images/imgs_words_ch/2.png b/python/onnxruntime_infer/test_images/rec_images/imgs_words_ch/2.pngsimilarity index 100%rename from python/test_images/rec_images/imgs_words_ch/2.pngrename to python/onnxruntime_infer/test_images/rec_images/imgs_words_ch/2.pngdiff --git a/python/test_images/rec_images/imgs_words_en/word_10.png b/python/onnxruntime_infer/test_images/rec_images/imgs_words_en/word_10.pngsimilarity index 100%rename from python/test_images/rec_images/imgs_words_en/word_10.pngrename to python/onnxruntime_infer/test_images/rec_images/imgs_words_en/word_10.pngdiff --git a/python/test_images/rec_images/imgs_words_japan/1.jpg b/python/onnxruntime_infer/test_images/rec_images/imgs_words_japan/1.jpgsimilarity index 100%rename from python/test_images/rec_images/imgs_words_japan/1.jpgrename to python/onnxruntime_infer/test_images/rec_images/imgs_words_japan/1.jpgdiff --git a/python/test_images/rec_images/imgs_words_korean/1.jpg b/python/onnxruntime_infer/test_images/rec_images/imgs_words_korean/1.jpgsimilarity index 100%rename from python/test_images/rec_images/imgs_words_korean/1.jpgrename to python/onnxruntime_infer/test_images/rec_images/imgs_words_korean/1.jpgdiff --git a/python/openvino_infer/README.md b/python/openvino_infer/README.mdnew file mode 100644index 0000000..f679cc7--- /dev/null+++ b/python/openvino_infer/README.md@@ -0,0 +1,6 @@+#### 安装+```bash+# Windows端+pip install openvino==2022.1.0+pip install openvino-dev==2022.1.0+```\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/__init__.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/__init__.pynew file mode 100644index 0000000..d7a3e0b--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/__init__.py@@ -0,0 +1,6 @@+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+# @File: __init__.py+# @Time: 2021/01/23 15:24:27+# @Author: Max+from .text_detect import TextDetector\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/run.sh b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/run.shnew file mode 100644index 0000000..99fc806--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/run.sh@@ -0,0 +1,9 @@+#! /bin/bash+set -e errexit++root_path=$(cd $(dirname $0); pwd)+image_path=${root_path}"/test_images/det_images/1.jpg"+model_path=${root_path}"/models/ch_ppocr_mobile_v2.0_det_infer.onnx"++python text_detect.py --image_path ${image_path} \+ --model_path ${model_path}\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/text_detect.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/text_detect.pynew file mode 100644index 0000000..2816f12--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/text_detect.py@@ -0,0 +1,153 @@+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+import argparse+import time++import cv2+import numpy as np+from openvino.runtime import Core++try:+ from .utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+except:+ from utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+++class TextDetector(object):+ def __init__(self, det_model_path):+ pre_process_list = [{+ 'DetResizeForTest': {+ 'limit_side_len': 736,+ 'limit_type': 'min'+ }+ }, {+ 'NormalizeImage': {+ 'std': [0.229, 0.224, 0.225],+ 'mean': [0.485, 0.456, 0.406],+ 'scale': '1./255.',+ 'order': 'hwc'+ }+ }, {+ 'ToCHWImage': None+ }, {+ 'KeepKeys': {+ 'keep_keys': ['image', 'shape']+ }+ }]++ self.preprocess_op = create_operators(pre_process_list)+ self.postprocess_op = DBPostProcess(thresh=0.3,+ box_thresh=0.5,+ max_candidates=1000,+ unclip_ratio=1.6,+ use_dilation=True)+ ie = Core()+ model_onnx = ie.read_model(det_model_path)+ compile_model = ie.compile_model(model=model_onnx,+ device_name='CPU')+ self.vino_session = compile_model.create_infer_request()++ def order_points_clockwise(self, pts):+ """+ reference from: https://github.com/jrosebr1/imutils/blob/master/imutils/perspective.py+ # sort the points based on their x-coordinates+ """+ xSorted = pts[np.argsort(pts[:, 0]), :]++ # grab the left-most and right-most points from the sorted+ # x-roodinate points+ leftMost = xSorted[:2, :]+ rightMost = xSorted[2:, :]++ # now, sort the left-most coordinates according to their+ # y-coordinates so we can grab the top-left and bottom-left+ # points, respectively+ leftMost = leftMost[np.argsort(leftMost[:, 1]), :]+ (tl, bl) = leftMost++ rightMost = rightMost[np.argsort(rightMost[:, 1]), :]+ (tr, br) = rightMost++ rect = np.array([tl, tr, br, bl], dtype="float32")+ return rect++ def clip_det_res(self, points, img_height, img_width):+ for pno in range(points.shape[0]):+ points[pno, 0] = int(min(max(points[pno, 0], 0), img_width - 1))+ points[pno, 1] = int(min(max(points[pno, 1], 0), img_height - 1))+ return points++ def filter_tag_det_res(self, dt_boxes, image_shape):+ img_height, img_width = image_shape[0:2]+ dt_boxes_new = []+ for box in dt_boxes:+ box = self.order_points_clockwise(box)+ box = self.clip_det_res(box, img_height, img_width)+ rect_width = int(np.linalg.norm(box[0] - box[1]))+ rect_height = int(np.linalg.norm(box[0] - box[3]))+ if rect_width <= 3 or rect_height <= 3:+ continue+ dt_boxes_new.append(box)+ dt_boxes = np.array(dt_boxes_new)+ return dt_boxes++ def __call__(self, img):+ ori_im = img.copy()+ data = {'image': img}+ data = transform(data, self.preprocess_op)+ img, shape_list = data+ if img is None:+ return None, 0+ img = np.expand_dims(img, axis=0)+ img = img.astype(np.float32)+ shape_list = np.expand_dims(shape_list, axis=0)++ start_time = time.time()+ self.vino_session.infer(inputs=[img])+ vino_preds = self.vino_session.get_output_tensor().data++ post_result = self.postprocess_op(vino_preds, shape_list)+ dt_boxes = post_result[0]['points']+ dt_boxes = self.filter_tag_det_res(dt_boxes, ori_im.shape)+ elapse = time.time() - start_time++ src_im = draw_text_det_res(dt_boxes, r'E:\PythonProjects\RapidOCR\python\test_images\long1.jpg')+ cv2.imwrite('det_results.jpg', src_im)+ return dt_boxes, elapse+++if __name__ == "__main__":+ parser = argparse.ArgumentParser()+ parser.add_argument('--image_path', type=str, help='image_path|image_dir')+ parser.add_argument('--model_path', type=str, help='model_path')+ args = parser.parse_args()++ text_detector = TextDetector(args.model_path)++ img, flag = check_and_read_gif(args.image_path)+ if not flag:+ img = cv2.imread(args.image_path)+ if img is None:+ raise ValueError(f"error in loading image:{args.image_path}")++ dt_boxes, elapse = text_detector(img)++ src_im = draw_text_det_res(dt_boxes, args.image_path)+ cv2.imwrite('det_results.jpg', src_im)+ print(f'图像已经保存为det_results.jpg了\t{elapse:.6f}s')+diff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/utils.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/utils.pynew file mode 100644index 0000000..dfbea6c--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_ir/utils.py@@ -0,0 +1,414 @@+"""+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+"""+# coding:utf-8+from __future__ import (absolute_import, division, print_function,+ unicode_literals)++import os+import sys++import cv2+import numpy as np+import pyclipper+import six+from shapely.geometry import Polygon+++class DecodeImage(object):+ """ decode image """++ def __init__(self, img_mode='RGB', channel_first=False, **kwargs):+ self.img_mode = img_mode+ self.channel_first = channel_first++ def __call__(self, data):+ img = data['image']+ if six.PY2:+ assert type(img) is str and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ else:+ assert type(img) is bytes and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ img = np.frombuffer(img, dtype='uint8')+ img = cv2.imdecode(img, 1)+ if img is None:+ return None+ if self.img_mode == 'GRAY':+ img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)+ elif self.img_mode == 'RGB':+ assert img.shape[2] == 3, 'invalid shape of image[%s]' % (+ img.shape)+ img = img[:, :, ::-1]++ if self.channel_first:+ img = img.transpose((2, 0, 1))++ data['image'] = img+ return data+++class NormalizeImage(object):+ """ normalize image such as substract mean, divide std+ """++ def __init__(self, scale=None, mean=None, std=None, order='chw', **kwargs):+ if isinstance(scale, str):+ scale = eval(scale)+ self.scale = np.float32(scale if scale is not None else 1.0 / 255.0)+ mean = mean if mean is not None else [0.485, 0.456, 0.406]+ std = std if std is not None else [0.229, 0.224, 0.225]++ shape = (3, 1, 1) if order == 'chw' else (1, 1, 3)+ self.mean = np.array(mean).reshape(shape).astype('float32')+ self.std = np.array(std).reshape(shape).astype('float32')++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)++ assert isinstance(img,+ np.ndarray), "invalid input 'img' in NormalizeImage"+ data['image'] = (+ img.astype('float32') * self.scale - self.mean) / self.std+ return data+++class ToCHWImage(object):+ """ convert hwc image to chw image+ """++ def __init__(self, **kwargs):+ pass++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)+ data['image'] = img.transpose((2, 0, 1))+ return data+++class KeepKeys(object):+ def __init__(self, keep_keys, **kwargs):+ self.keep_keys = keep_keys++ def __call__(self, data):+ data_list = []+ for key in self.keep_keys:+ data_list.append(data[key])+ return data_list+++class DetResizeForTest(object):+ def __init__(self, **kwargs):+ super(DetResizeForTest, self).__init__()+ self.resize_type = 0+ if 'image_shape' in kwargs:+ self.image_shape = kwargs['image_shape']+ self.resize_type = 1+ elif 'limit_side_len' in kwargs:+ self.limit_side_len = kwargs['limit_side_len']+ self.limit_type = kwargs.get('limit_type', 'min')++ if 'resize_long' in kwargs:+ self.resize_type = 2+ self.resize_long = kwargs.get('resize_long', 960)+ else:+ self.limit_side_len = 736+ self.limit_type = 'min'++ def __call__(self, data):+ img = data['image']+ src_h, src_w, _ = img.shape++ if self.resize_type == 0:+ # img, shape = self.resize_image_type0(img)+ img, [ratio_h, ratio_w] = self.resize_image_type0(img)+ elif self.resize_type == 2:+ img, [ratio_h, ratio_w] = self.resize_image_type2(img)+ else:+ # img, shape = self.resize_image_type1(img)+ img, [ratio_h, ratio_w] = self.resize_image_type1(img)+ data['image'] = img+ data['shape'] = np.array([src_h, src_w, ratio_h, ratio_w])+ return data++ def resize_image_type1(self, img):+ resize_h, resize_w = self.image_shape+ ori_h, ori_w = img.shape[:2] # (h, w, c)+ ratio_h = float(resize_h) / ori_h+ ratio_w = float(resize_w) / ori_w+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ # return img, np.array([ori_h, ori_w])+ return img, [ratio_h, ratio_w]++ def resize_image_type0(self, img):+ """+ resize image to a size multiple of 32 which is required by the network+ args:+ img(array): array with shape [h, w, c]+ return(tuple):+ img, (ratio_h, ratio_w)+ """+ limit_side_len = self.limit_side_len+ h, w, _ = img.shape++ # limit the max side+ if self.limit_type == 'max':+ if max(h, w) > limit_side_len:+ if h > w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ else:+ if min(h, w) < limit_side_len:+ if h < w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ resize_h = int(h * ratio)+ resize_w = int(w * ratio)++ resize_h = int(round(resize_h / 32) * 32)+ resize_w = int(round(resize_w / 32) * 32)++ try:+ if int(resize_w) <= 0 or int(resize_h) <= 0:+ return None, (None, None)+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ except:+ print(img.shape, resize_w, resize_h)+ sys.exit(0)+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)+ return img, [ratio_h, ratio_w]++ def resize_image_type2(self, img):+ h, w, _ = img.shape++ resize_w = w+ resize_h = h++ # Fix the longer side+ if resize_h > resize_w:+ ratio = float(self.resize_long) / resize_h+ else:+ ratio = float(self.resize_long) / resize_w++ resize_h = int(resize_h * ratio)+ resize_w = int(resize_w * ratio)++ max_stride = 128+ resize_h = (resize_h + max_stride - 1) // max_stride * max_stride+ resize_w = (resize_w + max_stride - 1) // max_stride * max_stride+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)++ return img, [ratio_h, ratio_w]+++def transform(data, ops=None):+ """ transform """+ if ops is None:+ ops = []+ for op in ops:+ data = op(data)+ if data is None:+ return None+ return data+++def check_and_read_gif(img_path):+ if os.path.basename(img_path)[-3:] in ['gif', 'GIF']:+ gif = cv2.VideoCapture(img_path)+ ret, frame = gif.read()+ if not ret:+ print("Cannot read {}. This gif image maybe corrupted.")+ return None, False+ if len(frame.shape) == 2 or frame.shape[-1] == 1:+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)+ imgvalue = frame[:, :, ::-1]+ return imgvalue, True+ return None, False+++def create_operators(op_param_list, global_config=None):+ """+ create operators based on the config++ Args:+ params(list): a dict list, used to create some operators+ """+ assert isinstance(+ op_param_list, list), ('operator config should be a list')+ ops = []+ for operator in op_param_list:+ assert isinstance(operator,+ dict) and len(operator) == 1, "yaml format error"+ op_name = list(operator)[0]+ param = {} if operator[op_name] is None else operator[op_name]+ if global_config is not None:+ param.update(global_config)+ op = eval(op_name)(**param)+ ops.append(op)+ return ops+++def draw_text_det_res(dt_boxes, img_path):+ src_im = cv2.imread(img_path)+ for box in dt_boxes:+ box = np.array(box).astype(np.int32).reshape(-1, 2)+ cv2.polylines(src_im, [box], True,+ color=(255, 255, 0), thickness=2)+ return src_im+++class DBPostProcess(object):+ """+ The post process for Differentiable Binarization (DB).+ """++ def __init__(self,+ thresh=0.3,+ box_thresh=0.7,+ max_candidates=1000,+ unclip_ratio=2.0,+ use_dilation=False,+ **kwargs):+ self.thresh = thresh+ self.box_thresh = box_thresh+ self.max_candidates = max_candidates+ self.unclip_ratio = unclip_ratio+ self.min_size = 3+ self.dilation_kernel = None if not use_dilation else np.array(+ [[1, 1], [1, 1]])++ def boxes_from_bitmap(self, pred, _bitmap, dest_width, dest_height):+ '''+ _bitmap: single map with shape (1, H, W),+ whose values are binarized as {0, 1}+ '''++ bitmap = _bitmap+ height, width = bitmap.shape++ outs = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST,+ cv2.CHAIN_APPROX_SIMPLE)+ if len(outs) == 3:+ img, contours, _ = outs[0], outs[1], outs[2]+ elif len(outs) == 2:+ contours, _ = outs[0], outs[1]++ num_contours = min(len(contours), self.max_candidates)++ boxes = []+ scores = []+ for index in range(num_contours):+ contour = contours[index]+ points, sside = self.get_mini_boxes(contour)+ if sside < self.min_size:+ continue+ points = np.array(points)+ score = self.box_score_fast(pred, points.reshape(-1, 2))+ if self.box_thresh > score:+ continue++ box = self.unclip(points).reshape(-1, 1, 2)+ box, sside = self.get_mini_boxes(box)+ if sside < self.min_size + 2:+ continue+ box = np.array(box)++ box[:, 0] = np.clip(+ np.round(box[:, 0] / width * dest_width), 0, dest_width)+ box[:, 1] = np.clip(+ np.round(box[:, 1] / height * dest_height), 0, dest_height)+ boxes.append(box.astype(np.int16))+ scores.append(score)+ return np.array(boxes, dtype=np.int16), scores++ def unclip(self, box):+ unclip_ratio = self.unclip_ratio+ poly = Polygon(box)+ distance = poly.area * unclip_ratio / poly.length+ offset = pyclipper.PyclipperOffset()+ offset.AddPath(box, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)+ expanded = np.array(offset.Execute(distance))+ return expanded++ def get_mini_boxes(self, contour):+ bounding_box = cv2.minAreaRect(contour)+ points = sorted(list(cv2.boxPoints(bounding_box)), key=lambda x: x[0])++ index_1, index_2, index_3, index_4 = 0, 1, 2, 3+ if points[1][1] > points[0][1]:+ index_1 = 0+ index_4 = 1+ else:+ index_1 = 1+ index_4 = 0+ if points[3][1] > points[2][1]:+ index_2 = 2+ index_3 = 3+ else:+ index_2 = 3+ index_3 = 2++ box = [+ points[index_1], points[index_2], points[index_3], points[index_4]+ ]+ return box, min(bounding_box[1])++ def box_score_fast(self, bitmap, _box):+ h, w = bitmap.shape[:2]+ box = _box.copy()+ xmin = np.clip(np.floor(box[:, 0].min()).astype(np.int), 0, w - 1)+ xmax = np.clip(np.ceil(box[:, 0].max()).astype(np.int), 0, w - 1)+ ymin = np.clip(np.floor(box[:, 1].min()).astype(np.int), 0, h - 1)+ ymax = np.clip(np.ceil(box[:, 1].max()).astype(np.int), 0, h - 1)++ mask = np.zeros((ymax - ymin + 1, xmax - xmin + 1), dtype=np.uint8)+ box[:, 0] = box[:, 0] - xmin+ box[:, 1] = box[:, 1] - ymin+ cv2.fillPoly(mask, box.reshape(1, -1, 2).astype(np.int32), 1)+ return cv2.mean(bitmap[ymin:ymax + 1, xmin:xmax + 1], mask)[0]++ def __call__(self, pred, shape_list):+ pred = pred[:, 0, :, :]+ segmentation = pred > self.thresh++ boxes_batch = []+ for batch_index in range(pred.shape[0]):+ src_h, src_w, ratio_h, ratio_w = shape_list[batch_index]+ if self.dilation_kernel is not None:+ mask = cv2.dilate(+ np.array(segmentation[batch_index]).astype(np.uint8),+ self.dilation_kernel)+ else:+ mask = segmentation[batch_index]+ boxes, scores = self.boxes_from_bitmap(pred[batch_index], mask,+ src_w, src_h)++ boxes_batch.append({'points': boxes})+ return boxes_batchdiff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/__init__.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/__init__.pynew file mode 100644index 0000000..d7a3e0b--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/__init__.py@@ -0,0 +1,6 @@+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+# @File: __init__.py+# @Time: 2021/01/23 15:24:27+# @Author: Max+from .text_detect import TextDetector\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/run.sh b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/run.shnew file mode 100644index 0000000..99fc806--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/run.sh@@ -0,0 +1,9 @@+#! /bin/bash+set -e errexit++root_path=$(cd $(dirname $0); pwd)+image_path=${root_path}"/test_images/det_images/1.jpg"+model_path=${root_path}"/models/ch_ppocr_mobile_v2.0_det_infer.onnx"++python text_detect.py --image_path ${image_path} \+ --model_path ${model_path}\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/text_detect.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/text_detect.pynew file mode 100644index 0000000..1029971--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/text_detect.py@@ -0,0 +1,150 @@+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+import argparse+import time++import cv2+import numpy as np+from openvino.runtime import Core++try:+ from .utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+except:+ from utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+++class TextDetector(object):+ def __init__(self, det_model_path):+ pre_process_list = [{+ 'DetResizeForTest': {+ 'limit_side_len': 736,+ 'limit_type': 'min'+ }+ }, {+ 'NormalizeImage': {+ 'std': [0.229, 0.224, 0.225],+ 'mean': [0.485, 0.456, 0.406],+ 'scale': '1./255.',+ 'order': 'hwc'+ }+ }, {+ 'ToCHWImage': None+ }, {+ 'KeepKeys': {+ 'keep_keys': ['image', 'shape']+ }+ }]++ self.preprocess_op = create_operators(pre_process_list)+ self.postprocess_op = DBPostProcess(thresh=0.3,+ box_thresh=0.5,+ max_candidates=1000,+ unclip_ratio=1.6,+ use_dilation=True)+ ie = Core()+ model_onnx = ie.read_model(det_model_path)+ compile_model = ie.compile_model(model=model_onnx,+ device_name='CPU')+ self.vino_session = compile_model.create_infer_request()++ def order_points_clockwise(self, pts):+ """+ reference from: https://github.com/jrosebr1/imutils/blob/master/imutils/perspective.py+ # sort the points based on their x-coordinates+ """+ xSorted = pts[np.argsort(pts[:, 0]), :]++ # grab the left-most and right-most points from the sorted+ # x-roodinate points+ leftMost = xSorted[:2, :]+ rightMost = xSorted[2:, :]++ # now, sort the left-most coordinates according to their+ # y-coordinates so we can grab the top-left and bottom-left+ # points, respectively+ leftMost = leftMost[np.argsort(leftMost[:, 1]), :]+ (tl, bl) = leftMost++ rightMost = rightMost[np.argsort(rightMost[:, 1]), :]+ (tr, br) = rightMost++ rect = np.array([tl, tr, br, bl], dtype="float32")+ return rect++ def clip_det_res(self, points, img_height, img_width):+ for pno in range(points.shape[0]):+ points[pno, 0] = int(min(max(points[pno, 0], 0), img_width - 1))+ points[pno, 1] = int(min(max(points[pno, 1], 0), img_height - 1))+ return points++ def filter_tag_det_res(self, dt_boxes, image_shape):+ img_height, img_width = image_shape[0:2]+ dt_boxes_new = []+ for box in dt_boxes:+ box = self.order_points_clockwise(box)+ box = self.clip_det_res(box, img_height, img_width)+ rect_width = int(np.linalg.norm(box[0] - box[1]))+ rect_height = int(np.linalg.norm(box[0] - box[3]))+ if rect_width <= 3 or rect_height <= 3:+ continue+ dt_boxes_new.append(box)+ dt_boxes = np.array(dt_boxes_new)+ return dt_boxes++ def __call__(self, img):+ ori_im = img.copy()+ data = {'image': img}+ data = transform(data, self.preprocess_op)+ img, shape_list = data+ if img is None:+ return None, 0+ img = np.expand_dims(img, axis=0)+ img = img.astype(np.float32)+ shape_list = np.expand_dims(shape_list, axis=0)++ start_time = time.time()+ self.vino_session.infer(inputs=[img])+ vino_preds = self.vino_session.get_output_tensor().data++ post_result = self.postprocess_op(vino_preds, shape_list)+ dt_boxes = post_result[0]['points']+ dt_boxes = self.filter_tag_det_res(dt_boxes, ori_im.shape)+ elapse = time.time() - start_time+ return dt_boxes, elapse+++if __name__ == "__main__":+ parser = argparse.ArgumentParser()+ parser.add_argument('--image_path', type=str, help='image_path|image_dir')+ parser.add_argument('--model_path', type=str, help='model_path')+ args = parser.parse_args()++ text_detector = TextDetector(args.model_path)++ img, flag = check_and_read_gif(args.image_path)+ if not flag:+ img = cv2.imread(args.image_path)+ if img is None:+ raise ValueError(f"error in loading image:{args.image_path}")++ dt_boxes, elapse = text_detector(img)++ src_im = draw_text_det_res(dt_boxes, args.image_path)+ cv2.imwrite('det_results.jpg', src_im)+ print(f'图像已经保存为det_results.jpg了\t{elapse:.6f}s')+diff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/utils.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/utils.pynew file mode 100644index 0000000..dfbea6c--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_onnx/utils.py@@ -0,0 +1,414 @@+"""+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+"""+# coding:utf-8+from __future__ import (absolute_import, division, print_function,+ unicode_literals)++import os+import sys++import cv2+import numpy as np+import pyclipper+import six+from shapely.geometry import Polygon+++class DecodeImage(object):+ """ decode image """++ def __init__(self, img_mode='RGB', channel_first=False, **kwargs):+ self.img_mode = img_mode+ self.channel_first = channel_first++ def __call__(self, data):+ img = data['image']+ if six.PY2:+ assert type(img) is str and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ else:+ assert type(img) is bytes and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ img = np.frombuffer(img, dtype='uint8')+ img = cv2.imdecode(img, 1)+ if img is None:+ return None+ if self.img_mode == 'GRAY':+ img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)+ elif self.img_mode == 'RGB':+ assert img.shape[2] == 3, 'invalid shape of image[%s]' % (+ img.shape)+ img = img[:, :, ::-1]++ if self.channel_first:+ img = img.transpose((2, 0, 1))++ data['image'] = img+ return data+++class NormalizeImage(object):+ """ normalize image such as substract mean, divide std+ """++ def __init__(self, scale=None, mean=None, std=None, order='chw', **kwargs):+ if isinstance(scale, str):+ scale = eval(scale)+ self.scale = np.float32(scale if scale is not None else 1.0 / 255.0)+ mean = mean if mean is not None else [0.485, 0.456, 0.406]+ std = std if std is not None else [0.229, 0.224, 0.225]++ shape = (3, 1, 1) if order == 'chw' else (1, 1, 3)+ self.mean = np.array(mean).reshape(shape).astype('float32')+ self.std = np.array(std).reshape(shape).astype('float32')++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)++ assert isinstance(img,+ np.ndarray), "invalid input 'img' in NormalizeImage"+ data['image'] = (+ img.astype('float32') * self.scale - self.mean) / self.std+ return data+++class ToCHWImage(object):+ """ convert hwc image to chw image+ """++ def __init__(self, **kwargs):+ pass++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)+ data['image'] = img.transpose((2, 0, 1))+ return data+++class KeepKeys(object):+ def __init__(self, keep_keys, **kwargs):+ self.keep_keys = keep_keys++ def __call__(self, data):+ data_list = []+ for key in self.keep_keys:+ data_list.append(data[key])+ return data_list+++class DetResizeForTest(object):+ def __init__(self, **kwargs):+ super(DetResizeForTest, self).__init__()+ self.resize_type = 0+ if 'image_shape' in kwargs:+ self.image_shape = kwargs['image_shape']+ self.resize_type = 1+ elif 'limit_side_len' in kwargs:+ self.limit_side_len = kwargs['limit_side_len']+ self.limit_type = kwargs.get('limit_type', 'min')++ if 'resize_long' in kwargs:+ self.resize_type = 2+ self.resize_long = kwargs.get('resize_long', 960)+ else:+ self.limit_side_len = 736+ self.limit_type = 'min'++ def __call__(self, data):+ img = data['image']+ src_h, src_w, _ = img.shape++ if self.resize_type == 0:+ # img, shape = self.resize_image_type0(img)+ img, [ratio_h, ratio_w] = self.resize_image_type0(img)+ elif self.resize_type == 2:+ img, [ratio_h, ratio_w] = self.resize_image_type2(img)+ else:+ # img, shape = self.resize_image_type1(img)+ img, [ratio_h, ratio_w] = self.resize_image_type1(img)+ data['image'] = img+ data['shape'] = np.array([src_h, src_w, ratio_h, ratio_w])+ return data++ def resize_image_type1(self, img):+ resize_h, resize_w = self.image_shape+ ori_h, ori_w = img.shape[:2] # (h, w, c)+ ratio_h = float(resize_h) / ori_h+ ratio_w = float(resize_w) / ori_w+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ # return img, np.array([ori_h, ori_w])+ return img, [ratio_h, ratio_w]++ def resize_image_type0(self, img):+ """+ resize image to a size multiple of 32 which is required by the network+ args:+ img(array): array with shape [h, w, c]+ return(tuple):+ img, (ratio_h, ratio_w)+ """+ limit_side_len = self.limit_side_len+ h, w, _ = img.shape++ # limit the max side+ if self.limit_type == 'max':+ if max(h, w) > limit_side_len:+ if h > w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ else:+ if min(h, w) < limit_side_len:+ if h < w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ resize_h = int(h * ratio)+ resize_w = int(w * ratio)++ resize_h = int(round(resize_h / 32) * 32)+ resize_w = int(round(resize_w / 32) * 32)++ try:+ if int(resize_w) <= 0 or int(resize_h) <= 0:+ return None, (None, None)+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ except:+ print(img.shape, resize_w, resize_h)+ sys.exit(0)+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)+ return img, [ratio_h, ratio_w]++ def resize_image_type2(self, img):+ h, w, _ = img.shape++ resize_w = w+ resize_h = h++ # Fix the longer side+ if resize_h > resize_w:+ ratio = float(self.resize_long) / resize_h+ else:+ ratio = float(self.resize_long) / resize_w++ resize_h = int(resize_h * ratio)+ resize_w = int(resize_w * ratio)++ max_stride = 128+ resize_h = (resize_h + max_stride - 1) // max_stride * max_stride+ resize_w = (resize_w + max_stride - 1) // max_stride * max_stride+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)++ return img, [ratio_h, ratio_w]+++def transform(data, ops=None):+ """ transform """+ if ops is None:+ ops = []+ for op in ops:+ data = op(data)+ if data is None:+ return None+ return data+++def check_and_read_gif(img_path):+ if os.path.basename(img_path)[-3:] in ['gif', 'GIF']:+ gif = cv2.VideoCapture(img_path)+ ret, frame = gif.read()+ if not ret:+ print("Cannot read {}. This gif image maybe corrupted.")+ return None, False+ if len(frame.shape) == 2 or frame.shape[-1] == 1:+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)+ imgvalue = frame[:, :, ::-1]+ return imgvalue, True+ return None, False+++def create_operators(op_param_list, global_config=None):+ """+ create operators based on the config++ Args:+ params(list): a dict list, used to create some operators+ """+ assert isinstance(+ op_param_list, list), ('operator config should be a list')+ ops = []+ for operator in op_param_list:+ assert isinstance(operator,+ dict) and len(operator) == 1, "yaml format error"+ op_name = list(operator)[0]+ param = {} if operator[op_name] is None else operator[op_name]+ if global_config is not None:+ param.update(global_config)+ op = eval(op_name)(**param)+ ops.append(op)+ return ops+++def draw_text_det_res(dt_boxes, img_path):+ src_im = cv2.imread(img_path)+ for box in dt_boxes:+ box = np.array(box).astype(np.int32).reshape(-1, 2)+ cv2.polylines(src_im, [box], True,+ color=(255, 255, 0), thickness=2)+ return src_im+++class DBPostProcess(object):+ """+ The post process for Differentiable Binarization (DB).+ """++ def __init__(self,+ thresh=0.3,+ box_thresh=0.7,+ max_candidates=1000,+ unclip_ratio=2.0,+ use_dilation=False,+ **kwargs):+ self.thresh = thresh+ self.box_thresh = box_thresh+ self.max_candidates = max_candidates+ self.unclip_ratio = unclip_ratio+ self.min_size = 3+ self.dilation_kernel = None if not use_dilation else np.array(+ [[1, 1], [1, 1]])++ def boxes_from_bitmap(self, pred, _bitmap, dest_width, dest_height):+ '''+ _bitmap: single map with shape (1, H, W),+ whose values are binarized as {0, 1}+ '''++ bitmap = _bitmap+ height, width = bitmap.shape++ outs = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST,+ cv2.CHAIN_APPROX_SIMPLE)+ if len(outs) == 3:+ img, contours, _ = outs[0], outs[1], outs[2]+ elif len(outs) == 2:+ contours, _ = outs[0], outs[1]++ num_contours = min(len(contours), self.max_candidates)++ boxes = []+ scores = []+ for index in range(num_contours):+ contour = contours[index]+ points, sside = self.get_mini_boxes(contour)+ if sside < self.min_size:+ continue+ points = np.array(points)+ score = self.box_score_fast(pred, points.reshape(-1, 2))+ if self.box_thresh > score:+ continue++ box = self.unclip(points).reshape(-1, 1, 2)+ box, sside = self.get_mini_boxes(box)+ if sside < self.min_size + 2:+ continue+ box = np.array(box)++ box[:, 0] = np.clip(+ np.round(box[:, 0] / width * dest_width), 0, dest_width)+ box[:, 1] = np.clip(+ np.round(box[:, 1] / height * dest_height), 0, dest_height)+ boxes.append(box.astype(np.int16))+ scores.append(score)+ return np.array(boxes, dtype=np.int16), scores++ def unclip(self, box):+ unclip_ratio = self.unclip_ratio+ poly = Polygon(box)+ distance = poly.area * unclip_ratio / poly.length+ offset = pyclipper.PyclipperOffset()+ offset.AddPath(box, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)+ expanded = np.array(offset.Execute(distance))+ return expanded++ def get_mini_boxes(self, contour):+ bounding_box = cv2.minAreaRect(contour)+ points = sorted(list(cv2.boxPoints(bounding_box)), key=lambda x: x[0])++ index_1, index_2, index_3, index_4 = 0, 1, 2, 3+ if points[1][1] > points[0][1]:+ index_1 = 0+ index_4 = 1+ else:+ index_1 = 1+ index_4 = 0+ if points[3][1] > points[2][1]:+ index_2 = 2+ index_3 = 3+ else:+ index_2 = 3+ index_3 = 2++ box = [+ points[index_1], points[index_2], points[index_3], points[index_4]+ ]+ return box, min(bounding_box[1])++ def box_score_fast(self, bitmap, _box):+ h, w = bitmap.shape[:2]+ box = _box.copy()+ xmin = np.clip(np.floor(box[:, 0].min()).astype(np.int), 0, w - 1)+ xmax = np.clip(np.ceil(box[:, 0].max()).astype(np.int), 0, w - 1)+ ymin = np.clip(np.floor(box[:, 1].min()).astype(np.int), 0, h - 1)+ ymax = np.clip(np.ceil(box[:, 1].max()).astype(np.int), 0, h - 1)++ mask = np.zeros((ymax - ymin + 1, xmax - xmin + 1), dtype=np.uint8)+ box[:, 0] = box[:, 0] - xmin+ box[:, 1] = box[:, 1] - ymin+ cv2.fillPoly(mask, box.reshape(1, -1, 2).astype(np.int32), 1)+ return cv2.mean(bitmap[ymin:ymax + 1, xmin:xmax + 1], mask)[0]++ def __call__(self, pred, shape_list):+ pred = pred[:, 0, :, :]+ segmentation = pred > self.thresh++ boxes_batch = []+ for batch_index in range(pred.shape[0]):+ src_h, src_w, ratio_h, ratio_w = shape_list[batch_index]+ if self.dilation_kernel is not None:+ mask = cv2.dilate(+ np.array(segmentation[batch_index]).astype(np.uint8),+ self.dilation_kernel)+ else:+ mask = segmentation[batch_index]+ boxes, scores = self.boxes_from_bitmap(pred[batch_index], mask,+ src_w, src_h)++ boxes_batch.append({'points': boxes})+ return boxes_batchdiff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/__init__.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/__init__.pynew file mode 100644index 0000000..d7a3e0b--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/__init__.py@@ -0,0 +1,6 @@+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+# @File: __init__.py+# @Time: 2021/01/23 15:24:27+# @Author: Max+from .text_detect import TextDetector\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/run.sh b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/run.shnew file mode 100644index 0000000..99fc806--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/run.sh@@ -0,0 +1,9 @@+#! /bin/bash+set -e errexit++root_path=$(cd $(dirname $0); pwd)+image_path=${root_path}"/test_images/det_images/1.jpg"+model_path=${root_path}"/models/ch_ppocr_mobile_v2.0_det_infer.onnx"++python text_detect.py --image_path ${image_path} \+ --model_path ${model_path}\ No newline at end of filediff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/text_detect.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/text_detect.pynew file mode 100644index 0000000..1029971--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/text_detect.py@@ -0,0 +1,150 @@+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+import argparse+import time++import cv2+import numpy as np+from openvino.runtime import Core++try:+ from .utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+except:+ from utils import (DBPostProcess, check_and_read_gif, create_operators,+ draw_text_det_res, transform)+++class TextDetector(object):+ def __init__(self, det_model_path):+ pre_process_list = [{+ 'DetResizeForTest': {+ 'limit_side_len': 736,+ 'limit_type': 'min'+ }+ }, {+ 'NormalizeImage': {+ 'std': [0.229, 0.224, 0.225],+ 'mean': [0.485, 0.456, 0.406],+ 'scale': '1./255.',+ 'order': 'hwc'+ }+ }, {+ 'ToCHWImage': None+ }, {+ 'KeepKeys': {+ 'keep_keys': ['image', 'shape']+ }+ }]++ self.preprocess_op = create_operators(pre_process_list)+ self.postprocess_op = DBPostProcess(thresh=0.3,+ box_thresh=0.5,+ max_candidates=1000,+ unclip_ratio=1.6,+ use_dilation=True)+ ie = Core()+ model_onnx = ie.read_model(det_model_path)+ compile_model = ie.compile_model(model=model_onnx,+ device_name='CPU')+ self.vino_session = compile_model.create_infer_request()++ def order_points_clockwise(self, pts):+ """+ reference from: https://github.com/jrosebr1/imutils/blob/master/imutils/perspective.py+ # sort the points based on their x-coordinates+ """+ xSorted = pts[np.argsort(pts[:, 0]), :]++ # grab the left-most and right-most points from the sorted+ # x-roodinate points+ leftMost = xSorted[:2, :]+ rightMost = xSorted[2:, :]++ # now, sort the left-most coordinates according to their+ # y-coordinates so we can grab the top-left and bottom-left+ # points, respectively+ leftMost = leftMost[np.argsort(leftMost[:, 1]), :]+ (tl, bl) = leftMost++ rightMost = rightMost[np.argsort(rightMost[:, 1]), :]+ (tr, br) = rightMost++ rect = np.array([tl, tr, br, bl], dtype="float32")+ return rect++ def clip_det_res(self, points, img_height, img_width):+ for pno in range(points.shape[0]):+ points[pno, 0] = int(min(max(points[pno, 0], 0), img_width - 1))+ points[pno, 1] = int(min(max(points[pno, 1], 0), img_height - 1))+ return points++ def filter_tag_det_res(self, dt_boxes, image_shape):+ img_height, img_width = image_shape[0:2]+ dt_boxes_new = []+ for box in dt_boxes:+ box = self.order_points_clockwise(box)+ box = self.clip_det_res(box, img_height, img_width)+ rect_width = int(np.linalg.norm(box[0] - box[1]))+ rect_height = int(np.linalg.norm(box[0] - box[3]))+ if rect_width <= 3 or rect_height <= 3:+ continue+ dt_boxes_new.append(box)+ dt_boxes = np.array(dt_boxes_new)+ return dt_boxes++ def __call__(self, img):+ ori_im = img.copy()+ data = {'image': img}+ data = transform(data, self.preprocess_op)+ img, shape_list = data+ if img is None:+ return None, 0+ img = np.expand_dims(img, axis=0)+ img = img.astype(np.float32)+ shape_list = np.expand_dims(shape_list, axis=0)++ start_time = time.time()+ self.vino_session.infer(inputs=[img])+ vino_preds = self.vino_session.get_output_tensor().data++ post_result = self.postprocess_op(vino_preds, shape_list)+ dt_boxes = post_result[0]['points']+ dt_boxes = self.filter_tag_det_res(dt_boxes, ori_im.shape)+ elapse = time.time() - start_time+ return dt_boxes, elapse+++if __name__ == "__main__":+ parser = argparse.ArgumentParser()+ parser.add_argument('--image_path', type=str, help='image_path|image_dir')+ parser.add_argument('--model_path', type=str, help='model_path')+ args = parser.parse_args()++ text_detector = TextDetector(args.model_path)++ img, flag = check_and_read_gif(args.image_path)+ if not flag:+ img = cv2.imread(args.image_path)+ if img is None:+ raise ValueError(f"error in loading image:{args.image_path}")++ dt_boxes, elapse = text_detector(img)++ src_im = draw_text_det_res(dt_boxes, args.image_path)+ cv2.imwrite('det_results.jpg', src_im)+ print(f'图像已经保存为det_results.jpg了\t{elapse:.6f}s')+diff --git a/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/utils.py b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/utils.pynew file mode 100644index 0000000..dfbea6c--- /dev/null+++ b/python/openvino_infer/ch_ppocr_mobile_v2_det_openvino_paddle/utils.py@@ -0,0 +1,414 @@+"""+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+"""+# coding:utf-8+from __future__ import (absolute_import, division, print_function,+ unicode_literals)++import os+import sys++import cv2+import numpy as np+import pyclipper+import six+from shapely.geometry import Polygon+++class DecodeImage(object):+ """ decode image """++ def __init__(self, img_mode='RGB', channel_first=False, **kwargs):+ self.img_mode = img_mode+ self.channel_first = channel_first++ def __call__(self, data):+ img = data['image']+ if six.PY2:+ assert type(img) is str and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ else:+ assert type(img) is bytes and len(+ img) > 0, "invalid input 'img' in DecodeImage"+ img = np.frombuffer(img, dtype='uint8')+ img = cv2.imdecode(img, 1)+ if img is None:+ return None+ if self.img_mode == 'GRAY':+ img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)+ elif self.img_mode == 'RGB':+ assert img.shape[2] == 3, 'invalid shape of image[%s]' % (+ img.shape)+ img = img[:, :, ::-1]++ if self.channel_first:+ img = img.transpose((2, 0, 1))++ data['image'] = img+ return data+++class NormalizeImage(object):+ """ normalize image such as substract mean, divide std+ """++ def __init__(self, scale=None, mean=None, std=None, order='chw', **kwargs):+ if isinstance(scale, str):+ scale = eval(scale)+ self.scale = np.float32(scale if scale is not None else 1.0 / 255.0)+ mean = mean if mean is not None else [0.485, 0.456, 0.406]+ std = std if std is not None else [0.229, 0.224, 0.225]++ shape = (3, 1, 1) if order == 'chw' else (1, 1, 3)+ self.mean = np.array(mean).reshape(shape).astype('float32')+ self.std = np.array(std).reshape(shape).astype('float32')++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)++ assert isinstance(img,+ np.ndarray), "invalid input 'img' in NormalizeImage"+ data['image'] = (+ img.astype('float32') * self.scale - self.mean) / self.std+ return data+++class ToCHWImage(object):+ """ convert hwc image to chw image+ """++ def __init__(self, **kwargs):+ pass++ def __call__(self, data):+ img = data['image']+ from PIL import Image+ if isinstance(img, Image.Image):+ img = np.array(img)+ data['image'] = img.transpose((2, 0, 1))+ return data+++class KeepKeys(object):+ def __init__(self, keep_keys, **kwargs):+ self.keep_keys = keep_keys++ def __call__(self, data):+ data_list = []+ for key in self.keep_keys:+ data_list.append(data[key])+ return data_list+++class DetResizeForTest(object):+ def __init__(self, **kwargs):+ super(DetResizeForTest, self).__init__()+ self.resize_type = 0+ if 'image_shape' in kwargs:+ self.image_shape = kwargs['image_shape']+ self.resize_type = 1+ elif 'limit_side_len' in kwargs:+ self.limit_side_len = kwargs['limit_side_len']+ self.limit_type = kwargs.get('limit_type', 'min')++ if 'resize_long' in kwargs:+ self.resize_type = 2+ self.resize_long = kwargs.get('resize_long', 960)+ else:+ self.limit_side_len = 736+ self.limit_type = 'min'++ def __call__(self, data):+ img = data['image']+ src_h, src_w, _ = img.shape++ if self.resize_type == 0:+ # img, shape = self.resize_image_type0(img)+ img, [ratio_h, ratio_w] = self.resize_image_type0(img)+ elif self.resize_type == 2:+ img, [ratio_h, ratio_w] = self.resize_image_type2(img)+ else:+ # img, shape = self.resize_image_type1(img)+ img, [ratio_h, ratio_w] = self.resize_image_type1(img)+ data['image'] = img+ data['shape'] = np.array([src_h, src_w, ratio_h, ratio_w])+ return data++ def resize_image_type1(self, img):+ resize_h, resize_w = self.image_shape+ ori_h, ori_w = img.shape[:2] # (h, w, c)+ ratio_h = float(resize_h) / ori_h+ ratio_w = float(resize_w) / ori_w+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ # return img, np.array([ori_h, ori_w])+ return img, [ratio_h, ratio_w]++ def resize_image_type0(self, img):+ """+ resize image to a size multiple of 32 which is required by the network+ args:+ img(array): array with shape [h, w, c]+ return(tuple):+ img, (ratio_h, ratio_w)+ """+ limit_side_len = self.limit_side_len+ h, w, _ = img.shape++ # limit the max side+ if self.limit_type == 'max':+ if max(h, w) > limit_side_len:+ if h > w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ else:+ if min(h, w) < limit_side_len:+ if h < w:+ ratio = float(limit_side_len) / h+ else:+ ratio = float(limit_side_len) / w+ else:+ ratio = 1.+ resize_h = int(h * ratio)+ resize_w = int(w * ratio)++ resize_h = int(round(resize_h / 32) * 32)+ resize_w = int(round(resize_w / 32) * 32)++ try:+ if int(resize_w) <= 0 or int(resize_h) <= 0:+ return None, (None, None)+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ except:+ print(img.shape, resize_w, resize_h)+ sys.exit(0)+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)+ return img, [ratio_h, ratio_w]++ def resize_image_type2(self, img):+ h, w, _ = img.shape++ resize_w = w+ resize_h = h++ # Fix the longer side+ if resize_h > resize_w:+ ratio = float(self.resize_long) / resize_h+ else:+ ratio = float(self.resize_long) / resize_w++ resize_h = int(resize_h * ratio)+ resize_w = int(resize_w * ratio)++ max_stride = 128+ resize_h = (resize_h + max_stride - 1) // max_stride * max_stride+ resize_w = (resize_w + max_stride - 1) // max_stride * max_stride+ img = cv2.resize(img, (int(resize_w), int(resize_h)))+ ratio_h = resize_h / float(h)+ ratio_w = resize_w / float(w)++ return img, [ratio_h, ratio_w]+++def transform(data, ops=None):+ """ transform """+ if ops is None:+ ops = []+ for op in ops:+ data = op(data)+ if data is None:+ return None+ return data+++def check_and_read_gif(img_path):+ if os.path.basename(img_path)[-3:] in ['gif', 'GIF']:+ gif = cv2.VideoCapture(img_path)+ ret, frame = gif.read()+ if not ret:+ print("Cannot read {}. This gif image maybe corrupted.")+ return None, False+ if len(frame.shape) == 2 or frame.shape[-1] == 1:+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)+ imgvalue = frame[:, :, ::-1]+ return imgvalue, True+ return None, False+++def create_operators(op_param_list, global_config=None):+ """+ create operators based on the config++ Args:+ params(list): a dict list, used to create some operators+ """+ assert isinstance(+ op_param_list, list), ('operator config should be a list')+ ops = []+ for operator in op_param_list:+ assert isinstance(operator,+ dict) and len(operator) == 1, "yaml format error"+ op_name = list(operator)[0]+ param = {} if operator[op_name] is None else operator[op_name]+ if global_config is not None:+ param.update(global_config)+ op = eval(op_name)(**param)+ ops.append(op)+ return ops+++def draw_text_det_res(dt_boxes, img_path):+ src_im = cv2.imread(img_path)+ for box in dt_boxes:+ box = np.array(box).astype(np.int32).reshape(-1, 2)+ cv2.polylines(src_im, [box], True,+ color=(255, 255, 0), thickness=2)+ return src_im+++class DBPostProcess(object):+ """+ The post process for Differentiable Binarization (DB).+ """++ def __init__(self,+ thresh=0.3,+ box_thresh=0.7,+ max_candidates=1000,+ unclip_ratio=2.0,+ use_dilation=False,+ **kwargs):+ self.thresh = thresh+ self.box_thresh = box_thresh+ self.max_candidates = max_candidates+ self.unclip_ratio = unclip_ratio+ self.min_size = 3+ self.dilation_kernel = None if not use_dilation else np.array(+ [[1, 1], [1, 1]])++ def boxes_from_bitmap(self, pred, _bitmap, dest_width, dest_height):+ '''+ _bitmap: single map with shape (1, H, W),+ whose values are binarized as {0, 1}+ '''++ bitmap = _bitmap+ height, width = bitmap.shape++ outs = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST,+ cv2.CHAIN_APPROX_SIMPLE)+ if len(outs) == 3:+ img, contours, _ = outs[0], outs[1], outs[2]+ elif len(outs) == 2:+ contours, _ = outs[0], outs[1]++ num_contours = min(len(contours), self.max_candidates)++ boxes = []+ scores = []+ for index in range(num_contours):+ contour = contours[index]+ points, sside = self.get_mini_boxes(contour)+ if sside < self.min_size:+ continue+ points = np.array(points)+ score = self.box_score_fast(pred, points.reshape(-1, 2))+ if self.box_thresh > score:+ continue++ box = self.unclip(points).reshape(-1, 1, 2)+ box, sside = self.get_mini_boxes(box)+ if sside < self.min_size + 2:+ continue+ box = np.array(box)++ box[:, 0] = np.clip(+ np.round(box[:, 0] / width * dest_width), 0, dest_width)+ box[:, 1] = np.clip(+ np.round(box[:, 1] / height * dest_height), 0, dest_height)+ boxes.append(box.astype(np.int16))+ scores.append(score)+ return np.array(boxes, dtype=np.int16), scores++ def unclip(self, box):+ unclip_ratio = self.unclip_ratio+ poly = Polygon(box)+ distance = poly.area * unclip_ratio / poly.length+ offset = pyclipper.PyclipperOffset()+ offset.AddPath(box, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)+ expanded = np.array(offset.Execute(distance))+ return expanded++ def get_mini_boxes(self, contour):+ bounding_box = cv2.minAreaRect(contour)+ points = sorted(list(cv2.boxPoints(bounding_box)), key=lambda x: x[0])++ index_1, index_2, index_3, index_4 = 0, 1, 2, 3+ if points[1][1] > points[0][1]:+ index_1 = 0+ index_4 = 1+ else:+ index_1 = 1+ index_4 = 0+ if points[3][1] > points[2][1]:+ index_2 = 2+ index_3 = 3+ else:+ index_2 = 3+ index_3 = 2++ box = [+ points[index_1], points[index_2], points[index_3], points[index_4]+ ]+ return box, min(bounding_box[1])++ def box_score_fast(self, bitmap, _box):+ h, w = bitmap.shape[:2]+ box = _box.copy()+ xmin = np.clip(np.floor(box[:, 0].min()).astype(np.int), 0, w - 1)+ xmax = np.clip(np.ceil(box[:, 0].max()).astype(np.int), 0, w - 1)+ ymin = np.clip(np.floor(box[:, 1].min()).astype(np.int), 0, h - 1)+ ymax = np.clip(np.ceil(box[:, 1].max()).astype(np.int), 0, h - 1)++ mask = np.zeros((ymax - ymin + 1, xmax - xmin + 1), dtype=np.uint8)+ box[:, 0] = box[:, 0] - xmin+ box[:, 1] = box[:, 1] - ymin+ cv2.fillPoly(mask, box.reshape(1, -1, 2).astype(np.int32), 1)+ return cv2.mean(bitmap[ymin:ymax + 1, xmin:xmax + 1], mask)[0]++ def __call__(self, pred, shape_list):+ pred = pred[:, 0, :, :]+ segmentation = pred > self.thresh++ boxes_batch = []+ for batch_index in range(pred.shape[0]):+ src_h, src_w, ratio_h, ratio_w = shape_list[batch_index]+ if self.dilation_kernel is not None:+ mask = cv2.dilate(+ np.array(segmentation[batch_index]).astype(np.uint8),+ self.dilation_kernel)+ else:+ mask = segmentation[batch_index]+ boxes, scores = self.boxes_from_bitmap(pred[batch_index], mask,+ src_w, src_h)++ boxes_batch.append({'points': boxes})+ return boxes_batchdiff --git a/python/openvino_infer/test_openvino.py b/python/openvino_infer/test_openvino.pynew file mode 100644index 0000000..308838b--- /dev/null+++ b/python/openvino_infer/test_openvino.py@@ -0,0 +1,59 @@+# !/usr/bin/env python+# -*- encoding: utf-8 -*-+# @File: test_openvino.py+# @Author: SWHL+# @Contact: [email protected]+import numpy as np++# Infer ONNXRuntime+# import cv2++# from ch_ppocr_mobile_v2_det import TextDetector++# det_model_path = 'models/ch_PP-OCRv2_det_infer.onnx'++# img_path = 'test_images/long1.jpg'+# img = cv2.imread(img_path)++# text_detect_onnx = TextDetector(det_model_path)+# dt_boxes, elapse = text_detect_onnx(img)+# print(f'onnxruntime: {elapse}')+++# # Infer ONNX+# import cv2+# from ch_ppocr_mobile_v2_det_openvino_onnx import TextDetector as TextDetectorVino++# det_model_path = 'models/ch_PP-OCRv2_det_infer.onnx'+# img_path = 'test_images/det_images/ch_en_num.jpg'+# img = cv2.imread(img_path)++# text_detect_vino = TextDetectorVino(det_model_path)+# dt_boxes, elapse = text_detect_vino(img)+# print(f'openvino ONNX: {elapse}\tdt_boxes:{np.mean(dt_boxes)}')++# # Infer IR+import cv2+from ch_ppocr_mobile_v2_det_openvino_ir import TextDetector as TextDetectorIR++det_model_path = 'models/IR/FP16/ch_PP-OCRv2_det_infer.xml'+img_path = 'test_images/long1.jpg'+img = cv2.imread(img_path)++text_detect_ir = TextDetectorIR(det_model_path)+dt_boxes, elapse = text_detect_ir(img)+print(f'FP16 openvino IR: {elapse}\tdt_boxes:{np.mean(dt_boxes)}')++# Infer paddle 有误+# RuntimeError: Can't SetBlob with name: x, because model input (shape={?,3,960,960}) and blob (shape=(1.3.12128.800)) are incompatible+# import cv2+# from ch_ppocr_mobile_v2_det_openvino_paddle import TextDetector as TextDetectorPaddle++# det_model_path = 'models/ch_PP-OCRv2_det_infer/ch_PP-OCRv2_det_infer/inference.pdmodel'+# img_path = 'test_images/long1.jpg'+# img = cv2.imread(img_path)++# text_detect_ir = TextDetectorPaddle(det_model_path)+# dt_boxes, elapse = text_detect_ir(img)+# print(f'openvino IR: {elapse}')+