训练框架 | 模型格式 | 目标运行时 | 编译后模型格式 |
Keras | h5 | Tf Serving | SavedModel |
|
| OpenVINO | IR |
|
| TensorRT | Plan |
|
| TF-Lite | tflite |
TensorFlow | Ckpt/pb | Tf Serving | SavedModel |
|
| OpenVINO | IR |
|
| TensorRT | Plan |
|
| TF-Lite | tflite |
PyTorch | pth | OpenVINO | IR |
|
| TensorRT | Plan |
训练框架 | 推理引擎 | 运行硬件 |
| TensorFlow Serving-1.14 | CPU/GPU |
| TensorFlow Serving-2.2 | CPU/GPU |
| OpenVINO-2019 | CPU |
| TensorRT-6 | GPU |
| TensorRT-7 | GPU |
| TF Lite-2.1 | CPU(X86/ARM) |
TensorFlow | TensorFlow Serving-1.14 | CPU/GPU |
| TensorFlow Serving-2.2 | CPU/GPU |
| OpenVINO-2019 | CPU |
| TensorRT-6 | GPU |
| TensorRT-7 | GPU |
| TF Lite-2.1 | CPU(X86/ARM) |
PyTorch | OpenVINO-2019 | CPU |
| TensorRT-6 | GPU |
1、支持多机多卡的模型训练和剪枝
2、可以配置的filter pruning实现,剪枝后能直接得到更小的推理模型;
3、基于小批量数据集的模型量化,支持TF-Lite和TF-TRT量化。
集成多种推理运行时。
集成的推理运行时及版本 | 运行硬件 |
TensorFlow Serving-1.14 | CPU/GPU |
TensorFlow Serving-2.2 | CPU/GPU |
OpenVINO-2019 | CPU |
TensorRT-6 | GPU |
TensorRT-7 | GPU |
TF Lite-2.1 | CPU(X86/ARM) |
Benchmark Test Framework for Deep Learning Model