Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 3 additions & 94 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,102 +1,11 @@
# OpenVINO Model API

[![PyPI](https://img.shields.io/pypi/v/openvino-model-api)](https://pypi.org/project/openvino-model-api)
[![Downloads](https://static.pepy.tech/personalized-badge/openvino-model-api?period=total&units=international_system&left_color=grey&right_color=green&left_text=PyPI%20Downloads)](https://pepy.tech/project/openvino-model-api)

[![Pre-Merge Test](https://github.com/open-edge-platform/model_api/actions/workflows/pre_commit.yml/badge.svg)](https://github.com/open-edge-platform/model_api/actions/workflows/pre_commit.yml)
[![Build Docs](https://github.com/open-edge-platform/model_api/actions/workflows/docs.yml/badge.svg)](https://github.com/open-edge-platform/model_api/actions/workflows/docs.yml)

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/open-edge-platform/model_api/badge)](https://scorecard.dev/viewer/?uri=github.com/open-edge-platform/model_api)

## Introduction

Model API is a set of wrapper classes for particular tasks and model architectures, simplifying data preprocess and postprocess as well as routine procedures (model loading, asynchronous execution, etc.). It is aimed at simplifying end-to-end model inference for different deployment scenarios, including local execution and serving. The Model API is based on the OpenVINO inference API.

## How it works

Model API searches for additional information required for model inference, data, pre/postprocessing, label names, etc. directly in OpenVINO Intermediate Representation. This information is used to prepare the inference data, process and output the inference results in a human-readable format.

Currently, ModelAPI supports models trained in [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions) framework.
Training Extensions embed all the metadata required for inference into model file. For models coming from other than Training Extensions frameworks metadata generation step is required before using ModelAPI.

## Supported model formats

- [OpenVINO IR](https://docs.openvino.ai/2025/documentation/openvino-ir-format.html)
- [ONNX](https://onnx.ai)

## Features

- Python API
- Synchronous and asynchronous inference
- Local inference and serving through the REST API
- Model preprocessing embedding for faster inference

## Installation

`pip install openvino-model-api`
# OpenVINO Model API and Model Converter

## Repository layout

This repository contains two independently installable Python subprojects:

- `model_api/` — the inference library published as `openvino-model-api` with a minimal runtime dependency set.
- `model_converter/` — conversion tooling published as `openvino-model-converter`, with conversion-time dependencies such as PyTorch, TorchVision, OpenVINO, ONNX, and NNCF.
- [model_api](model_api) — the inference library published as [openvino-model-api](https://pypi.org/project/openvino-model-api/) with a minimal runtime dependency set.
- [model_converter](model_converter) — conversion tooling published as `openvino-model-converter`, with conversion-time dependencies such as PyTorch, TorchVision, OpenVINO, ONNX, and NNCF.

Each subproject owns its own `pyproject.toml` and `uv.lock`.
Shared repository files, including this `README.md`, `LICENSE`, `CONTRIBUTING.md`, `SECURITY.md`, and CI workflows, remain at the repository root.

## Usage

```python
from model_api.models import Model

# Create a model wrapper from a compatible model generated by OpenVINO Training Extensions
# To work with an OVMS-served model, pass its endpoint instead of a file path, e.g. "localhost:8000/v2/models/ssdlite_mobilenet_v2"
model = Model.create_model("model.xml")

# Run synchronous inference locally
result = model(image) # image is numpy.ndarray

# Print results in model-specific format
print(f"Inference result: {result}")
```

## Prepare a model for `InferenceAdapter`

There are usecases when it is not possible to modify an internal `ov::Model` and it is hidden behind `InferenceAdapter`. For example the model can be served using [OVMS](https://github.com/openvinotoolkit/model_server). `create_model()` can construct a model from a given `InferenceAdapter`. That approach assumes that the model in `InferenceAdapter` was already configured by `create_model()` called with a string (a path or a model name). It is possible to prepare such model:

```python
model = DetectionModel.create_model("~/.cache/omz/public/ssdlite_mobilenet_v2/FP16/ssdlite_mobilenet_v2.xml")
model.save("serialized.xml")
```

## Usage with generic OpenVINO models

ModelAPI uses custom field in `rt_info/model_info` section of OpenVINO IR to store metadata required for preprocessing and postprocessing. If you have a generic OpenVINO model without such metadata, you can provide that metadata in `configuration` argument of `create_model()` method:

```python
from model_api.models import Model

# Create a model wrapper from a compatible model generated by OpenVINO Training Extensions
model = Model.create_model(
"model.xml",
configuration={
"model_type": "Segmentation",
"blur_strength": 1,
"labels": ["object"],
"soft_threshold": 0.5,
}
)

# Run synchronous inference locally
result = model(image) # image is numpy.ndarray

# Print results in model-specific format
print(f"Inference result: {result}")

# Save the model with metadata already embedded (passing configuration is not required anymore)
model.save("serialized_with_metadata.xml")
```

For more details please refer to the [examples](https://github.com/openvinotoolkit/model_api/tree/master/model_api/examples) of this project.
92 changes: 92 additions & 0 deletions model_api/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# OpenVINO Model API

[![PyPI](https://img.shields.io/pypi/v/openvino-model-api)](https://pypi.org/project/openvino-model-api)
[![Downloads](https://static.pepy.tech/personalized-badge/openvino-model-api?period=total&units=international_system&left_color=grey&right_color=green&left_text=PyPI%20Downloads)](https://pepy.tech/project/openvino-model-api)

[![Pre-Merge Test](https://github.com/open-edge-platform/model_api/actions/workflows/pre_commit.yml/badge.svg)](https://github.com/open-edge-platform/model_api/actions/workflows/pre_commit.yml)
[![Build Docs](https://github.com/open-edge-platform/model_api/actions/workflows/docs.yml/badge.svg)](https://github.com/open-edge-platform/model_api/actions/workflows/docs.yml)

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/open-edge-platform/model_api/badge)](https://scorecard.dev/viewer/?uri=github.com/open-edge-platform/model_api)

## Introduction

Model API is a set of wrapper classes for particular tasks and model architectures, simplifying data preprocess and postprocess as well as routine procedures (model loading, asynchronous execution, etc.). It is aimed at simplifying end-to-end model inference for different deployment scenarios, including local execution and serving. The Model API is based on the OpenVINO inference API.

## How it works

Model API searches for additional information required for model inference, data, pre/postprocessing, label names, etc. directly in OpenVINO Intermediate Representation. This information is used to prepare the inference data, process and output the inference results in a human-readable format.

Currently, ModelAPI supports models trained in [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions) framework.
Training Extensions embed all the metadata required for inference into model file. For models coming from other than Training Extensions frameworks metadata generation step is required before using ModelAPI.

## Supported model formats

- [OpenVINO IR](https://docs.openvino.ai/2025/documentation/openvino-ir-format.html)
- [ONNX](https://onnx.ai)

## Features

- Python API
- Synchronous and asynchronous inference
- Local inference and serving through the REST API
- Model preprocessing embedding for faster inference

## Installation

`pip install openvino-model-api`

## Usage

```python
from model_api.models import Model

# Create a model wrapper from a compatible model generated by OpenVINO Training Extensions
# To work with an OVMS-served model, pass its endpoint instead of a file path, e.g. "localhost:8000/v2/models/ssdlite_mobilenet_v2"
model = Model.create_model("model.xml")

# Run synchronous inference locally
result = model(image) # image is numpy.ndarray

# Print results in model-specific format
print(f"Inference result: {result}")
```

## Prepare a model for `InferenceAdapter`

There are usecases when it is not possible to modify an internal `ov::Model` and it is hidden behind `InferenceAdapter`. For example the model can be served using [OVMS](https://github.com/openvinotoolkit/model_server). `create_model()` can construct a model from a given `InferenceAdapter`. That approach assumes that the model in `InferenceAdapter` was already configured by `create_model()` called with a string (a path or a model name). It is possible to prepare such model:

```python
model = DetectionModel.create_model("~/.cache/omz/public/ssdlite_mobilenet_v2/FP16/ssdlite_mobilenet_v2.xml")
model.save("serialized.xml")
```

## Usage with generic OpenVINO models

ModelAPI uses custom field in `rt_info/model_info` section of OpenVINO IR to store metadata required for preprocessing and postprocessing. If you have a generic OpenVINO model without such metadata, you can provide that metadata in `configuration` argument of `create_model()` method:

```python
from model_api.models import Model

# Create a model wrapper from a compatible model generated by OpenVINO Training Extensions
model = Model.create_model(
"model.xml",
configuration={
"model_type": "Segmentation",
"blur_strength": 1,
"labels": ["object"],
"soft_threshold": 0.5,
}
)

# Run synchronous inference locally
result = model(image) # image is numpy.ndarray

# Print results in model-specific format
print(f"Inference result: {result}")

# Save the model with metadata already embedded (passing configuration is not required anymore)
model.save("serialized_with_metadata.xml")
```

For more details please refer to the [examples](https://github.com/openvinotoolkit/model_api/tree/master/model_api/examples) of this project.
2 changes: 1 addition & 1 deletion model_api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ maintainers = [
{name = "Intel(R) Corporation"},
]
description = "Model API: model wrappers and pipelines for inference with OpenVINO"
readme = "../README.md"
readme = "README.md"
classifiers = [
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.11",
Expand Down
Loading