How To Deploy PyTorch Models on Raspberry Pi AI Camera

This tutorial will guide you through deploying PyTorch models on a Raspberry Pi AI Camera.

The Raspberry Pi AI Camera by Sony comes with a collection of prebuilt AI models that you can deploy right away to run real-time inference applications. These models are already converted into the format required by the Raspberry Pi AI Camera, so you can get started quickly. If you’re new to the camera, check out our Getting Started with the Raspberry Pi AI Camera guide.

In addition to the preinstalled models, you can also convert and deploy your own custom models. Currently, both PyTorch and TensorFlow models can be converted, quantized, and packaged into the Raspberry Pi AI Camera format.

In this tutorial, we’ll walk through the process of taking a pretrained PyTorch model, converting it into the correct format, and deploying it on the Raspberry Pi AI Camera.

Step-by-Step Process to Deploy PyTorch Models on Raspberry Pi AI Camera

To deploy a PyTorch model on the Raspberry Pi AI Camera, you’ll go through three main steps:

1. Model Optimization – Optimize and compress the model using techniques such as Post-Training Quantization and Parameter Fine-Tuning with Gradients. This step reduces the model’s size and memory footprint, ensuring it fits within the memory of the Raspberry Pi AI Camera.

2. Model Compilation – Convert the optimized model into a binary file. This file contains both the model data and the hardware configuration details, which can be packaged and deployed into the Raspberry Pi AI Camera.

3. Model Packaging – Package the compiled model into an .rpk file. This is the firmware file that can be installed and run directly on the Raspberry Pi AI Cam

Tip: The first two steps (Optimization and Compilation) are best done on your computer, since they can be resource-intensive. The final step, Packaging, must be performed on the Raspberry Pi board itself.

Model deployment workflow on Raspberry Pi AI Camera, taken from Sony AITRIOS.

Requirements to Deploy PyTorch Models on the Raspberry Pi AI Camera

To optimize and compile PyTorch models for the Raspberry Pi AI Camera, you’ll need the right tools installed. Sony makes this easy by providing an all-in-one Python package called the Edge-MDT (Model Development Toolkit). This toolkit bundles everything you need to quantize, compress, and convert a model so it can run on the Raspberry Pi AI Camera.

Edge MDT will include:

• MCT (Model Compression Toolkit) – A Python library for quantizing and compressing neural network models.

• IMX500 Converter – A command-line tool for compiling optimized neural network models.

  
  

Before installing Edge-MDT, we recommend setting up a Python virtual environment. This keeps your dependencies clean and avoids conflicts with other projects. You can use any virtual environment tool you like, but in this guide, we’ll use uv. If you’re new to uv, check out our UV Tutorial for step-by-step instructions.

Since we will be working with PyTorch, we need to install Edge MDT’s PyTorch version:

Additionally, make sure you’ve installed both PyTorch’s torch and torchvision by following the official instructions.

How to Optimize Models for the Raspberry Pi AI Camera

To optimize and compress models, we’ll use the Model Compression Toolkit. This toolkit includes several techniques for quantizing models and offers a wide range of configuration options. With the right settings, you can strike a balance between reducing model size and maintaining accuracy.

In this guide, we’ll keep things simple and use the default optimization settings. However, we recommend exploring the full set of options in the MCT documentation to better understand how different configurations can improve performance or accuracy depending on your needs.

Create a new file named optimize.py and add the following:

We will now study each portion of the code.

1. Obtain the pretrained PyTorch model

Start by loading a pretrained model. In this example, we’ll use MobileNetV2, which has been trained on the ImageNet dataset. You can follow along with this model, or substitute it with your own custom PyTorch model if you prefer.

2. Prepare a representative dataset

A representative dataset is used by MCT to adjust the internal ranges of the model’s activations. Typically, this is just a subset of the training data. For simplicity, in this guide we’ll use the validation split of the ImageNet dataset.

First, download the ImageNet validation dataset:

Then, load the dataset in PyTorch:

3. Create a representative dataset generator

The MCT needs a function that can provide batches of images from the dataset during optimization. To do this, we’ll wrap the representative dataset inside a data loader and then create a small generator function that yields images.

4. Set the target hardware platform

MCT can optimize your model specifically for the hardware it will run on. Since the Raspberry Pi AI Camera uses the IMX500 sensor, we’ll configure MCT to target that platform.

5. Optimize the model

We will now use Post-Training Quantization to compress our model.

6. Export the model to ONNX format

Finally, you can save the compressed model to an ONNX file.

7. Evaluate the optimized model

Additionally, you can run accuracy tests on the quantized model and compare it with the original floating-point model.

Run optimize.py to generate the quantized mobilenetv2.onnx model:

How to Compile Models for the Raspberry Pi AI Camera

The next step is to compile the compressed model. For this, we’ll use the imxconv-pt command-line tool. All you need to do is provide the path to your ONNX model and the output directory where the compiled files will be saved:

After running the command, you’ll find several files in the output directory:

• packerOut.zip contains the compiled model files, which will later be packaged into firmware (.rpk) for the Raspberry Pi AI Camera.

• A memory report is generated in mobilenetv2_MemoryReport.json, with key details about the compiled model’s resource requirements.

If you encounter insufficient memory errors such as the following, it means your model is still too large for the Raspberry Pi AI Camera. In that case, you’ll need to apply further compression or experiment with different techniques in the Model Compression Toolkit.

Alternatively, you can use the --no-input-persistency option. This frees up additional memory for the model, but with the trade-off that input tensors will not persist during inference, making them unavailable later for debugging AI applications

How to Package Models for the Raspberry Pi AI Camera

The final step is packaging the models for deployment. For this step, go into your Raspberry Pi device and make sure to transfer the packerOut.zip file to it.

1. Install the IMX500 packager

Run the following command to install the tools needed to package your compiled AI models for the IMX500 sensor.

2. Package the model

The IMX500 packager takes the packerOut.zip file as input and generates a network.rpk in the specified output directory.

Note: You may still encounter insufficient memory errors during this final step. If that happens, you’ll need to further compress the model with the MCT before retrying.

How to Run PyTorch Models on the Raspberry Pi AI Camera

We can now go ahead and run the model on the Raspberry Pi AI Camera. For this, we will use the Picamera2 example scripts.

1. Download the examples

Clone the Picamera2 repository:

Travel to the AI Camera examples directory:

2. Install dependencies

You’ll need OpenCV to run the example applications, along with Picamera2 (which is already preinstalled on recent versions of Raspberry Pi OS).

3. Run real-time AI inference

Since MobileNetV2 is an image classification model, we will run the image classification example script.

After running the previous example, a window will appear showing your camera’s image classification results in real time. Be sure to customize the example or create one tailored to your own model and task.

Real-time image classification with the Raspberry Pi AI Camera.

Contact Us

Working with custom AI models? At RidgeRun.ai, we specialize in optimizing and pruning models for embedded systems like Raspberry Pi, helping you bring ambitious projects to life while keeping resource usage in check. Contact us at contactus@ridgerun.ai — let’s collaborate!