Judy@4pcba.com
7:30 AM - 7:30 PM
Monday to Saturday

OpenMV Cam: Technical Guide on Smart Vision Camera

Introduction to OpenMV

OpenMV is an open-source, low-cost machine vision platform that aims to make computer vision more accessible. The OpenMV project was started in 2015 with the goal of creating an easy-to-use, Python-powered camera module for hobbyists, students, and professionals alike.

The OpenMV Cam is a small, low-power microcontroller board with an integrated camera module. It runs MicroPython, a lean version of Python 3 optimized for microcontrollers. This allows you to write concise scripts to process images and detect objects in real-time, directly on the camera itself.

Key Features of OpenMV Cam

  • Arduino-size form factor (45mm x 36mm)
  • STM32F7 ARM Cortex M7 processor running at 216 MHz
  • 512 KB RAM, 2 MB flash storage
  • OV7725 camera module (640×480 pixels)
  • MicroSD card slot for storing images and scripts
  • USB and serial interfaces for programming and debugging
  • 10-pin JTAG header for advanced debugging
  • User-controllable RGB LED and two user buttons
  • Powered via USB or 3.7V LiPo battery

Getting Started with OpenMV

Hardware Setup

To get started with the OpenMV Cam, you’ll need:
1. OpenMV Cam board
2. Mini USB cable
3. MicroSD card (optional, for storing scripts and images)
4. 3.7V LiPo battery (optional, for untethered operation)

Simply connect the OpenMV Cam to your computer using the mini USB cable. The board will power on and a green LED will light up. Your computer should recognize it as a standard USB drive.

Software Setup

Next, you’ll need to install the OpenMV IDE (Integrated Development Environment) on your computer. The IDE is available for Windows, Mac, and Linux. It provides an intuitive interface for writing MicroPython scripts, flashing them to the OpenMV Cam, and viewing the camera’s output.

Download the latest release of the OpenMV IDE from the official OpenMV website:
https://openmv.io/pages/download

Launch the installer and follow the on-screen instructions. Once installation is complete, open the OpenMV IDE.

Connecting to the OpenMV Cam

With the OpenMV IDE open, connect your OpenMV Cam board to the computer via USB. The IDE should automatically detect the board and display its serial port and firmware version in the lower-right corner of the window.

If the IDE fails to detect the board, make sure the USB cable is properly connected and the board is powered on. You may need to select the correct serial port manually using the “Connect” button.

Running Your First Script

The OpenMV IDE comes with several example scripts to help you get started. To run one of these scripts:

  1. Click the “Examples” button in the left sidebar of the IDE.
  2. Select an example script, such as “01-Basics/helloworld.py”.
  3. Click the green “Play” button at the top of the IDE to run the script.

The script will be automatically compiled and flashed to the OpenMV Cam. The camera’s output will be displayed in a new window.

Congratulations, you’ve just run your first OpenMV script! Feel free to experiment with the other example scripts to get a feel for the various computer vision capabilities of the OpenMV Cam.

MicroPython Programming with OpenMV

MicroPython Basics

OpenMV scripts are written in MicroPython, a lean implementation of Python 3 optimized for microcontrollers. If you’re already familiar with Python, you’ll feel right at home with MicroPython. The syntax is almost identical, with some minor differences to account for the limited memory and processing power of microcontrollers.

Here’s a quick overview of the key differences between MicroPython and standard Python:

Feature MicroPython Standard Python
Syntax Mostly identical to Python 3 Python 3
Memory Management No automatic garbage collection Automatic garbage collection
Standard Library Limited subset of Python standard library Full Python standard library
External Libraries Limited selection, must be compiled for MicroPython Extensive selection through pip

For a more detailed comparison, refer to the official MicroPython documentation:
https://docs.micropython.org/en/latest/genrst/index.html

OpenMV Library

In addition to the standard MicroPython libraries, OpenMV provides its own library for interacting with the camera module and performing computer vision tasks. The OpenMV library is divided into several modules:

  • sensor: Control the camera sensor’s settings, such as resolution, frame rate, and color mode.
  • image: Perform operations on image objects, such as cropping, resizing, and color filtering.
  • tf: Classify images using pre-trained TensorFlow models.
  • pyb: Interact with the OpenMV board’s hardware, such as LEDs and timers.
  • ustruct: Pack and unpack binary data structures.
  • network: Use the ESP8266 WiFi module to connect to wireless networks.

For a full list of modules and their functions, refer to the OpenMV library documentation:
https://docs.openmv.io/library/index.html

Image Processing with OpenMV

One of the key features of the OpenMV Cam is its ability to perform image processing and computer vision tasks directly on the camera module. This allows for fast, real-time processing without the need for an external computer.

The image module provides a wide range of functions for manipulating image objects:

  • image.binary(): Convert an image to black and white based on a threshold value.
  • image.cartoon(): Apply a cartoon-like effect to an image.
  • image.find_edges(): Find edges in an image using the Canny edge detection algorithm.
  • image.find_blobs(): Find connected regions of similar pixels in an image.
  • image.find_keypoints(): Find keypoints in an image using the FAST algorithm.

Here’s an example script that demonstrates some basic image processing techniques:

import sensor, image, time

sensor.reset() # Initialize the camera sensor
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000) # Wait for settings to take effect

while(True):
    img = sensor.snapshot() # Take a picture and store it in img

    img.cartoon(seed_threshold=0.05, floating_thresholds=0.05) # Apply cartoon effect

    blobs = img.find_blobs([(0, 100, -128, 127, -128, 127)]) # Find red blobs
    for b in blobs:
        img.draw_rectangle(b[0:4], color=(0, 255, 0)) # Draw green rectangle around each blob

    img.draw_string(0, 0, "Red Blobs: " + str(len(blobs)), color=(0, 255, 0), scale=2) # Print number of blobs

    print(clock.fps()) # Print current FPS (frames per second) to console

This script first initializes the camera sensor and sets the pixel format and frame size. It then enters an infinite loop that takes a picture, applies a cartoon effect, finds red blobs in the image, and draws rectangles around them. Finally, it prints the number of blobs found and the current frames per second to the console.

Machine Learning with OpenMV

In addition to traditional computer vision techniques, the OpenMV Cam also supports machine learning inference using pre-trained TensorFlow models. This allows you to classify images or detect objects based on a pre-trained neural network.

The OpenMV IDE comes with several pre-trained models for common tasks, such as face detection, digit recognition, and color tracking. You can also train your own models using TensorFlow and convert them to a format compatible with the OpenMV Cam.

Here’s an example script that uses a pre-trained model to recognize handwritten digits:

import sensor, image, time, tf

sensor.reset() # Initialize the camera sensor
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to grayscale
sensor.set_framesize(sensor.QQVGA) # Set frame size to QQVGA (160x120)
sensor.skip_frames(time = 2000) # Wait for settings to take effect

net = tf.load('mnist_cnn.tflite') # Load pre-trained CNN model

while(True):
    img = sensor.snapshot() # Take a picture and store it in img

    img.gaussian(3) # Apply gaussian filtering
    img.median(3) # Apply median filtering

    for obj in tf.classify(net, img, min_scale=1.0, scale_mul=0.8, x_overlap=0.5, y_overlap=0.5):
        print("Digit: %d, Confidence: %f" % (obj.output(), obj.output()))
        img.draw_rectangle(obj.rect(), color=(255, 255, 255)) # Draw white rectangle around each digit
        img.draw_string(obj.x()+3, obj.y()-1, str(obj.output()), color=(255, 255, 255), scale=2) # Print digit

    print(clock.fps()) # Print current FPS (frames per second) to console

This script loads a pre-trained convolutional neural network (CNN) model for recognizing handwritten digits. It then enters an infinite loop that takes a picture, applies some image filtering, and runs the CNN model on the image. For each recognized digit, it prints the classification and confidence score to the console and draws a rectangle and label around the digit in the image.

Advanced OpenMV Techniques

Using Multiple Cameras

The OpenMV Cam M7 supports connecting up to two camera modules simultaneously. This allows you to perform stereo vision and depth estimation by comparing the images from two different viewpoints.

To use multiple cameras, simply connect an additional camera module to the secondary camera port on the OpenMV Cam M7. You can then access the secondary camera using the sensor2 module in your MicroPython scripts.

Here’s an example script that demonstrates capturing images from two cameras simultaneously:

import sensor, image, time, pyb

sensor.reset() # Initialize primary camera sensor
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000) # Wait for settings to take effect

sensor2 = pyb.I2C(2).scan()[0] # Initialize secondary camera sensor
sensor2.reset() # Reset secondary camera sensor
sensor2.set_pixformat(sensor2.RGB565) # Set pixel format to RGB565
sensor2.set_framesize(sensor2.QVGA) # Set frame size to QVGA (320x240)
sensor2.skip_frames(time = 2000) # Wait for settings to take effect

while(True):
    img = sensor.snapshot() # Take a picture with primary camera and store it in img
    img2 = sensor2.snapshot() # Take a picture with secondary camera and store it in img2

    # Perform image processing on img and img2

    img.draw_string(0, 0, "Primary Camera", color=(255, 255, 255), scale=2) # Label primary camera image
    img2.draw_string(0, 0, "Secondary Camera", color=(255, 255, 255), scale=2) # Label secondary camera image

    pyb.delay(500) # Pause for 500 milliseconds

This script initializes both the primary and secondary camera sensors and sets their pixel formats and frame sizes. It then enters an infinite loop that captures images from both cameras, performs any desired image processing, and labels the images before pausing for 500 milliseconds.

Interfacing with Other Hardware

The OpenMV Cam includes several I/O pins that allow you to connect external sensors and actuators. This enables you to create more advanced computer vision projects that interact with the physical world.

Some examples of external hardware you can connect to the OpenMV Cam include:

  • Servos and motors for robotic applications
  • Ultrasonic distance sensors for object avoidance
  • Infrared sensors for motion detection
  • LED strips for visual output

To control external hardware from your OpenMV scripts, you can use the pyb module to access the board’s I/O pins. Here’s an example script that demonstrates controlling a servo motor based on the position of a colored object:

import sensor, image, time, pyb

sensor.reset() # Initialize the camera sensor
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000) # Wait for settings to take effect

servo_pin = pyb.Pin("P0", pyb.Pin.OUT_PP) # Configure P0 as servo control pin
servo = pyb.Servo(servo_pin) # Create servo object

while(True):
    img = sensor.snapshot() # Take a picture and store it in img

    blobs = img.find_blobs([(255, 0, 0), (255, 50, 50)], area_threshold=200) # Find red blobs

    if len(blobs) > 0:
        biggest_blob = max(blobs, key=lambda b: b.area()) # Find the largest red blob
        img.draw_rectangle(biggest_blob[0:4], color=(0, 255, 0)) # Draw green rectangle around blob

        servo_angle = (biggest_blob.cx() - 160) * 180 // 320 # Calculate servo angle based on blob position
        servo.angle(servo_angle) # Set servo angle

        img.draw_string(0, 0, "Servo Angle: %d" % servo_angle, color=(0, 255, 0), scale=2) # Print servo angle

    print(clock.fps()) # Print current FPS (frames per second) to console

This script configures pin P0 as a servo control pin and creates a Servo object. It then enters an infinite loop that takes a picture, finds the largest red blob in the image, and calculates a servo angle based on the blob’s horizontal position. Finally, it sets the servo to the calculated angle and prints the angle to the console.

FAQ

What is the maximum resolution of the OpenMV Cam?

The standard OpenMV Cam M7 comes with an OV7725 camera module that has a maximum resolution of 640×480 pixels (VGA). However, the OpenMV Cam H7 supports higher resolution camera modules, such as the OV5640 (5 MP) and OV7670 (0.3 MP).

Can I use the OpenMV Cam for video streaming?

Yes, the OpenMV Cam supports video streaming over USB or WiFi. You can use the mjpeg module to compress camera frames into an MJPEG video stream and send it to a computer or web browser.

How long does the battery last on the OpenMV Cam?

The battery life of the OpenMV Cam depends on several factors, such as the camera resolution, frame rate, and image processing operations being performed. With a standard 500 mAh LiPo battery, you can expect the OpenMV Cam to last for approximately 2-3 hours of continuous operation.

Can I use the OpenMV Cam with Arduino or Raspberry Pi?

Yes, you can connect the OpenMV Cam to an Arduino or Raspberry Pi using the serial or I2C interface. This allows you to use the OpenMV Cam as a computer vision sensor in your Arduino or Raspberry Pi Projects.

What is the difference between the OpenMV Cam M7 and H7?

The OpenMV Cam M7 and H7 are two different models of the OpenMV Cam with different specifications. The M7 is the standard model with an STM32F7 processor, 512 KB RAM, and 2 MB flash storage. The H7 is a higher-end model with an STM32H7 processor, 1024 KB RAM, and 2 MB flash storage. The H7 also supports higher resolution camera modules and faster frame rates.

Conclusion

The OpenMV Cam is a powerful and versatile computer vision platform that makes it easy to develop real-time image processing and machine learning applications. With its intuitive MicroPython programming interface, extensive library of computer vision functions,