The rpicam-apps package apt show rpicam-apps comes with the rpicam-hello program and others. It uses DRM to show the preview in case the system has no graphical desktop.

rpicam-hello -t0 shows a preview, without -t0 is times out after 5 seconds

rpicam-jpeg --output "$(date +%Y-%m-%d_%H-%M-%S).jpg" to take pictures

rpicam-hello supports post processing https://www.raspberrypi.com/documentation/computers/camera_software.html#post-processing-with-rpicam-apps. Post processing is configured in a *.json file that will be passed via command line. A stage processes camera images. Multiple stage can operate sequentially and are put into the *.json file https://github.com/raspberrypi/rpicam-apps/tree/main/assets.or /usr/share/rpi-camera-assets hold such *.json files

cd /usr/share/rpi-camera-assets and rpicam-hello --post-process-file negate.json

The sample program can be started with the verbose -v 2 option, this prints metadata as for object detection and bounding boxes into the console: rpicam-hello -t 0s --post-process-file /usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30 -v 2

This is debug information and therefore not well documented and not recommended for productive solutions.

In edge computing the camera gets CPU power. The AI camera has the imx500 chip from Sony that has Iimage Signal Processor and a AI accelerator that can load a neuronal network.

See https://www.raspberrypi.com/products/ai-camera/ and https://www.raspberrypi.com/products/ai-camera/

To get the firmware sudo apt install imx500-all to get the firmware sudo reboot to get it recognized by the system

/usr/share/imx500-models/ holds neuronal networks in *.rpk format to be loaded into the AI camera

The object detection sample program is started as rpicam-hello -t 0s --post-process-file /usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30

The pose detection is started as rpicam-hello -t 0s --post-process-file /usr/share/rpi-camera-assets/imx500_posenet.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30

For advanced things https://mlsysbook.ai/kits/contents/raspi/raspi.html

picamera2 ttps://github.com/raspberrypi/picamera2https://pip-assets.raspberrypi.com/categories/652-raspberry-pi-camera-module-2/documents/RP-008156-DS-2-picamera2-manual.pdf?disposition=inlinehttps://pip-assets.raspberrypi.com/categories/652-raspberry-pi-camera-module-2/documents/RP-008156-DS-2-picamera2-manual.pdf?disposition=inline is based on libcamera and is a python library for Raspberry Pi cameras. As sudo dpkg-query -l | grep picamera2 shows, python3-picamera2 is already installed.

However dpkg -L python3-picamera2 shows that it got installed without its valuable examples.

Those example python scripts are a good start to create applications.

Therefore git clone https://github.com/raspberrypi/picamera2.git

Using picamera2 run python imx500_object_detection_demo.py --model /usr/share/imx500-models/imx500_network_ssd_mobilenetv2_fpnlite_320x320_pp.rpk

python imx500_object_detection_demo.py -h shows the help

--iou <values between 0.0 and 1.0> Intersection-over-Union threshold handles how overlapped boxes are dealt

--max-detections <n> defines maximum of boxes to be displayed

python imx500_object_detection_demo.py --print-intrinsics shows information and exits.

/usr/share/imx500-models/ or https://github.com/raspberrypi/imx500-models/tree/main contain the models and lists what example applications can be used

git clone https://github.com/raspberrypi/imx500-models.git to get the newest models as the yolo models that require the command line option to find the label file --labels picamera2/examples/imx500/assets/coco_labels.txt

https://github.com/raspberrypi/picamera2/tree/main/examples/imx500 contains the example programs for the AI camera.

For the object detection there are different versions

imx500_object_detection_demo_mp.py that uses multiprocess and gives smoother video. This demo comes with a menu that allows to take screen shots

imx500_object_detection_injection_demo.py allows customization by injecting the results into the camera pipeline

RTSP is a standard for streams it is robust but creates delays. WebRTC is commonly used for browsers and has little delay.

paths:
  stream1:
    source: rpiCamera

[Unit]
Description=MediaMTX RTSP server
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/opt/mediamtx/mediamtx /opt/mediamtx/mediamtx.yml
Restart=always
RestartSec=2

# Security / cleanliness
User=<user name>
WorkingDirectory=/opt/mediamtx
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target

paths:
  cam:
    source: udp://127.0.0.1:1234

[Unit]
Description=Raspberry Pi Camera Audio-Video Stream
After=network.target mediamtx.service
Requires=mediamtx.service

[Service]
User=<username>
Group=video
ExecStart=/bin/bash -c "rpicam-vid -t 0 -n --codec libav --low-latency --libav-format mpegts -o - | \
ffmpeg -y -thread_queue_size 32 -i pipe:0 \
-f alsa -thread_queue_size 32 -ac 1 -i plughw:2,0 \
-c:v copy -c:a aac -b:a 128k \
-fflags nobuffer+flush_packets -flags low_delay \
-f mpegts udp://127.0.0.1:1234?pkt_size=1316"

Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

rpicam-vid -t 0 -n --codec libav --low-latency --libav-format mpegts -o - | \
ffmpeg -y \
  -thread_queue_size 1024 -i pipe:0 \
  -f alsa -thread_queue_size 1024 -ac 1 -i plughw:2,0 \
  -c:v copy \
  -c:a libopus -ar 48000 -ac 2 \
  -f mpegts "udp://127.0.0.1:1234?pkt_size=1316"

#!/bin/bash
# RTSP stream using command from Raspberry Pi documentation

exec rpicam-vid -t 0 -n --codec libav --libav-format mpegts -o - | \
     cvlc stream:///dev/stdin --sout "#rtp{sdp=rtsp://:8554/stream1}"

[Unit]
Description=Raspberry Pi Camera RTSP Stream
After=network.target
Wants=network.target

[Service]
Type=simple
User=<username>
ExecStart=/opt/rtsp/rpi-rtsp-stream.sh
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target