Savant Samples Catalog ====================== This document provides a comprehensive overview of all available Savant samples. Each sample demonstrates specific computer vision and video processing capabilities using different models, adapters, and platform configurations. .. note:: Platform support is indicated as follows: - **X86 + L4T**: Both x86 and Jetson (L4T) platforms supported - **X86 only**: Only x86 platform supported - **L4T only**: Only Jetson (L4T) platform supported When not specified, the sample is implemented only for X86 platform by some reason. Computer Vision and AI Samples ------------------------------- Face Detection and Analysis ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Age Gender Recognition ^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Face detection using YOLOv8-Face model with 5 face landmarks (eyes, nose, mouth) - Age and gender estimation for detected faces - Face orientation calculation using landmarks - Face tracking with Nvidia Tracker **Auxiliary Features**: - Image preprocessing for model input - Real-time and capacity processing modes - Performance benchmarking support **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Age Gender Recognition `_ Face ReID ^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Facial recognition and re-identification using YOLOv8-Face and AdaFace models - Face gallery indexing and matching - Doorbell security system demonstration **Auxiliary Features**: - Index builder for face gallery management - Face preprocessing and feature vector extraction - HNSWLIB-based face matching **Adapters Used**: - :ref:`Video files source adapter ` (for index building) - :ref:`Always On RTSP sink adapter ` (for demo) - Client SDK integration **Link**: `Face ReID `_ People Detection and Tracking ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PeopleNet Detector ^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Person and face detection using Nvidia PeopleNet model - GPU-accelerated face blurring with OpenCV CUDA - Body-face matching and tracking **Auxiliary Features**: - Real-time and capacity processing modes - Flickering reduction with simple tracker - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `PeopleNet Detector `_ Traffic and Line Crossing Analysis ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Traffic Meter ^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Pedestrian line crossing detection and counting - Multiple detector model support (PeopleNet, YOLOv8m, YOLOv8s) - Direction-aware crossing detection - Grafana dashboard integration **Auxiliary Features**: - DeepStream-Yolo integration - Graphite metrics storage - Real-time dashboard visualization **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Traffic Meter `_ Intersection Traffic Meter ^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Vehicle crossing detection at city intersections (cars, trucks, buses) - Polygon-based intersection area definition - YOLOv8 model for vehicle detection - Multi-source and multi-polygon counting **Auxiliary Features**: - DeepStream-Yolo integration - Grafana dashboard with metrics visualization - Graphite storage backend **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Intersection Traffic Meter `_ Fisheye Line Crossing ^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Line crossing detection for fisheye camera footage - YOLOv7 OBB (Oriented Bounding Box) detector - Similari tracking library integration - Rotated bounding box detection **Auxiliary Features**: - Grafana dashboard integration - Multi-source line crossing analytics - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Fisheye Line Crossing `_ Area Object Counting ^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - People counting within user-defined areas - Multi-area simultaneous monitoring - Real-time area occupancy display **Auxiliary Features**: - Configurable area definitions - Real-time visualization - Multi-source processing **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Area Object Counting `_ Object Detection and Classification ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ License Plate Recognition ^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Car detection using YOLOv8 models - License plate detection using Nvidia LPD model - License plate text recognition using Nvidia LPR model - Vehicle and plate tracking **Auxiliary Features**: - Multi-stage detection pipeline - US license plate dictionary support - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `License Plate Recognition `_ Nvidia Car Classification ^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Car detection and tracking - Multi-attribute classification (type, color, make) - Reproduces deepstream-test2 functionality **Auxiliary Features**: - Multiple classification models - Track ID visualization - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Nvidia Car Classification `_ RT-DETR R50 Demo ^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Object detection using RT-DETR model - Real-time detection transformer architecture - DeepStream-Yolo integration **Auxiliary Features**: - ONNX model format support - Performance optimization - Real-time processing capability **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `RT-DETR `_ Advanced Computer Vision ~~~~~~~~~~~~~~~~~~~~~~~~ YOLOv8 Instance Segmentation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Person instance segmentation using YOLOv8-seg model - GPU and CPU converter options - Complex model output processing **Auxiliary Features**: - CuPy GPU acceleration support - NumPy/Numba CPU processing - cv2.cuda.GpuMat rendering - Performance optimization options **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `YOLOv8 Segmentation `_ Keypoint Detection ^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Human body keypoint detection using YOLOv8n-pose model - 17-point body pose estimation - Real-time pose visualization **Auxiliary Features**: - Ultralytics model integration - ONNX export pipeline - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Keypoint Detection `_ NanoSAM ^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Object segmentation using NanoSAM model - Point-based object identification - Custom model input handling - Multi-object color-coded segmentation **Auxiliary Features**: - Four-point interactive segmentation - Gradient mask visualization - Custom pyfunc integration - TensorRT engine customization **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `NanoSAM `_ Image Processing and Enhancement ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ AnimeGAN ^^^^^^^^ **Platform Support**: X86 only **Main Features**: - Anime-style image transformation using AnimeGANv2 - Hayao Miyazaki anime style application - Video style transfer **Auxiliary Features**: - PyTorch to ONNX conversion pipeline - ONNX model simplification - Frame replacement demonstration **Adapters Used**: - :ref:`Multi-stream source adapter ` - :ref:`Video files sink adapter ` **Link**: `AnimeGAN `_ Super Resolution ^^^^^^^^^^^^^^^^ **Platform Support**: X86 only **Main Features**: - Video super-resolution using NinaSR models - 360p to 1080p upscaling demonstration - Multiple scale factor support (2x, 3x, 4x) **Auxiliary Features**: - TorchSR model integration - Lightweight neural network approach - Quality enhancement visualization **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Super Resolution `_ OpenCV CUDA Background Removal ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Real-time background removal using OpenCV CUDA MOG2 - High-performance processing (500+ FPS capability) - GPU-accelerated background segmentation **Auxiliary Features**: - Hardware acceleration optimization - Real-time and capacity processing modes - Performance benchmarking **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `OpenCV CUDA Background Removal `_ Panoptic Driving Perception ^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Driving scene understanding using YOLOP model - Object detection and semantic segmentation - PyTorch inference in Savant - GPU memory interaction demonstration **Auxiliary Features**: - Torch hub integration - Multi-task learning approach - Real-time driving analysis **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Panoptic Driving Perception `_ Streaming and Adapter Samples ------------------------------ Source Adapters ~~~~~~~~~~~~~~~ MJPEG USB Camera ^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - USB camera MJPEG stream capture - Low-latency compressed video streaming - Hardware acceleration support **Auxiliary Features**: - NVJPEG hardware acceleration on Jetson - Software/hardware-assisted decoding on X86 - Configurable camera parameters **Adapters Used**: - :ref:`FFmpeg source adapter ` (USB camera) - :ref:`Always On RTSP sink adapter ` **Link**: `MJPEG USB Camera `_ Multiple RTSP Streams ^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Multiple RTSP stream ingestion - Multiplexed stream processing - PeopleNet model processing on multiple streams **Auxiliary Features**: - Stream multiplexing demonstration - Multi-source processing pipeline - LL-HLS output support **Adapters Used**: - :ref:`RTSP source adapter ` (multiple instances) - :ref:`Always On RTSP sink adapter ` **Link**: `Multiple RTSP `_ Multiple GigE Vision Cameras ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - GigE Vision camera integration - Multiple camera stream processing - Raw RGBA and HEVC-encoded frame support - GigE Vision Source Adapter demonstration **Auxiliary Features**: - Stream control API - Multi-format camera support - Real-time multi-camera processing **Adapters Used**: - :ref:`GigE camera source adapter ` (multiple instances) - :ref:`Always On RTSP sink adapter ` **Link**: `Multiple GigE `_ RTSP Camera Compatibility Test ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - RTSP camera compatibility testing - FFmpeg and Retina RTSP adapter variants - NVDEC and NVENC integration testing - Camera compatibility validation **Auxiliary Features**: - RTCP Sender Reports support (Retina adapter) - Cross-stream synchronization testing - Multiple adapter variant testing **Adapters Used**: - :ref:`FFmpeg source adapter ` OR Retina RTSP source adapter - :ref:`Always On RTSP sink adapter ` **Link**: `RTSP Camera Compatibility Test `_ AWS Kinesis Integration ^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Kinesis Video Stream integration - Frame export/import pipeline - MongoDB metadata storage **Auxiliary Features**: - Cloud streaming capabilities - Metadata synchronization - AWS service integration **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Kinesis Video Stream sink adapter ` - :ref:`Kinesis Video Stream source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `AWS Kinesis `_ Processing Control and Flow ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Conditional Video Processing ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Conditional processing based on Etcd parameters - Dynamic processing enable/disable - PeopleNet-based conditional encoding **Auxiliary Features**: - Etcd-based source control - Tag-based processing pipeline - DrawFunc and encoder control **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Conditional Video Processing `_ Buffer Adapter Demo ^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Buffer adapter functionality demonstration - Load spike simulation and handling - Frame buffering and dropping strategies **Auxiliary Features**: - Prometheus metrics integration - Grafana dashboard visualization - Performance monitoring **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Buffer adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Buffer Adapter `_ Auxiliary Streams ^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Multiple resolution stream generation - Auxiliary stream demonstration - Frame scaling to different resolutions **Auxiliary Features**: - Multi-resolution output (360p, 480p, 720p) - Encoder optimization - Upscaling demonstration **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` (multi-stream) **Link**: `Auxiliary Streams `_ Data Integration and APIs ------------------------- Kafka-Redis Adapter ~~~~~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Kafka-Redis adapter demonstration - Frame content storage in KeyDB/Redis - Metadata storage in Kafka - Pub-sub architecture implementation **Auxiliary Features**: - KeyDB alternative to Redis - Message broker integration - Distributed processing support **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Kafka-Redis sink adapter ` - :ref:`Kafka-Redis source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Kafka-Redis Adapter `_ Key-Value API ~~~~~~~~~~~~~ **Platform Support**: X86 only **Main Features**: - Embedded Key-Value store demonstration - REST API access to KV store - WebSocket subscription support **Auxiliary Features**: - HTTP API integration - Real-time data subscription - Protobuf serialization support **Adapters Used**: - :ref:`Video loop source adapter ` **Link**: `Key-Value API `_ Source Adapter with JSON Metadata ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - JSON metadata integration with video sources - External metadata injection - Metadata-video synchronization **Auxiliary Features**: - Custom metadata handling - JSON format support - Synchronized processing **Adapters Used**: - Media files source adapter (with JSON metadata) - see :ref:`Image File Source Adapter ` or :ref:`Video File Source Adapter ` - :ref:`Image files sink adapter ` **Link**: `Source Adapter with JSON Metadata `_ Development and Testing Tools ----------------------------- Template Sample ~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Complete development template for custom Savant modules - Basic pipeline with dev features - Client SDK demonstration **Auxiliary Features**: - Docker Compose and devcontainer setup - Jaeger tracing integration - PyFunc and DrawFunc templates - Hot-reload development workflow **Adapters Used**: - :ref:`Always On RTSP sink adapter ` - URI Input script integration **Link**: `Template `_ Bypass Model ~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Identity model for preprocessing troubleshooting - Data preprocessing demonstration - Model input/output comparison **Auxiliary Features**: - PyTorch to ONNX conversion - Aspect ratio maintenance - Preprocessing visualization **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Bypass Model `_ Pipeline Watchdog ~~~~~~~~~~~~~~~~~ **Platform Support**: X86 only **Main Features**: - Pipeline monitoring and watchdog functionality - Random processing delays simulation - Pipeline health monitoring **Auxiliary Features**: - Configurable delay parameters - Sink monitoring capabilities - Reliability testing **Adapters Used**: - Custom sink monitoring via Client SDK **Link**: `Pipeline Watchdog `_ Telemetry and Monitoring ~~~~~~~~~~~~~~~~~~~~~~~~ OpenTelemetry Example ^^^^^^^^^^^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - OpenTelemetry integration demonstration - Distributed tracing with Jaeger - Performance monitoring and debugging **Auxiliary Features**: - TLS support for telemetry collection - Span instrumentation examples - Error tracking and visualization - Custom telemetry collection **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Telemetry `_ Router Demo ^^^^^^^^^^^ **Platform Support**: X86 + L4T **Main Features**: - Stream routing demonstration - Multi-sink pipeline routing - Keyframe-based routing logic **Auxiliary Features**: - Screenshot generation - Video archiving - Frame filtering **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Video files sink adapter ` **Link**: `Router `_ Specialized Processing ---------------------- Pass-Through Processing ~~~~~~~~~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Minimal processing pipeline demonstration - Frame pass-through without modification - Pipeline overhead measurement **Auxiliary Features**: - Performance baseline establishment - Minimal latency processing - Throughput optimization **Adapters Used**: - :ref:`Video loop source adapter ` - :ref:`Always On RTSP sink adapter ` **Link**: `Pass-Through Processing `_ Original Resolution Processing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Processing at original video resolution - No scaling or resolution changes - Native resolution handling **Auxiliary Features**: - Resolution preservation - Quality maintenance - Performance optimization **Adapters Used**: - :ref:`Video loop source adapter ` (multiple instances for different resolutions) - :ref:`Always On RTSP sink adapter ` **Link**: `Original Resolution Processing `_ Source Shaper Sample ~~~~~~~~~~~~~~~~~~~~ **Platform Support**: X86 + L4T **Main Features**: - Video source shaping and preprocessing - Frame rate and resolution control - Source adaptation demonstration **Auxiliary Features**: - Dynamic source modification - Frame rate adjustment - Resolution scaling **Adapters Used**: - :ref:`Video loop source adapter ` (multiple instances for different sources) - :ref:`Always On RTSP sink adapter ` **Link**: `Source Shaper Sample `_ Getting Started --------------- To explore any of these samples: 1. **Prerequisites**: Ensure your environment is properly configured by running:: git clone https://github.com/insight-platform/Savant.git cd Savant git lfs pull ./utils/check-environment-compatible 2. Visit the sample page for detailed instructions and configuration options. Performance Notes ----------------- - **First-time execution**: Many samples require model compilation to TensorRT engines, which can take 10-40 minutes depending on the model complexity - **Platform optimization**: Samples are tested for their supported platforms with appropriate hardware acceleration For detailed information about each sample, including specific setup instructions and configuration options, please visit the individual sample links provided above.