Building Hybrid Pipelines
The section discusses how to pack multiple pipelines in a single pipeline, separating their compute spaces with custom primary ROI. In Savant, a developer can pack multiple sub-pipelines into a single pipeline, working either on all frames or conditionally based on ROI.
By default, the primary model analyzes the whole frame; however, under the hood, Savant creates the default top-level object covering the whole frame; thus, the models without specified input constraints can analyze it. It allows placing multiple primary models one after another and then their secondary models. The only requirement is non-overlapping unit names to avoid object collisions.
Normally, there is no difference how to place units if there are no cross dependencies between units; thus, the ordering is important only between elements of sub-pipelines.
consider placing sub-pipelines in element groups: it helps to develop and debug them independently.
When you need to process frames conditionally, based on per-stream information, e.g., handle cam-1 with a car processing sub-pipeline and cam-2 with a person processing sub-pipeline, a developer must place a special ROI-modifying custom pyfunc before other pipeline elements.
That pyfunc must modify ROI based on
source-id or other knowledge like per-frame attributes:
source-idis unknown it can be deleted to ensure the frame is not processed;
source-idis known and relates to the car processing sub-pipeline, set it to
source-idis known and relates to the person processing sub-pipeline, set it to
The primary models must accept corresponding ROIs rather than work on default ROI:
... element: nvinfer@detector name: CarDetector model: input: object: car.roi ... element: nvinfer@detector name: PersonDetector model: input: object: person.roi
Pros & Cons Of Hybrid Pipelines
easier to maintain deployments;
easier to route video streams (no stream duplication is needed);
more difficult to develop and troubleshoot, consider element groups;
increases end-to-end delay;
more difficult to plan compute resources when real-time processing is required.