Element Groups

A single module config can contain multiple variations of the pipeline through the use of conditionally initialized ElementGroup.

For example, the user can introduce alternative detector units without the need to duplicate all the other module parameters in a separate config file, as is done in the traffic meter sample.

Tip

Extra information on element groups in our blog on Medium.

Here’s a section of the module config from the beginning of the pipeline definition that adds a pyfunc unit as usual, and then defines a conditionally enabled group for the PeopleNet detector:

  batch_size: 1
  detector: ${oc.env:DETECTOR, 'yolov8m'}
  send_stats: True

pipeline:

  elements:
    # regular pyfunc unit definition
    - element: pyfunc
      module: samples.traffic_meter.line_crossing
      class_name: ConditionalDetectorSkip
      kwargs:
        config_path: ${oc.env:PROJECT_PATH}/samples/traffic_meter/line_config.yml

    # variant group: peoplenet detector
    # groups are included in the pipeline.elements alongside regular units or other groups
    - group:
        # enabled if env var DETECTOR==peoplenet

Later in the module config another group is defined for the YOLOV8m detector:

                layer_names: [output_bbox/BiasAdd, output_cov/Sigmoid]
                num_detected_classes: 3
                objects:
                  - class_id: ${parameters.detected_object.id}
                    label: ${parameters.detected_object.label}

    # variant group: yolov8m detector
    # groups are included in the pipeline.elements alongside regular units or other groups
    - group:

The groups are initialized based on whether their init_condition evaluates to True at the time of module initialization. In the example above which detector unit is initialized depends on the value of the DETECTOR environment variable: if it is set to peoplenet then the peoplenet detector is initialized, and similarly for the yolov8m and yolov8s detectors.

Note

The behaviour is static, not dynamic. Once the pipeline is initialized its contents do not change, so if the value of DETECTOR is changed after the pipeline is initialized, the detectors will not be swapped out.

In this example the detector units are conditionally initialized based on the value of an environment variable, but any expression on the config variables can be used as the condition. Expression evaluation is done in the same way as for any other config variable, standard OmegaConf interpolation and resolvers can be used, along with custom resolvers from Savant.

In this manner any of the supported Pipeline units can be conditionally initialized. In addition, the conditions can be applied to multiple units at the same time, by specifying them in the elements list of a single group.