Conversions Between GPU Memory Formats

When working with images, there are many ways to represent them as arrays of pixels. Working with different models you may encounter representation of an image using OpenCV GpuMat class, PyTorch tensor or CuPy array.

The Savant framework aims to use GPU efficiently without excessive data transfers. To achieve this, Savant provides functions for converting between different image representations. Data exchange is performed with zero-copying between different views, except for some cases of conversion to GpuMat OpenCV.

Conversion to OpenCV

From PyTorch tensor

pytorch_tensor_as_opencv_gpu_mat allows you to convert a PyTorch tensor into an OpenCV GpuMat. The input tensor must be on GPU, must have shape in HWC format and be in C-contiguous layout.

import torch
from savant.utils.memory_repr_pytorch import pytorch_tensor_as_opencv_gpu_mat

# original in HWC
pytorch_tensor = torch.randint(0, 255, size=(10, 20, 3), device='cuda').to(torch.uint8)
# map to opencv gpu mat (zero-copy)
opencv_gpu_mat = pytorch_tensor_as_opencv_gpu_mat(torch_tensor)

If the shape format of the tensor is different, you can transform it into the required format using e.g. Tensor.permute(). You should keep in mind that such transformations usually lead to data copying and additionally require the tensor to be converted to contiguous in memory layout. You can do this with Tensor.contiguous().

import torch
from savant.utils.memory_repr_pytorch import pytorch_tensor_as_opencv_gpu_mat

# original in CHW
tensor0 = torch.randint(0, 255, size=(3, 10, 20), device='cuda').to(torch.uint8)
# transform to HWC
tensor1 = tensor0.permute(1, 2, 0)
# to contiguous (copy)
tensor2 = tensor1.contiguous()
# map to opencv gpu mat (zero-copy)
gpu_mat = pytorch_tensor_as_opencv_gpu_mat(tensor2)

From CuPy array

cupy_array_as_opencv_gpu_mat allows you to convert a CuPy array into an OpenCV GpuMat. The input array must have shape in HWC format, 2 or 3 dimensions and be in C-contiguous layout.

import cupy as cp
from savant.utils.memory_repr import cupy_array_as_opencv_gpu_mat

# original in HWC
cupy_array = cp.random.randint(0, 255, (10, 20, 3)).astype(cp.uint8)
# map to opencv gpu mat (zero-copy)
opencv_gpu_mat = cupy_array_as_opencv_gpu_mat(cupy_array)

If the shape format of the array is different, you can transform it into the required format using e.g. cupy.transpose(). You should keep in mind that such transformations usually lead to data copying and additionally require the array to be converted to contiguous in memory layout. You can do this with cupy.ascontiguousarray().

import cupy as cp
from savant.utils.memory_repr import cupy_array_as_opencv_gpu_mat

# original in CHW
arr0 = cp.random.randint(0, 255, (3, 10, 20)).astype(cp.uint8)
# transform to HWC
arr1 = arr0.transpose((1, 2, 0))
# to contiguous (copy)
arr2 = cp.ascontiguousarray(arr1)
# map to opencv gpu mat (zero-copy)
gpu_mat = cupy_array_as_opencv_gpu_mat(arr2)

Conversion to PyTorch Tensor

From OpenCV GpuMat

opencv_gpu_mat_as_pytorch_tensor allows you to convert an OpenCV GpuMat into a PyTorch tensor on GPU.

import cv2
from savant.utils.memory_repr_pytorch import opencv_gpu_mat_as_pytorch_tensor

opencv_gpu_mat = cv2.cuda_GpuMat()
opencv_gpu_mat.upload(np.random.randint(0, 255, (10, 20, 3)).astype(np.uint8))
# zero-copy, HWC format
torch_tensor = opencv_gpu_mat_as_pytorch_tensor(opencv_gpu_mat)

From CuPy Array

Conversion from CuPy array to PyTorch tensor is performed by using standard PyTorch function torch.as_tensor.

import cupy as cp
import torch

cupy_array = cp.random.randint(0, 255, (10, 20, 3)).astype(cp.uint8)
# zero-copy, original array format
torch_tensor = torch.as_tensor(cupy_array)

Conversion to CuPy Array

From OpenCV GpuMat

opencv_gpu_mat_as_cupy_array allows you to convert an OpenCV GpuMat into a CuPy array.

import cv2
import cupy as cp
import numpy as np
from savant.utils.memory_repr import opencv_gpu_mat_as_cupy_array

opencv_gpu_mat = cv2.cuda_GpuMat()
opencv_gpu_mat.upload(np.random.randint(0, 255, (10, 20, 3)).astype(np.uint8))
# zero-copy, HWC format
cupy_array = opencv_gpu_mat_as_cupy_array(opencv_gpu_mat)

From PyTorch tensor

Conversion from PyTorch tensor to CuPy is performed by using standard CuPy function cupy.asarray .

import torch
import cupy as cp

torch_tensor = torch.randint(0, 255, size=(3, 10, 20), device='cuda').to(torch.uint8)
# zero-copy, original tensor format
cupy_array = cp.asarray(torch_tensor)