Abstract: A processing device encodes a frame of a video. The processing device determines a reference checksum of the frame. The processing device adds the reference checksum to supplemental metadata associated with the encoded frame of the video. The processing device transmits the encoded frame and the supplemental metadata including the reference checksum to a recipient. The recipient is to use the reference checksum to verify an integrity of the frame.

Abstract: Approaches presented herein provide systems and methods for generating a standardized specification and associated interface for application development. One or more microservices may be selected and graphically represented within an interface that receives connection information from one or more users. Connected microservices may have one or more configuration specifications that are auto-populated based, at least, on operation parameters for an associated application and/or related microservices. A development environment may provide for visual representations of connections between microservices along with configuration parameters and validation services. Deployment information may then be generated based on the configuration in the representations.

Abstract: Apparatuses, systems, and techniques for real-time persistent object tracking for intelligent video analytics systems are provided. A first object is tracked in an environment depicted by a first set of images. One or more predicted future states of the first object in the environment are obtained. A second object is detected in the environment depicted by a second set of images. A number of images of the second set of images exceeds a threshold number of images. A determination is made of whether a current state of the second object corresponds to at least one of the predicted future states of the first object. Responsive to a determination that a current state of the second object corresponds to at least one of the predicted future states of the first object, state data for the first object is updated based on the determined current state of the second object.

Abstract: Disclosed are apparatuses, systems, and techniques for implementing automatic runtime selection and tuning of MLM processing pipelines using stream augmentation. In one embodiment, the techniques include augmenting data stream(s) with auxiliary data to obtain an augmented data stream. The techniques further include performing an inference processing of the augmented data stream using a machine learning model (MLM) to obtain a characterization of a presence of the auxiliary data in the augmented data stream and adjusting one or more runtime settings of the MLM using the obtained characterization.

Abstract: Apparatuses, systems, and techniques for real-time persistent object tracking for intelligent video analytics systems. A state of a first object included in an environment may be tracked based on a first set of images depicting the environment. The first set of images may be generated during a first time period. It may be determined that the first object is not detected in the environment depicted in a second set of images. The second set of images may be generated during a second time period that is subsequent to the first time period. One or more predicted future states of the first object may be obtained in view of the state of the first object in the environment depicted in the first set of images. A second object may be detected in the environment depicted in a third set of images generated during a third time period that is subsequent to the second time period.

Abstract: In various examples, image areas may be extracted from a batch of one or more images and may be scaled, in batch, to one or more template sizes. Where the image areas include search regions used for localization of objects, the scaled search regions may be loaded into Graphics Processing Unit (GPU) memory and processed in parallel for localization. Similarly, where image areas are used for filter updates, the scaled image areas may be loaded into GPU memory and processed in parallel for filter updates. The image areas may be batched from any number of images and/or from any number of single- and/or multi-object trackers. Further aspects of the disclosure provide approaches for associating locations using correlation response values, for learning correlation filters in object tracking based at least on focused windowing, and for learning correlation filters in object tracking based at least on occlusion maps.

Abstract: Apparatuses, systems, and techniques for dynamically composable object tracker configuration for intelligent video analytics systems. A state of one or more objects included in an object is tracked using an object tracking application that implements an object tracker of a first type based on images depicting the environment. A request is received to perform tracking using a second object tracker type that is different from the first object tracker type. The object tracking application is configured to implement an object tracker of the second object tracker type in accordance with the request. The state of the objects in the environment is tracked using the object tracking application that implements the object tracker of the second object tracker type based on the images depicting the environment.

Abstract: Systems and methods for improved media stream processing. In at least one embodiment, a media stream is assigned to either a hardware processing engine or software processing engine based on a performance state of an application server and one or more parameters of the media stream.

Abstract: A first visual appearance descriptor associated with a first object in an environment is obtained based on a first set of images of a first time period. The first object is subsequently absent from the environment in a second set of images of a second time period. A second visual appearance descriptor associated with a second object is obtained based on a third set of images, of a third time period subsequent to the second time period. A compound similarity metric between the first and second objects is obtained in view of visual appearance similarity and motion similarity metrics. The visual appearance similarity metric corresponds to a degree of similarity between the first and second visual appearance descriptors. An identifier associated with the second object is updated to correspond to an identifier associated with the first object in response to determining that the compound similarity metric meets a threshold value.

Abstract: In various examples, image areas may be extracted from a batch of one or more images and may be scaled, in batch, to one or more template sizes. Where the image areas include search regions used for localization of objects, the scaled search regions may be loaded into Graphics Processing Unit (GPU) memory and processed in parallel for localization. Similarly, where image areas are used for filter updates, the scaled image areas may be loaded into GPU memory and processed in parallel for filter updates. The image areas may be batched from any number of images and/or from any number of single- and/or multi-object trackers. Further aspects of the disclosure provide approaches for associating locations using correlation response values, for learning correlation filters in object tracking based at least on focused windowing, and for learning correlation filters in object tracking based at least on occlusion maps.

Abstract: Apparatuses, systems, and techniques for managing lost objects in an intelligent video analytics system. A first set of application modules is executed for an object tracking application configured to track, based on images depicting an environment, a state of objects included in the environment. The first set of application modules is associated with a first object tracker type. A request is received to configure the object tracking application to execute a second set of application modules associated with a second object tracker type. The second set of application modules includes one or more application modules that are different from application modules of the first set of application modules. The object tracking application is configured to execute the second set of application modules in accordance with the request.

Abstract: Disclosed are apparatuses, systems, and techniques that improve memory and computational efficiency of remote direct memory accesses into a memory of a graphics processing unit. The techniques include but are not limited to receiving packets with video frame data, storing the plurality of packets in a memory of a network controller, processing the packets to obtain unit(s) of the video frame, storing the unit(s) representative of the video frame in a memory of a graphics processing unit (GPU), and extracting the data of the video frame from the units representative of the video frame, stored in the memory of the GPU, to render the video frame.

Abstract: In various examples, source archives may be classified, sorted, downloaded, and installed using command files for container image generation. For example, a user may select source archives for inclusion in a container build operation, and the system may classify the source archives as belonging to one of any number of classes and/or sub-classes. Where a size of the source archive is below an associated threshold, the source archive may be included in the container image using native context copy operations. In contrast, where the size of the source archive is above the associated threshold, the source archive may be included in the container image using a local HTTP auto copy operation—e.g., by creating a local HTTP server, and generating a command line such that a container builder retrieves the archive information via the local HTTP server.

Abstract: In various examples, a three-dimensional (3D) data processing pipeline for autonomous systems and applications is presented. Systems and methods are disclosed for 3D point cloud data processing fused with video analysis applications. Using the systems and methods described herein, processing of 3D data may be performed in different multimedia frameworks, allowing a user to use common libraries and/or to implement custom libraries on top of the existing system design. As a result, conventional 2D video processing may be combined with 3D data processing, to allow for data representing a flat 2D world to represent a rich 3D world. In this way, the fused 3D depth and/or range data with 2D camera image data allows for perception and/or vision that is more powerful, accurate, and precise.

Abstract: Apparatuses, systems, and techniques for continuous training of an object detection and/or classification model. A first image including a depiction of an environment based on a first set of conditions is identified. Object data associated with an object detected in the first image is obtained based on one or more outputs of a machine learning model. A determination is made of whether a level of confidence that an object corresponds to an object class satisfies a level of confidence criterion. If so, a first set of conditions corresponding to the environment depicted in the first image. One or more noise characteristics associated with a second image including a depiction of the environment is determined based on a difference between the first set of conditions and a second set of conditions of the second image. The first image is augmented based on the one or more determined noise characteristics to generate a third image.

Abstract: Apparatuses, systems, and techniques to balance processing load between a plurality of hardware accelerators. In at least one embodiment, operations performed on batches of frames of a video (e.g., as part of a video analytics pipeline) are distributed by a load balancer between a first hardware accelerator and a second hardware accelerator.

Abstract: A multi-stage multimedia inferencing pipeline may be set up and executed using configuration data including information used to set up each stage by deploying the specified or desired models and/or other pipeline components into a repository (e.g., a shared folder in a repository). The configuration data may also include information a central inference server library uses to manage and set parameters for these components with respect to a variety of inference frameworks that may be incorporated into the pipeline. The configuration data can define a pipeline that encompasses stages for video decoding, video transform, cascade inferencing on different frameworks, metadata filtering and exchange between models and display. The entire pipeline can be efficiently hardware-accelerated using parallel processing circuits (e.g., one or more GPUs, CPUs, DPUs, or TPUs). Embodiments of the present disclosure can integrate an entire video/audio analytics pipeline into an embedded platform in real time.

Abstract: Resources, such as machine learning models, can be updated for an application without any significant downtime for that application. For an application hosted at a network edge, the application can be deployed in a container and one or more model versions stored in local storage at the edge, which can be mounted into the container as necessary. When a different model version is to be used, a configuration change or new context can be used to trigger the application to automatically change to the different model version. This updating can be performed seamlessly, without any loss of data.

Abstract: In various examples, image areas may be extracted from a batch of one or more images and may be scaled, in batch, to one or more template sizes. Where the image areas include search regions used for localization of objects, the scaled search regions may be loaded into Graphics Processing Unit (GPU) memory and processed in parallel for localization. Similarly, where image areas are used for filter updates, the scaled image areas may be loaded into GPU memory and processed in parallel for filter updates. The image areas may be batched from any number of images and/or from any number of single- and/or multi-object trackers. Further aspects of the disclosure provide approaches for associating locations using correlation response values, for learning correlation filters in object tracking based at least on focused windowing, and for learning correlation filters in object tracking based at least on occlusion maps.