Abstract: A display has a screen that can simultaneously display different images in a plurality of display directions. An image generation unit generates, for each of one or more user viewpoint positions within a user space, which is forward from the screen of the display and in which one or more users viewing an image displayed on the screen can be present, a display image of a three-dimensional object, in which the three-dimensional object appears in the same manner as when the three-dimensional object in a virtual space is being viewed via the screen from the user viewpoint position. An image display unit causes the screen of the display to display, for each of the user viewpoint positions, the display image generated so as to correspond to the user viewpoint position, the display images being displayed in user viewpoint position directions that correspond to the user viewpoint positions.

Abstract: A prediction unit for an encoder or decoder implements matrix based intra prediction. Input boundary samples for a current block are downsampled to obtain reduced boundary samples for matrix multiplication and/or linear interpolation, or both. In one embodiment, downsampling is performed in a manner that aligns the reduced boundary samples with an output of a matrix multiplication unit of the prediction unit. In other embodiments, downsampling is performed without averaging. The embodiments reduce the complexity of the prediction unit and the latency of the encoder or decoder.

Abstract: One or more intra mode predictors of a current block are determined, where the one or more intra mode predictors are derived based on the neighboring samples of the current block via either a decoder side intra mode derivation (DIMD) or a template-based intra mode derivation (TIMD). Whether a position dependent intra prediction combination (PDPC) is to be applied is determined based on the one or more intra mode predictors. In response to the determining that the PDPC is to be applied, a fusion process is performed on the one or more intra mode predictors to combine the one or more intra mode predictors into a single intra mode predictor. The PDPC is applied based on the single intra mode predictor to generate a modified predictor of the current block. The current block is reconstructed based on the modified predictor generated by the PDPC.

Abstract: Systems and methods are provided for depth estimation from monocular images using a depth model with sparse range sensor data and uncertainty in the range sensor as inputs thereto. According to some embodiments, the methods and systems comprise receiving an image captured by an image sensor, where the image represents a scene of an environment. The method and systems also comprise deriving a point cloud representative of the scene of the environment from range sensor data, and deriving range sensor uncertainty from the range sensor data. Then a depth map can be derived for the image based on the point cloud and the range sensor uncertainty as one or more inputs into a depth model.

Abstract: Methods and systems for improving coding decoding efficiency of video by providing a syntax modeler, a buffer, and a decoder. The syntax modeler may associate a first sequence of symbols with syntax elements. The buffer may store tables, each represented by a symbol in the first sequence, and each used to associate a respective symbol in a second sequence of symbols with encoded data. The decoder decodes the data into a bitstream using the second sequence retrieved from a table.

Abstract: A method of decoding video data performed by an electronic device is provided. The method receives the video data and determines a block unit from a current frame included in the video data. The method further determines a plurality of neighboring sub-partitions selected from a plurality of neighboring sub-blocks and determines, from the current frame, more than one previously affine-coded block reconstructed in an affine-based mode prior to reconstructing the block unit. The method receives a plurality of neighboring motion vectors of the neighboring sub-partitions and a plurality of affine motion vectors of the affine-coded sub-blocks and derives a motion linear model based on the neighboring motion vectors and the neighboring positions of the neighboring sub-partitions and the affine motion vectors and the affine positions of the affine-coded sub-blocks. Then, the method determines a plurality of derived motion vectors based on the motion linear model for reconstructing the block unit.

Abstract: A method of decoding an image can include constructing an amvp candidate list using available motion vector candidates of a left, an above and a temporal motion vector candidate, selecting a motion vector predictor among motion vector candidates of the amvp candidates list using the amvp index, and generating a motion vector using the motion vector predictor and a differential motion vector. The method further includes generating a prediction block using the motion vector and a reference picture index, inversely quantizing a quantized block using a quantization parameter to generate a transformed block and inversely transforming the transformed block to generate a residual block, and generating a reconstructed block. Also, the quantization parameter is generated per quantization unit, a minimum size of the quantization unit is adjusted per picture, and the quantization parameter is generated using a differential quantization parameter and a quantization parameter predictor.

Abstract: A method and system for employing video analytics in providing a situational awareness alert is disclosed. The method includes automatically detecting that a first person has entered a coverage area of a security camera. The method also includes automatically determining that the first person is registered in a monitoring service associated with the coverage area. The method also includes determining that a plurality of environmental and geospatial parameters associated with the coverage area satisfy a situational awareness threshold applicable to the first person. In response to the situational awareness threshold being satisfied, video analytics is employed to analyze video captured by security camera(s) to determine that there exists a second person having a respective Point Of View (POV) that permits staring observation of the first person by the second person.

Abstract: An image decoding method performed by a decoding device according to the present document comprises the steps of deriving an intra prediction type for a current block from among intra prediction types; constituting an intra prediction mode candidate list of the current block on the basis of intra prediction modes of neighboring blocks of the current block; deriving an intra prediction mode of the current block on the basis of the intra prediction mode candidate list; and generating prediction samples of the current block on the basis of the intra prediction type and the intra prediction mode.

Abstract: A method for encoding a picture of a video sequence in a bit stream that constrains slice header processing overhead is provided. The method includes computing a maximum slice rate for the video sequence, computing a maximum number of slices for the picture based on the maximum slice rate, and encoding the picture wherein a number of slices used to encode the picture is enforced to be no more than the maximum number of slices.

Abstract: Systems, methods, and computer programs for eye tracking involving a camera that is operated according to eyelid position. In one embodiment, an inward-facing head-mounted camera captures images of an eye. A head-mounted device emits electromagnetic waves towards the eye and measures reflections of these waves from the eye. The average rate of reflection measurements is at least ten times the average rate of image capture. A computer detects the position of at least one of the eyelids covering the eye based on the reflection measurements. And the camera operation is adjusted according to the eyelid position in a manner that decreases, in the captured images, areas depicting the eyelids, and optionally increases areas depicting the iris.

Abstract: There is provided an image processing apparatus and an image processing method for enabling suppression of an increase in a memory capacity required for orthogonal transform and inverse orthogonal transform. A second transformation matrix is derived using a first transformation matrix, a prediction residual of an image is orthogonally transformed using the derived second transformation matrix, and coefficient data obtained by orthogonally transforming the prediction residual is encoded to generate a bit stream. The present disclosure can be applied to, for example, an image processing apparatus, an image encoding device, an image decoding device, or the like.

Abstract: Systems, methods, and computer programs for eye tracking. In one embodiment, an eye tracking system includes a head-mounted device that takes measurements indicative of a position of the eye of a user. A head-mounted camera captures an image of the eye. A computer calculates the position of the eye based on the measurements, utilizes the position of the eye to crop the image around the pupil, and provides a cropped image to a video-based eye tracker. Optionally, the size of the cropped image is less than a third of the size of its respective uncropped image. Optionally, the head-mounted device includes one or more of the following: a photosensor-oculography device (PSOG), an electrooculography device (EOG), an electromyography device (EMG), an optical flow sensor, and a range sensor. Optionally, the image-based eye tracker calculates, based on the cropped image, at least one of pupil diameter and pupillary response.

Abstract: Systems, methods, and devices of the various embodiments enable virtual displays of an item, such as automobile, to be generated. A photo studio can be used to capture a series of images of the item for annotation. The series of images of the item can then be sent to a computer system for display.

Abstract: An electronic apparatus performs a method of decoding video data. The method comprises: receiving, from bitstream, video data corresponding to a transform-skip mode coded block; decoding, from the video data, a first codeword, a second codeword and a first group of codewords for a pixel within the transform-skip mode coded block; deriving an initial level value from the first group of codewords; converting the first codeword into a remainder of the pixel in accordance with a predefined mapping relationship that is generated using a constant Rice parameter; converting the second codeword into a sign value of the remainder; and deriving a quantified residual of the pixel from the remainder, the sign value and the initial level value.

Abstract: A virtual reality/augmented reality (VR/AR) wearable assembly is described herein. The VR/AR wearable assembly includes a wearable frame adapted to be worn over a patient's eyes, a pair of photosensor oculography (PSOG) assemblies mounted to the wearable frame with each PSOG assembly including a display and an eye tracking assembly, and a processor coupled to each PSOG assembly. The processor is configured to execute an algorithm to continuously calibrate the eye tracking assembly including the steps of rendering a sequence of images on a corresponding display and calibrating a corresponding eye tracking assembly by establishing a display coordinate system associated with the corresponding display, determining predicted fixation locations and estimated gaze position locations within the display coordinate system, and determining an offset gaze position value by mapping the estimated gaze position location to the predicted fixation location within the display coordinate system.

Abstract: This disclosure relates to video coding and more particularly to techniques for signaling temporal sublayer information for coded video. According to an aspect of an invention, a flag indicating whether a syntax element specifying a maximum number of temporal sublayers that are present in each coded layer video sequence referring to a sequence parameter set is present in the sequence parameter set is signaled; and the syntax element specifying a maximum number of temporal sublayers is conditionally signaled based on the value of the flag.

Abstract: A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers of increasing depths as the distance between a given layer and the display surface increases. Data layers which are sampled using an effective resolution function to determine a suitable sampling rate and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

Abstract: The present technology relates to an image processing apparatus and method and a program that enable natural representation of light in an image in accordance with a viewpoint. The image processing apparatus calculates information indicating a change in a light source region between an input image and a viewpoint-converted image that is obtained by performing viewpoint conversion on the input image on the basis of a specified viewpoint, and causes a change in representation of light in the viewpoint-converted image on the basis of the calculated information indicating a change in the light source region. The present technology can be applied to an image display system that generates a pseudo stereoscopic image with motion parallax from one image.

Abstract: Various embodiments set forth eye tracking systems. In some embodiments, an eye tracking system includes a polarization volume hologram (PVH) combiner having a rolling k-vector design that provides relatively wide coverage of users whose eyeglasses prescriptions can vary. The PVH combiner can further include (1) fiducial regions created by differential patterning that generate dark regions in images captured of an eye, and/or (2) multiple regions that diffract light at angles to produce different perspectives in the captured images. The dark regions and/or different perspectives can be used to calibrate eye tracking. In addition, the PVH combiner can include off-axis lens regions that generate glints for the eye tracking.