Abstract: Provided are systems, methods, and computer-readable medium for operating a neural network. In various implementations, the neural network can receive an input image that includes an object to be identified. The neural network can generate a plurality of initial feature maps using a convolution layers, wherein a first initial feature maps is generated using the input image. The neural network can generate an up-sampled feature map using a de-convolution layer that takes an initial feature map as input, where the up-sampled feature map has a same resolution as the previous initial feature map. The neural network can combine the up-sampled feature map and the previous initial feature map, and use the combined feature map to more accurate identify the object.

Abstract: An apparatus includes a first sensor configured to generate first sensor data. The first sensor data is related to an occupant of a vehicle. The apparatus further includes a depth sensor and a processor. The depth sensor is configured to generate data corresponding to a volume associated with at least a portion of the occupant. The processor is configured to receive the first sensor data and to activate the depth sensor based on the first sensor data.

Abstract: Provided are systems, methods, and computer-readable medium for operating a neural network. In various implementations, the neural network can receive an input image that includes an object to be identified. The neural network can generate a plurality of initial feature maps using a convolution layers, wherein a first initial feature maps is generated using the input image. The neural network can generate an up-sampled feature map using a de-convolution layer that takes an initial feature map as input, where the up-sampled feature map has a same resolution as the previous initial feature map. The neural network can combine the up-sampled feature map and the previous initial feature map, and use the combined feature map to more accurate identify the object.

Abstract: Techniques and systems are provided for classifying objects in one or more video frames. For example, a plurality of object trackers maintained for a current video frame can be obtained. A plurality of classification requests can also be obtained. The classification requests are associated with a subset of object trackers from the plurality of object trackers, and are generated based on one or more characteristics associated with the subset of object trackers. Based on the obtained plurality of classification requests, an object tracker is selected from the subset of object trackers for object classification. For example, the object tracker can be selected from the subset of object trackers based on priorities assigned to the subset of object trackers. The object classification can then be performed for the selected at least one object tracker.

Abstract: An apparatus includes a first sensor configured to generate first sensor data. The first sensor data is related to an occupant of a vehicle. The apparatus further includes a depth sensor and a processor. The depth sensor is configured to generate data corresponding to a volume associated with at least a portion of the occupant. The processor is configured to receive the first sensor data and to activate the depth sensor based on the first sensor data.

Abstract: A mobile platform efficiently processes image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires image data, and determines whether there is a trigger event to transmit the image data to the server. The trigger event may be a change in the image data relative to previously acquired image data, e.g., a scene change in an image. When a change is present, the image data may be transmitted to the server for processing. The server processes the image data and returns information related to the image data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Abstract: A vision based tracking system in a mobile platform tracks objects using groups of detected lines. The tracking system detects lines in a captured image of the object to be tracked. Groups of lines are formed from the detected lines. The groups of lines may be formed by computing intersection points of the detected lines and using intersection points to identified connected lines, where the groups of lines are formed using connected lines. A graph of the detected lines may be constructed and intersection points identified. Interesting subgraphs are generated using the connections and the group of lines is formed with the interesting subgraphs. Once the groups of lines are formed, the groups of lines are used to track the object, e.g., by comparing the groups of lines in a current image of the object to groups of lines in a previous image of the object.

Abstract: A multi-user augmented reality (AR) system operates without a previously acquired common reference by generating a reference image on the fly. The reference image is produced by capturing at least two images of a planar object and using the images to determine a pose (position and orientation) of a first mobile platform with respect to the planar object. Based on the orientation of the mobile platform, an image of the planar object, which may be one of the initial images or a subsequently captured image, is warped to produce the reference image of a front view of the planar object. The reference image may be produced by the mobile platform or by, e.g., a server. Other mobile platforms may determine their pose with respect to the planar object using the reference image to perform a multi-user augmented reality application.

Abstract: A mobile platform efficiently processes sensor data, including image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires sensor data, such as image data, and determines whether there is a trigger event to transmit the sensor data to the server. The trigger event may be a change in the sensor data relative to previously acquired sensor data, e.g., a scene change in an image. When a change is present, the sensor data may be transmitted to the server for processing. The server processes the sensor data and returns information related to the sensor data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Abstract: A mobile platform efficiently processes image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires image data, and determines whether there is a trigger event to transmit the image data to the server. The trigger event may be a change in the image data relative to previously acquired image data, e.g., a scene change in an image. When a change is present, the image data may be transmitted to the server for processing. The server processes the image data and returns information related to the image data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Abstract: A homography between two captured images of a planar object is decomposed into at least one possible solution, and typically at least two ambiguous solutions. The ambiguity between the two solutions is removed, or a single solution validated, using measurements from orientation sensors. The measurements from orientation sensors may be used by comparing at least one of the yaw, pitch, and/or roll angles derived from a relative rotation matrix for the one or more solutions to a corresponding at least one of the yaw, pitch, and/or roll angles derived from the measurements from the orientation sensors.

Abstract: A multi-user augmented reality (AR) system operates without a previously acquired common reference by generating a reference image on the fly. The reference image is produced by capturing at least two images of a planar object and using the images to determine a pose (position and orientation) of a first mobile platform with respect to the planar object. Based on the orientation of the mobile platform, an image of the planar object, which may be one of the initial images or a subsequently captured image, is warped to produce the reference image of a front view of the planar object. The reference image may be produced by the mobile platform or by, e.g., a server. Other mobile platforms may determine their pose with respect to the planar object using the reference image to perform a multi-user augmented reality application.

Abstract: A multi-user augmented reality (AR) system operates without a previously acquired common reference by generating a reference image on the fly. The reference image is produced by capturing at least two images of a planar object and using the images to determine a pose (position and orientation) of a first mobile platform with respect to the planar object. Based on the orientation of the mobile platform, an image of the planar object, which may be one of the initial images or a subsequently captured image, is warped to produce the reference image of a front view of the planar object. The reference image may be produced by the mobile platform or by, e.g., a server. Other mobile platforms may determine their pose with respect to the planar object using the reference image to perform a multi-user augmented reality application.

Abstract: A reference patch of an unknown environment is generated on the fly for positioning and tracking. The reference patch is generated using a captured image of a planar object with two perpendicular sets of parallel lines. The planar object is detected in the image and axes of the world coordinate system are defined using the vanishing points for the two sets of parallel lines. The camera rotation is recovered based on the defined axes, and the reference patch of at least a portion of the image of the planar object is generated using the recovered camera rotation. The reference patch can then be used for vision based detection and tracking. The planar object may be detected in the image as sets of parallel lines or as a rectangle.

Abstract: A mobile station determines its orientation using an image of an object produced by the mobile station and a top view of that object obtained from an online server. The mobile station image is analyzed to identify lines on the object and to determine the direction of the lines with respect to the mobile station. The top view image, which may be a satellite image, is also analyzed to identify lines on the object that correspond to the lines identified in the mobile station image. The direction of the lines in the top view image are compared to the direction of lines in the mobile station image and based on their relative orientation the orientation of the mobile station may be determined. For example, the difference between the preliminary and corrected orientations may be stored as a calibration factor and used to correct subsequent orientation measurements from orientation sensors.

Abstract: A homography between two captured images of a planar object is decomposed into at least one possible solution, and typically at least two ambiguous solutions. The ambiguity between the two solutions is removed, or a single solution validated, using measurements from orientation sensors. The measurements from orientation sensors may be used by comparing at least one of the yaw, pitch, and/or roll angles derived from a relative rotation matrix for the one or more solutions to a corresponding at least one of the yaw, pitch, and/or roll angles derived from the measurements from the orientation sensors.

Abstract: A vision based tracking system in a mobile platform tracks objects using groups of detected lines. The tracking system detects lines in a captured image of the object to be tracked. Groups of lines are formed from the detected lines. The groups of lines may be formed by computing intersection points of the detected lines and using intersection points to identified connected lines, where the groups of lines are formed using connected lines. A graph of the detected lines may be constructed and intersection points identified. Interesting subgraphs are generated using the connections and the group of lines is formed with the interesting subgraphs. Once the groups of lines are formed, the groups of lines are used to track the object, e.g., by comparing the groups of lines in a current image of the object to groups of lines in a previous image of the object.

Abstract: A reference patch of an unknown environment is generated on the fly for positioning and tracking. The reference patch is generated using a captured image of a planar object with two perpendicular sets of parallel lines. The planar object is detected in the image and axes of the world coordinate system are defined using the vanishing points for the two sets of parallel lines. The camera rotation is recovered based on the defined axes, and the reference patch of at least a portion of the image of the planar object is generated using the recovered camera rotation. The reference patch can then be used for vision based detection and tracking. The planar object may be detected in the image as sets of parallel lines or as a rectangle.

Abstract: A mobile platform efficiently processes sensor data, including image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires sensor data, such as image data, and determines whether there is a trigger event to transmit the sensor data to the server. The trigger event may be a change in the sensor data relative to previously acquired sensor data, e.g., a scene change in an image. When a change is present, the sensor data may be transmitted to the server for processing. The server processes the sensor data and returns information related to the sensor data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Abstract: Entropy based image segmentation determines entropy values for pixels in an image based on intensity or edge orientation. One or more threshold values are determined as a fraction of the entropy distribution over the image. For example, high and/or low thresholds may be generated to identify regions in the image associated with trees or sky, respectively. The entropy values are compared to the threshold(s) from which regions within the image can be segmented. Intensity based entropy has no structural information, and thus, proximity based clustering and pruning of the entropy points is performed. A mask may be applied to the segmented regions to remove the regions from the image, which is useful in, e.g., objection recognition processes. Additionally, separate buildings may be identified and segmented using edge orientation entropy with clustering and pruning.