Abstract: The present disclosure is directed to a software tool that facilitates the presentation of a three-dimensional view of a construction project as well as the generation of various types of two-dimensional technical drawings based on this three-dimensional view. In one implementation, the software tool causes a computing device to engage in the following operations. The computing device may receive a user input identifying at least one first mesh and at least one second mesh, determine portions of each identified mesh that intersect with a two-dimensional plane, and generate a two-dimensional drawing including a display of relevant dimensioning information based on the determined portions.

Abstract: A virtual space configuration unit 23 detects, on the basis of depth information of an image of a real space around a user, a position of an object in the real space and arrange a virtual body at a position in a virtual space corresponding to the detected position of the object to configure information of the virtual space. An image generation unit 25 generates an image of the virtual space configured by the virtual space configuration unit 23. A depth comparison unit 22 compares the depth information of the image of the real space and depth information of the image of the virtual space to decide an unknown depth region. The image generation unit 25 displays information indicative of presence of an unknown object in the unknown depth region of the image of the virtual space.

Abstract: One aspect of the disclosure provides a method for rendering an image. The method includes: placing primitives of the image in a screen space; binning the primitives into tiles of the screen space that the primitives touch; and rasterizing the tiles. The aforementioned rasterizing includes shading a subset of the primitives binned to one of the tiles over multiple passes at multiple shading rates, each of the shading rates is based at least on a frequency at which a color being shaded at each pass changes across the screen space, and the subset of the primitives are cached in an on-chip memory of a processor rendering the image between the passes.

Abstract: An apparatus for performing 3D estimation on the basis of pictures of at least two different views includes a hypotheses provider, a similarity measure calculator, and a 3D information determiner. The hypotheses provider locally determines 3D information hypotheses for positions of a current picture of a first view on the basis of a pre-estimate which associates a 3D information estimate to each position of a picture of the first view. The similarity measure calculator calculates, for each position of the current picture of the first view, a hypotheses of the respective position by measuring a similarity between a region similarity measure for each of the 3D information of the current picture of the first view at the respective position and a corresponding region of a second view at a position displaced relative to the respective position by a respective 3D information hypothesis.

Abstract: The techniques introduced here enable a display system, such as an HMD device, to generate and display to a user a holographic structure matching a real-world structure. In some embodiments vertices, edges and planes of the holographic schematics are generated via the use of a peripheral tool that is positioned by a user. In other embodiments, other user input indicates the bounds of the holographic schematic. In response to user action, a holographic schematic is made to appear including corresponding real-world size measurements. The corresponding measurements are used to develop a holographic structure that integrates with the holographic schematic.

Abstract: Systems and methods for modifying a fitness plan are disclosed. The method may include providing a user input; receiving a user selection; identifying electronic plan data stored in a database; receiving the electronic plan data on a portable electronic device; generating an electronic schedule of fitness activities based on the electronic plan data; receiving a second user selection related to the electronic schedule; and modifying the electronic schedule based on the second user selection.

Abstract: A distance measurement system includes two or more line pattern generators (LPGs), a camera, and a processor. Each LPG emits a line pattern having a first set of dark portions separated by a respective first set of bright portions. A first line pattern has a first angular distance between adjacent bright portions, and a second line pattern has a second angular distance between adjacent bright portions. The camera captures at least one image of the first line pattern and the second line pattern. The camera is a first distance from the first LPG and a second distance from the second LPG. The processor identifies a target object illuminated by the first and second line patterns and determines a distance to the target object based on the appearance of the target object as illuminated by the first and second line patterns.

Abstract: A method is presented for detecting occlusions on a color camera. The method includes: capturing an image of a scene using the camera; analyzing intensity of electromagnetic radiation forming the at least one image, where the intensity of the electromagnetic radiation is analyzed across the electromagnetic spectrum; detecting an occlusion on a lens of the camera based on variation of intensity of the electromagnetic radiation across the electromagnetic spectrum; and tagging the image with an indicator of the occlusion, where the tagging occurs in response to detecting an occlusion on the camera.

Abstract: A system, method or compute program product for generating stereoscopic images. One of the methods includes identifying, in a first three-dimensional coordinate system of a first three-dimensional virtual environment, a location and orientation of a first virtual object that is a virtual stereoscopic display object; identifying an eyepoint pair in the first virtual environment; identifying, in a second three-dimensional coordinate system of a second three-dimensional virtual environment, a location and orientation of a second virtual object that is in the second virtual environment; for each eyepoint of the eyepoint pair, rendering an inferior image of the second virtual object; for each eyepoint of the eyepoint pair, render a superior image of the first virtual environment, comprising rendering, in the superior image for each eyepoint, the corresponding inferior image onto the virtual stereoscopic display object; and display, on a physical stereoscopic display, the first virtual environment.

Abstract: A process improves accuracy in mapping geodetic coordinates to image sensor coordinates via an image rational function. The image rational function is fitted to an image-to ground sensor model at an input grid of image u coordinates and image v coordinates, and further at geodetic longitudes, geodetic latitudes, and geodetic heights corresponding to the image u coordinates and the image v coordinates. This fitting generates fit residuals. The fit residuals are stored as a function of the image u coordinates, the image v coordinates, and the geodetic height coordinates. The fit residuals are applied to metadata associated with the image rational function. This application corrects for a residual error in a fit of the image rational function to the image-to-ground sensor model.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to a workstation computing system(s) (“workstation”) that receives camera pose data from a camera(s) disposed on a first headset device. The workstation receives camera image data comprising one or more images portraying a current view of the camera. The workstation generates a rendering of the 3D model of medical data for display within a composite image. The workstation generates one or more portions of the composite image based on the camera image data. The workstation generates and displays a composite image to include simultaneous and contiguous portrayal of the rendering of the 3D model of medical data with the one or more portions based on the camera image data.

Abstract: Disclosed are a method for providing a virtual reality image, and a program using the same. The method includes receiving virtual reality image data from a server and allocating the virtual reality image data to a first layer, generating a second layer containing control information of at least one controller, receiving movement information of the at least one controller, and determining a collision point between a predetermined object in the virtual reality image data and the control information based on the movement information, modifying the control information based on a normal vector at the collision point, and allocating combination data including the modified control information to the second layer, and synthesizing the first layer and the second layer with each other to generate a last image frame.

Abstract: A learning based speed planner for autonomous driving vehicles (ADV) is disclosed. An ADV is set into human-driving mode. Driving control elements are under control of a human driver, and other ADV logic is enabled. The ADV plans a route path on a segment of the route having an obstacle. ADV logic generates a station-time graph for the path of the segment, and a grid of cells to encompass the path and obstacle. A feature vector is generated from the grid. Human driving behavior is recorded as the ADV is navigated along the path. Recorded driving data for a large plurality of paths, obstacles and ADVs is transmitted to a server to generate a speed model. The speed model is downloaded to one or more ADVs for use in autonomous driving mode, to determine an initial speed to use in similar driving situations.

Abstract: Inventory on a rack of store shelves is monitored by a camera-equipped system that senses when items have been removed. Image data is desirably sensed at plural spectral bands, to enhance item identification by digital watermark and/or other image recognition techniques. The system can be alert to the presence of nearby shoppers, and change its mode of operation in response, e.g., suppressing flash illumination or suspending image capture. The system may self-calibrate to the geometry of shelving in its field of view, and affine-correct captured imagery based on the camera's viewpoint. A great many other features and arrangements are also detailed.

Abstract: According to at least some example embodiments, a computer-readable medium stores computer-executable program instructions that, when executed by a processor, cause the processor to perform operations including, obtaining plane information of a plane by using first distances from a terminal to a plurality of points on the plane; obtaining a normal vector of the plane by using direction information of the terminal measured by a direction sensor and the plane information; determining, based on the normal vector, a parameter of an object to be displayed on the plane; and displaying, on a display of the terminal, the object according to the determined parameter.

Abstract: A motherboard is capable of outputting image data and includes an image transmission port, an on-board video graphics array (VGA) card, a switching circuit, a control circuit, and a first network connection port. The image transmission port is configured to receive an external image signal. The on-board VGA card is configured to provide an internal image signal. The switching circuit selectively uses the external image signal or the internal image signal as image data. The control circuit selectively uses the image data or Ethernet network data as output data. The first network connection port is configured to transmit the output data.

Abstract: An exemplary system for monitoring an operational area, includes a processor that receive plural data streams. Each data stream can include a different spatial characteristic of the operational area. The processor also generates a three-dimensional (3D) virtual visualization of the operational area based on observational perspectives associated with the data streams and their associated spatial characteristics and dynamically prioritizes operational sub-regions within the operational area based on the spatial characteristics. The processor generates a signal encoded with data for verifying the 3D virtual visualization of the operational area including the prioritized operational sub-regions.

Abstract: An apparatus and method are presented comprising one or more sensors or cameras configured to rotate about a central motor. In some examples, the motor is configured to travel at a constant linear speed while the one or more cameras face downward and collect a set of images in a predetermined region of interest. The apparatus and method are configured for image acquisition with non-sequential image overlap. The apparatus and method are configured to eliminate gaps in image detection for fault-proof collection of imagery for an underwater survey. In some examples, long baseline (LBL) is utilized for mapping detected images to a location. In some examples, ultra-short baseline (USBL) is utilized for mapping detected images to a location. The apparatus and method are configured to utilize a simultaneous localization and mapping (SLAM) approach.

Abstract: Techniques are disclosed for translating a virtual object within a displayed scene in a digital environment in a realistic and intuitive manner. The realistic and intuitive translation is achieved due to new techniques that are used for determining locations of the virtual object in the digital scene as the virtual object is being translated responsive to user input. In some embodiments, a control plane is constructed based upon a point of interaction with a virtual object and an initial surface in the scene on which the virtual object is located. The constructed control plane is then used to determine locations in the scene for locating and displaying the virtual object as the virtual object is translated from its start position to an end position in the scene.

Abstract: In example embodiments, techniques are provided integrating 2D images into the display of a 3D reality mesh to recover lost context. A determinization is made whether there is level of detail (LOD) in the 3D reality mesh sufficient to provide full context for a desired view of the 3D reality mesh. When such a LOD does not exist, a subset of the source 2D images that intersect a view frustrum for the desired view is selected from the set of 2D images used to reconstruct the 3D reality mesh. The selected source 2D image is evaluated to determine if it would be visually appropriate for the desired view. If the selected source 2D image is determined to be visually appropriate, the 3D reality mesh is replaced with the selected source 2D image or a portion thereof.

Abstract: A decoding device, an encoding device and a method for point cloud decoding is disclosed. The method includes receiving a bitstream. The method also includes decoding the bitstream into a geometry frame and a texture frame. The geometry and texture frames represent include pixels representing points of the 3D point cloud from different layers. The method further includes deriving a set of missing geometry values from the pixels in the geometry frame and a set of missing texture values from the pixels in the texture frame. The method additionally includes generating a first set of frames representing geometry based on the geometry frame and the set of missing geometry values and generating a second set of frames representing texture based on the texture frame and the set of missing texture values. The method also includes generating the 3D point cloud using the first and second sets of frames.

Abstract: Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

Abstract: A display apparatus and a control method thereof are provided. The display apparatus includes a receiver configured to receive a command, a processor configured to receive a content image and generate a three dimensional (3D) image by mapping the content image to a 3D structure selected from a plurality of 3D structures, each having different geometrical forms, and a display configured to display one viewpoint of the 3D image according to the command.

Abstract: A depth-image processing device includes: a memory; and a processor coupled to the memory and configured to: generate, cased on a synthetic model in which a three-dimensional model of a human body and a three-dimensional model of an object are combined, a plurality of learning images in which a depth image that indicates a distance from a reference position to respective positions on the human body or to respective positions on the object, and a part image to identify any one of respective parts of the human body and a cart of the object are associated with each other, and learn an identifier in which a feature of the depth image and any one of a part of the human body and a part of the object are associated with each other, based on the learning images.

Abstract: Embodiments of a 3D web interaction system are disclosed that allow a user to select a content item from a browser, displayed in an artificial reality environment, and present a corresponding version of the content item in the artificial reality environment. The 3D web interaction system can create the version of the selected content item in different ways depending on whether the selected content item is associated with 3D content and, if so, the type of the associated 3D content. For example, the 3D web interaction system can create and present different versions of the selected content item depending on whether the selected content item is (a) not associated with 3D content, (b) associated with “environment content,” or (c) associated with one or more 3D models.

Abstract: An image processing apparatus obtains a plurality of images based on image capturing performed by a plurality of image capturing apparatuses, receives an input corresponding to operation of a first device, receives an input corresponding to operation of a second device different from the first device; and generates, based on the plurality of images obtained, a first virtual viewpoint image corresponding to a first virtual viewpoint determined based on the input corresponding to the operation of the first device, wherein a second virtual viewpoint image corresponding to a second virtual viewpoint determined based on the input received by the second receiving unit is displayed within the first virtual viewpoint image.

Abstract: A method and system is provided for visualizing and navigating cybersecurity information. A hypertree is displayed on a display device of a computerized system. The hypertree includes a plurality of nodes linked by edges, one or more of the nodes representing cybersecurity incidents, and one or more of the nodes representing elements or artifacts of cybersecurity incidents, the edges representing a specific relationship between the nodes linked by the edges. The computerized system displays an interactive navigation aid to enable a user to navigate the hypertree, and receives a navigation command from the user through the interactive navigation aid. The computerized system modifies the displayed hypertree in response to the navigation command. The navigation command comprises selective elimination or restoration of edges or nodes on the hypertree so as to enable the user to readily visualize interrelationships between the displayed nodes that are significant to a cybersecurity investigation or response.

Abstract: An HMD device identifies a pose of the device and identifies a subset of a plurality of camera viewpoints of a light-field based on the pose. The HMD device interpolates image data of the light-field based on the pose and the subset of the plurality of camera viewpoints to generate an interpolated view; and displays at the HMD device an image based on the interpolated view. By interpolating based on the subset of camera viewpoints, the HMD device can reduce processing overhead and improve the user experience.

Abstract: A method, an apparatus, a device, and a medium for calibrating a posture of a moving obstacle are provided. The method includes: obtaining a 3D map, the 3D map including first static obstacles; selecting a target frame of data, the target frame of data including second static obstacles and one or more moving obstacles; determining posture information of each of the one or more moving obstacles in a coordinate system of the 3D map; registering the target frame of data with the 3D map; determining posture offset information of the target frame of data in the coordinate system according to a registration result; calibrating the posture information of each of the one or more moving obstacles according to the posture offset information; and adding each of the one or more moving obstacles after the calibrating into the 3D map.

Abstract: Disclosed herein is a program for controlling an information processing device that receives an operation on an object to be possessed disposed in a virtual space from a user, the program causing the information processing device to execute rendering a spatial image that depicts an aspect of an interior of the virtual space on a basis of a position and a direction of a given viewpoint, receiving a housing operation of adding the object to be possessed to user's personal belongings from the user, and controlling so as to dispose a list of personal belongings to which the object to be possessed has been added at a position in the virtual space determined depending on the position and the direction of the viewpoint at a time of receiving the housing operation.

Abstract: A method of generating a model for 3D printing, includes selecting a target object within a virtual environment; sampling the target object to form a point cloud, the point cloud comprising points corresponding to an outer surface of the target object and also one or more internal features of the target object; rendering the point cloud from a plurality of viewpoints using voxels in place of the points in the point cloud; detecting which voxels and hence which points of the cloud were rendered over the plurality of renders; forming a surface-only point cloud comprising those points of the cloud that were rendered; and generating a model for 3D printing based on the surface-only point cloud.

Abstract: A graphics processing system includes a graphics processor, the graphics processor generating a G-buffer (geometry-buffer) by performing a first pass of deferred shading; and a memory storing the G-buffer, wherein the graphics processor includes: a map manager to generate a first fragment map indicating fragments to be shaded and generate a second fragment map by adding or removing fragments to or from the first fragment map based on the G-buffer; and a shades to shade fragments according to the second fragment map.

Abstract: This invention relates to panning in a three dimensional environment on a mobile device. In an embodiment, a computer-implemented method for navigating a virtual camera in a three dimensional environment on a mobile device having a touch screen. A user input is received indicating that an object has touched a first point on a touch screen of the mobile device and the object has been dragged to a second point on the touch screen. A first target location in the three dimensional environment is determined based on the first point on the touch screen. A second target location in the three dimensional environment is determined based on the second point on the touch screen. Finally, a three dimensional model is moved in the three dimensional environment relative to the virtual camera according to the first and second target locations.

Abstract: A system, method or compute program product for generating composite images. One of the systems includes a capture device to capture an image of a physical environment; and one or more storage devices storing instructions that are operable, when executed by one or more processors of the system, to cause the one or more processors to: obtain an image of the physical environment as captured by the capture device, identify a visually-demarked region on a surface in the physical environment as depicted in the image, process the image to generate a composite image of the physical environment that includes a depiction of a virtual object, wherein a location of the depiction of the virtual object in the composite image is based on a location of the depiction of the visually-demarked region in the image, and cause the composite image to be displayed for a user.

Abstract: An electronic device and method are herein disclosed. The electronic device includes a first camera with a first field of view (FOV), a second camera with a second FOV that is narrower than the first FOV, and a processor configured to capture a first image with the first camera, the first image having a union FOV, capture a second image with the second camera, determine an overlapping FOV between the first image and the second image, generate a disparity estimate based on the overlapping FOV, generate a union FOV disparity estimate, and merge the union FOV disparity estimate with the overlapping FOV disparity estimate.

Abstract: This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Abstract: A mechanism is described for facilitating smart point cloud reconstruction of objects in visual scenes in computing environments. An apparatus of embodiments, as described herein, includes one or more processors including one or more graphics processors, and photo-consistency logic to perform line searches on cloud points of an object to enhance photo-consistency between multiple camera views associated with a visual hull encompassing the object in a scene captured by multiple cameras coupled to the one or more processors. The apparatus may further include refinement and application logic to perform a final reconstruction of the object based on the enhanced photo-consistency, where the scene having the reconstructed object is displayed at a display device.

Abstract: A method of calibrating stiffness of support structures of an assembly including a main structure and support structures includes modeling the assembly using finite elements, with a theoretical stiffness of the support structures, in moderate support load conditions, calculating calculated reaction forces for each support structure and measuring corresponding measured reaction forces. The stiffness of each support structure is calibrated by determining a calculated stiffness of each support structure based on comparison between the calculated reaction forces and the measured reaction forces. The modeling and simple measurements during an operational phase, in which the support loads are moderate, thus enable the calibration of the stiffness of the support structures. An associated method of modeling the assembly and a corresponding computer program are also disclosed.

Abstract: Various embodiments disclosed herein relate to systems and methods for analyzing and manipulating images and video. Methods as disclosed herein may include retrieving, using a processor, a multi-view interactive digital media representation (MIDMR) from a storage location, the MIDMR including a content model and a context model, the content model characterizing an object, and the context model characterizing scenery surrounding the object. The methods may also include receiving, using the processor, at least one dynamic content input associated with the retrieved MIDMR, the dynamic content input being received while a user is interacting with the MIDMR. The methods may further include implementing, using the processor, one or more modifications associated with the MIDMR based, at least in part, on the received at least one dynamic content input, the one or more modifications modifying a presentation and functionality of the MIDMR.

Abstract: A system and method are provided for determining a size of a virtual object in a virtual environment. The virtual object may represent an object in physical space. The system and method may obtain image data of the object from a camera, obtain camera metadata indicative of an angle of view of the camera, estimate an object distance between the object and the camera, and estimate a physical size of the object in physical space by determining an image size of the object in the image data, and determining a relation between the image size and the physical size of the object on the basis of the camera metadata and the object distance. Accordingly, the size of the virtual object in the virtual environment may be determined in accordance with the physical size of the object. Virtual objects may thus be given realistic relative sizes in the virtual environment. A further advantage may be that there is no need for a calibrated system of multiple cameras and a calibrated rendering environment.

Abstract: A target object detecting unit 13 that detects a target object existing within a predetermined distance from an HMD 200 from a moving image of a real world captured by a camera 202 installed in the HMD 200 displaying a virtual space image and an image superimposition unit 15 that causes an image of a predetermined range including the target object to be displayed superimposed on a virtual space image are provided, and when a user performs a predetermined work with a hand within a range photographed in the camera 202 installed in the HMD 200, the captured image of the predetermined range including the hand is displayed superimposed on the virtual space image, and thus the user can appropriately perform the predetermined work while looking at the captured image displayed superimposed on the virtual space image even while wearing the HMD 200.

Abstract: A method and a terminal device for providing a suspend button display are disclosed in order to improve display flexibility of a suspend button. In the solutions, when a terminal device detects that a target object performs a sliding operation on a suspend button, the terminal device may control the suspend button to present an effect of a dynamic change. In this way, the suspend button may present a plurality of display forms. Therefore, according to the method, the display flexibility of the suspend button can be improved, so that visual experience of a user is improved.

Abstract: Aspects of the present disclosure include a computing device for adaptive calibration for dynamic rotation. In an example, a computing device may include an orientation sensor to generate orientation information corresponding to an orientation of the computing device. The computing device monitor a rotation of the computing device based on the orientation information and determine a resting rotational angle of the computing device does not match a desired endpoint orientation angle. The computing device may set the endpoint orientation angle equal to the resting rotational angle and map a set of image orientation angles of an image according to the endpoint orientation angle and a second endpoint orientation angle. The computing device may determine the computing device is rotating based on the orientation information and cause dynamic display of the image based on the set of image orientation angles in response to a rotation of the computing device.

Abstract: Provided as an information processing apparatus that includes: a search unit to search for, on a basis of position information of a user, a predetermined point specified in accordance with a user status and scenario progress from surroundings of a user; and an output control unit that performs control in a manner that a voice of a character corresponding to the scenario progress performs guidance to the predetermined point that has been searched.

Abstract: Methods, systems, and media for enhancing one or more publications by receiving live video captured by a user, the live video comprising video of a publication, the publication comprising copyrighted content; identifying at least one first trigger in the live video, identifying one or more first three-dimensional, interactive media associated with the at least one first trigger and pertaining to the copyrighted content, and presenting to the user the first three-dimensional, interactive media; and identifying at least one second trigger in the first three-dimensional, interactive media, identifying one or more second three-dimensional, interactive media associated with the at least one second trigger and pertaining to the copyrighted content, and presenting to the user the second three-dimensional, interactive media to progressively deepen and enrich the engagement with the copyrighted content of the publication.

Abstract: An augmented reality device includes a logic machine and a storage machine holding instructions executable by the logic machine to, for one or more real-world surfaces represented in a three-dimensional representation of a real-world environment of the augmented reality device, fit a virtual two-dimensional plane to the real-world surface. A request to place a virtual three-dimensional object on the real-world surface is received. For each of a plurality of candidate placement locations on the virtual two-dimensional plane, the candidate placement location is evaluated as a valid placement location or an invalid placement location for the virtual three-dimensional object. An invalidation mask is generated that defines the valid and invalid placement locations on the virtual two-dimensional plane.

Abstract: A device for visualizing a 3D object includes a processor configured to provide an image in a 2D projection plane, and to project an initial 3D object from an initial plane with an inverse projection transformation in the 2D projection plane of the image to achieve an inverse 2D object. The inverse projection transformation is a projection transformation, where a vanishing point is at the other side of the 2D projection plane than the initial object. The processor is further configured to point-mirror the inverse 2D object to achieve a mirrored non-inverse 2D object, and to project the mirrored non-inverse 2D object back to the initial plane to provide a corrected 3D object. Further, the processor is configured to project the corrected 3D object again to the 2D projection plane of the image to provide a final 3D object appearing to be non-inversely projected.

Abstract: Motion and video data from vehicle sensors and camera arrays attached to a vehicle collect video and sensor data along a path driven by the vehicle. A system processes such data to produce high-accuracy structured map data, as might be used to precisely locate a moving vehicle in its environment. Positions are calculated from the sensor data. The positions are updated based on the video data. When loops in the vehicle path are detected, a loop closure error is calculated and used to update the positions as well as to reduce bias in the sensors when calculating future positions. Positions of features in the video are used to create or update structured map data.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing an edge prediction neural network and edge-guided colorization neural network to transform grayscale digital images into colorized digital images. In one or more embodiments, the disclosed systems apply a color edge prediction neural network to a grayscale image to generate a color edge map indicating predicted chrominance edges. The disclosed systems can present the color edge map to a user via a colorization graphical user interface and receive user color points and color edge modifications. The disclosed systems can apply a second neural network, an edge-guided colorization neural network, to the color edge map or a modified edge map, user color points, and the grayscale image to generate an edge-constrained colorized digital image.

Abstract: An obstacle detection method and apparatus are provided. The obstacle detection method provided includes: obtaining a to-be-detected image; determining a road surface area and a non-road surface area in the to-be-detected image according to pixel information contained in the to-be-detected image; respectively determining an outermost layer contour line of the road surface area and a contour line of the non-road surface area; and when the contour line of at least one non-road surface area is located in the area contained in the outermost layer contour line of the road surface area, determining a physical object contained in the at least one non-road surface area as an obstacle. The present application is applied to a process of detecting an obstacle.