Abstract: A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Abstract: A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Abstract: Aspects of the present disclosure are directed to creating a skybox for an artificial reality (“XR”) world from a two-dimensional (“2D”) image. The 2D image is scanned and split into at least two portions. The portions are mapped onto the interior of a virtual enclosed 3D shape, for example, a virtual cube. A generative adversarial network (GAN) interpolates from the information in the areas mapped from the portions to fill in at least some unmapped areas of the interior of the 3D shape. The 3D shape can be placed in a user's XR world to become the skybox surrounding that world.

Abstract: A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Abstract: A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Abstract: When an image is projected from 3D, the viewpoint of objects in the image, relative to the camera, must be determined. Since the image itself will not have sufficient information to determine the viewpoint of the various objects in the image, techniques to estimate the viewpoint must be employed. To date, neural networks have been used to infer such viewpoint estimates on an object category basis, but must first be trained with numerous examples that have been manually created. The present disclosure provides a neural network that is trained to learn, from just a few example images, a unique viewpoint estimation network capable of inferring viewpoint estimations for a new object category.

Abstract: Technologies for image processing based on a creation workflow for creating a type of images are provided. Both multi-stage image generation as well as multi-stage image editing of an existing image are supported. To accomplish this, one system models the sequential creation stages of the creation workflow. In the backward direction, inference networks can backward transform an image into various intermediate stages. In the forward direction, generation networks can forward transform an earlier-stage image into a later-stage image based on stage-specific operations. Advantageously, this technical solution overcomes the limitations of the single-stage generation strategy with a multi-stage framework to model different types of variation at various creation stages. Resultantly, both novices and seasoned artists can use these technologies to efficiently perform complex artwork creation or editing tasks.

Abstract: A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Abstract: This disclosure includes technologies for image processing based on a creation workflow for creating a type of images. The disclosed technologies can support both multi-stage image generation as well as multi-stage image editing of an existing image. To accomplish this, the disclosed system models the sequential creation stages of the creation workflow. In the backward direction, inference networks can backward transform an image into various intermediate stages. In the forward direction, generation networks can forward transform an earlier-stage image into a later-stage image based on stage-specific operations. Advantageously, the disclosed technical solution overcomes the limitations of the single-stage generation strategy with a multi-stage framework to model different types of variation at various creation stages. Resultantly, both novices and seasoned artists can use the disclosed technologies to efficiently perform complex artwork creation or editing tasks.

Abstract: Computer vision systems and methods for image to image translation are provided. The system receives a first input image and a second input image and applies a content adversarial loss function to the first input image and the second input image to determine a disentanglement representation of the first input image and a disentanglement representation of the second input image. The system trains a network to generate at least one output image by applying a cross cycle consistency loss function to the first disentanglement representation and the second disentanglement representation to perform multimodal mapping between the first input image and the second input image.

Abstract: When an image is projected from 3D, the viewpoint of objects in the image, relative to the camera, must be determined. Since the image itself will not have sufficient information to determine the viewpoint of the various objects in the image, techniques to estimate the viewpoint must be employed. To date, neural networks have been used to infer such viewpoint estimates on an object category basis, but must first be trained with numerous examples that have been manually created. The present disclosure provides a neural network that is trained to learn, from just a few example images, a unique viewpoint estimation network capable of inferring viewpoint estimations for a new object category.

Abstract: A geometric structure analyzing method, a geometric structure analyzing system, and a computer program product are provided, to analyze a two-dimensional geometric structure of a model composed of at least one magnetic building block. A magnetic field intensity image of the model is obtained, and a shape of the magnetic field intensity image is used as a contour of the model. The contour of the model is skeletonized to obtain the two-dimensional geometric structure of the model, and the two-dimensional geometric structure is displayed on a display panel. Therefore, a user is allowed to control the two-dimensional geometric structure on the display panel by manipulating the model, to achieve interactive effects including visual and tactile feedbacks.

Abstract: A geometric structure analyzing method, a geometric structure analyzing system, and a computer program product are provided, to analyze a two-dimensional geometric structure of a model composed of at least one magnetic building block. A magnetic field intensity image of the model is obtained, and a shape of the magnetic field intensity image is used as a contour of the model. The contour of the model is skeletonized to obtain the two-dimensional geometric structure of the model, and the two-dimensional geometric structure is displayed on a display panel. Therefore, a user is allowed to control the two-dimensional geometric structure on the display panel by manipulating the model, to achieve interactive effects including visual and tactile feedbacks.

Abstract: A fabrication method of nanoparticles is provided. A substrate having a plurality of pillar structures is provided and then a plurality of ring structures is formed to surround the plurality of the pillar structures. The inner wall of each ring structure surrounds the sidewall of each pillar structure. A portion of each pillar structure is removed to reduce the height of each pillar structure and to expose the inner wall of each ring structure. The ring structures are separated from the pillar structures to form a plurality of nanoparticles. Surface modifications are applied to the ring structures before the ring structures are separated from the pillar structures on the substrate.

Abstract: A fabrication method of nanoparticles is provided. A substrate having a plurality of pillar structures is provided and then a plurality of ring structures is formed to surround the plurality of the pillar structures. The inner wall of each ring structure surrounds the sidewall of each pillar structure. A portion of each pillar structure is removed to reduce the height of each pillar structure and to expose the inner wall of each ring structure. The ring structures are separated from the pillar structures to form a plurality of nanoparticles. Surface modifications are applied to the ring structures before the ring structures are separated from the pillar structures on the substrate.

Abstract: A method for forming a localized surface plasmon resonance (LSPR) sensor is disclosed, including providing a substrate, forming a metal thin film on the substrate and irradiating the metal thin film with a laser to form a plurality of metal nanoparticles, wherein the metal nanoparticles have a fixed orientation.

Abstract: A transformer includes a primary winding coil, a winding frame member, multiple first three-dimensional conductive pieces, a second three-dimensional conductive piece, a magnetic core assembly and a fixing plate. The winding frame member includes a first winding frame and a second winding frame for winding the primary winding coil thereon. The first three-dimensional conductive pieces are respectively sheathed around the first winding frame and the second winding frame of the winding frame member. The second three-dimensional conductive piece is arranged between the first three-dimensional conductive pieces. The magnetic core assembly is partially embedded into the first three-dimensional conductive pieces, the first winding frame, the second winding frame and the second three-dimensional conductive piece.

Abstract: A transformer includes a primary winding coil, a winding frame member, multiple first three-dimensional conductive pieces, a second three-dimensional conductive piece, a magnetic core assembly and a fixing plate. The winding frame member includes a first winding frame and a second winding frame for winding the primary winding coil thereon. The first three-dimensional conductive pieces are respectively sheathed around the first winding frame and the second winding frame of the winding frame member. The second three-dimensional conductive piece is arranged between the first three-dimensional conductive pieces. The magnetic core assembly is partially embedded into the first three-dimensional conductive pieces, the first winding frame, the second winding frame and the second three-dimensional conductive piece.