Abstract: Systems, methods, and computer-readable storage media for resizing images using seam carving techniques are disclosed. The methods may facilitate efficient re-computation of the energy of an image between seam carving iterations in a resizing operation by re-computing only the energy of pixels and/or seams in the neighborhood of removed and/or replicated seams. The neighborhood and pixels for which energy values are updated may be dependent on the energy function employed by the seam carving techniques. The efficient re-computation of image energy may facilitate the use of seam carving techniques in user-interactive environments. The methods may also be used to pre-compute a retargeting matrix, usable in performing retargeting operations on an input image without re-computing the energy and/or the lowest cost seam(s) of the image between iterations.

Abstract: Methods and apparatus for manipulating digital images. A warping module is described that enables the manipulation of a surface by selectively deforming portions of the surface while maintaining local rigidity. The user may position multiple control points on a surface to constrain deformation. The user may specify multiple properties (e.g., translation, rotation, depth, and scale) at each control point. A mesh may be overlaid on the surface. The warping module may perform an initialization in which the properties are propagated other vertices in the mesh to generate an initial deformed mesh. The warping module may then perform an iterative optimization operation on the deformed mesh to improve the deformation while retaining local rigidity. Thus, instead of moving every pixel in the surface, the warping module moves or adjusts coordinates of the vertices of the mesh. The surface is then deformed according to the deformed mesh.

Abstract: Systems, methods, and computer-readable storage media for editing the frequency content of an image may provide an intuitive interface for examining and/or modifying the frequency content. The methods may include accessing image data, performing a wavelet transform of the image data (or another MRA technique) to produce a time-frequency representation (TFR) of the image representing frequency content in multiple frequency bands, and displaying an indication of the frequency content of the image by displaying a sub-image of the TFR or numerical data representing the frequency content at selected pixels. In response to receiving input specifying a desired frequency content modification, the methods may include performing a Fourier transform of the image data to produce frequency content data, modifying the frequency content data, and performing an inverse transformation of the modified data to produce modified image data. The modification may be applied globally or to a selected portion of the image.

Abstract: Systems, methods, and computer-readable storage media for resizing images using seam carving techniques may include generation of a partial solution matrix by at least partially isolating dependencies between sub-problems of a dynamic programming problem corresponding to its solution within different regions of an input image. The number and/or shape of the isolated (or partially isolated) sub-problems may be dependent on the access pattern used by a dynamic programming operation to identify seams in the input image. Multiple sub-problems may be processed independently and in parallel on respective processor core(s) or threads thereof to generate the partial solution matrix. The partial solution matrix may then be processed to identify one or more low-cost seams of the input image. The methods may be implemented as stand-alone applications or as program instructions implementing components of a graphics application, executable by a CPU and/or GPU configured for parallel processing.

Abstract: Various methods and apparatus for removing artifacts in frequency domain processing of light-field images are described. Methods for the reduction or removal of the artifacts are described that include methods that may be applied during frequency domain processing and a method that may be applied during post-processing of resultant angular views. The methods may be implemented in software as or in a light-field frequency domain processing module. The described methods include an oversampling method to determine the correct centers of slices, a phase multiplication method to determine the correct centers of slices, a method to exclude low-energy slices, and a cosmetic correction method.

Abstract: A system and method for expansion and reduction of images uses an absolute value associated with each pixel of an input image (e.g., a color and/or intensity value) to determine a respective energy value for each pixel. For example, a given color or range of colors (e.g., skin tones, or other high-priority colors) may be assigned higher energy values than other colors and/or color ranges, and may be protected during image reduction and/or expansion. These energy values may be used to determine a cost associated with various seams of the image, which may represent the priority of the seams in the image. One or more low-cost seams may be identified for removal or replication to produce a resized image. The methods may be used in conjunction with an automated skin tone detector or a user interface that allows selection of one or more high priority colors or color ranges.

Abstract: A system and method for expansion and reduction of images uses a look-up table to define an arbitrary mapping of data (e.g., pixel values) representing an image to respective energy values. Each pixel value may represent an absolute color or intensity value, a difference in color or intensity values, or an average, derivative, minimum, or maximum of two or more pixel values. The energy values may then be used to determine one or more low-cost seams of the image to be removed for an image reduction operation or replicated for an image expansion operation, where the cost of each seam is dependent on the energy values of the pixels of the seam. The look-up table may be used to apply a threshold and/or cap on the energy values mapped to pixel values. The look-up table may also provide a mechanism for reconfiguring mappings, thresholds, and/or caps.

Abstract: A system and method for expansion and reduction of images uses pyramidal retargeting to reduce complexity in image resizing. An image pyramid may be generated dependent on a function of pixel data or energy values for each pixel of an input image. An image resizing solution may be developed by applying seam carving techniques to a lower-resolution version of the input image represented by the image pyramid, and may identify bands in the higher-resolution input image to be resized. A resizing operation may then be applied the bands based on local content. The resizing operation may include seam carving techniques, scaling techniques, and/or hybrid resizing techniques, and may be selectable by a user. Identified bands may be expanded to obtain a smoother solution. Different bands may be expanded by different amounts, based on local content. The number of bands and the expansion factors may be configurable based on user input.

Abstract: A system and method for expansion and reduction of images in an image editing application uses the frequency of pixels values (e.g., absolute color and/or intensity values) of an input image to determine respective energy values for each pixel of the image. The energy values may vary inversely with the frequency of the pixel values. The energy values may be used to determine one or more low-cost seams of the image to be removed for an image reduction operation or replicated for an image expansion operation, where the cost of each seam is dependent on the energy values of the pixels of the seam. Determining the frequency of pixel values of the image may involve determining the number of pixels having an absolute pixel value in each of a plurality of pixel value ranges, and the absolute pixel values included in each range may be configurable by the user.

Abstract: A system and method for expansion and reduction of images uses a hybrid resizing technique that combines seam carving and image scaling techniques to reduce or expand an image. Seam carving techniques may be used to remove or add seams having an average or total energy cost below a configurable threshold, where the cost of each seam is dependent on the energy values of the pixels of the seam. If a target size and/or ratio for the resized version of the image is not reached by removing or adding these seams, the hybrid resizing technique may apply standard or custom image scaling techniques to further reduce or expand the image to achieve the target size and/or ratio. The hybrid technique may be implemented by program instructions of an image editing application, and the cost threshold may be configurable by a user through a user interface of the image editing application.

Abstract: A system and method for expansion and reduction of images uses variable seam replication to expand an image. Seam carving techniques may be used to identify one or more low-cost seams of an input image, and these low-cost seams may be replicated to produce a resized version of the image. A different replication factor may be applied to different ones of the low-cost seams, dependent on the average or total energy value of each seam. For example, the lowest cost seam may be replicated twice as many times as the next lowest cost seam. The replication factor applied to each seam may be dependent on the number of low-cost seams identified for replication, which may be configurable by users. A configurable look-up table may map seam costs to replication factors, and may be accessed to determine a respective replication factor to be applied to each of the identified low-cost seams.

Abstract: Various methods and apparatus for removing artifacts in frequency domain processing of light-field images are described. Methods for the reduction or removal of the artifacts are described that include methods that may be applied during frequency domain processing and a method that may be applied during post-processing of resultant angular views. The methods may be implemented in software as or in a light-field frequency domain processing module. The described methods include an oversampling method to determine the correct centers of slices, a phase multiplication method to determine the correct centers of slices, a method to exclude low-energy slices, and a cosmetic correction method.

Abstract: A system and method for expansion and reduction of images uses a hybrid resizing technique that combines seam carving and image scaling techniques to reduce or expand an image. Seam carving techniques may be used to identify one or more low-cost seams of an input image, and these low-cost seams may be scaled up or down to expand or reduce the overall input image and produce a resized image. A different scaling factor may be applied to different ones of the low-cost seams, dependent on the average or total energy value of each of the seams. The scaling factor applied to each seam may be dependent on the number of low-cost seams identified for scaling, which may be configurable by a user. A configurable look-up table may map seam costs to scaling factors, and may be accessed to determine a respective scaling factor to be applied to each identified seam.

Abstract: Systems, methods, and computer-readable storage media for resizing images using seam carving techniques may access a pre-computed retargeting matrix for an input image that associates each pixel of the input image with a cost order of a horizontal and/or vertical seam of which the pixel is a part. The retargeting matrix may be divided into sub-matrices corresponding to horizontal or vertical strips of the image, in the direction of a desired seam carving operation, for parallel processing on multiple processors or threads thereof. The retargeting matrix and/or sub-matrices may be usable in performing one or more reduction or expansion operations on the input image without re-computation of the matrices between retargeting iterations. The methods may include pre-computing the retargeting matrix and/or sub-matrices.

Abstract: A system and method for expansion and reduction of images may apply resizing techniques independently to different regions of an input image to produce a resized version of the image having a specified target size and/or ratio. A content-aware resizing technique may be applied to some regions of the image. Each region may be reduced or expanded by the same amount based on local content, or different regions may be resized by different amounts to meet a resizing target for the input image. The same resizing technique may be applied to each region, or different resizing techniques may be applied to different regions. A given region or unselected portion of the image may not be resized at all. The techniques may be implemented by program instructions of an image editing application, and the definition of the regions and/or the selection of locally applied techniques may be configurable by a user.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: Sub pixel image alignment includes mapping first pixels from a first image and second pixels from a second image to a coordinate system and applying one or more sub-pixel shifts to the mapped first pixels. For each sub-pixel shift, an overall energy is calculated and is based on a plurality of gradients that represent changes in a channel value among the shifted first pixels and the mapped second pixels. The sub-pixel alignment further includes determining the sub-pixel shift that provides the lowest overall energy.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: A method, system, and computer-readable storage medium are disclosed for color conversion of a digital image. The digital image comprises a source set of pixels. A perceptual distance may be determined between the source set of pixels and a respective destination set of pixels for each of a plurality of rendering intents. A rendering intent corresponding to the smallest perceptual distance may be selected automatically. The source set of pixels may be converted to an output set of pixels using the selected rendering intent.