Fourier ptychographic imaging systems, devices, and methods
Systems, devices, and methods of Fourier ptychographic imaging configured for illuminating a specimen being imaged from a plurality of incidence angles, for acquiring variably-illuminated, low-resolution intensity images of the specimen, and for reconstructing a high-resolution image of the specimen by iteratively determining the high resolution image that is self-consistent with the variably-illuminated, low-resolution intensity images, for example, by updating overlapping regions of variably-illuminated, low-resolution intensity images in Fourier space.
Latest California Institute of Technology Patents:
- Systems for flight control on a multi-rotor aircraft
- SYSTEMS AND METHODS FOR DETECTING ABNORMALITIES IN ELECTRICAL AND ELECTROCHEMICAL ENERGY UNITS
- Method of producing thin enzyme-based sensing layers on planar sensors
- Systems and Methods for Electrochemical Hydrogen Looping
- Systems and Methods for a Global Positioning System using GNSS Signals and Stokes Parameters
This is a non-provisional application of, and claims priority to, U.S. Provisional Patent Application No. 61/720,258 entitled “Breaking the Spatial Product Barrier via Non-Interferometric Aperture-Synthesizing Microscopy (NAM),” filed on Oct. 30, 2012 and to U.S. Provisional Patent Application No. 61/847,472 entitled “Fourier Ptychographic Microscopy,” filed on Jul. 17, 2013. These provisional applications are hereby incorporated by reference in their entirety for all purposes.
This non-provisional application is related to the following and commonly-assigned patent application, which is hereby incorporated by reference in its entirety for all purposes:
U.S. patent application Ser. No. 14/065,305, entitled “Fourier Ptychographic X-ray Imaging Systems, Devices, and Methods,” filed on Oct. 28, 2013.
STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSORED RESEARCH OR DEVELOPMENTThis invention was made with government support under Grant No. D0007307 awarded by the National Institutes of Health. The government has certain rights in the invention.
BACKGROUND OF THE INVENTIONEmbodiments of the present disclosure generally relate to wide field-of-view, high-resolution digital imaging techniques. More specifically, certain embodiments relate to Fourier ptychographic imaging (FPI) devices, systems and methods for wide-field, high-resolution imaging.
The throughput of a conventional imaging platform (e.g., microscope) is generally limited by the space-bandwidth product defined by its optical system. The space-bandwidth product refers to the number of degrees of freedom (e.g., number of resolvable pixels) that the optical system can extract from an optical signal, as discussed in Lohmann, A. W., Dorsch, R. G., Mendlovic, D., Zalevsky, Z. & Ferreira, C., “Space-bandwidth product of optical signals and systems,” J. Opt. Soc. Am. A 13, pages 470-473 (1996), which is hereby incorporated by reference in its entirety. A conventional microscope typically operates with a space-bandwidth product on the order of 10 megapixels, regardless of the magnification factor or numerical aperture (NA) of its objective lens. For example, a conventional microscope with a ×20, 0.40 NA objective lens has a resolution of 0.8 mm and a field-of-view of 1.1 mm in diameter, which corresponds to a space-bandwidth product of about 7 megapixels. Prior attempts to increase space-bandwidth product of conventional microscopes have been confounded by the scale-dependent geometric aberrations of their objective lenses, which results in a compromise between image resolution and field-of-view. Increasing the space-bandwidth product of conventional imaging platforms may be limited by: 1) scale-dependent geometric aberrations of its optical system, 2) constraints of the fixed mechanical length of the relay optics and the fixed objective parfocal length, and/or 3) availability of gigapixel digital recording devices.
Some attempts to increase the spatial-bandwidth product using interferometric synthetic aperture techniques are described in Di, J. et al., “High resolution digital holographic microscopy with a wide field of view based on a synthetic aperture technique and use of linear CCD scanning,” Appl. Opt. 47, pp. 5654-5659 (2008); Hillman, T. R., Gutzler, T., Alexandrov, S. A., and Sampson, D. D., “High-resolution, wide-field object reconstruction with synthetic aperture Fourier holographic optical microscopy,” Opt. Express 17, pp. 7873-7892 (2009); Granero, L., Micó, V., Zalevsky, Z., and García, J., “Synthetic aperture superresolved microscopy in digital lensless Fourier holography by time and angular multiplexing of the object information,” Appl. Opt. 49, pp. 845-857 (2010); Kim, M. et al., “High-speed synthetic aperture microscopy for live cell imaging,” Opt. Lett. 36, pp. 148-150 (2011); Turpin, T., Gesell, L., Lapides, J., and Price, C., “Theory of the synthetic aperture microscope,” pp. 230-240; Schwarz, C. J., Kuznetsova, Y., and Brueck, S., “Imaging interferometric microscopy,” Optics letters 28, pp. 1424-1426 (2003); Feng, P., Wen, X., and Lu, R., “Long-working-distance synthetic aperture Fresnel off-axis digital holography,” Optics Express 17, pp. 5473-5480 (2009); Mico, V., Zalevsky, Z., García-Martínez, P., and García, J., “Synthetic aperture superresolution with multiple off-axis holograms,” JOSA A 23, pp. 3162-3170 (2006); Yuan, C., Zhai, H., and Liu, H., “Angular multiplexing in pulsed digital holography for aperture synthesis,” Optics Letters 33, pp. 2356-2358 (2008); Mico, V., Zalevsky, Z., and García, J., “Synthetic aperture microscopy using off-axis illumination and polarization coding,” Optics Communications, pp. 276, 209-217 (2007); Alexandrov, S., and Sampson, D., “Spatial information transmission beyond a system's diffraction limit using optical spectral encoding of the spatial frequency,” Journal of Optics A: Pure and Applied Optics 10, 025304 (2008); Tippie, A. E., Kumar, A., and Fienup, J. R., “High-resolution synthetic-aperture digital holography with digital phase and pupil correction,” Opt. Express 19, pp. 12027-12038 (2011); Gutzler, T., Hillman, T. R., Alexandrov, S. A., and Sampson, D. D., “Coherent aperture-synthesis, wide-field, high-resolution holographic microscopy of biological tissue,” Opt. Lett. 35, pp. 1136-1138 (2010); and Alexandrov, S. A., Hillman, T. R., Gutzler, T., and Sampson, D. D., “Synthetic aperture Fourier holographic optical microscopy,” Phil. Trans. R. Soc. Lond. A 339, pp. 521-553 (1992), which are hereby incorporated by reference in their entirety. Most of these attempts use setups that record both intensity and phase information using interferometric holography approaches, such as off-line holography and phase-shifting holography. The recorded data is then synthesized in the Fourier domain in a deterministic manner.
These previous attempts to increase spatial-bandwidth product using interferometric synthetic aperture techniques have limitations. For example, interferometric holography recordings typically used in these techniques require highly-coherent light sources. As such, the reconstructed images tend to suffer from various coherent noise sources, such as speckle noise, fixed pattern noise (induced by diffraction from dust particles and other optical imperfections in the beam path), and multiple interferences between different optical interfaces. The image quality is, therefore, not comparable to that of a conventional microscope. On the other hand, the use of an off-axis holography approach sacrifices useful spatial-bandwidth product (i.e., the total pixel number) of the image sensor, as can be found in Schnars, U. and Jüptner, W. P. O., “Digital recording and numerical reconstruction of holograms,” Measurement Science and Technology, 13, R85 (2002), which is hereby incorporated by reference in its entirety. Another limitation is that interferometric imaging may be subjected to uncontrollable phase fluctuations between different measurements. Hence, a priori and accurate knowledge of the specimen location may be needed for setting a reference point in the image recovery process (also known as phase referring). Another limitation is that previously reported attempts require mechanical scanning, either for rotating the sample or for changing the illumination angle. Therefore, precise optical alignments, mechanical control at the sub-micron level, and associated maintenances are needed for these systems. In terms of the spatial-bandwidth product, these systems present no advantage as compared to a conventional microscope with sample scanning and image stitching. Another limitation is that previous interferometric synthetic aperture techniques are difficult to incorporate into most existing microscope platforms without substantial modifications. Furthermore, color imaging capability has not been demonstrated on these platforms. Color imaging capability has proven pivotal in pathology and histology applications.
In microscopy, a large spatial-bandwidth product is highly desirable for biomedical applications such as digital pathology, haematology, phytotomy, immunohistochemistry, and neuroanatomy. A strong need in biomedicine and neuroscience to digitally image large numbers of histology slides for analysis has prompted the development of sophisticated mechanical scanning microscope systems and lensless microscopy set-ups. Typically, these systems increase their spatial-bandwidth product using complex mechanical means that have high precision and accurate components to control actuation, optical alignment and motion tracking. These complex components can be expensive to fabricate and difficult to use.
Previous lensless microscopy methods such as digital in-line holography and contact-imaging microscopy also present certain drawbacks. For example, conventional digital in-line holography does not work well for contiguous samples and contact-imaging microscopy requires a sample to be in close proximity to the sensor. Examples of digital in-line holography devices can be found in Denis, L., Lorenz, D., Thiebaut, E., Fournier, C. and Trede, D., “Inline hologram reconstruction with sparsity constraints,” Opt. Lett. 34, pp. 3475-3477 (2009); Xu, W., Jericho, M., Meinertzhagen, I., and Kreuzer, H., “Digital in-line holography for biological applications,” Proc. Natl Acad. Sci. USA 98, pp. 11301-11305 (2001); and Greenbaum, A. et al., “Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy,” Sci. Rep. 3, page 1717 (2013), which are hereby incorporated by reference in their entirety. Examples of contact-imaging microscopy can be found in Zheng, G., Lee, S. A., Antebi, Y., Elowitz, M. B. and Yang, C., “The ePetri dish, an on-chip cell imaging platform based on subpixel perspective sweeping microscopy (SPSM),” Proc. Natl Acad. Sci. USA 108, pp. 16889-16894 (2011); and Zheng, G., Lee, S. A., Yang, S. & Yang, C., “Sub-pixel resolving optofluidic microscope for on-chip cell imaging,” Lab Chip 10, pages 3125-3129 (2010), which are hereby incorporated by reference in their entirety.
BRIEF SUMMARY OF THE INVENTIONEmbodiments of the present disclosure provide Fourier ptychographic imaging (FPI) methods, devices, and systems for wide-field, high-resolution imaging as used in applications such as, for example, digital pathology, haematology, semiconductor wafer inspection, and X-ray and electron imaging. An example of an FPI device is a Fourier ptychographic microscope (FPM), which may also be referred to as employing non-interferometric aperture-synthesizing microscopy (NAM).
In some embodiments, an FPI system includes a variable illuminator, optical element, radiation detector, and a processor. The variable illuminator illuminates a specimen from a plurality of N different incidence angles at different sample times. The optical element filters light issuing from the specimen. The radiation detector captures a plurality of variably-illuminated (perspective) low-resolution intensity images. The processor iteratively stitches together the variably-illuminated, low-resolution images of overlapping regions in Fourier space to recover a wide-field, high-resolution image. In certain embodiments, the FPI device may also correct for aberrations and digitally refocus the complex high-resolution image, which can digitally extend the depth of focus of the FPI system beyond the physical limitations of its optical element.
One embodiment provides a Fourier ptychographic imaging device comprising a variable illuminator for providing illumination to a specimen from a plurality of incidence angles, an optical element for filtering illumination issuing from the specimen, and a detector for acquiring a plurality of variably-illuminated, low-resolution intensity images of the specimen based on light filtered by the optical element. The Fourier ptychographic imaging device also comprises a processor for computationally reconstructing a high-resolution image of the specimen by iteratively updating overlapping regions in Fourier space with the variably-illuminated, low-resolution intensity images. In one case, the variable illuminator is a two-dimensional matrix of light elements (e.g., light-emitting diodes), each light element providing illumination from one of the plurality of incidence angles. In one example, each light element of the two-dimensional matrix comprises three quasi-monochromatic light sources and the processor computationally reconstructs high-resolution images corresponding to the three quasi-monochromatic light sources, and combines the high-resolution images to generate a color high-resolution image.
Another embodiment provides a method of Fourier ptychographic imaging. The method illuminates a specimen being imaged from a plurality of incidence angles using a variable illuminator and filters light issuing from (e.g., scattered by) the specimen using an optical element. The method also captures a plurality of variably-illuminated, low-resolution intensity images of the specimen using a detector. Also, the method computationally reconstructs a high-resolution image of the specimen by iteratively updating overlapping regions of variably-illuminated, low-resolution intensity images in Fourier space. In one case, the method initializes a current high-resolution image in Fourier space, filters an filtering an overlapping region of the current high-resolution image in Fourier space to generate a low-resolution image for an incidence angle of the plurality of incidence angles, replaces the intensity of the low-resolution image with an intensity measurement, and updates the overlapping region in Fourier space with the low-resolution image with measured intensity. In this case, the filtering, replacing, and updating steps may be performed for the plurality of incidence angles. In another case, the method divides each variably-illuminated, low-resolution intensity image into a plurality of variably-illuminated, low-resolution intensity tile images, recovers a high-resolution image for each tile by iteratively updating overlapping regions of variably-illuminated, low-resolution intensity tile images in Fourier space, and combines the high resolution images of the tiles to generate the high-resolution image of the specimen.
Another embodiment provides a method of Fourier ptychographic imaging that receives a plurality of variably-illuminated, low-resolution intensity images of a specimen and computationally reconstructs a high-resolution image of the specimen by iteratively updating overlapping regions of variably-illuminated, low-resolution intensity images in Fourier space. In one case, the method divides each variably-illuminated, low-resolution intensity image into a plurality of variably-illuminated, low-resolution intensity tile images, recovers a high-resolution image for each tile by iteratively updating overlapping regions of variably-illuminated, low-resolution intensity tile images in Fourier space, and combines the high-resolution images of the tiles. In another case, the method initializes a current high-resolution image in Fourier space, filters an overlapping region of the current high-resolution image in Fourier space to generate a low-resolution image for an incidence angle of the plurality of incidence angles, replaces intensity of the low-resolution image with an intensity measurement, and updates the overlapping region in Fourier space with the low-resolution image with measured intensity. In this case, the filtering, replacing, and updating steps may be performed for the plurality of incidence angles.
Embodiments of the present invention will be described below with reference to the accompanying drawings. Although embodiments of FPI systems, devices, and methods may be described herein with respect to illumination with visible light radiation, these FPI systems, devices, and methods may also be used with other forms of radiation such as, for example, acoustic waves, Terahertz waves, microwaves, and X-rays.
Some embodiments include an FPI system comprising a variable illuminator, optical element, radiation detector, and a processor. The variable illuminator successively illuminates a specimen being imaged with plane waves at a plurality of N different incidence angles. The optical element filters light issuing from the specimen. The optical element may be, for example, an objective lens that accepts light issuing from the specimen based on its numerical aperture. In some cases, the optical element may be a low numerical aperture objective lens that provides a corresponding narrow acceptance angle and increased depth of field. The radiation detector detects light filtered by the optical element and captures a plurality of N low-resolution intensity images corresponding to the N different incidence angles. The processor iteratively stitches together overlapping low-resolution intensity images in Fourier space to recover a wide-field, high-resolution image of the specimen. In certain embodiments, the FPI device can also digitally refocus the complex high-resolution image to accommodate for defocus and aberrations in its optical element, which may digitally extend the depth of focus of the FPI system beyond the physical limitations of its optical element.
In certain aspects, an FPI method, performed by the FPI system, comprises a measurement process, a recovery process, and an optional display process. During the measurement process, the specimen is successively illuminated from the plurality of N incidence angles and corresponding low-resolution intensity images are acquired. During the recovery process, one or more high-resolution, wide field-of-view images are recovered based on the low-resolution intensity measurements. During the optional display process, the recovered images and other output is provided to the user of the FPI system on a display.
I. Introduction to FPI Systems and Devices
Although embodiments of FPI devices and systems are described herein with respect to visible light radiation (illumination), other forms of radiation (e.g., X-ray) may be used in certain cases.
During a measurement process, the variable illuminator 110 provides illumination from a plurality of N incidence angles, (θxi, θyi), i=1 to N to the specimen 20. Illumination from variable illuminator 110 may be altered (e.g., blocked, reduced intensity, modified wavelength/phase, modified polarization, etc.) by specimen 20 provided to the FPI device 100. The optical element can receive light from the variable illuminator, for example, as issuing from the specimen 20 and can filter the light it receives. For example, the optical element 130 may be in the form of an objective lens, which accepts light within its acceptance angle to act as a filter. In some cases, the optical element 130 may be an objective lens having a low numerical aperture (e.g., NA of about 0.08) to provide a corresponding narrow acceptance angle and allow for an increased depth of field. The radiation detector 140 can receive the filtered light from the optical element 130 and can record an intensity distribution at the radiation detector 140 at N sample times, ti=1 to N, to capture a plurality of N low-resolution two-dimensional intensity images of the specimen area.
In
Processor 210 is in electronic communication with CRM 220 (e.g., memory) to be able to transmit signals with image data in order to store to and retrieve image data from the CRM 220. Processor 210 is shown in electronic communication with display 230 to be able to send image data and instructions to display the wide-field, high-resolution image of the specimen area and other output, for example, to a user of the FPI system 10. As shown by a dotted line, variable illuminator 110 may optionally be in electronic communication with processor 210 to send instructions for controlling variable illuminator 110. As used herein, electronic communication between components of FPI system 10 may be in wired or wireless form.
In certain embodiments, the FPI device comprises an in-focus plane 122 and a sample plane 124. An in-focus plane 122 can refer to the focal plane of the optical element of the corresponding FPI device. The FPI device includes an x-axis and a y-axis at the in-focus plane 122, and a z-axis orthogonal to the in-focus plane 122. The in-focus plane is defined at an x-y plane at z=0. A sample plane 124 can refer to the plane at which the FPI device may computationally reconstruct the high-resolution wide field-of-view image. The FPI device captures the low-resolution images at the in-focus plane 122. Generally, the sample plane 124 is parallel to the in-focus plane 122. In some embodiments, the sample plane 124 may be coincident to the in-focus plane 122. In an autofocus embodiment, the FPI system 10 may perform an FPI method that can determine the location of the specimen 20 to locate the sample plane 124 at the specimen 20 in order to focus the high-resolution wide field-of-view image at the specimen 20.
In
In
In
A variable illuminator can refer to a device that provides incident radiation from a plurality of N different incidence angles (θxi, θyi), i=1 to N, in succession. Suitable values of N may range from 2 to 1000. In most embodiments, the variable illuminator includes a light element of one or more radiation sources providing illumination at a particular sample time. In most cases, each light element is approximated as providing plane wave illumination to the specimen 20 from a single incidence angle. For example, the incidence angle θxi at reference point, P, in
Although the radiation source or radiation sources are usually coherent radiation sources, incoherent radiation source(s) may also be used and computational corrections may be applied. In embodiments that use visible light radiation, each radiation source is a visible light source. Some examples of a source of visible light include an LCD pixel and a pixel of an LED display. In embodiments that use other forms of radiation, other sources of radiation may be used. For example, in embodiments that use X-ray radiation, the radiation source may comprise an X-ray tube and a metal target. As another example, in embodiments that use microwave radiation, the radiation source may comprise a vacuum tube. As another example, in embodiments that use acoustic radiation, the radiation source may be an acoustic actuator. As another example, in embodiments that use Terahertz radiation, the radiation source may be a Gunn diode. One skilled in the art would contemplate other sources of radiation.
In many embodiments, the properties (e.g., wavelength(s), frequency(ies), phase, amplitude, polarity, etc.) of the radiation provided by the variable illuminator at different incidence angles, (θxi, θyi), i=1 to N, is approximately uniform. In other embodiments, the properties may vary at the different incidence angles, for example, by providing n different wavelengths λ1, . . . , λn during the measurement process. In one embodiment, the variable illuminator 110 may provide RGB illumination of three wavelengths λ1, λ2, and λ3 corresponding to red, green, blue colors, respectively. In embodiments that use Terahertz radiation, the frequencies of the radiation provided by the variable illuminator 110 may be in the range of 0.3 to 3 THz. In embodiments that use microwave radiation, the frequencies of the radiation provided by the variable illuminator may be in the range of 100 MHz to 300 GHz. In embodiments that use X-ray radiation, the wavelengths of the radiation provided by the variable illuminator may be in the range of 0.01 nm to 10 nm. In embodiments that use acoustic radiation, the frequencies of the radiation provided by the variable illuminator may be in the range of 10 Hz to 100 MHz.
In some embodiments, the variable illuminator comprises a plurality of N stationary light elements at different spatial locations (e.g., variable illuminator 110(a) in
In embodiments having a variable illuminator comprising a plurality of N stationary light elements, the light elements may be arranged in the form of a one-dimensional array, a two-dimensional matrix, a hexagonal array, or other suitable arrangement capable of providing the illumination from the plurality of incidence angles. Some examples of matrices of stationary light elements are an LCD or an LED matrix. The light elements are designed with the appropriate spacing and designed to illuminate as required to provide the plurality of incidence angles. In some embodiments, the variable illuminator may be in the form of a two-dimensional matrix having dimensions such as for example, 10×10, 32×32, 100×100, 50×10, 20×60, etc. As an illustration example,
In embodiments having a variable illuminator comprising a moving light element, the moving light element may be moved to a plurality of N positions. The spatial locations of these N positions may be in the form of a one-dimensional array, a two-dimensional matrix, a hexagonal array, or other suitable arrangement capable of providing the illumination from the plurality of incidence angles. Some examples of matrix dimensions may be 10×10, 32×32, 100×100, 50×10, 20×60, etc.
The variable illuminator provides radiation incident to the specimen 20 at a plurality of incidence angles (θxi, θyi), i=1 to N. In one embodiment, the difference between two neighboring incidence angles in the plurality of incidence angles has a value in the range between 10% and 90% of the acceptance angle defined by the numerical aperture of the optical element in the form of an objective lens. In one embodiment, the difference between two adjacent incidence angles in the plurality of incidence angles has a value in the range between 33% and 66% of the acceptance angle defined by the numerical aperture of the optical element in the form of an objective lens. In one embodiment, the difference between two adjacent incidence angles in the plurality of incidence angles has a value that is less than 76% of the acceptance angle defined by the numerical aperture of the optical element in the form of an objective lens. In one embodiment, the difference between adjacent incidence angles in the plurality of incidence angles is about ⅓ of the acceptance angle defined by the numerical aperture of the optical element in the form of an objective lens. In one embodiment, the range of incidence angles, defined by a difference between the largest and smallest incidence angles, may be about equal to the effective numerical aperture consistent with the spatial resolution of the final full field-of-view high-resolution image.
The light elements of the variable illuminator are illuminated in an order defined by illumination instructions. In one embodiment, the illumination instructions determine the order of illuminating light elements in the form of a two-dimensional matrix of light elements. In this embodiment, the illumination instructions may first define a center light element. The illumination instructions may then instruct to illuminate the center light element (e.g., LED) first, then illuminate the 8 light elements surrounding the center light element going counterclockwise, then illuminate the 16 light elements surrounding the previous light element going counterclockwise, and so on until the N light elements have been illuminated from the plurality of N incidence angles (θxi, θyi), i=1 to N. In another embodiment, the illumination instructions determine another order of illuminating light elements in the form of a two-dimensional matrix of light elements. In this embodiment, the variable illumination instructions may define a light element in the matrix that is closest to the specimen. The illumination instructions may then instruct to illuminate the light element closest to the specimen, and then illuminate the light element next closest to the specimen, and then illuminate the light element next closest, and so on until the N light elements have been illuminated from the plurality of N incidence angles (θxi, θyi), i=1 to N.
In certain embodiments, the FPI device can image at least a portion of specimen 20 provided to the FPI device for imaging. In certain cases, the specimen 20 may comprise one or more objects. Each object may be a biological or inorganic entity. Examples of biological entities include whole cells, cell components, microorganisms such as bacteria or viruses, cell components such as proteins, thin tissue sections, etc. In some cases, the specimen 20 may be provided to the FPI device in a medium such as a liquid. In most cases, the specimen 20 is a stationary specimen. The specimen 20 is provided to the FPI device at a location capable of receiving illumination from the variable illuminator and so that light issuing from the specimen 20 is received by the optical element.
In certain embodiments, the FPI system 10 may comprise a receptacle for the specimen 20 with a specimen surface 126 for receiving a specimen 20. The specimen surface 126 may be part of a component of the FPI device 100, such as, for example, a surface of the variable illuminator 110. Alternatively, the specimen surface 126 may be a separate component from the FPI device 100 and/or FPI system 10. For example, the specimen surface 126 may a surface of a slide or a dish. This receptacle and specimen surface 126 may not be included in other embodiments.
In certain embodiments, one or more of the full field-of-view low-resolution images captured by the FPI device may be divided into one or more low-resolution tile images. In these cases, the processor can computationally reconstruct a high-resolution image for each tile independently, and then combine the tile images to generate the full field-of-view high-resolution image. This capability of independent processing of the tile images allows for parallel computing. In these embodiments, each tile may be represented by a two-dimensional area. The FPI system 10 uses an FPI method that assumes plane wave illumination over the area of each tile. In rectilinear spatial coordinates, each tile may be represented as a rectangular area (e.g., square area). In polar spatial coordinates, each tile may be a circular area or an oval area. In rectilinear spatial coordinates, the full field-of view low resolution image may be divided up into a two-dimensional matrix of tiles. In some embodiments, the dimensions of a two-dimensional square matrix of tiles may be in powers of two when expressed in number of pixels of the radiation sensor such as, for example, a 256 by 256 matrix, a 64×64 matrix, etc. In most cases, the tiles in the matrix will have approximately the same dimensions.
The FPI device also comprises an optical element that acts a low-pass filter. For example, the optical element may be an objective lens that only accepts light within a range of incidence angles based on its numerical aperture (NA). In many embodiments, the optical element is in the form of a low NA objective lens to provide narrow acceptance angle and high depth of field. In one embodiment, the optical element is a low NA objective lens has a low NA of about 0.08. In another embodiment, the optical element is a low NA objective lens has a low NA in the range between about 0.01 and about 0.1. In an embodiment of certain illustrated examples, the optical element is a 2× objective lens with an NA of about 0.08.
In embodiments that use X-ray radiation, an X-ray optical element may be needed, such as, for example, a grazing incidence mirror or zone plane. In embodiments that use acoustic radiation, a particular optical element may be needed such as, for example, an acoustic lens. In embodiments that use Terahertz radiation, a particular optical element may be needed such as, for example, a Teflon lens. In embodiments that use microwave radiation, a particular optical element may be needed such as, for example, a microwave lens antenna.
In certain embodiments, the FPI device has an initial depth of focus associated with the inherent depth of field of its optical element. A specimen provided to an FPI device of embodiments may be considered in focus when the sample plane is within the initial depth of focus of the optical element. Conversely, the specimen may be considered out-of-focus when the sample plane 124 is located outside of the initial depth of focus. Using an FPI method with digital refocusing of embodiments, the depth of focus of the FPI device may be extended beyond the limitations of the inherent depth of field of its optical element.
A radiation detector can refer to a device that can sense intensity of the radiation incident upon the radiation detector and can record spatial images based on the intensity pattern of the incident radiation. The radiation detector may record the images during a measurement process with a duration that includes at least the plurality of N sample times, ti=1 to N. For an FPI device using visible light radiation, the radiation detector 140 may be, for example, in the form of a charge coupled device (CCD), a CMOS imaging sensor, an avalanche photo-diode (APD) array, a photo-diode (PD) array, or a photomultiplier tube (PMT) array. For an FPI device using THz radiation, the radiation detector may be, for example, an imaging bolometer. For an FPI device using microwave radiation, the radiation detector may be, for example, an antenna. For an FPI device using X-ray radiation, the radiation detector may be, for example, an x-ray sensitive CCD. For an FPI device using acoustic radiation, the radiation detector may be, for example, a piezoelectric transducer array. These radiation detectors and others are commercially available. In certain color imaging embodiments, the radiation detector may be a color detector e.g. an RGB detector. In other color imaging embodiments, the radiation detector need not be a color detector. In certain embodiments, the radiation detector may be a monochromatic detector.
A sample time can refer to a time that the radiation detector can capture a low-resolution image. In many embodiments, each sample time ti, and associated captured low-resolution intensity image correspond to a particular incidence angle (θxi, θyi). The radiation detector may capture any suitable number N (e.g., 10, 20, 30, 50, 100, 1000, 10000, etc.) of low-resolution intensity images. The radiation detector may have a sampling rate or may have different sampling rates at which the radiation detector samples data. In some cases, sampling may be at a constant rate. In other cases, sampling may be at a variable rate. Some suitable examples of sample rates range from 0.1 to 1000 frames per second.
The radiation detector may have discrete radiation detecting elements (e.g., pixels). The radiation detecting elements may be of any suitable size (e.g., 1-10 microns) and any suitable shape (e.g., circular, rectangular, square, etc.). For example, a CMOS or CCD element may be 1-10 microns and an APD or PMT light detecting element may be as large as 1-4 mm. In one embodiment, the radiation detecting element is a square pixel having a size of 5.5 um.
The radiation detector can determine intensity image data related to captured low-resolution images. For example, the image data may include an intensity distribution. Image data may also include the sample time that the light was captured, or other information related to the intensity image.
Fourier space can refer to the mathematical space spanned by wavevectors kx and ky, being the coordinate space in which the two-dimensional Fourier transforms of the spatial images created by the FPI reside. Fourier space also can refer to the mathematical space spanned by wavevectors kx and ky in which the two-dimensional Fourier transforms of the spatial images collected by the radiation sensor reside.
Each of the low-resolution images captured by the radiation detector is associated with a region in Fourier space. This region in Fourier space can be defined by an approximated optical transfer function of the optical element and also by the incidence angle. If the optical element is an objective lens, for example, the low-resolution image in Fourier space may be the circular region defined by the approximated optical transfer function of the objective lens as a circular pupil with a radius of NA*k0, where k0 equals 2π/λ (the wave number in vacuum). In this example, the region is centered about the wave vector (kxi, kyi) associated with the corresponding incidence angle. In this example, the plurality of N low-resolution images are associated with a plurality of N regions centered about the plurality of N incidence angles in Fourier space.
In Fourier space, neighboring regions may share an overlapping area over which they sample the same Fourier domain data. The overlapping area between adjacent regions in Fourier space may be determined based on the values of the corresponding incidence angles. In most embodiments, the N incidence angles are designed so that the neighboring regions in Fourier space overlap by a certain amount of overlapping area. For example, the values of the N incidence angles may be designed to generate a certain amount of overlapping area for faster convergence to a high-resolution solution in the recovery process. In one embodiment, the overlapping area between neighboring regions may have an area that is in the range of 2% to 99.5% of the area of one of the regions. In another embodiment, the overlapping area between neighboring regions may have an area that is in the range of 65% to 75% the area of one of the regions. In another embodiment, the overlapping area between neighboring regions may have an area that is about 65% of the area of one of the regions.
The FPI system 10 of
The processor 210 (e.g., microprocessor) may execute instructions stored on the CRM 220 to perform one or more functions of the FPI system 10. For example, the processor 210 may execute instructions to perform one or more steps of the recovery process of the FPI method. As another example, the processor 210 may execute illumination instructions for illuminating light elements of the variable illuminator. As another example, the processor 210 may execute instructions stored on the CRM 220 to perform one or more other functions of the FPI system 10 such as, for example, 1) interpreting image data from the plurality of low-resolution images, 2) generating a high-resolution image from the image data, and 3) displaying one or more images or other output from the FPI method on the display 230.
The CRM (e.g., memory) 220 can store instructions for performing some of the functions of the FPI system 10. The instructions are executable by the processor 210 or other processing components of the FPI system 10. The CRM 220 can also store the low-resolution and high-resolution image, and other data produced by the FPI system 10.
The FPI system 10 also includes a display 230 in electronic communication with the processor 210 to receive data (e.g., image data) and provide output data (e.g., images) to an operator of the FPI system 10. The image display 230 may be a color display or a black and white display. In addition, the display 230 may be a two-dimensional display or a three-dimensional display. In one embodiment, the display 230 may be capable of displaying multiple views.
Modifications, additions, or omissions may be made to FPI system 10 or FPI device 100 without departing from the scope of the disclosure. In addition, the components of FPI system 10 or the FPI device 100 may be integrated or separated according to particular needs. For example, the computing device 200 or components thereof may be integrated into the FPI device 100. In some embodiments, the processor 210 or other suitable processor may be part of the FPI device 100. In some cases, the processor 210 may be integrated into the radiation detector 140 so that the radiation detector 140 performs the functions of the processor 210. As another example, the CRM 220 and/or display 230 may be omitted from the FPI system 10 in certain cases.
II. FPI Device Configurations
In certain embodiments, FPI devices (e.g., FPI device 100(a) of
FPI device 100(a) further comprises an optical element 130(a) (e.g., objective lens) and a radiation detector 140(a) having a sensing surface 142. Although radiation detector 140 is shown at a distance away from optical element 130(a), radiation detector 140 may optionally be located at the optical element 130(a). The FPI device 100(a) also includes an in-focus plane 122 at z=0 and a sample plane 124 at z=z0. The FPI device 100(a) includes an x-axis and a y-axis (not shown) at the in-focus plane 122, and a z-axis orthogonal to the in-focus plane 122. The FPI device 100(a) also includes a distance d between the variable illuminator 110(a) and the sample plane 124. In the illustrated example, specimen 20 has been located at a specimen surface 126 for imaging. In other embodiments, specimen 20 may be in other locations for imaging purposes.
In
In certain embodiments, components of an FPI system 10 may be in modular form to be placed in communication with components of a conventional microscope or other conventional imaging device to transform the conventional device into an FPI system 10.
In
The FPI device 110(a) does not require a scanning mechanism for variable illumination. Other embodiments may include a scanning mechanism. For example, the FPI device 110(b) in
FPI device 100(b) further comprises an optical element 130(b) and a radiation detector 140(b) having a sensing surface 142. Although radiation detector 140(b) is shown at a distance away from optical element 130(b), radiation detector 140(b) may optionally be located at the optical element 130(b). The FPI device 100(b) also includes an in-focus plane 122 at z=0 and a sample plane 124 at z=z0. The FPI device 100(b) includes an x-axis and a y-axis (not shown) at the in-focus plane 122, and a z-axis orthogonal to the in-focus plane 122. The FPI device 100(b) also includes a distance d between the variable illuminator 110(b) and the sample plane 124. In the illustrated example, specimen 20 has been located at a specimen surface 126 for imaging. In other embodiments, specimen 20 may be in other locations for imaging purposes.
In
In
Although radiation detector 140(c) is shown at a distance away from optical element 130(c), radiation detector 140(c) may optionally be located at the optical element 130(c). The FPI device 100(c) also includes an in-focus plane 122 at z=0 and a sample plane 124 at z=z0. The FPI device 100(c) includes an x-axis and a y-axis (not shown) at the in-focus plane 122, and a z-axis orthogonal to the in-focus plane 122. The FPI device 100(c) also includes a distance d between the variable illuminator 110(c) and the sample plane 124.
In
In
III. Exemplary FPI Methods
In embodiments, an FPI method comprises a measurement process, a recovery process, and an optional display process. In the measurement process, the specimen is illuminated from a plurality of incidence angles using the variable illuminator, the optical element filters light issuing from the specimen, and the radiation detector captures a plurality of low resolution intensity images based on the filtered light. In the recovery process, the intensity of each low resolution image obtained by inverse Fourier transformation and filtering of the high resolution reconstruction in Fourier space is replaced by the low-resolution intensity measurement and the corresponding region of the high-resolution reconstruction in Fourier space is iteratively updated. In the recovery process, the high-resolution image of the specimen may be computationally reconstructed based on a plurality of N low-resolution intensity images. In the optional display process, images and other output is provided to a display 220.
In certain embodiments, the FPI method may alternate between two working domains: the spatial (x-y) domain and Fourier (kx-ky) domain, where k represents the wavenumber.
In certain embodiments, the FPI method may use the concept of oblique incidence, which provides that illuminating the specimen 20 (e.g., a thin specimen) by an oblique plane wave with a wavevector (kxi, kyi) is equivalent to shifting the center of the image spectrum by (kx,ky) in the Fourier domain. According to oblique incidence, the low-resolution image in the Fourier domain is shifted from normal incidence by (kx, ky), which corresponds to the incidence angle applied by the variable illuminator.
In certain embodiments, the FPI method may provide that the filtering function of the optical element (i.e. the coherent optical transfer function) in Fourier space is a circular pupil with a radius of NA×k0, where k0=2π/λ is the wave number in vacuum. That is, the FPI method may update in Fourier space circular regions defined by this filtering function of the optical element. In these embodiments, the FPI method uses this filtering function to omit image data outside this region.
In certain embodiments, the specimen 20 may be placed at the sample plane 124 at z=z0 where the in-focus plane 122 of the optical element is located at position z=0. In other words, the image captured is not the specimen image at the specimen profile itself; it is the specimen profile propagated by −z0 distance at the in-focus plane of the optical element. In these embodiments, the FPI method may digitally re-focus the specimen 20 propagating the image data by the z0 distance back to the sample plane 124, without having to mechanically move the specimen in the z-direction. These propagating step(s) can be performed by multiplying a phase factor in Fourier space. These steps can extend the imaging depth of focus of the FPI system 10 and correct the chromatic aberration of the optical element.
At step 1100, the variable illuminator provides illumination to a specimen area from a plurality of N incidence angles, (θxi, θyi), i=1 . . . N, at N sample times. In most cases, the recovery process assumes plane wave illumination. The variable illuminator may provide illumination according to illumination instructions that define the order of the illumination angles. The wave vector in x and y directions can be denoted as kxi and kyi. In certain cases, the variable illuminator may provide illumination of different wavelengths at different sample times. For example, the variable illuminator 110 may provide RGB illumination of three wavelengths λ1, λ2, and λ3 corresponding to red, green, blue colors, respectively, for a color imaging embodiment.
At step 1200, the optical element (e.g., a low-NA microscope objective lens) filters light it receives. For example, the optical element can filter light issuing from the specimen 20. The optical element may be an objective lens that filters light by accepting light within a range of incidence angles according to its numerical aperture (NA). In some cases, the optical element may be a low-NA objective lens (e.g., a 2×, 0.08 NA objective lens) of a conventional microscope.
At step 1300, the radiation detector receives a projection of the filtered light from the optical element and captures a snapshot intensity distribution measurement at each of the N sample times, ti=1 to N to acquire a plurality of N low-resolution intensity images. Each low resolution intensity image sampled by the radiation detector is associated with a region in Fourier space. In many embodiments, the variable illuminator provides illumination from certain incidence angles at step 1100 to generate overlapping areas between neighboring regions in Fourier space. In one embodiment, the variable illuminator provides illumination to provide an overlapping area between neighboring regions of 2% to 99.5% of the area of one of the regions. In another embodiment, the variable illuminator provides illumination to provide an overlapping area between neighboring regions of 65% to 75% of the area of one of the regions. In one embodiment, the variable illuminator provides illumination to provide an overlapping area between neighboring regions of about 65% of the area of one of the regions.
In steps 1400 and 1500, a high-resolution image of the specimen area may be computationally reconstructed from the plurality of N low-resolution intensity distribution measurements, Ilm(kix, kyi) (indexed by their illumination wavevector, kxi, kyi, with i=1, 2, . . . N) captured at step 1300.
At step 1400, a high-resolution image: √{square root over (Ih)}eiφ
At step 1500, the high-resolution image of the specimen area is computationally reconstructed by iteratively combining low-resolution intensity measurements in Fourier space using the processor 210 of the FPI system 10.
At optional step 1600, the display 230 may receive image data such as the high-resolution image data to √{square root over (Ih)}eiφ
At step 1510, the processor 210 performs low-pass filtering of the high-resolution image √{square root over (Ih)}eiφ
At optional step 1520, using the processor 210, the low-resolution image, √{square root over (Il)}eiφ
At step 1530, using the processor 210, the computed amplitude component √{square root over (Ilf)} of the low-resolution image at the in-focus plane, √{square root over (Ilf)}eiφ
At optional step 1540, using the processor 210, the updated low-resolution image √{square root over (Ilfm)}eiφ
At step 1550, using the processor 210, a Fourier transform is applied to the updated target image propagated to the sample plane: √{square root over (Ils)}eiφ
At step 1560, the processor 210 determines whether steps 1510 through 1560 have been completed for all incidence angles. If steps 1510 through 1560 have not been completed for all incidence angles, steps 1510 through 1560 are repeated for the next incidence angle.
In most embodiments, the neighboring regions in Fourier space, which are iteratively updated for each incidence angle, overlap each other. In the overlapping area between updated overlapping regions, the FPI system 10 has multiple samplings over the same Fourier space. The incidence angles determine the area of the overlapping area. In one embodiment, the overlapping area between neighboring regions may have an area that is between 2% to 99.5% of the area of one of the neighboring regions. In another embodiment, the overlapping area between neighboring regions may have an area that is between 65% to 75% of the area of one of the neighboring regions. In another embodiment, the overlapping area between neighboring regions may have an area that is about 65% of the area of one of the neighboring regions. In certain embodiments, each overlapping region has the same area.
At step 1570, the processor 210 determines whether the high-resolution solution has converged (step 1570). For example, the processor 210 may determine whether the high-resolution solution may have converged to a self-consistent solution. In one case, the processor 210 compares the previous high-resolution solution of the previous iteration or initial guess to the present high-resolution solution, and if the difference is less than a certain value, the solution may have converged to a self-consistent solution. If processor 210 determines that the solution has not converged, then steps 1510 through 1570 are repeated. In one embodiment, steps 1510 through 1560 are repeated once. In other embodiments, steps 1510 through 1560 are repeated twice or more. If the solution has converged, the processor 210 transforms the converged solution in Fourier space to the spatial domain to recover a high-resolution image √{square root over (Ih)}eiφ
In
In embodiments, the FPI method can include digital refocusing by including optional steps 1520 and 1540. The digital refocusing feature of steps 1520 and 1540 propagates the image from the in-focus plane 122 z=0 to the sample plane 124 at z=z0. Digital refocusing may be needed when the specimen 20 is located at the sample plane 124 at z=z0, while the in-focus plane 122 of the optical element (e.g., objective lens) is located at position z=0. In other words, digital refocusing may be needed when the specimen 20 is out-of-focus by the amount of z0.
In one embodiment, an FPI system 10 with the FPI device 100(a) shown in
In one embodiment, an FPI system 10 with the FPI device 100(a) shown in
A. FPI Methods with Tile Imaging
In some embodiments, an FPI method with tile imaging divides the captured low-resolution images into a plurality of low-resolution tile images, independently acquires a high-resolution image for each of the tiles, and then combines the high-resolution tile images to generate a full field-of-view high-resolution image. In some cases, the high-resolution tile images may be combined with an image blending process. An example of an image blending process is alpha blending which can be found in PCT publication WO1999053469, entitled “A system and method for performing blending using an over sampled buffer,” filed on Apr. 7, 1999, which is hereby incorporated by reference in its entirety. Since the high-resolution images of the tiles may be acquired independently, this FPI method may allow for parallel computing, which may reduce computation time, and may also reduce memory requirements. Moreover, the light from each light element may be accurately treated as a plane wave for each tile. The incident wavevector for each tile can be expressed as:
where (xc,yc) is the central position of each tile of the full field-of-view low-resolution image, (xi,yi) is the position of the ith light element, and h is the distance between the illuminator and the specimen. Furthermore, this FPI method can assign a specific aberration-correcting pupil function to each tile in some cases.
In one embodiment, an FPI system 10 comprises a computing device 200 in the form of a personal computer having a processor 210 in the form of an Intel i7 CPU. In one experiment, this FPI system 10 performed an FPI method with tile imaging that divides the low-resolution images into tiles of 196×196 pixels. In one case, the processing time for each small image (converting 196 by 196 raw pixels to 1960 by 1960 pixels) is about 3.75 seconds using Matlab. In this case, the processing time for creating the full field-of-view gigapixel image is about 10 minutes when parallel computing using all 4 cores of the CPU. In other embodiments, an FPI system 10 can have a processor in the form of a GPU unit, which may reduce processing time by ten-fold. In other embodiments, the processor 210 may perform steps of the FPI method using another programming language, such as C++, which may reduce processing time. An example of this FPI system can be found in Zheng, G., Horstmeyer, R., and Yang, C., “Wide-field, high-resolution Fourier ptychographic microscopy,” Nature Photonics (July, 2013), which is hereby incorporated by reference in its entirety.
At step 1350, the processor 210 divides the full field-of-view into a plurality of tiles such as, for example, a two-dimensional matrix of tiles. The dimensions of a two-dimensional square matrix of tiles may be in powers of two such as, for example, a 256 by 256 matrix, a 64×64 matrix, etc. In one example, the processor may divide up a full field of view of 5,280×4,380 pixels into tiles having an area of 150×150 pixels.
Next, the processor 210 initializes the high-resolution image: √{square root over (Ih)}eiφ
At step 2500(1) . . . step 2500(M), the processor 210 computationally reconstructs a high-resolution image of each tile (1 to M) independently using parallel computing. The processor 210 computationally reconstructs the high-resolution image of each tile by iteratively combining low-resolution intensity images in Fourier space as described with reference to steps 1510, 1530, 1550, 1560, and 1570 shown in
At step 2590, the processor 210 combines the high-resolution tile images into a full field-of view high-resolution image. In some cases, combining tile images comprises an imaging-blending process such as, for example, alpha blending.
At optional step 1600, the image data of the recovered high-resolution two-dimensional image of the specimen area is displayed on the display 230.
In one embodiment, the FPI method with tile imaging may further comprise a procedure that accounts for differences in incident angles between different tiles based on the distance between the tiles and each light element.
B. FPI Methods with Digital Refocusing
Another limitation of conventional high-NA microscopes is the limited depth-of field. As an example, the depth-of-field of a conventional microscope with a 20× objective lens with 0.4 NA is about 5 μm. With a conventional microscope, the resolution degrades as the specimen moves away from the in-focus plane 122 due to its limited depth-of-field. To achieve optimal resolution using a conventional microscope, the operator needs to move the stage to mechanically bring the specimen back into focus. In this regard, a precise mechanical stage is needed in the conventional microscope to bring a specimen into the in-focus position with sub-micron accuracy.
In certain embodiments, an FPI system 10 implements an FPI method in which a specimen can be refocused digitally rather than mechanically. In these cases, the FPI method computationally refocuses the out-of-focus specimen 20 during the recovery process. Using digital refocusing, the FPI system 10 can expand its depth-of focus beyond the physical limitations of its optical element.
Digital refocusing with FPI methods represents another significant improvement over conventional microscopes. Digital refocusing can correct: 1) cases where specimens are not perfectly aligned over the entire field-of-view and 2) for chromatic aberration digitally (assigning different defocused distances for different colors).
C. FPI Methods with Digital Auto-Focusing
During operation of an FPI system 10, the z-position of the specimen 20 may not be known a priori. In certain embodiments, an FPI method may include a digital auto-focusing step that determines the z-position of the specimen 20 and uses this z-position to digitally refocus. For example, the FPI method of
Auto-focusing index: 1/Σabs(√{square root over (Ilf)}−√{square root over (Ilfm)}) (Eqn. 2)
Where: √{square root over (Ilf)} is the amplitude image from the low-pass filtering, and
√{square root over (Ilfm)} is the actual low-resolution measurement
The summation in Eqn. 2 is for all oblique incidence angles. After the FPI method computes the estimated z-position of the specimen 20, the FPI method can digitally refocus to the estimated z-position. In some cases, the high-resolution image solution has been found to converge better when using an accurate z-position.
D. FPI Methods with Digital Wavefront Correction
Although FPI methods do not require phase information as input, FPI methods of certain embodiments accommodate phase during iterative reconstruction. In these embodiments, the depth of focus can be extended beyond that of the objective lens using a numerical strategy to compensate for aberrations in the pupil function. Examples of such aberration compensation strategies can be found in Gutzler, T., Hillman, T. R., Alexandrov, S. A. and Sampson, D. D., “Coherent aperture-synthesis, wide-field, high-resolution holographic microscopy of biological tissue,” Opt. Lett. 35, pp. 1136-1138 (2010), and Colomb, T. et al., “Automatic procedure for aberration compensation in digital holographic microscopy and applications to specimen shape compensation,” Appl. Opt. 45, 851-863 (2006), which are hereby incorporated by reference in their entirety. The digital correction process digitally introduces a phase map to the coherent optical transfer function to compensate for aberrations at the pupil plane during the iterative image recovery process.
ei·φ(k
where kx and ky are the wavenumbers at the pupil plane, z0 is the defocus distance, and NA is the numerical aperture of the optical element.
At step 1605, the processor 210 multiplies by a phase factor ei·φ(k
At step 1610, the processor 210 performs low-pass filtering of the high-resolution image √{square root over (Ih)}eiφ
At step 1630, using the processor 210, the computed amplitude component √{square root over (Ilf)} of the low-resolution image at the in-focus plane, √{square root over (Ilf)}eiφ
At step 1645, the processor 210 multiplies by an inverse phase factor e−i·φ(k
At step 1650, using the processor 210, a Fourier transform is applied to the updated target image propagated to the sample plane: √{square root over (Ils)}eiφ
At step 1660, the processor 210 determines whether steps 1605 through 1650 have been completed for all incidence angles. If steps 1605 through 1650 have not been completed for all incidence angles, steps 1605 through 1650 are repeated for the next incidence angle.
In most embodiments, the neighboring regions in Fourier space, which are iteratively updated for each incidence angle, overlap each other. In the overlapping area between updated overlapping regions, the FPI system 10 has multiple samplings over the same Fourier space. The incidence angles determine the area of the overlapping area. In one embodiment, the overlapping area between neighboring regions may have an area that is between 2% to 99.5% of the area of one of the neighboring regions. In another embodiment, the overlapping area between neighboring regions may have an area that is between 65% to 75% of the area of one of the neighboring regions. In another embodiment, the overlapping area between neighboring regions may have an area that is about 65% of the area of one of the neighboring regions. In certain embodiments, each overlapping region has the same area.
At step 1670, the processor 210 determines whether the high-resolution solution has converged. For example, the processor 210 may determine whether the high-resolution solution may have converged to a self-consistent solution. In one case, the processor 210 compares the previous high-resolution solution of the previous iteration or initial guess to the present high-resolution solution, and if the difference is less than a certain value, the solution may have converged to a self-consistent solution. If processor 210 determines that the solution has not converged, then steps 1605 through 1670 are repeated. In one embodiment, steps 1605 through 1670 are repeated once. In other embodiments, steps 1605 through 1670 are repeated twice or more. If the solution has converged, the processor 210 transforms the converged solution in Fourier space to the spatial domain to recover a high-resolution image √{square root over (Ih)}eiφ
If the defocus distance is unknown, the FPI method can digitally adjust the ‘z’ parameter to different values based on a computation of the auto-focusing index from Eqn. 4. The FPI method can then reconstruct the corresponding images, and select the sharpest image. This approach can also be extended to image a tiled sample. In this case, the FPI method can digitally adjust the ‘z’ parameter to achieve acuity for each tiled region of the whole image and combine the in-focus regions to form a fully focused image of the full field of view.
In other embodiments, alternative digital multiplicative phase factors can be included in multiplication steps 1605 and 1645 to correct for a variety of aberrations, as long as the factors correctly model the employed optics.
E. Estimated Computational Cost of the Recovery Process
The computational cost of the recovery process of the illustrated example of
IV. Exemplary FPI systems with Color Imaging and/or Phase Imaging
A. Color Imaging
Color imaging capability is pivotal in pathology and histology. In certain embodiments, an FPI system 10 capable of color imaging comprises a variable illuminator with an LED having light elements that can provide red, green, and blue illuminations. The FPI method combines the high-resolution image results from red, green, and blue LED illumination into each corresponding color channel to form a final high-resolution color image. Three images are generated corresponding to red, green, and blue, which are combined to form a high resolution color image.
To demonstrate an implementation of this FPI system 10 for digital pathology applications, this color imaging system was used to acquire color high-resolution images of a pathology slide (human adenocarcinoma of breast section, Carolina).
B. Phase Imaging
Phase imaging is useful in numerous applications. For example, phase profile of a histopathology slide may contain information about the molecular scale organization of tissue and can be used as an intrinsic marker for cancer diagnosis as discussed in Wang, Z., Tangella, K., Balla, A. and Popescu, G., “Tissue refractive index as marker of disease,” Journal of Biomedical Optics 16, 116017-116017 (2011), which is hereby incorporated by reference in its entirety. As another example, phase information of a sample can also be used to measure the optical path length of cells or organelles to determine the growth pattern of cell cultures or to perform blood screening, as discussed in Lue, N. et al., “Live Cell Refractometry Using Hilbert Phase Microscopy and Confocal Reflectance Microscopy,” The Journal of Physical Chemistry A, 113, pp. 13327-13330 (2009), Mir, M. et al., “Optical measurement of cycle-dependent cell growth,” Proceedings of the National Academy of Sciences 108, pp. 13124-13129 (2011), and Mir, M. et al., “Blood screening using diffraction phase cytometry,” Journal of Biomedical Optics 15, pp. 027016-027014 (2010), which are hereby incorporated by reference in their entirety.
Most conventional full-field phase imaging techniques use interferometry, which requires sophisticated and well-designed optical alignments. Compared to these conventional techniques, the FPI methods of embodiments are easy and cost-effective solution for researchers and clinicians to incorporate phase imaging functionality into their current microscope systems. The Figures in
V. Further Simulation Results
Some results of numerical simulations of FPI systems performing FPI methods, according to embodiments of the invention, are provided in this section and elsewhere in the disclosure. In many embodiments, the FPI systems use FPI methods to reconstruct a high-resolution image with the use of a low NA lens.
As an example,
VI. Throughput Enhancement Factor
Based on the pixel size ratio between the largest pixel sizes of the raw low-resolution images and the largest pixel sizes of the high-resolution images, an enhancement factor of using an FPI system can be expressed as:
Enhancement factor=2·NAsyn/NAobj (Eqn. 3)
The larger the Enhancement factor, the higher the system throughput.
In certain embodiments, the FPI system 10 may have a certain sampling condition to provide a particular enhancement factor. Generally, the sampling condition may be associated with two questions: 1) given the NA of an objective lens, what is the largest pixel size that can be used for acquiring the low resolution intensity images; 2) given the synthetic NA of the reconstructed image, what is the largest pixel size that can be used for representing the reconstructed intensity image. Since the FPI method of embodiments can recover both intensity and phase information, the answer to question 1 is the same as the sampling condition for coherent optical systems: λ/(2·NAobj). For question 2, the synthetic NA is for the electric field E (with amplitude and phase). The final reconstruction, on the other hand, is for the intensity profile I (I=E·E*, where ‘*’ denotes for complex conjugate). Such a multiplication of electric field in the spatial domain corresponds to a convolution operation in the Fourier domain. As such, the passband of the reconstructed intensity image double in the Fourier domain. Therefore, the largest pixel size that can be used for representing the reconstructed intensity image is λ/(4·NAobj) at the sample plane.
In certain implementations, an FPI device 100(a) includes an optical element in the form of a 2× objective with 0.08 NA and radiation detector in the form of an image sensor with 5.5 μm pixel size. From the answer to question 2, the largest pixel size that can be used at the image plane is 2·λ/(2·NAobj)=5.88 μm for blue light. The 5.5 μm pixel size of the image sensor is, therefore, in accord with such a sampling condition. On the other hand, based on the answer to question 2, pixel sizes (at the sample plane) of the reconstructed image are 0.34 μm, 0.30 μm and 0.28 μm for red, green and blue wavelengths. For simplicity, a reconstructed pixel size of 0.275 μm was used for these three wavelengths in the implementation, corresponding to an enhancement factor of 10.
VII. Advantages of the FPI system
Embodiments of the invention may provide one or more of the following technical advantages. Some examples of these advantages are provided below.
1. Object Support in Fourier Domain.
In one aspect, FPI methods impose object support constraints in the Fourier domain, which provides large FOV and higher signal-to-noise ratio (with focusing elements) without mechanical scanning. In conventional ptychography systems, object support is provided by the confined illumination in the spatial domain. In these conventional systems, the specimen must be mechanically scanned through the desired field-of-view.
2. No phase measurements
In most embodiments, the FPI methods do not require phase measurements of the specimen. Conventional interfermometric synthetic aperture microscopes require phase measurements in their detection schemes, as can be described in Rodenburg, J. M. and Bates, R. H. T. “The theory of super-resolution electron microscopy via Wigner-distribution deconvolution,” Phil. Trans. R. Soc. Lond. A 339, 521-553 (1992), H. M. L. and Rodenburg, J. M., “Movable aperture lensless transmission microscopy, a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004), Rodenburg, J. M. et al., “Hard-X-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98, 034801 (2007), Thibault, P. et al., “High-resolution scanning X-ray diffraction microscopy,” Science 321, 379-382 (2008), Dierolf, M. et al., “Ptychographic coherent diffractive imaging of weakly scattering specimens,” New J. Phys. 12, 035017 (2010), Maiden, A. M., Rodenburg, J. M. and Humphry, M. J., “Optical ptychography: a practical implementation with useful resolution,” Opt. Lett. 35, 2585-2587 (2010), Humphry, M., Kraus, B., Hurst, A., Maiden, A. and Rodenburg, J., “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nat. Commun. 3, 730 (2012), which are hereby incorporated by reference in their entirety. Because no measured phase information is needed, The FPI system eliminates the design challenges associated with conventional interferometric detection schemes.
3. Spatial-bandwidth product beyond the physical limitations of optical element
One advantage may be that the FPI systems of certain embodiments may increase the spatial-bandwidth product beyond the physical limitations of its optics while avoiding the drawbacks of previous attempts to do so with conventional devices, in a simple and cost-effective manner. These systems use FPI methods to decouple field-of-view from resolution of the optics, which allows these systems to generate a high-resolution image with a low NA lens. Thus, the throughput of these FPI systems is not limited by the spatial-bandwidth product of their optics. In one exemplary embodiment, an FPI system has been implemented that produces a 1.6 gigapixel system with a maximum NA of 0.5, a field-of-view of 120 mm2, and a resolution-invariant imaging depth of 0.3 mm.
4. Scalable Throughput.
As discussed, the resolution and FOV are not coupled in FPI systems of embodiments. In one embodiment, an FPI system can be scaled for a desired throughput by using an objective lens having a particular NA and by using a particular number N of illuminations. For example, the throughput of an FPI system may be increased by reducing the NA of the objective lens and/or by increasing the number N of illuminations. In one embodiment example, an FPI system provides two orders of magnitude higher throughput than existing bright-field microscopes without the use of mechanical scanning and/or phase measurements
5. Modular form that can be Readily Implemented With Conventional Microscope
In certain cases, components of the FPI system may be in modular form to be readily implemented with components of a conventional microscope or other conventional imaging device. These implementations may have the potential to broadly impact digital pathology, hematology, phytotomy, immunochemistry and forensic photography. For example, modular components of an FPI system may include a variable illuminator (e.g., a simple light-emitting diode (LED) matrix) and a processor with instructions for executing the FPI method.
6. No mechanical scanning.
In certain embodiments, FPI methods do not need mechanical scanning. Unlike conventional synthetic-aperture and scanning-based wide FOV microscopy techniques, mechanical scanning is not required for FPI methods of embodiments. As such, it simplifies the platform design, reduces associated costs and allows for a higher throughput limit.
7. Capable of color imaging.
Another advantage may be that the FPI system of certain embodiments can be used to generate color images, which is pivotal in pathology and histology applications. In embodiments, an FPI system may be capable of color imaging by having a variable illuminator that provides illumination of different colors (e.g., color LED matrix). If digital refocusing is implemented, the computationally reconstructed image can remain free of any chromatic aberrations of the optics.
8. Digital Refocusing and Expanded Depth of Focus
To achieve optimal resolution in a conventional microscope, a stage must be used to mechanically bring an out-of-focus specimen into focus. In certain embodiments, FPI methods can computationally refocus the image digitally, rather than mechanically. This aspect is especially useful for cases where specimens are not perfectly aligned over the entire FOV. Using digital refocusing, the FPI system can expand the depth of focus beyond the optical limitations of the optical element. For example, an FPI method of one embodiment can been used to extend the depth of focus of the objective lens of a conventional microscope from about 80 μm to about 0.3 mm. This increased depth of focus can provide the additional advantage of providing a large tolerance to microscope slide placement errors, which improves accuracy over conventional systems.
9. Phase Imaging Capability.
A phase profile of a biological sample contains information about the molecular scale organization of sample and can be used as an intrinsic marker for different applications. The ability to perform phase imaging is also useful for digital image processing, such as cell segmentation and cell counting. Conventional full-field phase imaging techniques require fairly sophisticated and well-designed optical alignments. In certain embodiments, an FPI method provides an easy and cost-effective solution for researchers and clinicians to incorporate phase imaging functionality into their current microscope systems. The FPI method can reconstruct the complex image of the sample. In certain embodiments, the FPI method can generate a phase image by taking the phase angle of the complex image.
10. X-Ray and THz applications
In certain embodiments, FPI methods can be extended to Terahertz and X-ray imaging applications, where lenses are poor and of very limited numerical aperture.
VIII. FPI Systems for X-Ray Imaging
In certain embodiments, an FPI system 10 may be configured for X-ray imaging. In these embodiments, the Fourier ptychographic X-ray imaging system 10 comprises an FPI device with an X-ray radiation source (e.g., X-ray tube and metal target).
In one embodiment, a Fourier ptychographic X-ray imaging system 10 may include the FPI device 100(c) shown in
In another embodiment, a Fourier ptychographic X-ray imaging system 10 may include the FPI device 100(d) shown in
The Fourier ptychographic X-ray imaging system 10 may perform the FPI method described with respect to
By implementing X-ray imaging, these Fourier ptychographic X-ray imaging systems 10 provide additional capabilities. For example, these systems may provide larger penetration depths into the specimen. As another example, X-ray imaging may allow for elemental and chemical identification on a scale of 10 nm or less.
IX. Subsystems
The various components previously described in the Figures may operate using one or more of the subsystems to facilitate the functions described herein. Any of the components in the Figures may use any suitable number of subsystems to facilitate the functions described herein. Examples of such subsystems and/or components are shown in a
In some embodiments, an output device such as the printer 2430 or display 230 of the FPI system 10 can output various forms of data. For example, the FPI system 10 can output 2D color/monochromatic images (intensity and/or phase), data associated with these images, or other data associated with analyses performed by the FPI system 10.
Modifications, additions, or omissions may be made to any of the above-described FPI methods and their associated features, e.g., features described with respect to
It should be understood that the present invention as described above can be implemented in the form of control logic using computer software in a modular or integrated manner. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and/or methods to implement the present invention using hardware and a combination of hardware and software.
Any of the software components or functions described in this application, may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C++ or Perl using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands on a CRM, such as a random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a CD-ROM. Any such CRM may reside on or within a single computational apparatus, and may be present on or within different computational apparatuses within a system or network.
Although the foregoing disclosed embodiments have been described in some detail to facilitate understanding, the described embodiments are to be considered illustrative and not limiting. It will be apparent to one of ordinary skill in the art that certain changes and modifications can be practiced within the scope of the appended claims.
One or more features from any embodiment may be combined with one or more features of any other embodiment without departing from the scope of the disclosure. Further, modifications, additions, or omissions may be made to any embodiment without departing from the scope of the disclosure. The components of any embodiment may be integrated or separated according to particular needs without departing from the scope of the disclosure.
Claims
1. A Fourier ptychographic imaging device, comprising:
- an array of light elements, each light element comprising at least one light source, wherein the array of light elements is configured to provide illumination sequentially from each of a plurality of incidence angles to a specimen being imaged during operation;
- an optical element comprising at least one lens, wherein the optical element is configured to receive light issuing from the specimen being illuminated and is configured to filter the light received by propagating light based on an acceptance angle of the at least one lens;
- a radiation detector configured to receive light from the optical element and to acquire a plurality of variably-illuminated, low-resolution intensity images of the specimen, the radiation detector configured to acquire each low-resolution intensity image based on light issuing directly from the specimen and filtered by the optical element while the specimen is illuminated at one of the plurality of incidence angles; and
- a processor configured to reconstruct a high-resolution image of the specimen by iteratively determining a high-resolution solution, wherein reconstructing the high-resolution image comprises computationally recovering phase image information from the variably-illuminated, low-resolution intensity images, wherein iteratively determining the high-resolution solution comprises:
- (i) using a current high-resolution solution to generate data in each a plurality of regions in the Fourier domain, the plurality of regions associated with the plurality of incidence angles;
- (ii) inverse Fourier transforming the data in a region of the plurality of regions to generate image data, replacing amplitudes of the image data with square root of an intensity measurement from at least one of the plurality of variably-illuminated, low-resolution intensity images corresponding to a location of the region in Fourier space to produce updated image data, Fourier transforming the updated image data to produce Fourier-transformed updated data, and updating data in the region of the Fourier domain with the Fourier-transformed updated data;
- (iii) repeating operation in (ii) for each region of the plurality of regions in the Fourier domain to determine an updated high-resolution solution; and
- (iv) repeating operations in (iii) if a comparison of data derived from the updated high-resolution solution and data derived from the current high-resolution solution is more than a value,
- wherein the high-resolution image reconstructed has higher resolution that the low-resolution intensity images.
2. The Fourier ptychographic imaging device of claim 1, wherein the optical element is an objective that comprises the at least one lens.
3. The Fourier ptychographic imaging device of claim 1, wherein one or more sets of adjacent regions of the plurality of regions in the Fourier domain are overlapping in Fourier space.
4. The Fourier ptychographic imaging device of claim 1, wherein a difference between the two most adjacent incidence angles in the plurality of incidence angles is between 10% and 90% of the acceptance angle corresponding to a numerical aperture of the at least one lens.
5. The Fourier ptychographic imaging device of claim 1, wherein a difference between the two most adjacent incidence angles in the plurality of incidence angles is between 33% and 66% the acceptance angle corresponding to a numerical aperture of the at least one lens.
6. The Fourier ptychographic imaging device of claim 1, wherein a difference between the two most adjacent incidence angles in the plurality of incidence angles is less than 76% of the acceptance angle corresponding to a numerical aperture of the at least one lens.
7. The Fourier ptychographic imaging device of claim 1, wherein the array of light elements is a two-dimensional matrix of light elements, each light element providing illumination from one of the plurality of incidence angles.
8. The Fourier ptychographic imaging device of claim 7, wherein each light element is a set of one or more light-emitting diodes.
9. The Fourier ptychographic imaging device of claim 1, wherein the array of light elements is a hexagonal array of light elements, each light element providing illumination from one of the plurality of incidence angles.
10. The Fourier ptychographic imaging device of claim 9, wherein each light element is a set of one or more light-emitting diodes.
11. The Fourier ptychographic imaging device of claim 3, wherein the one or more sets of adjacent regions overlap by between 20% and 90% in area.
12. The Fourier ptychographic imaging device of claim 3, wherein the one or more sets of adjacent regions overlap by between 2% and 99.5% in area.
13. The Fourier ptychographic imaging device of claim 3, wherein the one or more sets of adjacent regions overlap by between 33% and 66% in area.
14. The Fourier ptychographic imaging device of claim 1, wherein the processor is also configured to automatically refocus the high-resolution image to an in-focus plane of the specimen.
15. The Fourier ptychographic imaging device of claim 1, wherein the processor is part of the radiation detector.
16. The Fourier ptychographic imaging device of claim 1, further comprising a display for displaying the high-resolution image.
17. A method of Fourier ptychographic imaging, comprising:
- causing a specimen to be illuminated from each of a plurality of incidence angles using an array of light elements, each light element comprising at least one light source, wherein light issuing from the specimen is filtered during illumination using an optical element comprising at least one lens, wherein the light is filtered based on an acceptance angle of the at least one lens;
- causing a plurality of variably-illuminated, low-resolution intensity images of the specimen to be acquired using a radiation detector that receives light from the optical element, wherein each low-resolution intensity image is captured based on light issuing from the specimen and filtered by the optical element while the specimen is illuminated at one of the plurality of incidence angles; and
- reconstructing a high-resolution image of the specimen by iteratively determining a high-resolution solution, wherein reconstructing the high-resolution image comprises computationally recovering phase image information from the variably-illuminated, low-resolution intensity images, wherein the high-resolution image reconstructed has higher resolution that the low-resolution intensity images, wherein iteratively determining the high-resolution solution comprises:
- (i) using a current high-resolution solution to generate data in each of a plurality of regions in the Fourier domain, the plurality or regions associated with the plurality of incidence angles;
- (ii) inverse Fourier transforming the data in a region of the plurality of regions to generate image data, replacing amplitudes of the image data with square root of an intensity measurement from at least one of the plurality of variably-illuminated, low-resolution intensity images associated with an illumination angle corresponding to a location of the region in Fourier space to produce updated image data, Fourier transforming the updated image data to produce Fourier-transformed updated data, and updating data in the region of the Fourier domain with the Fourier-transformed updated imago data;
- (iii) repeating operation in (ii) for each region of the plurality of regions in the Fourier domain to determine an updated high-resolution solution; and
- (iv) repeating operations in (iii) if a comparison of data derived from the updated high-resolution solution and data derived from the current high-resolution solution is more than a value.
18. The method of Fourier ptychographic imaging of claim 17, wherein one or more sets of adjacent regions of the plurality of regions in the Fourier domain are overlapping regions in Fourier space.
19. The method of Fourier ptychographic imaging of claim 18, wherein reconstructing the high-resolution image of the specimen comprises:
- dividing each variably-illuminated, low-resolution intensity image into a plurality of variably-illuminated, low-resolution intensity tile images;
- reconstructing a high-resolution tile image for each tile; and
- combining the high-resolution tile images of the tiles.
20. The method of Fourier ptychographic imaging of claim 17, further comprising refocusing the high-resolution image.
21. The method of Fourier ptychographic imaging of claim 17, further comprising automatically focusing the high-resolution image.
22. The method of Fourier ptychographic imaging of claim 17, further comprising inverse Fourier transforming data of the updated high-resolution solution to produce the high-resolution image of the specimen.
23. The method of Fourier ptychographic imaging of claim 17, wherein (ii) further comprises multiplying the data in each of the plurality of regions with a phase factor.
24. A method of Fourier ptychographic imaging, comprising:
- receiving a plurality of variably-illuminated, low-resolution intensity images of a specimen from an image sensor, wherein each low-resolution intensity image is captured based on light issuing from the specimen and filtered by an optical element while the specimen is illuminated sequentially at each respective one of a plurality of incidence angles; and
- reconstructing a high-resolution image of the specimen by iteratively determining a high-resolution solution, wherein reconstructing the high-resolution image comprises computationally recovering phase image information from the variably-illuminated, low-resolution intensity images, wherein the high-resolution image reconstructed has higher resolution that the low-resolution intensity images, wherein iteratively determining the high-resolution solution comprises: (i) using a current high-resolution solution to generate data in each of a plurality of regions in the Fourier domain, the plurality of regions associated with the plurality of incidence angles; (ii) inverse Fourier transforming the data a region of the plurality of regions to generate image data, replacing amplitudes of the image data with square root an intensity measurement from at least one of the plurality of variably-illuminated, low-resolution intensity images associated with an illumination angle corresponding to a location of the region in Fourier space to produce updated image data, Fourier transforming the updated image data to produce Fourier-transformed updated data, and updating data the region of the Fourier domain with the Fourier-transformed updated data; (iii) repeating operation in (ii) for each region of the plurality of regions in the Fourier domain to determine an updated high-resolution solution; and (iv) repeating operations in (iii) if a comparison of data derived from the updated high-resolution solution and data derived from the current high-resolution solution is more than a value.
25. The method of Fourier ptychographic imaging of claim 24, one or more sets of adjacent regions of the plurality of regions in the Fourier domain are overlapping in Fourier space.
26. The method of Fourier ptychographic imaging of claim 25, wherein reconstructing the high-resolution image of the specimen comprises:
- dividing each variably-illuminated, low-resolution intensity image into a plurality of variably-illuminated, low-resolution intensity tile images;
- reconstructing a high-resolution tile image for each tile; and
- combining the high-resolution tile images of the tiles.
27. The method of Fourier ptychographic imaging of claim 24, further comprising inverse Fourier transforming data of the updated high-resolution solution to produce the high-resolution image of the specimen.
28. The method of Fourier ptychographic imaging of claim 24, wherein (i) further comprises multiplying the data in each of the plurality of regions with a phase factor.
29. The method of Fourier ptychographic imaging of claim 25, wherein each region corresponds to an approximate optical transfer function of the optical element.
30. The Fourier ptychographic imaging device of claim 1, wherein (i) further comprises multiplying the data in each of the plurality of regions with a phase factor.
31. The Fourier ptychographic imaging device of claim 1, wherein iteratively determining the high-resolution solution further comprises inverse Fourier transforming data of the updated high-resolution solution to produce the high-resolution image of the specimen.
| 5475527 | December 12, 1995 | Hackel et al. |
| 6144365 | November 7, 2000 | Young et al. |
| 6154196 | November 28, 2000 | Fleck et al. |
| 6320174 | November 20, 2001 | Tafas et al. |
| 6320648 | November 20, 2001 | Brueck et al. |
| 6747781 | June 8, 2004 | Trisnadi |
| 6759949 | July 6, 2004 | Miyahara |
| 6905838 | June 14, 2005 | Bittner |
| 7436503 | October 14, 2008 | Chen et al. |
| 7460248 | December 2, 2008 | Kurtz et al. |
| 7706419 | April 27, 2010 | Wang et al. |
| 7738095 | June 15, 2010 | Gardner, Jr. et al. |
| 7787588 | August 31, 2010 | Yun et al. |
| 8271251 | September 18, 2012 | Schwartz et al. |
| 8313031 | November 20, 2012 | Vinogradov |
| 8497934 | July 30, 2013 | Milnes et al. |
| 8624968 | January 7, 2014 | Hersee et al. |
| 8654201 | February 18, 2014 | Lim et al. |
| 8942449 | January 27, 2015 | Maiden |
| 9029745 | May 12, 2015 | Maiden |
| 9426455 | August 23, 2016 | Horstmeyer et al. |
| 9497379 | November 15, 2016 | Ou et al. |
| 9829695 | November 28, 2017 | Kim et al. |
| 9864184 | January 9, 2018 | Ou et al. |
| 9892812 | February 13, 2018 | Zheng et al. |
| 9983397 | May 29, 2018 | Horstmeyer et al. |
| 9993149 | June 12, 2018 | Chung et al. |
| 9998658 | June 12, 2018 | Ou et al. |
| 10162161 | December 25, 2018 | Horstmeyer et al. |
| 10168525 | January 1, 2019 | Kim et al. |
| 10222605 | March 5, 2019 | Kim et al. |
| 10228550 | March 12, 2019 | Ou et al. |
| 10401609 | September 3, 2019 | Ou et al. |
| 10419665 | September 17, 2019 | Ou et al. |
| 10568507 | February 25, 2020 | Chung et al. |
| 10606055 | March 31, 2020 | Horstmeyer et al. |
| 10652444 | May 12, 2020 | Horstmeyer et al. |
| 20010055062 | December 27, 2001 | Shioda et al. |
| 20020141051 | October 3, 2002 | Vogt et al. |
| 20030116436 | June 26, 2003 | Amirkhanian et al. |
| 20030118223 | June 26, 2003 | Rahn et al. |
| 20040057094 | March 25, 2004 | Olszak et al. |
| 20040146196 | July 29, 2004 | Van Heel |
| 20040190762 | September 30, 2004 | Dowski, Jr. et al. |
| 20050211912 | September 29, 2005 | Fox |
| 20060098293 | May 11, 2006 | Garoutte et al. |
| 20060158754 | July 20, 2006 | Tsukagoshi et al. |
| 20060173313 | August 3, 2006 | Liu et al. |
| 20060291707 | December 28, 2006 | Kothapalli et al. |
| 20070057184 | March 15, 2007 | Uto et al. |
| 20070133113 | June 14, 2007 | Minabe |
| 20070159639 | July 12, 2007 | Teramura et al. |
| 20070171430 | July 26, 2007 | Tearney et al. |
| 20070189436 | August 16, 2007 | Goto et al. |
| 20070206200 | September 6, 2007 | Lindner et al. |
| 20070269826 | November 22, 2007 | Geddes |
| 20080101664 | May 1, 2008 | Perez |
| 20080182336 | July 31, 2008 | Zhuang et al. |
| 20090046164 | February 19, 2009 | Shroff et al. |
| 20090079987 | March 26, 2009 | Ben-Ezra et al. |
| 20090125242 | May 14, 2009 | Choi et al. |
| 20090284831 | November 19, 2009 | Schuster et al. |
| 20090316141 | December 24, 2009 | Feldkhun |
| 20100135547 | June 3, 2010 | Lee et al. |
| 20100271705 | October 28, 2010 | Hung |
| 20110075928 | March 31, 2011 | Jeong et al. |
| 20110181869 | July 28, 2011 | Yamaguchi et al. |
| 20110192976 | August 11, 2011 | Own et al. |
| 20110235863 | September 29, 2011 | Maiden |
| 20110255163 | October 20, 2011 | Merrill et al. |
| 20120069344 | March 22, 2012 | Liu |
| 20120099803 | April 26, 2012 | Ozcan et al. |
| 20120105618 | May 3, 2012 | Brueck et al. |
| 20120118967 | May 17, 2012 | Gerst |
| 20120157160 | June 21, 2012 | Ozcan et al. |
| 20120176673 | July 12, 2012 | Cooper |
| 20120182541 | July 19, 2012 | Canham |
| 20120218379 | August 30, 2012 | Ozcan et al. |
| 20120248292 | October 4, 2012 | Ozcan et al. |
| 20120250032 | October 4, 2012 | Wilde et al. |
| 20120281929 | November 8, 2012 | Brand et al. |
| 20130057748 | March 7, 2013 | Duparre et al. |
| 20130083886 | April 4, 2013 | Carmi et al. |
| 20130093871 | April 18, 2013 | Nowatzyk et al. |
| 20130094077 | April 18, 2013 | Brueck |
| 20130100525 | April 25, 2013 | Chiang et al. |
| 20130170767 | July 4, 2013 | Choudhury et al. |
| 20130182096 | July 18, 2013 | Boccara et al. |
| 20130223685 | August 29, 2013 | Maiden |
| 20140007307 | January 2, 2014 | Routh, Jr. et al. |
| 20140029824 | January 30, 2014 | Shi et al. |
| 20140043616 | February 13, 2014 | Maiden et al. |
| 20140050382 | February 20, 2014 | Adie et al. |
| 20140085629 | March 27, 2014 | Bodkin et al. |
| 20140126691 | May 8, 2014 | Zheng et al. |
| 20140133702 | May 15, 2014 | Zheng et al. |
| 20140152801 | June 5, 2014 | Fine |
| 20140153692 | June 5, 2014 | Larkin et al. |
| 20140160236 | June 12, 2014 | Ozcan |
| 20140160488 | June 12, 2014 | Zhou |
| 20140217268 | August 7, 2014 | Schleipen et al. |
| 20140267674 | September 18, 2014 | Mertz et al. |
| 20140347672 | November 27, 2014 | Pavillon et al. |
| 20140368812 | December 18, 2014 | Humphry et al. |
| 20150036038 | February 5, 2015 | Horstmeyer et al. |
| 20150044098 | February 12, 2015 | Smart et al. |
| 20150054979 | February 26, 2015 | Ou et al. |
| 20150160450 | June 11, 2015 | Ou et al. |
| 20150264250 | September 17, 2015 | Ou et al. |
| 20150286042 | October 8, 2015 | Hilbert et al. |
| 20150331228 | November 19, 2015 | Horstmeyer et al. |
| 20160088205 | March 24, 2016 | Horstmeyer et al. |
| 20160110584 | April 21, 2016 | Remiszewski et al. |
| 20160156880 | June 2, 2016 | Teich et al. |
| 20160178883 | June 23, 2016 | Horstmeyer et al. |
| 20160202460 | July 14, 2016 | Zheng |
| 20160210763 | July 21, 2016 | Horstmeyer et al. |
| 20160216208 | July 28, 2016 | Kim et al. |
| 20160216503 | July 28, 2016 | Kim et al. |
| 20160266366 | September 15, 2016 | Chung et al. |
| 20160320595 | November 3, 2016 | Horstmeyer et al. |
| 20160320605 | November 3, 2016 | Ou et al. |
| 20160341945 | November 24, 2016 | Ou et al. |
| 20170061599 | March 2, 2017 | Remiszewski et al. |
| 20170146788 | May 25, 2017 | Waller et al. |
| 20170178317 | June 22, 2017 | Besley et al. |
| 20170188853 | July 6, 2017 | Nakao et al. |
| 20170273551 | September 28, 2017 | Chung et al. |
| 20170299854 | October 19, 2017 | Kim et al. |
| 20170354329 | December 14, 2017 | Chung et al. |
| 20170363853 | December 21, 2017 | Besley |
| 20170371141 | December 28, 2017 | Besley |
| 20180045569 | February 15, 2018 | Nath et al. |
| 20180048811 | February 15, 2018 | Waller et al. |
| 20180078447 | March 22, 2018 | Viner et al. |
| 20180078448 | March 22, 2018 | Shockley, Jr. et al. |
| 20180088309 | March 29, 2018 | Ou et al. |
| 20180120553 | May 3, 2018 | Leshem et al. |
| 20180231761 | August 16, 2018 | Dai et al. |
| 20180307017 | October 25, 2018 | Horstmeyer et al. |
| 20180316855 | November 1, 2018 | Ou et al. |
| 20180329194 | November 15, 2018 | Small et al. |
| 20180348500 | December 6, 2018 | Naaman, III et al. |
| 20180373016 | December 27, 2018 | Leshem et al. |
| 20190049712 | February 14, 2019 | Kim et al. |
| 20190056578 | February 21, 2019 | Horstmeyer et al. |
| 20190077610 | March 14, 2019 | Flammann |
| 20190097523 | March 28, 2019 | Schaefer |
| 20190097524 | March 28, 2019 | Lin |
| 20190137753 | May 9, 2019 | Chan et al. |
| 20190317311 | October 17, 2019 | Kim et al. |
| 20190331902 | October 31, 2019 | Ou et al. |
| 20190391382 | December 26, 2019 | Chung et al. |
| 1688254 | October 2005 | CN |
| 1932565 | March 2007 | CN |
| 1311392 | April 2007 | CN |
| 101408623 | April 2009 | CN |
| 101680848 | March 2010 | CN |
| 101743519 | June 2010 | CN |
| 101868740 | October 2010 | CN |
| 101872033 | October 2010 | CN |
| 101957183 | January 2011 | CN |
| 102608597 | July 2012 | CN |
| 102753935 | October 2012 | CN |
| 103096804 | May 2013 | CN |
| 103154662 | June 2013 | CN |
| 103201648 | July 2013 | CN |
| 103377746 | October 2013 | CN |
| 104181686 | December 2014 | CN |
| 104200449 | December 2014 | CN |
| 2007-299604 | November 2007 | JP |
| 2008-147629 | June 2008 | JP |
| 2010-012222 | January 2010 | JP |
| 10-1998-0075050 | November 1998 | KR |
| 201428339 | July 2014 | TW |
| WO9953469 | October 1999 | WO |
| WO 2002/102128 | December 2002 | WO |
| WO 2003/062744 | July 2003 | WO |
| WO 2008-116070 | September 2008 | WO |
| WO 2011-093043 | August 2011 | WO |
| WO 2012/037182 | March 2012 | WO |
| WO 2014/070656 | May 2014 | WO |
| WO 2015/017730 | February 2015 | WO |
| WO 2015/027188 | February 2015 | WO |
| WO 2016/090331 | June 2016 | WO |
| WO 2016/106379 | June 2016 | WO |
| WO 2016/118761 | July 2016 | WO |
| WO 2016/123156 | August 2016 | WO |
| WO 2016/123157 | August 2016 | WO |
| WO 2016/149120 | September 2016 | WO |
| WO 2016/187591 | November 2016 | WO |
| WO 2017/066198 | April 2017 | WO |
| WO 2017081539 | May 2017 | WO |
| WO 2017081540 | May 2017 | WO |
| WO 2017081542 | May 2017 | WO |
- International Search Report and Written Opinion dated Feb. 21, 2014 in PCT/US2013/067068.
- International Search Report and Written Opinion dated Dec. 5, 2014 in PCT/US2014/052351.
- International Search Report and Written Opinion dated Nov. 13, 2014 in PCT/US2014/049297.
- Alexandrov, S. et al., “Spatial information transmission beyond a system's diffraction limit using optical spectral encoding of the spatial frequency,” Journal of Optics A: Pure and Applied Optics 10, 025304 (2008).
- Alexandrov, S. A. et al., “Synthetic Aperture Fourier holographic optical microscopy,” Phys. Rev. Lett. 97, 168102 (2006).
- Arimoto, H. et al. “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157-159 (2001).
- Brady, D. et al., “Multiscale gigapixel photography,” Nature 486, 386-389 (2012).
- Colomb, T. et al., “Automatic procedure for aberration compensation in digital holographic microscopy and applications to specimen shape compensation,” Appl. Opt. 45, 851-863 (2006).
- Denis, L. et al., “Inline hologram reconstruction with sparsity constraints,” Opt. Lett. 34, pp. 3475-3477 (2009).
- Di, J. et al., “High resolution digital holographic microscopy with a wide field of view based on a synthetic aperture technique and use of linear CCD scanning,” Appl. Opt. 47, pp. 5654-5659 (2008).
- Dierolf, M. et al., “Ptychographic coherent diffractive imaging of weakly scattering specimens,” New J. Phys. 12, 035017 (2010).
- Faulkner, H. et al., “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
- Feng, P. et al., “Long-working-distance synthetic aperture Fresnel off-axis digital holography,” Optics Express 17, pp. 5473-5480 (2009).
- Fienup, J. R., “Phase retrieval algorithms: a comparison,” Appl. Opt. 21, 2758-2769 (1982).
- Fienup, J. R., “Reconstruction of a complex-valued object from the modulus of its Fourier transform using a support constraint,” J. Opt. Soc. Am. A 4, 118-123 (1987).
- Fienup, J. R., “Reconstruction of an object from the modulus of its Fourier transform,” Opt. Lett. 3, 27-29 (1978).
- Fienup, J. R., “Lensless coherent imaging by phase retrieval with an illumination pattern constraint,” Opt. Express 14, 498-508 (2006).
- Goodman, J., “Introduction to Fourier Optics,” Roberts & Company Publication, Third Edition, chapters 1-6, pp. 1-172 (2005).
- Granero, L. et al., “Synthetic aperture superresolved microscopy in digital lensless Fourier holography by time and angular multiplexing of the object information,” Appl. Opt. 49, pp. 845-857 (2010).
- Greenbaum, A. et al., “Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy,” Sci. Rep. 3, p. 1717 (2013).
- Guizar-Sicairos, M., “Phase retrieval with transverse translation diversity: a nonlinear optimization approach,” Opt. Express 16, 7264-7278 (2008).
- Gustafsson, M. G., “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82-87 (2000).
- Gutzler, T. et al., “Coherent aperture-synthesis, wide-field, high-resolution holographic microscopy of biological tissue,” Opt. Lett. 35, pp. 1136-1138 (2010).
- Hillman, T. R. et al., “High-resolution, wide-field object reconstruction with synthetic aperture Fourier holographic optical microscopy,” Opt. Express 17, pp. 7873-7892 (2009).
- Hong, S-H. et al., “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483-491 (2004).
- Humphry, M. et al., “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nat. Commun. 3, 730 (2012).
- Kim, M. et al., “High-speed synthetic aperture microscopy for live cell imaging,” Opt. Lett. 36, pp. 148-150 (2011).
- Levoy, M., Ng, R., Adams, A., Footer, M. & Horowitz, M., “Light field microscopy,” ACM Trans. Graphics 25, 924-934 (2006).
- Levoy, M., Zhang, Z. & McDowall, I., “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144-162 (2009).
- Lohmann, A. W., Dorsch, R. G., Mendlovic, D., Zalevsky, Z. & Ferreira, C., “Space-bandwidth product of optical signals and systems,” J. Opt. Soc. Am. A 13, pp. 470-473 (1996).
- Lue, N. et al., “Live Cell Refractometry Using Hilbert Phase Microscopy and Confocal Reflectance Microscopy,” The Journal of Physical Chemistry A, 113, pp. 13327-13330 (2009).
- Maiden, A. M. et al., “Optical ptychography: a practical implementation with useful resolution,” Opt. Lett. 35, 2585-2587 (2010).
- Maiden et al., “Superresolution imaging via ptychography,” Journal of the Optical Society of America A. Apr. 2011, vol. 28 No. 4, pp. 604-612.
- Miao et al., “High Resolution 3D X-Ray Diffraction Microscopy,” Physical Review Letters, Aug. 19, 2002, vol. 89, No. 8, pp. 1-4.
- Mico, V. et al., “Synthetic aperture microscopy using off-axis illumination and polarization coding,” Optics Communications, pp. 276, 209-217 (2007).
- Mico, V. et al., “Synthetic aperture superresolution with multiple off-axis holograms,” JOSA A 23, pp. 3162-3170 (2006).
- Mir, M. et al., “Blood screening using diffraction phase cytometry,” Journal of Biomedical Optics 15, pp. 027016-027014 (2010).
- Mir, M. et al., “Optical measurement of cycle-dependent cell growth,” Proceedings of the National Academy of Sciences 108, pp. 13124-13129 (2011).
- Reinhard, E. et al., “High Dynamic Range Imaging: Acquisition, Display, and Image-based Lighting” (Morgan Kaufmann, 2010).
- Rodenburg, J. M. et al., “The theory of super-resolution electron microscopy via Wigner-distribution deconvolution,” Phil. Trans. R. Soc. Lond. A 339, 521-553 (1992).
- Rodenburg, J. M. et al., “Hard-X-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98, 034801 (2007).
- Schnars, U. et al., “Digital recording and numerical reconstruction of holograms,” Measurement Science and Technology, 13, R85 (2002).
- Schwarz, C. J. et al., “Imaging interferometric microscopy,” Optics letters 28, pp. 1424-1426 (2003).
- Thibault, P. et al., “High-resolution scanning X-ray diffraction microscopy,” Science 321, 379-382 (2008).
- Tippie, A.E., Kumar, A., and Fienup, J.R., “High-resolution synthetic-aperture digital holography with digital phase and pupil correction,” Opt. Express 19, pp. 12027-12038 (2011).
- Turpin, T. et al., “Theory of the synthetic aperture microscope,” pp. 230-240 (1995).
- Tyson, R., “Principles of Adaptive Optics” (CRC Press, 2010).
- Wang, Z. et al., “Tissue refractive index as marker of disease,” Journal of Biomedical Optics 16, 116017-116017 (2011).
- Wu, J. et al., “Wide field-of-view microscope based on holographic focus grid illumination,” Opt. Lett. 35, 2188-2190 (2010).
- Wu, J. et al., “Focal plane tuning in wide-field-of-view microscope with Talbot pattern illumination,” Opt. Lett. 36, 2179-2181 (2011).
- Ou X. et al., “Embedded pupil function recovery for Fourier ptychographic microscopy,” Optics Express 22 (5), pp. 4960-4972 (2014).
- Xu, W. et al., “Digital in-line holography for biological applications,” Proc. Natl Acad. Sci. USA 98, pp. 11301-11305 (2001).
- Yuan, C. et al., “Angular multiplexing in pulsed digital holography for aperture synthesis,” Optics Letters 33, pp. 2356-2358 (2008).
- Zheng, G. et al., “Wide-field, high-resolution Fourier ptychographic microscopy,” Nature Photonics (2013).
- Zheng, G. et al., “The ePetri dish, an on-chip cell imaging platform based on subpixel perspective sweeping microscopy (SPSM),” Proc. Natl Acad. Sci. USA 108, pp. 16889-16894 (2011).
- Zheng, G. et al., “Sub-pixel resolving optofluidic microscope for on-chip cell imaging,” Lab Chip 10, pp. 3125-3129 (2010).
- Zheng, G. et al., “Microscopy refocusing and dark-field imaging by using a simple LED array,” Opt. Lett. 36, 3987-3989 (2011).
- Zheng, G. et al., “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21, 15131-15143 (2013).
- “Immersion Media”, Olympus, Microscopy Resource Center, http://www.olympusmicro.com/primer/anatomy/immersion.html.
- U.S. Appl. No. 15/081,659, filed Mar. 25, 2016 entitled “Fourier Ptychographic Retinal Imaging Methods and Systems”.
- U.S. Office Action dated Oct. 5, 2015 in U.S. Appl. No. 14/065,305.
- U.S. Notice of Allowance dated Dec. 4, 2015 in U.S. Appl. No. 14/065,305.
- U.S. Notice of Allowance dated Jan. 14, 2016 in U.S. Appl. No. 14/448,850.
- U.S. Notice of Allowance dated Jan. 22, 2016 in U.S. Appl. No. 14/466,481.
- U.S. Notice of Allowance dated Apr. 13, 2016 in U.S. Appl. No. 14/448,850.
- U.S. Notice of Allowance dated Apr. 22, 2016 in U.S. Appl. No. 14/466,481.
- U.S. Office Action dated Jul. 14, 2016 in U.S. Appl. No. 15/007,196.
- U.S. Notice of Allowance dated Aug. 23, 2016 in U.S. Appl. No. 14/466,481.
- U.S. Office Action dated Sep. 16, 2016 I U.S. Appl. No. 14/065,305.
- U.S. Notice of Allowance dated Nov. 2, 2016 in U.S. Appl. No. 14,572,493.
- U.S. Office Action dated Nov. 22, 2016 in U.S. Appl. No. 15/003,559.
- U.S. Supplemental Notice of Allowance dated Dec. 12, 2016 in U.S. Appl. No. 14/572,493.
- U.S. Notice of Allowance dated Jan. 13, 2017 in U.S. Appl. No. 14/065,305.
- U.S. Final Office Action dated Jan. 23, 2017 in U.S. Appl. No. 15/007,196.
- International Preliminary Report on Patentability dated May 14, 2015 in PCT/US2013/067068.
- European Third-Party Observations, dated Jan. 20, 2016 in EP Application No. 13851670.3.
- European Extended Search Report dated Mar. 31, 2016 in EP Application No. 13851670.3.
- International Preliminary Report on Patentability dated Mar. 3, 2016 issued in PCT/US2014/052351.
- International Preliminary Report on Patentability dated Feb. 11, 2016 issued in PCT/US2014/049297.
- International Search Report and Written Opinion dated Feb. 22, 2016 issued in PCT/US2015/064126.
- International Search Report and Written Opinion dated Apr. 19, 2016 issued in PCT/US2015/067498.
- International Search Report and Written Opinion dated May 4, 2016 issued in PCT/US2016/015001.
- International Search Report and Written Opinion dated May 11, 2016 issued in PCT/US2016/015002.
- International Search Report and Written Opinion dated Jun. 27, 2016 issued in PCT/US2016/022116.
- International Search Report and Written Opinion dated Jun. 30, 2016 issued in PCT/US2016/014343.
- International Search Report and Wrtitten Opinion dated Sep. 5, 2016 issued in PCT/US2016/033638.
- Chinese Office Action [Description in English] dated Jul. 11, 2016 issued in Application No. CN 201380068831.6.
- Chinese Office Action [Description in English] dated Dec. 13, 2016 issued in Application No. CN201480057911.6.
- “About Molemap,” About Us—Skin Cancer Mole Check NZ, pp. 1-2. [retrieved Oct. 23, 2015 ] [http://molemap.net.au/about-us/].
- Abramomwitz, M., et al, “Field Curvature,” Olympus Microscopy Resource Center 2012, pp. 1-3. [retrieved on Feb. 24, 2016 ] [http://www.olympusmicro.com/primer/anatomy/fieldcurvature.html].
- “Age-Related Macular Degeneration (AMD) | National Eye Institute.” 2010 Table, retrieved Apr. 5, 2016, pp. 1-8. [https://www.nei.nih.gov/eyedata/amd#top.].
- Balan, R., et al, “On signal reconstruction without phase, Applied and Computational Harmonic Analysis 20,” No. 3 (2006): 345-356.
- Balan, R., et al, “Painless reconstruction from magnitudes of frame coefficients,” J Fourier Anal Appl 15:488-501 (2009).
- Bauschke, H.H., et al, “Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization,” J Opt Soc Am A 19:1334-1345 (2002).
- Becker, S., et al, “Templates for convex cone problems with applications to sparse signal recovery,” Technical report, Department of Statistics, Stanford University, (2010), 48 Pages.
- Betti, R., et al, “Observational study on the mitotic rate and other prognostic factors in cutaneous primary melanoma arising from naevi and from melanoma de novo,” Journal of the European Academy of Dermatology and Venereology, 2014.
- Bian, L., et al, “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express 23:4856-4866 (2015).
- Bian, Z., et al, “Adaptive system correction for robust Fourier ptychographic imaging,” Optics express, 2013. 21(26): p. 32400-32410.
- BioTek® Brochure: BioTek's Multi-Mode Microplate Reading Techonologies, BioTek Instruments, Inc. 2016, pp. 1-2. [http://www.biotek.com].
- Bishara W. et al “Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array,” Lab Chip 11(7), Apr. 7, 2011, pp. 1276-1279.
- Bishara, W., et al, “Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution,” Opt. Express 18(11), May 24, 2010, pp. 11181-11191.
- Blum, A., et al, “Clear differences in hand-held dermoscopes,” JDDG: Journal der Deutschen Dermatologischen Gesellschaft, 2006, 4(12): p. 1054-1057.
- Blum, A., et al, Dermatoskopie von Hauttumoren: Auflichtmikroskopie; Dermoskopie; digitate Bildanalyse; mit 28 Tabellen. 2003: Springer DE, Chapter 4 “Dermatoskopisch sichtbare Strukturen” p. 15-66.
- Born, M., et al, “Principles of Optics: Electromagnetic theory of propagation, interference and diffraction of light,” 7th Ed., 1999, Cambridge Univ. Press, pp. 1-31.
- Burer, S., et al, “A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization,” Math Program, Ser B 95:329-357 (2003).
- Burer, S., et al, “Local minima and convergence in low-rank semidefinite programming. Math Program,” Ser A 103:427-444 (2005).
- Candes, E.J., et al, “Phase retrieval via matrix completion,” SIAM J. Imaging Sci. 6:199-225 (2012).
- Candes, E.J., et al, “Phase retrieval via Wirtinger flow: theory and algorithms,” IEEE Trans. Info. Theory 61:1985-2007 (2015).
- Candes, E.J., et al, “PhaseLift: exact and stable signal recovery from magnitude measurements via convex programming.,” Comm Pure Appl Math 66:1241-1274 (2013).
- Carroll, J., “Adaptive optics retinal imaging: applications for studying retinal degeneration,” Arch Ophthalmol, vol. 126, No. 6, Jun. 2008, pp. 857-858. [retrieved Feb. 24, 2016].
- Chao, W. et al, “Soft X-ray microscopy at a spatial resolution better than 15 nm,” Nature Letters, vol. 435/30, Jun. 2005 pp. 1210-1213.
- Chen, T., et al, “Polarization and phase shifting for 3D scanning of translucent objects,” Proc. CVPR, (2007).
- Chin, L., et al, “Malignant melanoma: genetics and therapeutics in the genomic era,” Genes & development, 2006, 20(16): p. 2149-2182.
- Choi, W., et al, “Tomographic phase microscopy,” Nature Methods 4(9) (2007), pp. 1-3 Published Online Aug. 12, 2007.
- Chung, J., et al, “Counting White Blood Cells from a Blood Smear Using Fourier Ptychographic Microscopy,” PLoS One 10(7) e0133489 Jul. 17, 2015, pp. 1-10.
- Chung, J., et al, “Wide field-of-view fluorescence image deconvolution with aberration-estimation from Fourier ptychography,” Feb. 1, 2016, vol. 7, No. 2, Biomedical Optics Express pp. 352-368.
- De Sa, C., et al, “Global convergence of stochastic gradient descent for some non convex matrix problems,” Proc. 32nd Int. Conf. Machine Learning (2015), 10 pp.
- Debailleul, M., et al, “High-resolution three-dimensional tomographic diffractive microscopy of transparent inorganic and biological samples,” Optic Letters, vol. 34, No. 1, Jan. 1, 2009, pp. 79-81.
- Dierolf, M., et al, “Ptychographic X-ray computed tomography at the nanoscale,” Nature, vol. 467, Sep. 23, 2010, pp. 436-439.
- “Doctor Mole—Skin Cancer App,” App to check skin cancer by Dr. Mole, retrieved on Oct. 23, 2015, pp. 1-1. [http://www.doctormole.com].
- Dong, S., et al, “FPscope: a field-portable high-resolution microscope using a cellphone lens,” Biomed. Opt. Express 5(10), Oct. 1, 2014, pp. 3305-3310.
- Dong, S., et al, “High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography,” Opt. Express 22(17), Aug. 25, 2014, pp. 20856-20870.
- Dong, S., et al, “Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging,” pp. 13586-13599 (Jun. 2, 2014).
- Eldar, Y.C., et al, “Sparse phase retrieval from short-time Fourier measurements,” IEEE Signal Processing Letters 22, No. 5 (2015): 638-642.
- Emile, O., et al, “Rotating polarization imaging in turbid media,” Optics Letters 21(20), (1996).
- Faulkner, H.M.L., and Rodenburg, J.M., “Error tolerance of an iterative phase retrieval algorithm for moveable illumination microscopy,” Ultramicroscopy 103(2), 153-164 (2005).
- Fazel M., “Matrix rank minimization with applications,” PhD Thesis (Stanford University, Palo Alto, CA). (2002).
- Fienup, J. R., “Invariant error metrics for image reconstruction,” Appl. Opt. 36(32), 8352-8357 (1997).
- Gan, X., et al, “Image enhancement through turbid media under a microscope by use of polarization gating methods,” JOSA A 16(9), (1999).
- Gerke T.D., et al, “Aperiodic volume optics,” Nature Photonics, vol. 4, Feb. 7, 2010, pp. 188-193.
- Ghosh, A., et al, “Multiview face capture using polarized spherical gradient illumination,” ACM Transactions on Graphics 30(6) (2011).
- Godara, P., et al, “Adaptive optics retinal imaging: emerging clinical applications.,” Optom. Vis. Sci., vol. 87, No. 12, pp. 930-941, Dec. 2010.
- Goodson, A.G., et al, “Comparative analysis of total body and dermatoscopic photographic monitoring of nevi in similar patient populations at risk for cutaneous melanoma,” Dermatologic Surgery, 2010. 36(7): p. 1087-1098.
- Grant, M., et al, “CVX: Matlab software for disciplined convex programming,” version 2.0 beta. http://cvxr.com/cvx, (Sep. 2013), 3 pages.
- Gruev, V., et al, “Dual-tier thin film polymer polarization imaging sensor,” Optics Express, vol. 18, No. 18, 12 pages (2010).
- Gunturk, B.K., et al, “Image Restoration: Fundamentals and Advances,” vol. 7, Chapter 3, pp. 63-68 (CRC Press, 2012).
- Haigh, S. J., et al, (2009) “Atomic structure imaging beyond conventional resolution limits in the transmission electron microscope,” Physical Review Letters 103, Sep. 18, 2009, pp. 126101.1-126101.4.
- Han, C., et al, “Wide Field-of-View On-Chip Talbot Fluorescence Microscopy for Longitudinal Cell Culture Monitoring from within the Incubator” Anal. Chem. 85(4), Jan. 28, 2013, pp. 2356-2360.
- Hofer, H., et al, “Dynamics of the eye's wave aberration,” J. Opt. Soc. Am. A, vol. 18, No. 3, Mar. 2001, pp. 497-506.
- Hofer, H., et al, “Organization of the human trichromatic cone mosaic.,” J. Neurosci., vol. 25, No. 42, Oct. 2005, pp. 9669-9679.
- Hoppe, W., “Diffraction in inhomogeneous primary wave fields. 1. Principle of phase determination from electron diffraction interference,” Acta Crystallogr. A25, 495-501 1969.
- Horstmeyer, R., et al, “A phase space model of Fourier ptychographic microscopy,” Optics Express, 2014. 22(1): p. 338-358.
- Horstmeyer, R., et al, “Digital pathology with fourier ptychography,” Computer Medical Imaging and Graphics, vol. 42, Jun. 2015, pp. 38-43.
- Horstmeyer, R., et al, “Overlapped fourier coding for optical aberration removal,” Manuscript in preparation, 19 pages (2014).
- Hüe, F., et al, “Wave-front phase retrieval in transmission electron microscopy via ptychography,” Phys. Rev. B 82, 121415 (2010).
- Essen BioScience, “Real-time, quantitative live-cell analysis, IncuCyte® ZOOM System,” 2016 Brochure, retrieved Feb. 25, 2016, pp. 1-4. [http://www.essenbioscience.com/media/uploads/files/8000-0333-E00-IncuCyte_ZOOM_brochure.pdf].
- Jaganathan, K., et al, “Recovery of sparse 1-D signals from the magnitudes of their Fourier transform,” IEEE International Symposium on Information Theory Proceedings (2012): 1473-1477.
- Jaganathan, K., et al, “Phase retrieval with masks using convex optimization,” IEEE International Symposium on Information Theory Proceedings (2015): 1655-1659.
- Jaganathan, K., et al, “STFT Phase retrieval: uniqueness guarantees and recovery algorithms,” arXiv preprint arXiv:1508.02820 (2015).
- Joeres, S., et al, “Retinal imaging with adaptive optics scanning laser ophthalmoscopy in unexplained central ring scotoma.,” Arch. Ophthalmol., vol. 126, No. 4, Apr. 2008, pp. 543-547.
- Jung, J.H., et al, “Microfluidic-integrated laser-controlled microactuators with on-chip microscopy imaging functionality,” Lab Chip 14 (19), Oct. 7, 2014, pp. 3781-3789.
- Kay, D. B., et al, “Outer retinal structure in best vitelliform macular dystrophy,” JAMA Ophthalmol, vol. 131(9), Sep. 1, 2014, pp. 1207-1215.
- Kim, J., et al, “Incubator embedded cell culture imaging system (EmSight) based on Fourier ptychographic microscopy,” Biomedical Optics Express, vol. 7, No. 8, Aug. 1, 2016, pp. 3097-3110.
- Kirkland, A.I., et al, “Multiple beam tilt microscopy for super resolved imaging;” Japanese Society of Electron Microscopy: Journal of Electron Microscopy, vol. 46, No. 1, 1997, pp. 11-22.
- Kirkland, A.I., et al, “Super-resolution by aperture synthesis: tilt series reconstruction in CTEM,” Ultramicroscopy 57, (1995) 355-374, in final form Oct. 2, 1994; 1995 Elsevier Science B.V. SSDI 0304-3991(94)00191-x.
- Kittler, H., et al, “Morphologic changes of pigmented skin lesions: a useful extension of the ABCD rule for dermatoscopy,” Journal of the American Academy of Dermatology, 1999. 40(4): p. 558-562.
- Kozak, I., “Retinal imaging using adaptive optics technology.,” Saudi J. Ophthalmol. Off. J. Saudi Ophthalmol. Soc., vol. 28, No. 2, Feb. 26, 2014, pp. 117-122.
- Lauer, V., “New Approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomography microscope,” Journal of Microscopy, The Royal Microscopical Society 2001, vol. 205, Pt 2, Feb. 2002, pp. 165-176.
- Lee, K., et al, “Synthetic Fourier transform light scattering,” Optics Express 21, vol. 21, No. 19, Sep. 23, 2013, pp. 22453-22463.
- Li X., et al, “Sparse signal recovery from quadratic measurements via convex programming,” SIAM Journal on Mathematical Analysis 45, No. 5 (2013): 3019-3033.
- LUXEXCEL® Brochure, “LUXEXCEL: 3D Printing Service Description” [retrieved on Mar. 7, 2016] pp. 1-5, [http://www.luxexcel.com].
- “Lytro,” Lytro-Home, retrieved Oct. 23, 2015, pp. 1-6. [https://www.lytro.com/].
- Ma, W., et al, “Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination,” University of Southern California, Institute for Creative Technologies, 12 pages (2007).
- Mahajan, V. N., “Zernike circle polynomials and optical aberrations of systems with circular pupils,” Appl. Opt. 33(34), 8121-8124 (1994).
- Maiden, A. M., et al, “A new method of high resolution, quantitative phase scanning microscopy,” in: M.T. Postek, D.E. Newbury, S.F. Platek, D.C. Joy (Eds.), SPIE Proceedings of Scanning Microscopy, 7729, 2010.
- Maiden, A. M., et al, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109(10), 1256-1262 (2009).
- Marchesini S., et al, “A unified evaluation of iterative projection algorithms for phase retrieval,” Rev Sci Instrum 78:011301 (2007).
- Marchesini S., et al, “Augmented projections for ptychographic imaging,” Inverse Probl 29:115009 (2013).
- Marrison, J., et al, “Ptychography—a label free, high-contrast imaging technique for live cells using quantitative phase information,” Sci. Rep. 3, 2369 (2013).
- Medoff, B.P., et al, “Iterative convolution backprojection algorithms for image reconstruction from limited data,” J. Opt. Soc. Am. vol. 73, No. 11, Nov. 1983, pp. 1493-1500.
- “Melafind,” Optics by Carl Zeiss, MELA Sciences 2015, retrieved Oct. 23, 2015, pp. 1-4. [http://www.melafind.com/].
- Meyer, R.R., et al, “A new method for the determination of the wave aberration function of high-resolution TEM,” Elsevier B.V., Ultramicroscopy 99, 2004, pp. 115-123.
- Moreno, I., “Creating a desired lighting pattern with an LED array,” 8th International Conference on Solid State Lighting, Proceedings of SPIE, vol. 7058, 2008, pp. 1-9.
- Mrejen, S., et al, “Adaptive optics imaging of cone mosaic abnormalities in acute macular neuroretinopathy,” Ophthalmic Surgery, Lasers & Imaging Retina, vol. 45, No. 6, Nov./Dec. 2014, pp. 562-569.
- Nayar, S. K., et al, “Fast separation of direct and global components of a scene using high frequency illumination,” ACM Transactions on Graphics 25(3) (2006).
- Ng, R. et al, “Light field photography with a hand-held plenoptic camera”, Computer Science Technical Report CSTR, 2005. 2(11).
- Nomura, H., and Sato, T., “Techniques for measuring aberrations in lenses used in photolithography with printed patterns,” Appl. Opt. 38(13), 2800-2807 (1999).
- Ohlsson, H., et al, “Compressive phase retrieval from squared output measurements via semidefinite programming,” arXiv:1111.6323 (2011).
- Ou, X., et al, “High numerical aperture Fourier ptychography: principle, implementation and characterization,” Opt. Express 23:3472-3491 (2015).
- Ou, X. et al, “Quantitative phase imaging via Fourier ptychographic microscopy,” Optics Letters, 2013. 38(22): p. 4845-4848.
- Ou. X., et al, “Embedded pupil function recovery for Fourier ptychographic microscopy,” submitted Dec. 26, 2013; 13 pp.
- Pacheco, S., et al, “Reflective Fourier Ptychography,” J. Biomed. Opt. 21(2), pp. 026010-1-026010-7, (Feb. 18, 2016). [http://biomedicaloptics.spiedigitallibrary.org].
- Recht, B., et al, “Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization,” SIAM Review 52, No. 3 (2010): 471-501.
- Rodenburg, J., “Ptychography and related diffractive imaging methods,” Adv. Imaging Electron Phys.150, 87-184 (2008).
- Rossi, E.A., et al, “In vivo imaging of retinal pigment epithelium cells in age related macular degeneration.,” Biomed. Opt. Express, vol. 4, No. 11, pp. 2527-2539, Jan. 2013.
- Rowe, M., et al, “Polarization-difference imaging: a biologically inspired technique for observation through scattering media,” Optics Letters, vol. 20, No. 6, 3 pages (1995).
- Schechner, Y., “Multiplexing for Optimal Lighting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, No. 8, 1339-1354 (2007).
- Shechner, Y., et al, “Polarization-based vision through haze,” Applied Optics 42(3), (2003).
- Shechtman, Y., et al, “Sparsity based sub-wavelength imaging with partially incoherent light via quadratic compressed sensing,” Opt Express 19:14807-14822 (2011).
- Siegel, R., et al, “Cancer statistics 2013,” CA: a cancer journal for clinicians, 2013. 63(1): p. 11-30.
- Stoecker, W., et al, “Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection: Representative Lesion Sets and the Role for Adjunctive Technologies,” JAMA Dermatology, 2013. 149(7): p. 884.
- Sun, D., et al, “Estimating a signal from a magnitude spectrogram via convex optimization,” arXiv:1209.2076 (2012).
- Sun, J., et al, “Coded multi-angular illumination for Fourier ptychography based on Hadamard codes,” 5 pages (2015).
- Tam, K., et al, “Tomographical imaging with limited-angle input,” J. Opt. Soc. Am., vol. 71, No. 5, May 1981, pp. 582-592.
- Thibault, P. et al, “Probe retrieval in ptychographic coherent diffractive imaging,” Ultramicroscopy 109(4), 338-343 (2009).
- Thomas, L., et al, “Semiological value of ABCDE criteria in the diagnosis of cutaneous pigmented tumors,” Dermatology, 1998. 197(1): p. 11-17.
- Tian, L., et al, “Multiplexed Coded Illumination for Fourier Ptychography with an LED Array Microscope,” Optical Society of America, 14 pages (2014).
- Vulovic, M., et al, “When to use the projection assumption and the weak-phase object approximation in phase contrast cryo-EM,” Ultramicroscopy 136, vol. 136, Jan. 2014, pp. 61-66.
- Waldspurger, I., et al, “Phase recovery, maxcut and complex semidefinite programming,” Mathematical Programming 149, No. 1-2 (2015): 47-81.
- Wang, Q., et al, “Adaptive Optics Microperimetry and OCT Images Show Preserved Function and Recovery of Cone Visibility in Macular Telangiectasia Type 2 Retinal Lesions,” Investigative Ophthalmology Visual Science, vol. 56, No. 2, Feb. 2015, pp. 778-786.
- Watanabe, M., et al, “Telecentric optics for focus analysis,” IEEE trans. pattern. anal. mach. intell., 19 1360-1365 (1997).
- Wesner, J., et al, “Reconstructing the pupil function of microscope objectives from the intensity PSF,” in Current Developments in Lens Design and Optical Engineering III, R. E. Fischer, W. J. Smith, and R. B. Johnson, eds., Proc. SPIE 4767, 32-43 (2002).
- Williams, A., et al, “Fourier ptychographic microscopy for filtration-based circulating tumor cell enumeration and analysis,” Journal of Biomedical Optics, vol. 19(6), Jun. 20, 2014, pp. 066007.1-066007.8.
- Wolf, J., et al, “Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection,” JAMA Dermatology, 2013, 149(7): p. 885-885.
- Zeiss, C., “Microscopy: Cells Need the Perfect Climate, System Solutions for Live Cell Imaging under Physiological Conditions,” BioSciences, Feb. 2008, pp. 1-42. [http://www.zeiss.de/incubation].
- Zhang, Y., et al, “Self-learning based fourier ptychographic microscopy,” Optics Express, 16pgs (2015).
- Zhang, Y., et al, “Photoreceptor Perturbation Around Subretinal Drusenoid Deposits as Revealed by Adaptive Optics Scanning Laser Ophthalmoscopy,” Am. J. Ophthalmol., Author manuscript, vol. 158, No. 3, Sep. 1, 2015, pp. 584-596.
- Zheng, G., et al, “0.5 gigapixel microscopy using a flatbed scanner,” Biomed. Opt. Express 5, 1-8 (2014).
- Chung, J., et al, “Wide-field Fourier ptychographic microscopy using laser illumination source,” Optical Society of America, 13 pgs., Mar. 23, 2016.
- Guo, K., et al, “Optimization of sampling pattern and the design of Fourier ptychographic illuminator,” Optics Express, vol. 23, No. 5, Mar. 9, 2015, pp. 6171-6180.
- Phillips, Z., et al, “Multi-Contrast Imaging and Digital Refocusing on a Mobile Microscope with a Domed LED Array,” PLoS One, 10 (5), May 13, 2015, pp. 1-13.
- Horstmeyer, R., et al, “Standardizing the resolution claims for coherent microscopy,” Nature Photonics, vol. 10, pp. 68-71, Feb. 2016.
- Horstmeyer, R., et al, “Solving ptychography with a convex relaxation,” New Journal of Physics, vol. 17, May 27, 2015, pp. 1-14.
- Lu, H., et al, “Quantitative phase imaging and complex field reconstruction by pupil modulation differential phase contrast,” Optics Express, vol. 24, No. 22, pp. 25345-25361 (Oct. 31, 2016).
- Ou, X., et al, “Aperture scanning Fourier ptychographic microscopy,” Biomedical Optics Express, vol. 7, No. 8, pp. 3140-3150 (Aug. 1, 2016).
- Horstmeyer, R., et al, “Diffraction tomography with Fourier ptychography,” Optica, vol. 3, No. 8, pp. 827-835 (Aug. 2016).
- Bian, L., et al, “Fourier ptychographic reconstruction using Poisson maximum likelihood and truncated Wirtinger gradient,” NPG: Scientific Reports 6, article No. 27384, Jun. 10, 2016, pp. 1-10.
- Wu, J., et al, “Harmonically matched grating-based full-field quantitative high-resolution phase microscope for observing dynamics of transparent biological samples,” Optical Society of America, vol. 15, No. 26, pp. 18141-18155 (Dec. 19, 2007).
- Wu, J., et al, “Paired-angle-rotation scanning optical coherence tomography forward-imaging probe,” Optics Letters, vol. 31, No. 9, pp. 1265-1267 (May 2006).
- Preliminary Amendment dated Apr. 25, 2016 filed in U.S. Appl. No. 14/710,947.
- Preliminary Amendment dated Nov. 28, 2016 filed in U.S. Appl. No. 15/206,859.
- Preliminary Amendment dated Mar. 17, 2014 filed in U.S. Appl. No. 14/065,305.
- Preliminary Amendment dated Nov. 28, 2016 filed in U.S. Appl. No. 15/209,604.
- U.S. Office Action dated Feb. 21, 2017 in U.S. Appl. No. 14/960,252.
- U.S. Supplemental Notice of Allowability dated Mar. 2, 2017 in U.S. Appl. No. 14/065,305.
- U.S. Notice of Allowance dated Mar. 8, 2017 in U.S. Appl. No. 14/572,493.
- U.S. Office Action dated Mar. 13, 2017 in U.S. Appl. No. 14/658,019.
- U.S. Notice of Allowance dated Mar. 22, 2017 in U.S. Appl. No. 15/007,196.
- U.S. Office Action dated Mar. 24, 2017 in U.S. Appl. No. 14/710,947.
- U.S. Notice of Allowance dated Mar. 31, 2017 in U.S. Appl. No. 14/572,493.
- U.S. Notice of Allowance dated Jun. 9, 2017 in U.S. Appl. No. 14/065,305.
- U.S. Notice of Allowance dated Jun. 9, 2017 in U.S. Appl. No. 15/206,859.
- U.S. Notice of Allowance dated Jun. 9, 2017 in U.S. Appl. No. 15/007,196.
- U.S. Notice of Allowance dated Jun. 20, 2017 in U.S. Appl. No. 14/572,493.
- U.S. Supplemental Notice of Allowance dated Jun. 28, 2017 in U.S. Appl. No. 15/206,859.
- U.S. Final Office Action dated Jul. 27, 2017 in U.S. Appl. No. 15/003,559.
- U.S. Notice of Allowance dated Aug. 16, 2017 in U.S. Appl. No. 15/209,604.
- U.S. Notice of Allowance dated Sep. 1, 2017 in U.S. Appl. No. 15/206,859.
- U.S. Office Action dated May 19, 2017 in U.S. Appl. No. 15/081,659.
- Extended European Search Report dated Feb. 16, 2017 issued in Application No. 14837844.1.
- Extended European Search Report dated Feb. 15, 2017 issued in Applicatoin No. 14832857.8.
- Chinese Second Office Action [Description in English] dated Jan. 22, 2017 issued in Application No. CN201380068831.6.
- International Preliminary Report on Patentability dated Jun. 15, 2017 issued in Application No. PCT/US2015/064126.
- European Office Action dated May 16, 2017 issued in European Patent Application No. 13851670.3.
- International Preliminary Report on Patentability dated Jul. 6, 2017 issued in Application No. PCT/US2015/067498.
- International Preliminary Report on Patentability dated Aug. 3, 2017 issued in Application No. PCT/US2016/014343.
- International Preliminary Report on Patentability dated Aug. 10, 2017 issued in Application No. PCT/US2016/015001.
- International Preliminary Report on Patentability dated Aug. 10, 2017 issued in Application No. PCT/US2016/015002.
- Chinese Third Office Action [Summary in English] dated Jul. 24, 2017 issued in Application No. 201380068831.6.
- Chinese First Office Action [Summary in English] dated Aug. 2, 2017 issued in Application No. CN 201480054301.0.
- Australian Office Action dated Sep. 18, 2017 issued in Application No. AU 2014296034.
- International Preliminary Report on Patentability dated Sep. 28, 2017 issued in Application No. PCT/US2016/022116.
- Bunk, O., et al, “Influence of the overlap parameter on the convergence of the ptychographical iterative engine,” Ultramicroscopy, vol. 108, (2008), pp. 481-487. <doi: 10.1016/j.ultramic.2007.08.003>.
- Greenbaum, A., et al, “Field-portable wide-field microscopy of dense samples using multi-height pixel super-resolution based lensfree imaging,” Lab Chip, The Royal Society of Chemistry, vol. 12, No. 7, Jan. 31, 2012, pp. 1242-1245. [retrieved on Feb. 27, 2016] <URL:http://dx.doi.org/10.1039/C2LC21072J>.
- Hoppe, W., “Diffraction in inhomogeneous primary wave fields. 1. Principle of phase determination from electron diffraction interference.” Acta Crystallographica Section a—Crystal Physics Diffraction Theoretical and General Crystallography, A25, Jan. 1, 1969, pp. 495-501. (English Machine Translation Incl.).
- Kawata, S. et al, “Optical microscope tomography. I. Support constraint,” Journal Optical Society America A, vol. 4, No. 1, Jan. 1987, pp. 292-297. <doi:10.1364/JOSAA.4.000292>.
- Kim, M., et al, “High-speed synthetic aperture microscopy for live cell imaging,” Optics Letters, vol. 36, No. 2, Jan. 15, 2011, pp. 148-150. <doi:10.1364/OL.36.000148>.
- Reinhard, E., et al, “High Dynamic Range Imaging: Acquisition, Display, and Image-based Lighting” Second Edition § 5.2 HDR Image Capture: Morgan Kaufmann, May 28, 2010, pp. 148-151. <ISBN: 9780123749147>.
- Rodenburg, J. M., et al, “A phase retrieval algorithm for shifting illumination,” Applied Physics Letters, vol. 85, No. 20, Nov. 15, 2004, pp. 4795-4797. <doi: 10.1063/1.1823034>.
- Sankaranarayanan, Aswin C., et al, “CS-MUVI: Video Compressive Sensing for Spatial-Multiplexing Cameras,” Proceedings of the IEEE International Conference Computational Photography (ICCP), Apr. 2012, pp. 11. <doi:10.1109/ICCPhot.2012.6215212>.
- Tian, L., et al, “3D differential phase-contrast microscopy with computational illumination using an LED array,” Optics Letters, vol. 39, No. 5, Mar. 1, 2014, pp. 1326-1329. <doi:10.1364/OL39.001326>.
- Tian, L., et al, “Computional illumination for high-speed in vitro Fourier ptychographic microscropy,” Optica: Research Article, vol. 2, No. 10, Oct. 14, 2015, pp. 904-911. <doi:10.1364/OPTICA.2.000904>.
- Yeh, et al., “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Optics Express, vol. 23, No. 26, Dec. 28, 2015, pp. 33214-33240. <doi: 10.1364/OE.23.033214>.
- Zheng, G., “Fourier Ptychographic Imaging: A MATLAB tutorial,” IOP Concise Physics, Morgan & Claypool Publication, San Rafael, CA., May 2016, pp. 96. <ISBN: 978-1-6817-4272-4 (ebook)> <doi: 10.1088/978-1-6817-4273-1>.
- Zheng, G., et al, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nature Photonics, vol. 7, Sep. 2013, Published Online Jul. 28, 2013, pp. 739-746. <doi:10.1038/NPHOTON.2013.187>.
- U.S. Appl. No. 15/620,674, filed Jun. 12, 2017, Chung, J. et al.
- U.S. Appl. No. 15/636,494, filed Jun. 28, 2017, Kim, J. et al.
- Office Action dated Aug. 31, 2017 in U.S. Appl. No. 15/636,494.
- Preliminary Amendment dated Jun. 13, 2018 filed in U.S. Appl. No. 15/820,295.
- Notice of Allowance dated Dec. 4, 2017 in U.S. Appl. No. 14/065,305.
- Notice of Allowance dated Jan. 26, 2018 in U.S. Appl. No. 15/209,604.
- Notice of Allowance dated Jan. 23, 2018 in U.S. Appl. No. 15/206,859.
- Notice of Allowance dated Oct. 11, 2017 in U.S. Appl. No. 14/572,493.
- U.S. Final Office Action dated Jan. 17, 2018 in U.S. Appl. No. 14/658,019.
- U.S. Final Office Action dated Dec. 28, 2017 in U.S. Appl. No. 14/710,947.
- U.S. Final Office Action dated Dec. 14, 2017 in U.S. Appl. No. 14/960,252.
- U.S. Notice of Allowance dated Sep. 20, 2017 in U.S. Appl. No. 15/007,196.
- U.S. Notice of Allowance dated Jun. 27, 2018 in U.S. Appl. No. 15/636,494.
- U.S. Office Action dated Nov. 30, 2017 in U.S. Appl. No. 15/007,159.
- U.S. Notice of Allowance dated Jul. 16, 2018 in U.S. Appl. No. 15/007,159.
- Office Action dated Apr. 4, 2018 issued in U.S. Appl. No. 15/003,559.
- Office Action dated Nov. 3, 2017 in U.S. Appl. No. 15/068,389.
- Office Action Interview Summary dated May 3, 2018 in U.S. Appl. No. 15/068,389.
- Final Office Action dated Jun. 6, 2018 issued in U.S. Appl. No. 15/068,389.
- Office Action dated May 19, 2017 in U.S. Appl. No. 15/081,659.
- Notice of Allowance dated Oct. 20, 2017 in U.S. Appl. No. 15/081,659.
- Notice of Allowance dated Feb. 9, 2018 in U.S. Appl. No. 15/081,659.
- Office Action dated Apr. 13, 2018 issued in U.S. Appl. No. 15/160,941.
- European Extended Search Report dated Jun. 6, 2018 issued in Application No. 15865492.1.
- Extended European Search Report dated Jul. 3, 2018 issued in Application No. EP 15874344.3.
- Japanese Office Action dated Oct. 17, 2017 issued in Application No. 2015-539884.
- Chinese Office Action [Summary in English] dated Oct. 26, 2017 issued in CN 201480057911.6.
- International Preliminary Report on Patentability dated Nov. 30, 2017 issued in PCT/US2016/033638.
- Australian Examination Report 1/Office Action dated Jan. 18, 2018 issued in AU 2014308673.
- Chinese First Office Action dated Feb. 24, 2018 issued in CN 201680003937.1.
- Abrahamsson, S., et al., “Fast multicolor 3D imaging using aberration-corrected mulitfocus microscopy,” Brief Communications: Nature Methods, vol. 10, No. 1, Jan. 2013, pp. 60-65. <doi:10.1038/nmeth.2277>.
- Chai, A., et al, “Array imaging using intensity-only measurements,” IOP Publishing: Inverse Problems, vol. 27, No. 1, Jan. 2011, pp. 1-16. <doi:10.1088/0266-5611/27/1/015005>.
- Holloway, J., et al. “SAVI: Synthetic apertures for long-range, subdiffraction-limited visible imaging using Fourier ptychography,” Science Advances | Research Article, vol. 3, No. 4, Apr. 14, 2017, pp. 1-11. <doi:10.1126/sciadv.1602564> [retrieved on Nov. 28, 2017] <URL:http://advances.sciencemag.org/>.
- Jacques, et al., “Imaging Superficial Tissues With Polarized Light,” Lasers in Surgery and Medicine, vol. 26, No. 2, Apr. 25, 2000, pp. 119-129.
- Jensen, et al. “Types of imaging, Part 2: An Overview of Fluorescence Microscopy.” The Anatomical Record, vol. 295, No. 10, Oct. 1, 2012, pp. 1621-1627.
- Kner, P., “Phase diversity for three-dimensional imaging,” Journal of the Optical Society of America A, vol. 30, No. 10, Oct. 1, 2013, pp. 1980-1987. <doi:10.1364/JOSAA.30.001980>.
- Sarder, et al. “Deconvolution Methods for 3-D Fluorescence Microscopy Images,” IEEE Signal Processing Magazine, vol. 23, No. 3, May 2006, pp. 32-45.
- Wills, S., “Synthetic Apertures for the Optical Domain,” Optics & Photonics News Article [webpage], The Optical Society (OSA), Apr. 18, 2017, pp. 2. <URL:https://www.osa-opn.org/home/newsroom/2017/april/synthetic_apertures_for_the_optical_domain/>.
- U.S. Appl. No. 15/963,966, filed Apr. 26, 2018, Ou et al.
- U.S. Appl. No. 15/959,050, filed Apr. 20, 2018, Horstmeyer et al.
- Preliminary Amendment filed Jul. 11, 2018 in U.S. Appl. No. 15/959,050.
- U.S. Office Action dated Dec. 26, 2018 in U.S. Appl. No. 15/963,966.
- U.S. Notice of Allowance dated Apr. 19, 2019 in U.S. Appl. No. 15/963,966.
- U.S. Office Action dated Dec. 26, 2018 in U.S. Appl. No. 15/959,050.
- U.S. Notice of Allowance dated Sep. 17, 2018 in U.S. Appl. No. 15/820,295.
- U.S. Notice of Allowance dated Jan. 14, 2019 in U.S. Appl. No. 15/820,295.
- U.S. Notice of Allowance dated Apr. 16, 2019 in U.S. Appl. No. 15/820,295.
- U.S. Office Action dated Oct. 4, 2018 in U.S. Appl. No. 14/658,019.
- U.S. Notice of Allowance dated Jul. 25, 2018 in U.S. Appl. No. 14/710,947.
- U.S. Office Action dated Mar. 8, 2019 in U.S. Appl. No. 16/171,270.
- U.S. Office Action dated Dec. 13, 2018 in U.S. Appl. No. 14/960,252.
- U.S. Notice of Allowance dated Oct. 5, 2018 in U.S. Appl. No. 15/636,494.
- U.S. Office Action dated Apr. 4, 2019 in U.S. Appl. No. 16/162,271.
- U.S. Office Action dated Sep. 7, 2018 in U.S. Appl. No. 14/979,154.
- U.S. Final Office Action dated Dec. 10, 2018 issued in U.S. Appl. No. 15/003,559.
- U.S. Office Action dated Jan. 17, 2019 issued in U.S. Appl. No. 15/068,389.
- U.S. Notice of Allowance dated Oct. 19, 2018 issued in U.S. Appl. No. 15/160,941.
- U.S. Notice of Allowance dated Jan. 15, 2019 issued in U.S. Appl. No. 15/620,674
- U.S. Notice of Allowance dated Apr. 29, 2019 issued in U.S. Appl. No. 15/620,674.
- Chinese Third Office Action dated Jul. 13, 2018 issued in CN 201480057911.6.
- Japanese First Office Action dated Aug. 7, 2018 issued in Application No. JP 2016-531919.
- Chinese First Office Action dated Jan. 28, 2019 issued in CN 201580072950.8.
- Extended European Search Report dated Aug. 8, 2018 issued in Application No. EP 16744002.3.
- European Extended Search Report dated Aug. 14, 2018 issued in EP 16744003.1.
- Chinese First Office Action dated Dec. 28, 2018 issued in Application No. CN 201680005491.6.
- Chinese Office Action [Description in English] dated May 31, 2016 issued in Application No. CN 201380068831.6.
- Chinese First Office Action dated Apr. 19, 2019 issued in Application No. CN 201680006738.6.
- Chinese First Office Action dated Dec. 26, 2018 issued in Application No. CN 201580067354.0.
- Extended European Search Report dated Sep. 12, 2018 issued in Application No. EP 16740769.1.
- Chinese Second Office Action dated Jul. 3, 2018 issued in Application No. CN 201480054301.0.
- Extended European Search Report dated Oct. 25, 2018 issued in Application No. EP 16765505.9.
- Chinese First Office Action dated Apr. 19, 2019 issued in Application No. CN 201680014898.5.
- International Search Report and Written Opinion dated Feb. 22, 2019 issued in PCT/US2018/059059.
- Godden, T.M. et al., “Ptychographic microscope for three-dimensional imaging,” Optics Express, vol. 22, No. 10, May 19, 2014, pp. 12513-12523.
- Maiden, A.M., et al., “Ptychographic transmission microscopy in three dimensions using a multi-slice approach,” Journal of the Optical Society of America A., vol. 29, No. 8, Aug. 1, 2012, pp. 1606-1614.
- Pankajakshan, P., “Blind Deconvolution for Confocal Laser Scanning Microscopy,” Doctoral dissertation, Universite Nice Sophia Antipolis, 2009. <URL: https://tel.archives-ouvertes.fr/tel-00474264>.
- U.S. Appl. No. 16/242,934, filed Jan. 8, 2019, Kim et al.
- U.S. Appl. No. 16/252,465, filed Jan. 18, 2019, Ou et al.
- U.S. Notice of Allowance dated Sep. 17, 2019 in U.S. Appl. No. 14/960,252.
- U.S. Notice of Allowance dated Sep. 16, 2019 issued in U.S. Appl. No. 15/068,389.
- U.S. Notice of Allowance dated Sep. 25, 2019 issued in U.S. Appl. No. 15/620,674.
- U.S. Office Action dated Oct. 11, 2019 issued in U.S. Appl. No. 16/179,688.
- Desjardins, et al., “Angle-resolved Optical Coherence Tomography with sequential selectivity for speckle reduction” Optics Express, vol. 15, No. 10, May 14, 2007, pp. 6200-6209.
- Li, et al., “GPU accelerated parallel FFT processing for Fourier transform hyperspectral imaging,” Applied Optics, vol. 54, No. 13, May 1, 2015, pp. D91-D98.
- U.S. Appl. No. 16/572,497, filed Sep. 16, 2019, Ou et al.
- U.S. Office Action dated Sep. 23, 2019 issued in U.S. Appl. No. 16/252,465.
- U.S. Final Office Action dated Jun. 3, 2019 in U.S. Appl. No. 15/959,050.
- U.S. Final Office Action dated May 30, 2019 in U.S. Appl. No. 14/658,019.
- U.S. Notice of Allowance dated Aug. 12, 2019 in U.S. Appl. No. 14/960,252.
- U.S. Final Office Action dated May 30, 2019 in U.S. Appl. No. 14/979,154.
- U.S. Office Action dated Jun. 26, 2019 issued in U.S. Appl. No. 15/003,559.
- U.S. Final Office Action dated Jun. 19, 2019 issued in U.S. Appl. No. 15/068,389.
- U.S. Notice of Allowance dated Aug. 14, 2019 issued in U.S. Appl. No. 15/620,674.
- Adie, et al., “Computational adaptive optics for broadband optical interferometric tomography of biological tissue,” Proc. Natl. Acad. Sci. USA 109, 7175-7180 (May 8, 2012).
- Bian, et al., “Content adaptive illumination for Fourier ptychography,” Opt. Lett., vol. 39, (Aug. 2014) pp. 6648-6651.
- Bioucas-Dias, et al., “Total variation-based image deconvolution: a majorization-minimization approach,” ICASSP (2), pp. 861-864 (May 14, 2006).
- Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light Sci. Appl. 3, e165 (Apr. 25, 2014 ).
- Chung, et al., “Computational aberration compensation by coded-aperture-based correction of aberration obtained from optical Fourier coding and blur estimation,” Optica, vol. 6, May 10, 2019, pp. 647-661.
- Dowski, et al., “Extended depth of field through wavefront coding,” Appl. Opt., vol. 34, Apr. 10, 1995, pp. 1859-1866.
- Evered, et al., “Accuracy and perceptions of virtual microscopy compared with glass slide microscopy in cervical cytology,” Cytopathology, vol. 22, Feb. 2, 2010, pp. 82-87.
- Fergus, et al., “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787-794 (2006).
- Fienup and Miller, “Aberration correction by maximizing generalized sharpness metrics,” J. Opt. Soc. Am. A 20, pp. 609-620 (Apr. 2003).
- Fried, D.L.,“Anisoplanatism in adaptive optics,” J. Opt. Soc. Am. 72, pp. 52-61 (Jan. 1982).
- G. Gunjala, S. Sherwin, A. Shanker, and L. Waller, “Aberration recovery by imaging a weak diffuser,” Opt. Express 26, 21054-21068 (Aug. 6, 2018).
- G. McConnell, J. Trägårdh, R. Amor, J. Dempster, E. Reid, and W. B. Amos, “A novel optical microscope for imaging large embryos and tissue volumes with sub-cellular resolution throughout,” eLife 5, e18659 (Sep. 23, 2016).
- G. Muyo and A. R. Harvey, “Wavefront coding for athermalization of infrared imaging systems,” Proc. SPIE 5612, 227-235 (Dec. 2004).
- G. Muyo, et al., “Infrared imaging with a wavefront-coded singlet lens,” Opt. Express, vol. 17, Nov. 5, 2009, pp. 21118-21123.
- Ginner, et al., “Holographic line field en-face OCT with digital adaptive optics in the retina in vivo,” Biomed. Opt. Express 9, 472-485 (Feb. 1, 2018).
- Ginner, et al., “Noniterative digital aberration correction for cellular resolution retinal optical coherence tomography in vivo,” Optica, vol. 4, Aug. 2017, pp. 924-931.
- Gustafsson, M.,“Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. USA 102, 13081-13086 (Sep. 13, 2005).
- H. Hofer, L. Chen, G. Y. Yoon, B. Singer, Y. Yamauchi, and D. R. Williams, “Improvement in retinal image quality with dynamic correction of the eye's aberrations,” Opt. Express 8, 631-643 (May 21, 2001).
- Hillmann, et al., “Aberration-free volumetric high-speed imaging of in vivo retina,” Sci. Rep. 6, 35209 (Oct. 20, 2016).
- Kamal, et al., “In situ retrieval and correction of aberrations in moldless lenses using Fourier ptychography,” Opt. Express, vol. 26, No. 3, pp. 2708-2719 (Feb. 5, 2018).
- Kuang, et al., “Digital micromirror device-based laserillumination Fourier ptychographic microscopy,” Opt. Express 23, 26999-27010 (Oct. 19, 2015).
- Kubala, et al., “Reducing complexity in computational imaging systems,” Opt. Express 11, 2102-2108 (2003).
- Kumar, et al., “Subaperture correlation based digital adaptive optics for full field optical coherence tomography,” Opt. Express, vol. 21, May 6, 2013, pp. 10850-10866.
- Kundur, et al., “Blind Image Deconvolution,” IEEE Signal Process. Mag. 13(3), pp. 43-64 (May 1996).
- Levin et al., “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
- Levin, et al., “Understanding blind deconvolution algorithms,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, No. 12, Dec. 2011, pp. 2354-2367.
- Li, et al., “Separation of threedimensional scattering effects in tilt-series Fourier ptychography,” Ultramicroscopy 158, 1-7 (Jun. 14, 2015).
- Marcos, et al., “Vision science and adaptive optics, the state of the field,” Vis. Res. 132, 3-33 (Mar. 2017).
- Martins da Silva et al., “Photosensitivity and epilepsy: current concepts and perspectives—a narrative review,” Seizure 50, 209-218 (2017).
- Neumaier, “Solving ill-conditioned and singular linear systems: a tutorial on regularization,” SIAM Rev. 40, 636-666 (1998).
- Pan, et al., “Subwavelength resolution Fourier ptychography with hemispherical digital condensers,” Opt. Express 26, 23119-23131 (Sep. 3, 2018).
- Pan, et al., “System calibration method for Fourier ptychographic microscopy,” J. Biomed. Opt. 22, 096005 (Sep. 12, 2017).
- Pan, et al., “Three-dimensional space optimization for near-field ptychography,” Opt. Express 27, 5433-5446 (Feb. 18, 2019).
- Qian, et al., “Large-scale 3D imaging of insects with natural color,” Opt. Express 27, 4845-4857 (Feb. 18, 2019).
- Reinig, et al., “Adaptative optics microscopy enhances image quality in deep layers of CLARITY processed brains of YFP-H mice” Proc., of SPIE, vol. 9690, (Mar. 9, 2016) pp. 969008-1-969008-12. <doi:10.1117/12.2213283>.
- Rha, et al., “Adaptive optics flood-illumination camera for high speed retinal imaging,” Opt. Express 14, 4552-4569 (2006).
- Shemonski, et al., “Computational high-resolution optical imaging of the living human retina,” Nat. Photonics 9, 440-443 (Jun. 22, 2015).
- Soulez, et al., “Blind deconvolution of 3D data in wide field fluorescence microscopy” In 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI) May 2, 2012, pp. 1735-1738.
- Sun, et al., “Efficient positional misalignment correction method for Fourier ptychographic microscopy,” Biomed. Opt. Express 7, 1336-1350 (Mar. 17, 2016).
- Sun, et al., “Resolution-enhanced Fourier ptychographic microscopy based on high-numerical-aperture illuminations,” Sci. Rep. 7, 1187 (Apr. 26, 2017).
- Sun, et al., “Sampling criteria for Fourier ptychographic microscopy in object space and frequency space,” Opt. Express 24, 15765-15781 (Jul. 11, 2016).
- Thiébaut and Conan, “Strict a priori constraints for maximumlikelihood blind deconvolution,” J. Opt. Soc. Am. A, vol. 12, No. 3, Mar. 1995, pp. 485-492.
- Tian and Waller, “3D intensity and phase imaging from light field measurements in an LED array microscope,” Optica 2, 104-111 (Jan. 28, 2015).
- Tomer et al., “Advanced CLARITY for rapid and high-resolution imaging of intact tissues,” Nat. Protoc., vol. 9, No. 7, Jul. 2014, pp. 1682-1697.
- Wade, et al., “A fast, robust pattern recognition system for low light level image registration and its application to retinal imaging,” Opt. Express 3, 190-197 (1998).
- Williams, D., “Imaging Single Cells in the Living Retina,” Vis. Res. 51, pp. 1379-1396 (Jul. 1, 2011).
- Yaroslaysky, “Image Resampling and Building Continuous Image Models”, Chapter 6, Theoretical Foundations of Digital Imaging Using MATLAB , pp. 293-342 (CRC Press, 1 edition, Nov. 26, 2012).
- Yuan, et al., “Image deblurring with blurred/noisy image pairs,” ACM Trans. Graph. 26, 1 (2007).
- Zhou, et al., “What are good apertures for defocus deblurring?” in 2009 IEEE International Conference on Computational Photography (IEEE, Apr. 16-17, 2009), pp. 1-8.
- U.S. Appl. No. 16/552,948, filed Aug. 27, 2019, Chung et al.
- US Ex Parte Quayle Action dated Aug. 8, 2019 issued in U.S. Appl. No. 16/242,934.
- U.S. Notice of Allowance dated Nov. 20, 2019 in U.S. Appl. No. 15/959,050.
- U.S. Office Action dated Jan. 9, 2020 in U.S. Appl. No. 14/658,019.
- U.S. Notice of Allowance dated Jan. 6, 2020 in U.S. Appl. No. 14/960,252.
- U.S. Notice of Allowance dated Nov. 4, 2019 issued in U.S. Appl. No. 16/242,934.
- U.S. Notice of Allowance dated Dec. 9, 2019 in U.S. Appl. No. 16/162,271.
- U.S. Notice of Allowance dated Jan. 17, 2020 issued in U.S. Appl. No. 15/003,559.
- U.S. Notice of Allowance dated Jan. 24, 2020 issued in U.S. Appl. No. 15/068,389.
- Chinese Second Office Action Dec. 31, 2019 issued in CN 201580072950.8.
- Chinese Second Office Action dated Nov. 12, 2019 issued in Application No. CN 201680005491.6.
- Chinese Second Office Action dated Nov. 28, 2019 issued in Application No. CN 201680006738.6.
- U.S. Notice of Allowance dated Feb. 18, 2020 issued in U.S. Appl. No. 16/242,934.
- U.S. Notice of Allowance dated Mar. 2, 2020 in U.S. Appl. No. 14/797,154.
- U.S. Notice of Allowance dated Apr. 3, 2020 in U.S. Appl. No. 16/162,271.
- U.S. Notice of Allowance dated Apr. 7, 2020 issued in U.S. Appl. No. 16/179,688.
- Chinese Second Office Action dated Mar. 19, 2020 issued in Application No. CN 201680014898.5.
Type: Grant
Filed: Oct 28, 2013
Date of Patent: Jun 9, 2020
Patent Publication Number: 20140118529
Assignee: California Institute of Technology (Pasadena, CA)
Inventors: Guoan Zheng (Vernon, CT), Changhuei Yang (Alhambra, CA), Roarke Horstmeyer (San Marino, CA)
Primary Examiner: Albert Kir
Application Number: 14/065,280
International Classification: G06K 9/36 (20060101); G02B 27/58 (20060101); G21K 7/00 (20060101); G06K 9/00 (20060101); G02B 21/36 (20060101); G01B 1/00 (20060101); G02B 21/00 (20060101);
