Patents Issued in April 13, 2017
  • Publication number: 20170103266
    Abstract: A verification method of a fraudulent activity taking place at a self-checkout terminal is disclosed. The method verifies the fraudulent activity by confirming an incident of the fraudulent activity with multiple data generated by monitoring the transaction area and the bagging area of the self-checkout terminal. A human validation is optionally performed to verify machine-identified incidents.
    Type: Application
    Filed: December 22, 2016
    Publication date: April 13, 2017
    Inventors: Joshua Migdal, Matthew Farrow, Malay Kundu
  • Publication number: 20170103267
    Abstract: To address problems that video imaging systems and platforms face when analyzing image and video content for detection and feature extraction, a solution is provided in which accumulating significant amounts of data suitable for training and learning analytics is leveraged to improve over time, the classifiers used to perform the detection and feature extraction, by employing a larger search space and generate additional and more complex classifiers through distributed processing. A distributed learning platform is therefore provided, which is configured for operating on large scale data, in a true big data paradigm. The learning platform is operable to empirically estimate a set of optimal feature vectors and a set of discriminant functions using a parallelizable learning algorithm. A method of adding new data into a database utilized by such a learning platform is also provided.
    Type: Application
    Filed: December 16, 2016
    Publication date: April 13, 2017
    Inventors: Akshaya K. MISHRA, Nicholas MILLER, Justin A. EICHEL, Nicholas JANKOVIC, Kurtis N. MCBRIDE, Anthony I. BRIJPAUL
  • Publication number: 20170103268
    Abstract: An alignment tool is disclosed configured to provide accurate tire alignment analysis based on the comparison of video recordings capturing a direction of travel and data obtained from various vehicle on-board components.
    Type: Application
    Filed: October 9, 2015
    Publication date: April 13, 2017
    Inventors: Omar Santamaria, Rodrigo Felix
  • Publication number: 20170103269
    Abstract: A system and method for providing laser camera fusion for identifying and tracking a traffic participant that include receiving an image of a surrounding environment of a vehicle from a vehicle camera system and a set of object coordinates of at least one object determined within the surrounding environment of the vehicle from a vehicle laser projection system. The system and method also include determining a portion of the image as object space based on the image and the set of object coordinates and filtering the object space to identify a traffic related object. Additionally, the system and method include determining a three dimensional position of the traffic related object and classifying the traffic related object as at least one of: the traffic participant, or a non-traffic participant. The system and method further include tracking the traffic participant based on a three dimensional position of the traffic related object classified as the traffic participant.
    Type: Application
    Filed: October 7, 2015
    Publication date: April 13, 2017
    Inventors: Bernd Heisele, Alper Ayvaci
  • Publication number: 20170103270
    Abstract: A controller for an autonomous vehicle receives an image stream from one or more imaging devices. The controller identifies vehicle images in the image stream. Vehicle images are compared to the color, shape, badging, markings, license plate, and driver of the autonomous vehicle. If the vehicle image is determined to match the autonomous vehicle, then the vehicle image is ignored as a potential obstacle. The location of a reflective surface that generated the vehicle image may be determined and added to a set of potential obstacles. The color and shape of a vehicle in a vehicle image may be evaluated first. Only if the color and shape in the vehicle image match the autonomous vehicle are other factors such as badging, markings, license plate, and driver considered. Vehicle images not matching the autonomous vehicle are included in a set of potential obstacles.
    Type: Application
    Filed: October 7, 2015
    Publication date: April 13, 2017
    Inventors: Brielle Reiff, Madeline Jane Schrier, Nithika Sivashankar
  • Publication number: 20170103271
    Abstract: This invention provides a driving assistance system for a vehicle, with which an intersection having poor visibility can be detected with a high degree of precision and a driver can be alerted thereto while suppressing cost and time requirements. A sample determination unit identifies an image captured by a front camera as detection subject sample data relating to an intersection having poor visibility when the detection subject is detected by a stereo camera, a distance calculation unit, and a first object detection unit but the detection subject is not detected in a position of the detected detection subject by the front camera and a second object detection unit, and identifies the image captured by the front camera as non-detection subject sample data when the detection subject is detected in the position of the detected detection subject by the front camera and the second object detection unit.
    Type: Application
    Filed: April 4, 2016
    Publication date: April 13, 2017
    Applicant: Mitsubishi Electric Corporation
    Inventor: Tomoya KAWAGOE
  • Publication number: 20170103272
    Abstract: In an information processing device mounted on a vehicle, a sign recognition section detects a traffic sign from image data captured by and transmitted from an in-vehicle camera. A judgment section detects a direction to a prohibited roadway/lane based on the detected traffic sign. To the prohibited roadway/lane the vehicle is prohibited from approaching and entering. An operation detection section predicts, as a predicted driver's operation, whether or not the driver intends to move the vehicle in the detected direction to the prohibited roadway/lane. A warning control section instructs a display and an audio speaker to provide warning information to the driver when the predicted driver's operation indicates that the driver intends to move the own vehicle toward the detected direction to the prohibited roadway/lane.
    Type: Application
    Filed: September 30, 2016
    Publication date: April 13, 2017
    Inventor: Shota Satomura
  • Publication number: 20170103273
    Abstract: Techniques are described for vehicle or entity tracking. In one example, techniques include receiving a first set of data, said data comprising video or images from a first mobile data source, determining that said first set of data matches a second set of data in a first data source, the second set of data corresponding to a vehicle or entity being tracked; requesting a third set of data, said data comprising video or images from a second mobile data source, and determining a confidence level based on how well said first and third data sets match said second data set.
    Type: Application
    Filed: October 8, 2015
    Publication date: April 13, 2017
    Inventors: Hernan A Cunico, Asima Silva
  • Publication number: 20170103274
    Abstract: A traffic light detecting device includes a camera capturing an image in the travelling direction of a vehicle, and a traffic light detecting unit detecting a traffic light from the image. The unit sets reference signals the number of which is a multiple of three, and phases of which are shifted each other by a time obtained by dividing a cycle of an alternating current power supplied to the traffic light by the multiple of three, and adjusts a phase of the reference signal which is closest to the phase of the alternating current power, to the phase thereof. The unit extracts from the image a synchronized pixel with a luminance which varies in synchronization with the reference signal having the phase adjusted to the phase thereof, and judges that a traffic light is present at a position of the synchronized pixel.
    Type: Application
    Filed: March 10, 2014
    Publication date: April 13, 2017
    Inventors: Takahiko Oki, Hidekazu Nishiuchi
  • Publication number: 20170103275
    Abstract: A traffic signal recognition apparatus acquires image data by imaging the surroundings of a vehicle, detects a self-position of the vehicle, and detects, in the image, a traffic signal around the vehicle. The traffic signal recognition apparatus identifies two or more of the traffic signal predicted to be captured in the image, based on information on the self-position and map information containing positional information on the traffic signals, and assigns a priority level to each of the two or more traffic signals thus identified, based on a possibility of the traffic signal being blocked out of sight. When the two or more traffic signals are identified, the traffic signal detected in the image is a traffic signal with a highest priority level among the two or more traffic signals.
    Type: Application
    Filed: May 20, 2014
    Publication date: April 13, 2017
    Applicant: Nissan Motor Co., Ltd.
    Inventors: Daiki Yamanoi, Toshiyuki Andou, Hidekazu Nishiuchi, Haruo Matsuo, Hiroshi Satoh
  • Publication number: 20170103276
    Abstract: A method may comprise obtaining image data of a plurality of digital image frames captured of an assumed eye having an iris and a pupil while illuminating the assumed eye from different directions; obtaining comparison data dependent on similarities in parts of image data of at least two of the plurality of digital image frames corresponding to an area of interest of the assumed eye; and automatically determining, on the basis of the comparison data, presence of a verification feature in the assumed eye indicating a real eye.
    Type: Application
    Filed: October 8, 2015
    Publication date: April 13, 2017
    Inventors: Jyri Rauhala, Pertti Husu
  • Publication number: 20170103277
    Abstract: A method, computer program product, and a computer system is provided. A processor determines a first viewpoint of the point of interest, the first viewpoint includes a first image group captured from a first location. A processor retrieves an image from an online repository. In response to content of the image matching content of the first image group, a processor assigns a location of the image to the first viewpoint. A processor determines a second viewpoint of the point of interest, the second viewpoint includes a second image group captured from a second location. A processor determines a first ranking score for the first viewpoint based, in part, on the first image group. A processor determines a second ranking score for the second viewpoint based, at least in part, on the second image group. A processor generates a ranked list of the first viewpoint and the second viewpoint based.
    Type: Application
    Filed: December 20, 2016
    Publication date: April 13, 2017
    Inventors: James E. Bostick, John M. Ganci, JR., Martin G. Keen, Sarbajit K. Rakshit
  • Publication number: 20170103278
    Abstract: A method, computer program product, and a computer system is provided. A processor determines a first viewpoint of the point of interest, the first viewpoint includes a first image group captured from a first location. A processor retrieves an image from an online repository. In response to content of the image matching content of the first image group, a processor assigns a location of the image to the first viewpoint. A processor determines a second viewpoint of the point of interest, the second viewpoint includes a second image group captured from a second location. A processor determines a first ranking score for the first viewpoint based, in part, on the first image group. A processor determines a second ranking score for the second viewpoint based, at least in part, on the second image group. A processor generates a ranked list of the first viewpoint and the second viewpoint based.
    Type: Application
    Filed: December 20, 2016
    Publication date: April 13, 2017
    Inventors: James E. Bostick, John M. Ganci, JR., Martin G. Keen, Sarbajit K. Rakshit
  • Publication number: 20170103279
    Abstract: Systems and methods generate a projected image at an optimal exposure time. Images are captured different exposure times. Pixels that satisfy an intensity threshold percentage for each image are selected. The intensity values of the selected pixels are then evaluated to determine whether the selected pixels are distributed above a lower intensity threshold and below an upper intensity threshold. The linear relationship is projected to determine an optimal exposure time that has an optimal exposure time duration that exceeds each exposure time duration associated with each of the captured images when the linear relationship exists between each of the captured images. A projected image associated with the optimal exposure time is generated from one or more of the captured images.
    Type: Application
    Filed: October 14, 2016
    Publication date: April 13, 2017
    Inventors: Suk HONG, Nikki JARRETT, Eric HOMMEMA, Jason ARAVICH, Yanpeng CAO, Benyong SHI, Kok Siong TEO
  • Publication number: 20170103280
    Abstract: A processing apparatus includes a luminance information obtainer that obtains luminance information for each of a plurality of images, each of which is shot respectively when the light source arranged at mutually different positions illuminates an object, and a normal information determiner that determines normal information of the object from a plurality of normal candidates calculated using three or more polarization images with polarization states different in each other on the basis of the luminance information.
    Type: Application
    Filed: September 27, 2016
    Publication date: April 13, 2017
    Inventors: Yuichi Kusumi, Chiaki Inoue, Yoshiaki Ida
  • Publication number: 20170103281
    Abstract: In one embodiment, a system includes: a processor; and logic in and/or executable by the processor to cause the processor to: generate a first feature vector based on a digital image captured by a mobile device; compare the first feature vector to a plurality of reference feature matrices; classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparison; determine one or more object features of the object based at least in part on the particular object class; and detect one or more additional objects belonging to the particular object class based on the determined object feature(s). The one or more additional objects are depicted either in the digital image or another digital image received by the mobile device. Corresponding computer program products are also disclosed.
    Type: Application
    Filed: December 20, 2016
    Publication date: April 13, 2017
    Inventors: Jan W. Amtrup, Anthony Macciola, Steve Thompson, Jiyong Ma, Alexander Shustorovich, Christopher W. Thrasher
  • Publication number: 20170103282
    Abstract: A method, system and computer program product for encoding an image is provided. The image that needs to be represented is represented in the form of a Gaussian pyramid which is a scale-space representation of the image and includes several pyramid images. The feature points in the pyramid images are identified and a specified number of feature points are selected. The orientations of the selected feature points are obtained by using a set of orientation calculating algorithms. A patch is extracted around the feature point in the pyramid images based on the orientations of the feature point and the sampling factor of the pyramid image. The boundary patches in the pyramid images are extracted by padding the pyramid images with extra pixels. The feature vectors of the extracted patches are defined. These feature vectors are normalized so that the components in the feature vectors are less than a threshold.
    Type: Application
    Filed: December 21, 2016
    Publication date: April 13, 2017
    Inventors: Mark A. Ruzon, Raghavan Manmatha, Donald Ranguay
  • Publication number: 20170103283
    Abstract: A method, computer program product, and a computer system is provided. A processor determines a first viewpoint of the point of interest, where the first viewpoint includes a first image group captured from a first location. A processor determines a second viewpoint of the point of interest, where the second viewpoint includes a second image group captured from a second location. A processor determines a first ranking score for the first viewpoint based on the first image group. A processor determines a second ranking score for the second viewpoint based on the second image group. A processor generates a ranked list of the first viewpoint and the second viewpoint based on the first ranking score and the second ranking score.
    Type: Application
    Filed: October 9, 2015
    Publication date: April 13, 2017
    Inventors: James E. Bostick, John M. Ganci, JR., Martin G. Keen, Sarbajit K. Rakshit
  • Publication number: 20170103284
    Abstract: Systems and methods are provided for identifying a subset of exemplar images in a set of exemplary images that may be used to generate an image classifier having one or more desirable performance characteristics. A computing system may select every possible subset of exemplar images from a set of exemplar images and may generate a classifier for each selected subset of images. The computing system may use each generated classifier to classify one or more images in order to generate performance statistics related to how each classifier performed when classifying the one or more images. The computing system may identify a particular subset of exemplar images that is associated with performance statistics of a classifier that satisfies one or more desired performance characteristics.
    Type: Application
    Filed: October 7, 2015
    Publication date: April 13, 2017
    Inventors: Daniel Scott Rosen, Cambron Neil Carter
  • Publication number: 20170103285
    Abstract: A method and a device for detecting copies or near-copies of images, comprises receiving an initial image, converting the initial image to grayscale, resizing the grayed image to a reduced image having a plurality of rows and an even number of columns, computing an overall signature for the reduced image, and determining whether the initial image is a copy or near-copy of an image according to the result of a comparison between the overall signature of the reduced image and reference image signatures. The step of computing the overall signature comprises the steps of computing a row signature for each row of the reduced image, the computation being based on a comparison of values obtained statistically across subsets of symmetrical pixels in each row, and concatenating the row signatures in order to obtain an overall signature.
    Type: Application
    Filed: August 11, 2016
    Publication date: April 13, 2017
    Inventors: Hervé LE BORGNE, Etienne GADESKI, Adrian POPESCU
  • Publication number: 20170103286
    Abstract: The methods and systems described herein relate to a method of processing and storing images. The method is executed by a server and comprises: acquiring, by the server, a first image and a second image; determining, by the server, a first mapping information between the first image and the second image; generating, by the server, a reference image based on the first image and the second image, the reference image comprising a set of synthetic pixels; computing, by the server, a first parameter and a second parameter, the first parameter being at least partially indicative of a visual dissimilarity between the first pixels of the first image and the respective synthetic pixels and the second parameter being at least partially indicative of the visual dissimilarity between the second pixels of the second image and the respective synthetic pixels.
    Type: Application
    Filed: September 27, 2016
    Publication date: April 13, 2017
    Inventor: Alexander Alexandrovich OSIPOV
  • Publication number: 20170103287
    Abstract: Systems and methods are provided for generating a pseudo-CT prediction model using multi-channel MR images. An exemplary system may include a processor configured to retrieve training data including multiple MR images and at least one CT image for each of a plurality of training subjects. For each training subject, the processor may determine at least one tissue parameter map based on the multiple MR images and obtain CT values based on the at least one CT image. The processor may also generate the pseudo-CT prediction model based on the tissue parameter maps and the CT values of the plurality of training subjects.
    Type: Application
    Filed: October 13, 2015
    Publication date: April 13, 2017
    Inventor: Xiao HAN
  • Publication number: 20170103288
    Abstract: A print control device includes a receiver that receives a print job and a processor that generates image data by rasterizing data of each page of the print job in parallel with a reception of the print job and controls a page order for rasterizing the data of each page. The processor rasterizes the data of each page in an order from a front page until a finish of a reception of data of a predetermined page determined according to a binding format of an output document to be prepared by the print job. After the reception of the data of the predetermined page is finished, the processor controls the page order to optimally rasterize the data of each page according to the binding format.
    Type: Application
    Filed: September 15, 2016
    Publication date: April 13, 2017
    Applicant: Konica Minolta, Inc.
    Inventor: Kouichirou Matsushita
  • Publication number: 20170103289
    Abstract: When a job for executing both of a one-sided printing and a two-sided printing, the lowering of the productivity is suppressed and an output result a user desires is output.
    Type: Application
    Filed: December 20, 2016
    Publication date: April 13, 2017
    Inventor: Yuuki MAEDA
  • Publication number: 20170103290
    Abstract: A method for managing a data center that includes racks arranged in aisles, includes guiding an operator, by a mobile terminal, to a desired device of a rack. The guiding step includes: indicating, on a screen of the mobile terminal, a route to follow to arrive near the device; once the operator is near the device, reading, by a reading application of the mobile terminal, an electronic marker pattern placed on a first rack facing the operator to determine if the operator is facing the rack including the desired device; if not, repeating the reading operation on the rack directly adjacent to the first rack; once the rack is identified, reading, by the reading application of the mobile terminal, an optical marker pattern placed on the rack so as to obtain a height reference and thus locate the desired device; and acting upon the desired device using the mobile terminal.
    Type: Application
    Filed: March 20, 2015
    Publication date: April 13, 2017
    Inventors: Christophe GUIONNEAU, Matthieu ISOARD, Xavier PLATTARD
  • Publication number: 20170103291
    Abstract: An apparatus (1) for the creation of products incorporating devices for radio frequency identification (RFID), said apparatus (1) being configured to operate according to a continuous processing line which includes, in sequence: a supply unit (2) of a substrate material; an add-chip unit (4), having means for the application of chips on the substrate material (41); an aerial-making assembly comprising: an aerial posing unit (5), configured to deposit a metal wire at each chip; an electrical connection unit, configured for electrically connecting the metal wire and a respective chip therebetween; a coupling unit (7), adapted to place a coating material on the substrate material; and a plurality of reading means (42, 62, 9) of each chip, arranged along said processing line, wherein said unit of aerial posing (5) comprises a plurality of devices (100) for realizing an aerial, wherein each device (100) is provided with a putting wire head (150) arranged at one end of a main body (101) and having a transla
    Type: Application
    Filed: October 6, 2016
    Publication date: April 13, 2017
    Inventors: Marcello LOLLI, Massimiliano CARLUCCI, Marco CARLUCCI
  • Publication number: 20170103292
    Abstract: A piece of baggage including a body of the piece of baggage, a display integrated in the body of the piece of baggage and configured as an electronic tag. An indicating area, having flight and personal data required for a flight. The indicating area has a length (A) ranging between 160 and 200 mm and a width (B) ranging between 38 and 60 mm.
    Type: Application
    Filed: December 7, 2016
    Publication date: April 13, 2017
    Applicant: RIMOWA Electronic Tag GmbH
    Inventors: Jan REH, Dieter MORSZECK
  • Publication number: 20170103293
    Abstract: A method of securing a redundant initial two-dimensional barcode includes providing an initial message having a unique identifier, encoding the initial message as a set of predefined symbols so as to generate a redundant initial two-dimensional barcode having a redundant zone, generating a unique secret character string having a predefined fixed size, encoding the unique secret character string as a set of predefined symbols identical to the symbols used to generate the redundant initial two-dimensional barcode, and inserting the encoded unique secret character string into a redundant zone of the redundant initial two-dimensional barcode, each symbol of said encoded unique secret character string having a predefined position, so as to create a final two-dimensional barcode.
    Type: Application
    Filed: October 11, 2016
    Publication date: April 13, 2017
    Inventor: Marc Pic
  • Publication number: 20170103294
    Abstract: The present invention is generally directed towards a card and package assembly and methods of making the same. Card and package assemblies in accordance with some embodiments of the present invention may include a package, a data card, the data card packaged at least in part within the package, and an activation indicia, the activation indicia comprising a first portion printed on the package and a second portion printed on the data card. Methods of packaging a data card in accordance with some embodiments of the present invention may include steps of manufacturing or otherwise obtaining a data card, manufacturing or otherwise obtaining a package, determining an activation indicia, packaging the data card at least in part within the package, and printing the activation indicia in part on the data card and in part on the package.
    Type: Application
    Filed: December 20, 2016
    Publication date: April 13, 2017
    Inventors: Merrill Brooks SMITH, Chandilyn Smith
  • Publication number: 20170103295
    Abstract: A support device, which supports a belt-shaped sheet wound around a hollow cylindrical shaft core so as to feed the belt-shaped sheet, includes a support shaft that is insertable into a hollow part of the shaft core, a frame that supports the support shaft, an antenna surface perpendicular to an axis line of the support shaft, and a loop antenna provided to the antenna surface. The shaft core includes a non-contact data carrier capable of at least one of storing and transmitting predetermined data. The loop antenna includes a loop unit coiled to define a loop surrounding the axis line. When the shaft core is seen through the loop antenna along the axis line, the loop unit is disposed in the vicinity of the shaft core.
    Type: Application
    Filed: March 18, 2015
    Publication date: April 13, 2017
    Inventors: Kaori MATSUSHITA, Katsumi KATAKURA, Masayuki NARUKAWA, Kenichi WATANABE, Naoya OKAMOTO
  • Publication number: 20170103296
    Abstract: A wireless input device comprises a plurality of RFID tag units and keys. Each RFID tag unit is coupled with two conductive wires. Only when the two conductive wires are in a closed-loop status, the RFID tag unit can generate a responsive RF signal corresponding to a scanning signal generated by a host device. Each key is formed with a switching mechanism connectable to the conductive wires of at least one of the RFID tag units. When any one of the keys is pressed, the conductive wires connectable to the pressed key will be switched to the closed-loop status from an open-loop status, and when that pressed key is released, the conductive wires connectable to the released key will be switched back to said open-loop status. Therefore, the wireless input device can be operated without the need of internal electric power.
    Type: Application
    Filed: October 7, 2015
    Publication date: April 13, 2017
    Inventors: Richard Hwang, Memiee L. Hwang
  • Publication number: 20170103297
    Abstract: In a tracking device, a crescent-shaped PCB partially encircles a battery to minimize thickness of the device. A speaker and an LED emit alerts upon command of a control apparatus or in response to motion or temperature sensed by sensor. A local network has one hub to tracking devices and a wider area network has multiple hubs for more detailed tracking of devices. A wide area network tracks devices anywhere and stores data of each tracking device including its last known position and its sensor data.
    Type: Application
    Filed: December 16, 2016
    Publication date: April 13, 2017
    Applicant: PB Inc.
    Inventors: Daniel Daoura, Nicholas Pearson-Franks
  • Publication number: 20170103298
    Abstract: A method for implementing a convolutional neural network (CNN) accelerator on a target includes identifying characteristics and parameters for the CNN accelerator. Resources on the target are identified. A design for the CNN accelerator is generated in response to the characteristics and parameters of the CNN accelerator and the resources on the target.
    Type: Application
    Filed: October 9, 2015
    Publication date: April 13, 2017
    Inventors: Andrew Chaang Ling, Gordon Raymond Chiu, Utku Aydonat
  • Publication number: 20170103299
    Abstract: A method for implementing a convolutional neural network (CNN) accelerator on a target includes utilizing one or more processing elements to implement a standard convolution layer. A configuration of the CNN accelerator is modified to change a data flow between components on the CNN accelerator. The one or more processing elements is utilized to implement a fully connected layer in response to the change in the data flow.
    Type: Application
    Filed: February 6, 2016
    Publication date: April 13, 2017
    Inventors: Utku Aydonat, Gordon Raymond Chiu, Andrew Chaang Ling
  • Publication number: 20170103300
    Abstract: A neural network unit configurable to first/second/third configurations has N narrow and N wide accumulators, multipliers and adders. Each multiplier performs a narrow/wide multiply on first and second narrow/wide inputs to generate a narrow/wide product. A first adder input receives a corresponding narrow/wide accumulator's output and third input receives a widened corresponding narrow multiplier's narrow product in the third configuration. In the first configuration, each narrow/wide adder performs a narrow/wide addition on the first and second inputs to generate a narrow/wide sum for storage into the corresponding narrow/wide accumulator. In the second configuration, each wide adder performs a wide addition on the first and a second input to generate a wide sum for storage into the corresponding wide accumulator. In the third configuration, each wide adder performs a wide addition on the first, second and third inputs to generate a wide sum for storage into the corresponding wide accumulator.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103301
    Abstract: A processor has an instruction fetch unit that fetches ISA instructions from memory and execution units that perform operations on instruction operands to generate results according to the processor's ISA. A hardware neural network unit (NNU) execution unit performs computations associated with artificial neural networks (ANN). The NNU has an array of ALUs, a first memory that holds data words associated with ANN neuron outputs, and a second memory that holds weight words associated with connections between ANN neurons. Each ALU multiplies a portion of the data words by a portion of the weight words to generate products and accumulates the products in an accumulator as an accumulated value. Activation function units normalize the accumulated values to generate outputs associated with ANN neurons. The ISA includes at least one instruction that instructs the processor to write data words and the weight words to the respective first and second memories.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103302
    Abstract: A neural network unit. A register holds an indicator that specifies narrow and wide configurations. A first memory holds rows of 2N/N narrow/wide weight words in the narrow/wide configuration. A second memory holds rows of 2N/N narrow/wide data words in the narrow/wide configuration. An array of neural processing units (NPU) is configured as 2N/N narrow/wide NPUs and to receive the 2N/N narrow/wide weight words of rows from the first memory and to receive the 2N/N narrow/wide data words of rows from the second memory in the narrow/wide configuration. In the narrow configuration, the 2N NPUs perform narrow arithmetic operations on the 2N narrow weight words and the 2N narrow data words received from the first and second memories. In the wide configuration, the N NPUs perform wide arithmetic operations on the N wide weight words and the N wide data words received from the first and second memories.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103303
    Abstract: A neural network unit has at least one RAM, an output buffer and an array of neural processing units that: read first time step context layer node values from the output buffer; read second time step input layer node values from the RAM; generate second time step hidden layer node values based on the read input and context layer node values; output the hidden layer node values to the output buffer rather than to the RAM; read the hidden layer node values from the output buffer; generate second time step context layer node values based on the read hidden layer node values; output the context layer node values to the output buffer rather than to the RAM; generate output layer node values using the hidden layer node values; write the output layer node values to the RAM; and repeat for a sequence of time steps.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS, KYLE T. O'BRIEN
  • Publication number: 20170103304
    Abstract: A neural network unit includes a register programmable with a control value, a plurality of neural processing units (NPU), and a plurality of activation function units (AFU). Each NPU includes an arithmetic logic unit (ALU) that performs arithmetic and logical operations on a sequence of operands to generate a sequence of results and an accumulator into which the ALU accumulates the sequence of results as an accumulated value. Each AFU includes a first module that performs a first function on the accumulated value to generate a first output, a second module that performs a second function on the accumulated value to generate a second output, the first function is distinct from the second function, and a multiplexer that receives the first and second outputs and selects one of the two outputs based on the control value programmed into the register.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103305
    Abstract: An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each of the N words. N processing units (PU) are arranged as N/J mutually exclusive PU groups. Each PU group has an associated OBWG. Each PU includes an accumulator and an arithmetic unit that performs operations on inputs, which include the accumulator output, to generate a first result for accumulation into the accumulator. Activation function units selectively perform an activation function on the accumulator outputs to generate results for provision to the N output buffer words. For each PU group, four of the J PUs and at least one of the activation function units compute an input gate, a forget gate, an output gate and a candidate state of a Long Short Term Memory (LSTM) cell, respectively, for writing to respective first, second, third and fourth words of the associated OBWG.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. Glenn HENRY, Terry PARKS, Kyle T. O'BRIEN
  • Publication number: 20170103306
    Abstract: An array of N processing units (PU) each has: an accumulator; an arithmetic unit performs an operation on first, second and third inputs to generate a result to store in the accumulator, the first input receives the accumulator output; a weight input is received by the second input to the arithmetic unit; a multiplexed register has first and second data inputs, an output received by the third input to the arithmetic unit, and a control input that controls the data input selection. The multiplexed register output is also received by an adjacent PU's multiplexed register second data input. The N PU's multiplexed registers collectively operate as an N-word rotater when the control input specifies the second data input. Respective first/second memories hold W/D rows of N weight/data words and provide the N weight/data words to the corresponding weight/multiplexed register first data inputs. A sequencer controls the multiplexer and memories.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103307
    Abstract: A processor includes a front-end portion that issues instructions to execution units that execute the issued instructions. A hardware neural network unit (NNU) execution unit includes a first memory that holds data words associated with artificial neural networks (ANN) neuron outputs, a second memory that holds weight words associated with connections between ANN neurons, and a third memory that holds a program comprising NNU instructions that are distinct, with respect to their instruction set, from the instructions issued to the NNU by the front-end portion of the processor. The program performs ANN-associated computations on the data and weight words. A first instruction instructs the NNU to transfer NNU instructions of the program from architectural general purpose registers to the third memory. A second instruction instructs the NNU to invoke the program stored in the third memory.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103308
    Abstract: Technical solutions are described to accelerate training of a multi-layer convolutional neural network. According to one aspect, a computer implemented method is described. A convolutional layer includes input maps, convolutional kernels, and output maps. The method includes a forward pass, a backward pass, and an update pass that each include convolution calculations. The described method performs the convolutional operations involved in the forward, the backward, and the update passes based on a first, a second, and a third perforation map respectively. The perforation maps are stochastically generated, and distinct from each other. The method further includes interpolating results of the selective convolution operations to obtain remaining results. The method includes iteratively repeating the forward pass, the backward pass, and the update pass until the convolutional neural network is trained. Other aspects such as a system, apparatus, and computer program product are also described.
    Type: Application
    Filed: October 8, 2015
    Publication date: April 13, 2017
    Inventors: LELAND CHANG, SUYOG GUPTA
  • Publication number: 20170103309
    Abstract: Technical solutions are described to accelerate training of a multi-layer convolutional neural network. According to one aspect, a computer implemented method is described. A convolutional layer includes input maps, convolutional kernels, and output maps. The method includes a forward pass, a backward pass, and an update pass that each include convolution calculations. The described method performs the convolutional operations involved in the forward, the backward, and the update passes based on a first, a second, and a third perforation map respectively. The perforation maps are stochastically generated, and distinct from each other. The method further includes interpolating results of the selective convolution operations to obtain remaining results. The method includes iteratively repeating the forward pass, the backward pass, and the update pass until the convolutional neural network is trained. Other aspects such as a system, apparatus, and computer program product are also described.
    Type: Application
    Filed: November 30, 2015
    Publication date: April 13, 2017
    Inventors: LELAND CHANG, SUYOG GUPTA
  • Publication number: 20170103310
    Abstract: A neural network unit (NNU) includes N neural processing units (NPU). Each NPU has an arithmetic unit and an accumulator. First and second multiplexed registers of the N NPUs collectively selectively operate as respective first and second N-word rotaters. First and second memories respectively hold rows of N weight/data words and provide the N weight/data words of a row to corresponding ones of the N NPUs. The NPUs selectively perform: multiply-accumulate operations on rows of N weight words and on a row of N data words, using the second N-word rotater; convolution operations on rows of N weight words, using the first N-word rotater, and on rows of N data words, the rows of weight words being a data matrix, and the rows of data words being elements of a convolution kernel; and pooling operations on rows of N weight words, using the first N-word rotater.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS
  • Publication number: 20170103311
    Abstract: A neural network unit has a first memory that holds elements of a data matrix and a second memory that holds elements of a convolution kernel. An array of neural processing units (NPU) each have a multiplexed register that receives a corresponding element of a row from the first memory and that also receives the multiplexed register output of an adjacent NPU. A register receives a corresponding element of a row from the second memory. An arithmetic unit receives the outputs of the register, the multiplexed register and an accumulator and performs a multiply-accumulate operation on them. For each sub-matrix of a plurality of sub-matrices of the data matrix, each arithmetic unit selectively receives either the element from the first memory or the adjacent NPU multiplexed register output and performs a series of the multiply-accumulate operations to accumulate into the accumulator a convolution of the sub-matrix with the convolution kernel.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS, KYLE T. O'BRIEN
  • Publication number: 20170103312
    Abstract: An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each. N processing units (PU) are arranged as N/J mutually exclusive PU groups each having an associated OBWG. Each PU has an accumulator, arithmetic unit, and first and second multiplexed registers each having at least J+1 inputs. A first input receives a memory operand and the other J inputs receive the J words of the associated OBWG. Each accumulator provides its output to a respective OBWG. Each arithmetic unit performs an operation on the first and second multiplexed register outputs and accumulator output to generate a result for accumulation into the accumulator. A mask input to the output buffer controls which words, if any, of the N words retain their current value or are updated with their respective accumulator output. Each PU group operates as a recurrent neural network LSTM cell.
    Type: Application
    Filed: April 5, 2016
    Publication date: April 13, 2017
    Inventors: G. GLENN HENRY, TERRY PARKS, KYLE T. O'BRIEN
  • Publication number: 20170103313
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.
    Type: Application
    Filed: December 22, 2016
    Publication date: April 13, 2017
    Inventors: Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu
  • Publication number: 20170103314
    Abstract: A circuit for performing neural network computations for a neural network, the circuit comprising: a systolic array comprising a plurality of cells; a weight fetcher unit configured to, for each of the plurality of neural network layers: send, for the neural network layer, a plurality of weight inputs to cells along a first dimension of the systolic array; and a plurality of weight sequencer units, each weight sequencer unit coupled to a distinct cell along the first dimension of the systolic array, the plurality of weight sequencer units configured to, for each of the plurality of neural network layers: shift, for the neural network layer, the plurality of weight inputs to cells along the second dimension of the systolic array over a plurality of clock cycles and where each cell is configured to compute a product of an activation input and a respective weight input using multiplication circuitry.
    Type: Application
    Filed: December 22, 2016
    Publication date: April 13, 2017
    Inventor: Jonathan Ross
  • Publication number: 20170103315
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.
    Type: Application
    Filed: December 22, 2016
    Publication date: April 13, 2017
    Inventors: Gregory Michael Thorson, Christopher Aaron Clark, Dan Luu