Abstract: Disclosed examples to detect and annotate backedges in data-flow graphs include: a characteristic detector to store a node characteristic identifier in memory in association with a first node of a dataflow graph; a characteristic comparator to compare the node characteristic identifier with a reference criterion; and a backedge identifier generator to generate a backedge identifier indicative of a backedge between the first node and a second node of the dataflow graph based on the comparison, the memory to store the backedge identifier in association with a connection arc between the first and second nodes.

Abstract: A computer-implemented method, computer system and computer program product dynamically manage failure in an edge computing environment. According to the method, a request for executing a task may be sent to a first edge device according to a defined process, where the defined process is used to schedule tasks to be executed on edge devices. In response to the first edge device failing to execute the task, the defined process may be suspended. Then, a request for executing the task may be sent to a second edge device. A task result that is received first may be taken as the task result for the task, where the task result is from either the first edge device or the second edge device. And, continuing the rest of the defined process.

Abstract: According to one embodiment, a method that supports queries deploying operators based on multiple programming languages is described. A sequence of operators associated with a query is identified, where the sequence of operators includes at least two neighboring operators including a first operator based on a first programming language and a second operator based on a second programming language that is different from the first programming language. Thereafter, a schema associated with the first operator and a schema associated with the second operator is determined along with the compatibility between the schema of the first operator and the schema of the second operator. A query error message is generated in response to incompatibility between the first operator schema and the second operator schema. Compatibility is determined when an output generated by execution of the first operator provides machine data needed as input for execution of the second operator.

Abstract: A graphics rendering processor receives data related to a display and a user's gaze which is directed at the display. The user gaze may be detected based on inputs received from an optical sensor, such as a near-infrared sensor. The processor then renders different portions of the display based on the user gaze, such that an area where the user gaze is directed will receive higher rendering priority than an area at which the user gaze is not directed. In a processor with multiple cores which differ in precision, operation cost, etc. a controller may determine what portion of the display to render on which cores, based on the detected user gaze, content, or a combination thereof.

Abstract: Described herein includes a calculation scenario of a calculation engine that efficiently partitions data for processing at separate hosts, including in parallel, and unions intermediate results from such separate processing when required for further processing. Such parallel processing of partitions can allow for faster processing times, and such unioning of data only when required for further processing can limit the transferring of data that results in slower processing.

Abstract: An apparatus and method of operating a data processing apparatus are disclosed. The apparatus comprises data processing circuitry to perform data processing operations in response to a sequence of instructions, wherein the data processing circuitry is capable of performing speculative execution of at least some of the sequence of instructions. A cache structure comprising entries stores temporary copies of data items which are subjected to the data processing operations and speculative execution tracking circuitry monitors correctness of the speculative execution and responsive to indication of incorrect speculative execution to cause entries in the cache structure allocated by the incorrect speculative execution to be evicted from the cache structure.

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as map-reduce functions, using overlapping symbols. Storage nodes are configured to partition data units into symbols that include an overlap data portion of an adjacent symbol and erasure encode the symbols. The storage nodes may then decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as serial functions. Storage nodes are configured to partition decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, process the subunits using a serial function to generate intermediate contexts, and send the intermediate context to a next storage node for continued processing using the serial function.

Abstract: Techniques are disclosed for managing a series of blocks in a distributed system. One embodiment presented herein includes a computer-implemented method, which includes dividing the series of blocks into a plurality of groups. The method further includes distributing the plurality of groups to a plurality of processors. The plurality of processors may apply one or more functions to each group of the plurality of groups in parallel. The method further includes receiving, from the plurality of processors, results of the one or more functions. The method further includes merging the results to generate combined results. The combined results may be used in processing data.

Abstract: An apparatus and method for loop flattening and reduction in a SIMD pipeline including broadcast, move, and reduction instructions. One embodiment of a processor comprises: a decoder to decode a broadcast instruction to generate a decoded broadcast instruction identifying a plurality of operations, the broadcast instruction including an opcode and first and second source operands, and having a split value associated therewith; and execution circuitry to execute the operations of the decoded broadcast instruction to copy a first data element specified by the first source operand to each of a first set of contiguous data element locations in a destination register and to copy a second data element specified by the second source operand to a second set of contiguous data element locations in the destination register, wherein the first and second sets of contiguous data element locations are determined in accordance with the split value.

Abstract: A computer processor includes an issue queue to receive an instruction, and one or more execution units to generate a condition code bit corresponding to the instruction. A branch condition queue is in signal communication with the issue queue, and receives the instruction from the issue queue before the at least one execution unit generates the condition code bit.

Abstract: An apparatus for data transfer includes a first node connected to a bus to communicate bidirectionally, and second nodes connected in series to the bus. Each second node lacks an internal clock and has fixed pads including a power, a ground, and signal pads to transfer a data frame, return data pads for a return signal, select pads for a selection signal and clock pads for a clock signal. Each second node is indexed by a hop count in the frame that is incremented each time the frame is transferred in topological order. Each second node is selectable using a mode defined by a combination of the hop count, a mask field and an address field in the frame. The signal pads are used for frame transfer in a selected mode controlled by a combination of the selection, clock and return signals.

Abstract: An apparatus has an input interface for receiving instruction execution information from processing circuitry, and trace generation circuitry for generating from the instruction execution information a trace stream. The instruction sequence from the processing circuitry includes at least one branch-future instruction that effectively turns an instruction identified by the branch-future instruction into a branch, and in particular causes the processing circuitry to branch to a target address identified by the branch-future instruction when that identified instruction is encountered within the instruction sequence.

Abstract: A method of scheduling a dedicated processing resource includes: obtaining source code of an application to be compiled; extracting, during compiling of the source code, metadata associated with the application, the metadata indicating an amount of the dedicated processing resource required by the application; and obtaining, based on the metadata, the dedicated processing resource allocated to the application. In this manner, performance of the dedicated processing resource scheduling system and resource utilization is improved.

Abstract: Example storage systems, storage nodes, and methods provide parallel storage node processing of data functions, such as map-reduce functions. Storage nodes are configured to decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits in parallel using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Processor instruction scheduling by: providing a set of program instructions, selecting instructions for reordering from the set of program instructions, reordering the instructions according to instruction properties, assigning sequential instruction tags to the instructions, tagging the instructions for completion as a group in a completion table; and executing the instructions.

Abstract: Computer technology facilitating optimization of a container image is provided. In one example, a computer-implemented method includes: analyzing a source container image to obtain a content of the source container image and building an initial target container image using a set of base image layers based on the content of the source container image. The computer-implemented method also includes determining content difference between the initial target container image and the source container image, and building a final target container image based on the initial target container image and the content difference.

Abstract: Particular embodiments described herein provide for an electronic device that can be configured to identify a process running on the electronic device, assign a reputation to the process if the process has a known reputation, determine if the process includes executable code, determine a reputation for the executable code, and combine the reputation for the executable code with the reputation assigned to the process to create a new reputation for the process.

Abstract: A dependency analyzer for a data processing system comprising at least one computer hardware processor configured to generate dependency information among variables, which may appear in any of multiple programs written in different source languages. The data processing system may parse each program, regardless of the language in which the module was written. Parsed information about each program may be recorded in a first-type data structure and then may be converted to a format representing dependencies among variables. Dependency information for each of the plurality of programs may be expressed as a combination of language independent constructs, which may be processed together, to generate dependency information for the data processing system. The dependency information may be recorded in a dependency data structure and further used for operations, such as data quality checking and change control for the data processing program.

Abstract: Systems, apparatuses, and methods may provide for technology to process graphical data, and to modify a runtime environment in a parallel computing platform for a graphic environment.

Abstract: A transient IR-drop waveform measurement system and method for a high speed integrated circuit are provided. The system includes all-digital elements and is based on a ring oscillator in GHz. Through oscillation with a Fast Ring Oscillator, sampling with an Edge Detector and counting with a Ripple Counter, a width and a peak of an IR-drop waveform are obtained. Moreover, a power supply network is adapted during a clock cycle through sending an adaptation signal to a connected dynamic voltage frequency scaling (DVFS) system. The measurement method includes 11 steps. The measurement system has following features: 1) IR-drop peak/width measurement ability; 2) low fabrication and test cost; 3) high accuracy and sensitivity; 4) early adaptation ability. Therefore, the measurement system can be used alone for chip monitoring or testing, in order to reduce a power supply noise disturbance to a chip.

Abstract: An apparatus in one embodiment comprises at least one processing device having a processor coupled to a memory. The processing device is configured to distribute in-memory computations across at least first and second nodes of respective distinct data processing clusters of a plurality of data processing clusters over at least one network, and to aggregate results of the distributed in-memory computations for delivery to a requesting client device. The data processing clusters are associated with respective distinct data zones, and the first and second nodes of the respective distinct data processing clusters are configured to perform corresponding portions of the distributed in-memory computations utilizing respective ones of first and second in-memory datasets locally accessible within their respective data zones. The in-memory computations in some embodiments illustratively comprise Spark computations, such as Spark Core batch computations.

Abstract: Example implementations relate to scaling a processing system. An example implementation includes receiving an application having a number of operators for performing a service in the processing system. A metric of the processing system may be monitored while the application runs, and the processing system may be scaled where the metric surpasses a threshold. In an example, the processing system may be scaled by increasing or decreasing the number of operators of the application.

Abstract: An example method of initializing a plurality of processors in a hardware platform of computing device for use by system software executing on the hardware platform includes: parsing a descriptor table that has been loaded into memory from firmware to identify an original boot protocol for initializing at least one secondary processor of the plurality of processors; creating at least one mailbox structure in the memory associated with the at least one secondary processor; causing the at least one secondary processor to execute secondary processor initialization code stored in the memory, the secondary processor initialization code implementing a mailbox-based boot protocol that uses the at least one mailbox structure to initialize the at least one secondary processor; and modifying the descriptor table to identify the mailbox-based boot protocol for initializing the at least one secondary processor in place of the original boot protocol.

Abstract: Disclosed herein provides enhancements for generating images for deployment in processing environments. In one implementation a method of preparing service images for large scale data processing environments includes identifying a first service image, and identifying metadata that defines runtime requirements for deploying the service in data processing environments. The method further provides generating scripts for deploying the service based on the metadata, and generating a second service image for the service, wherein the second service image includes the scripts.

Abstract: Provided are a MapReduce apparatus and a MapReduce control apparatus and method. The MapReduce apparatus includes a map calculator configured to perform a map calculation, a combiner configured to aggregate and summarize map calculation results of the map calculator, a rack combiner configured to aggregate and summarize map calculation results of MapReduce apparatuses connected to the same rack switch according to a policy of a MapReduce control apparatus, and a reduce calculator configured to receive the aggregated and summarized results of the combiner or the rack combiner and perform a reduce calculation.

Abstract: Methods and apparatuses relating to distributed memory hazard detection and error recovery are described. In one embodiment, a memory circuit includes a memory interface circuit to service memory requests from a spatial array of processing elements for data stored in a plurality of cache banks; and a hazard detection circuit in each of the plurality of cache banks, wherein a first hazard detection circuit for a speculative memory load request from the memory interface circuit, that is marked with a potential dynamic data dependency, to an address within a first cache bank of the first hazard detection circuit, is to mark the address for tracking of other memory requests to the address, store data from the address in speculative completion storage, and send the data from the speculative completion storage to the spatial array of processing elements when a memory dependency token is received for the speculative memory load request.

Abstract: A method of determining an execution order of memory operations performed by a processor includes executing at least one single-instruction, multiple-data (SIMD) scatter operation by the processor to store data to a memory. The method further includes executing one or more instructions by the processor to determine the execution order of a set of memory operations. The set of memory operations includes the at least one SIMD scatter operation.

Abstract: An apparatus and method for loop flattening and reduction in a SIMD pipeline including broadcast, move, and reduction instructions.

Abstract: Systems and methods for offloading data transformation from a host to a hybrid solid state drive (HSSD) are described. In one such method, the HSSD receives initial data from the host and stores the data at a first non-volatile memory (NVM). The HSSD receives a transformation command from the host to offload data transformation. The HSSD copies the data from the first NVM to a second NVM that is configured to provide a finer granularity of data access than that of the first NVM. Then the HSSD transforms the data at the second NVM utilizing the configured processing circuit. The HSSD may store the result in the first NVM and/or second NVM, and send it to the host.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for profiling and configuring work on a cluster of computer nodes.

Abstract: Provided are a computer program product, system, and method for allocating physical nodes for processes in an execution plan. An execution plan is generated indicating a plurality of processes. A resource requirement is generated indicating requested physical nodes and an assignment of the processes to execute on the requested physical nodes. A determination is made from the resource requirement of a resource allocation of physical nodes for the requested physical nodes and the processes. The execution plan is updated to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the received resource allocation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a data entity that causes a processing unit to process a computational graph. In one aspect, method includes the actions of receiving data identifying a computational graph, the computational graph including a plurality of nodes representing operations; obtaining compilation artifacts for processing the computational graph on a processing unit; and generating a data entity from the compilation artifacts, wherein the data entity, when invoked, causes the processing unit to process the computational graph by executing the operations represented by the plurality of nodes.

Abstract: Examples of reducing data in a network are disclosed. In one example implementation according to aspects of the present disclosure, method may include receiving, by a network device, data from a mapper system. The method may then include performing, by the network device, a reduction function on the data received from the mapper system to reduce the data. The method may also include transmitting, by the network device, the reduced data to a reducer system.

Abstract: A method of generating a hardware design to calculate a modulo value for any input value in a target input range with respect to a constant value d using one or more range reduction stages. The hardware design is generated through an iterative process that selects the optimum component for mapping successively increasing input ranges to the target output range until a component is selected that maps the target input range to the target output range. Each iteration includes generating hardware design components for mapping the input range to the target output range using each of a plurality of modulo preserving range reduction methods, synthesizing the generated hardware design components, and selecting one of the generated hardware design components based on the results of the synthesis.

Abstract: A memory management module receives a request to access a page in a memory, sends the request to a memory controller controlling the memory if the page is available in the memory, and if the page is unavailable, (i) does not send the request to the memory controller, and (ii) generates a first exception. A hypervisor intercepts the first exception and sends a second exception to an operating system. The operating system includes a handler to, in response to the second exception, selectively request the memory controller to obtain the page from a storage device into the memory, and to suspend execution of a first thread issuing the request on a processor until the page becomes available in the memory; and a kernel to schedule execution of a second thread on the processor until the page becomes available, or to idle the processor until the page becomes available.

Abstract: An operator split mechanism analyzes code in a streaming application according to specified split criteria to determine when an operator in the streaming application can be split. At compile-time, when an operator satisfies the split criteria, the operator split mechanism splits the operator according to the split criteria. In an integrated development environment (IDE), the operator split mechanism determines when an operator satisfies the split criteria, and splits the operator according to the split criteria. The operator split mechanism can operate in an automatic mode where operators are split without further input from the user, or in a more interactive mode where the operator split mechanism provides recommendations and options to a user, who makes appropriate selections, and the operator split mechanism then functions according to the selections by the user.

Abstract: Disclosed herein provides enhancements for generating large scale processing framework (LSPF) images for deployment in processing environments. In one implementation a method of preparing LSPF service images for large scale data processing environments includes identifying a first LSPF service image, and identifying metadata that defines runtime requirements for deploying the LSPF service in data processing environments. The method further provides generating scripts for deploying the LSPF service based on the metadata, and generating a second LSPF service image for the LSPF service, wherein the second LSPF service image includes the scripts.

Abstract: The disclosed techniques may employ components referred to herein as atoms for computing and maintaining of states. Unlike traditional actors, atoms may be capable of binding to other atoms to form a bound combination of atoms, referred to herein as a molecule. In some examples, while bound to other atoms, an atom may operate in a manner that is different from traditional actors. For example, in some cases, atoms that are bound to one another may be prohibited from concurrently performing different operations on their own separate states. Additionally, bound atoms may be operable to collectively (e.g., synchronously) perform shared operations on their associated states. Furthermore, a shared operation performed on the states of bound atoms may be performed atomically. Also, in some examples, bound atoms may be capable of communicating synchronously with one another and of synchronously accessing each other's states.

Abstract: Systems, apparatuses and methods may provide for technology that detects, by a current stage of a hardware pipeline, a flush request with respect to a first resource and executes, by the current stage, one or more transactions associated with a second resource. Additionally, the current stage may conduct one or more flush operations with respect to the first resource, wherein the one or more transactions associated with the second resource are executed after detection of the flush request and before the one or more flush operations.

Abstract: Embodiments of the present invention relate to an architecture that uses hierarchical statistically multiplexed counters to extend counter life by orders of magnitude. Each level includes statistically multiplexed counters. The statistically multiplexed counters includes P base counters and S subcounters, wherein the S subcounters are dynamically concatenated with the P base counters. When a row overflow in a level occurs, counters in a next level above are used to extend counter life. The hierarchical statistically multiplexed counters can be used with an overflow FIFO to further extend counter life.

Abstract: A complex event query specification may be constructed that a complex event processing engine uses to analyze an event stream for an occurrence of a complex event. Event data stored in a distributed file system are mapped to respective instances of the complex event processing engine in respective distributed event streams. The distributed event streams are analyzed by the independently executing complex event processing engines in accordance with the complex event query specification. The occurrence of the complex event in any of the distributed event streams is indicated at the output of the complex event processing engines.

Abstract: A memory swap operation comprises writing information about a process in which a page fault occurred, into a temporary memory using a processor of a host, copying a page in which the page fault occurred, from a memory device recognized as a swap memory into a main memory of the host, and after completing the copying of the page, resuming the process in which the page fault occurred, using the information about the process, written in the temporary memory.

Abstract: The present invention relates to a method and device for visually implementing a software code. To this end, a method for visually implementing a software code according to the present invention comprises the steps of: generating, by a code block generation unit, a code block used for implementing a software code by the unit of block depending on a requirement and a function; and setting, by a code block setting unit, a code block attribute or an internal attribute code included in the code block on the basis of information input from a user, wherein the step of setting the code block attribute or the internal attribute code comprises the step of including function information on the code block, description information on the function information, and the internal attribute code in the code block.

Abstract: A parallel operation system includes a first node including a first processor configured to execute a first process, a second processor configured to execute a second process, and a first memory, and a second node including a third processor configured to execute a third process, a fourth processor configured to execute a fourth process, and a second memory, and a first signal line that transfers synchronization information between at least one of the first and second processors and at least one of the third and fourth processors, wherein when the first process is to be synchronized with the third process, at least one of the first and the third processors using the first signal line to execute a first synchronization process.

Abstract: Methods and apparatus are provided for serializing data. A computing device can generate a serialization buffer (SB). The SB can specify fields storing data and corresponding offsets, with an offset referring to a location in the SB storing the corresponding field. The SB can access a designated field in the SB by determining a designated offset for the designated field, determining a starting location based on the designated offset, and accessing data at the starting location. A distinct copy of the SB can be stored on a storage device.

Abstract: Provided are a computer program product, system, and method for allocating physical nodes for processes in an execution plan. An execution plan is generated indicating a plurality of processes. A resource requirement is generated indicating requested physical nodes and an assignment of the processes to execute on the requested physical nodes. A determination is made from the resource requirement of a resource allocation of physical nodes for the requested physical nodes and the processes. The execution plan is updated to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the received resource allocation.

Abstract: A system generates a plurality of price master data lookup procedures that are capable of being executed in parallel. The plurality of price master data lookup procedures is stored in a repository in an in-memory system. The in-memory system includes a parallel processor. The system accesses the repository of price master data lookup procedures, and retrieves price master data in parallel using the price master data lookup procedures. The system then calculates a price using the price master data retrieved from the price master database in the in-system memory.

Abstract: A computer-implemented method for processing input data in a mapreduce framework includes: receiving, in the mapreduce framework, a data processing request for input data; initiating, based on the data processing request, a map operation on the input data by multiple mappers in the mapreduce framework, each of the mappers using an aggregator to partially aggregate the input data into one or more intermediate key/value pairs; initiating a reduce operation on the intermediate key/value pairs by multiple reducers in the mapreduce framework, wherein, without sorting the intermediate key/value pairs, those of the intermediate key/value pairs with a common key are handled by a same one of the reducers, each of the reducers using the aggregator to aggregate the intermediate key/value pairs into one or more output values; and providing the output values in response to the data processing request.