Abstract: The technology disclosed relates to maintaining throughput of a stream processing framework while increasing processing load. In particular, it relates to defining a container over at least one worker node that has a plurality workers, with one worker utilizing a whole core within a worker node, and queuing data from one or more incoming near real-time (NRT) data streams in multiple pipelines that run in the container and have connections to at least one common resource external to the container. It further relates to concurrently executing the pipelines at a number of workers as batches, and limiting simultaneous connections to the common resource to the number of workers by providing a shared connection to a set of batches running on a same worker regardless of the pipelines to which the batches in the set belong.

Abstract: The technology disclosed relates to maintaining throughput of a stream processing framework while increasing processing load. In particular, it relates to defining a container over at least one worker node that has a plurality workers, with one worker utilizing a whole core within a worker node, and queuing data from one or more incoming near real-time (NRT) data streams in multiple pipelines that run in the container and have connections to at least one common resource external to the container. It further relates to concurrently executing the pipelines at a number of workers as batches, and limiting simultaneous connections to the common resource to the number of workers by providing a shared connection to a set of batches running on a same worker regardless of the pipelines to which the batches in the set belong.

Abstract: The technology disclosed relates to discovering multiple previously unknown and undetected technical problems in fault tolerance and data recovery mechanisms of modern stream processing systems. In addition, it relates to providing technical solutions to these previously unknown and undetected problems. In particular, the technology disclosed relates to discovering the problem of modification of batch size of a given batch during its replay after a processing failure. This problem results in over-count when the input during replay is not a superset of the input fed at the original play. Further, the technology disclosed discovers the problem of inaccurate counter updates in replay schemes of modern stream processing systems when one or more keys disappear between a batch's first play and its replay. This problem is exacerbated when data in batches is merged or mapped with data from an external data store.

Abstract: The technology disclosed provides a novel and innovative technique for compact deployment of application code to stream processing systems. In particular, the technology disclosed relates to obviating the need of accompanying application code with its dependencies during deployment (i.e., creating fat jars) by operating a stream processing system within a container defined over worker nodes of whole machines and initializing the worker nodes with precompiled dependency libraries having precompiled classes. Accordingly, the application code is deployed to the container without its dependencies, and, once deployed, the application code is linked with the locally stored precompiled dependencies at runtime. In implementations, the application code is deployed to the container running the stream processing system between 300 milliseconds and 6 seconds. This is drastically faster than existing deployment techniques that take anywhere between 5 to 15 minutes for deployment.

Abstract: The technology disclosed relates to managing processing of long tail task sequences in a stream processing framework. In particular, it relates to operating a computing grid that includes a plurality of physical threads which processes data from one or more near real-time (NRT) data streams for multiple task sequences, and queuing data from the NRT data streams as batches in multiple pipelines using a grid-coordinator that controls dispatch of the batches to the physical threads. The method also includes assigning a priority-level to each of the pipelines using a grid-scheduler, wherein the grid-scheduler initiates execution of a first number of batches from a first pipeline before execution of a second number of batches from a second pipeline, responsive to respective priority levels of the first and second pipelines.

Abstract: The technology disclosed relates to managing processing of long tail task sequences in a stream processing framework. In particular, it relates to operating a computing grid that includes a plurality of physical threads which processes data from one or more near real-time (NRT) data streams for multiple task sequences, and queuing data from the NRT data streams as batches in multiple pipelines using a grid-coordinator that controls dispatch of the batches to the physical threads. The method also includes assigning a priority-level to each of the pipelines using a grid-scheduler, wherein the grid-scheduler initiates execution of a first number of batches from a first pipeline before execution of a second number of batches from a second pipeline, responsive to respective priority levels of the first and second pipelines.

Abstract: The technology disclosed relates to managing resource allocation to task sequences in a stream processing framework. In particular, it relates to operating a computing grid that includes machine resources, with heterogeneous containers defined over whole machines and some containers including multiple machines. It also includes initially allocating multiple machines to a first container, initially allocating first set of stateful task sequences to the first container, running the first set of stateful task sequences as multiplexed units of work under control of a container-scheduler, where each unit of work for a first task sequence runs to completion on first machine resources in the first container, unless it overruns a time-out, before a next unit of work for a second task sequence runs multiplexed on the first machine resources. It further includes automatically modifying a number of machine resources and/or a number assigned task sequences to a container.

Abstract: The technology disclosed relates to maintaining throughput of a stream processing framework while increasing processing load. In particular, it relates to defining a container over at least one worker node that has a plurality workers, with one worker utilizing a whole core within a worker node, and queuing data from one or more incoming near real-time (NRT) data streams in multiple pipelines that run in the container and have connections to at least one common resource external to the container. It further relates to concurrently executing the pipelines at a number of workers as batches, and limiting simultaneous connections to the common resource to the number of workers by providing a shared connection to a set of batches running on a same worker regardless of the pipelines to which the batches in the set belong.

Abstract: The technology disclosed relates to discovering multiple previously unknown and undetected technical problems in fault tolerance and data recovery mechanisms of modern stream processing systems. In addition, it relates to providing technical solutions to these previously unknown and undetected problems. In particular, the technology disclosed relates to discovering the problem of modification of batch size of a given batch during its replay after a processing failure. This problem results in over-count when the input during replay is not a superset of the input fed at the original play. Further, the technology disclosed discovers the problem of inaccurate counter updates in replay schemes of modern stream processing systems when one or more keys disappear between a batch's first play and its replay. This problem is exacerbated when data in batches is merged or mapped with data from an external data store.

Abstract: The technology disclosed improves existing streaming processing systems by allowing the ability to both scale up and scale down resources within an infrastructure of a stream processing system. In particular, the technology disclosed relates to a dispatch system for a stream processing system that adapts its behavior according to a computational capacity of the system based on a run-time evaluation. The technical solution includes, during run-time execution of a pipeline, comparing a count of available physical threads against a set number of logically parallel threads. When a count of available physical threads equals or exceeds the number of logically parallel threads, the solution includes concurrently processing the batches at the physical threads. Further, when there are fewer available physical threads than the number of logically parallel threads, the solution includes multiplexing the batches sequentially over the available physical threads.

Abstract: The technology disclosed relates to providing strong ordering in multi-stage processing of near real-time (NRT) data streams. In particular, it relates to maintaining current batch-stage information for a batch at a grid-scheduler in communication with a grid-coordinator that controls dispatch of batch-units to the physical threads for a batch-stage. This includes operating a computing grid, and queuing data from the NRT data streams as batches in pipelines for processing over multiple stages in the computing grid.

Abstract: Technologies are described herein for determining the display of equal spacing guides between diagram shapes. The nearest neighbors of each shape in a diagram are determined, and the distances between each shape and its nearest neighbors are calculated. When an active shape in the diagram is positioned or moved, spacing guides are displayed between shapes having equidistant spacing when the calculated distances between shapes become equal to a distance between the active shape and one of its nearest neighbors.

Abstract: A request may be received to trigger an animation action in response to reaching a bookmark during playback of a media object. In response to the request, data is stored defining a new animation timeline configured to perform the animation action when playback of the media object reaches the bookmark. When the media object is played back, a determination is made as to whether the bookmark has been encountered. If the bookmark is encountered, the new animation timeline is started, thereby triggering the specified animation action. An animation action may also be added to an animation timeline that triggers a media object action at a location within a media object. When the animation action is encountered during playback of the animation timeline, the specified media object action is performed on the associated media object.

Abstract: Tools and techniques for media portability and compatibility for different destination platforms are provided. These tools may receive commands to launch a media portability capability, and may receive source media as input for transformation. These tools may also receive indications of profile settings for specifying how to transform the source media for enhanced portability on destination systems for playback. The source media may be transformed in response to the profile setting, with the transformed media inserted into a document. The tools may then distribute the document to the destination system for playback.

Abstract: A presentation program provides an authoring tool allowing users to indicate animation sequences to be applied to an object in a document for purposes of creating or editing animation sequences. The user can directly manipulate the object on an editing pane, and the manipulations are interpreted as applying an animation class type. Different animation effects can be further associated with object for the particular animation class type. The user can select a particular animation effect and define the layout as a key frame that defines the animation sequence to be applied to the object at a given time during playback. The user can further manipulate the object and define subsequent key frames, and upon playback, the presentation program will interpolate the locations of the object between key frames as necessary. The user can further define the time period between key frames that is to be applied during playback.

Abstract: Media timeline interaction may be provided. An electronic presentation may comprise a media object. A user may select the media object within a presentation application and use an on-object user interface in conjunction with the application's user interface to modify the media object. The user may also display the modified media object within the presentation application.

Abstract: Technologies are described herein for determining the display of equal spacing guides between diagram shapes. The nearest neighbors of each shape in a diagram are determined, and the distances between each shape and its nearest neighbors are calculated. When an active shape in the diagram is positioned or moved, spacing guides are displayed between shapes having equidistant spacing when the calculated distances between shapes become equal to a distance between the active shape and one of its nearest neighbors.

Abstract: User actions, content, and other elements related to a presentation document are received. These elements are analyzed to generate recommendations for improving a presentation document. The presentation document may be modified in accordance with the recommendations.

Abstract: Media timeline interaction may be provided. An electronic presentation may comprise a media object. A user may select the media object within a presentation application and use an on-object user interface in conjunction with the application's user interface to modify the media object. The user may also display the modified media object within the presentation application.

Abstract: Tools and techniques for media portability and compatibility for different destination platforms are provided. These tools may receive commands to launch a media portability capability, and may receive source media as input for transformation. These tools may also receive indications of profile settings for specifying how to transform the source media for enhanced portability on destination systems for playback. The source media may be transformed in response to the profile setting, with the transformed media inserted into a document.