Abstract: Method, system, and computer program product to enforce application dependencies, by building a data model based on inputs specifying relationships between a plurality of applications, generating, based on the data model, a list of candidate executions for a first application having a specified relationship with a second application, and outputting a visual representation of the data model, the visual representation comprising a plurality of objects representing the plurality of applications and arranged to represent the relationships between the applications, the list of candidate executions, and one or more user interface elements allowing user navigation between the first application and the second application, of the list of candidate executions.

Abstract: Method, system, and computer program product to enforce application dependencies, by building a data model based on inputs specifying relationships between a plurality of applications, generating, based on the data model, a list of candidate executions for a first application having a specified relationship with a second application, and outputting a visual representation of the data model, the visual representation comprising a plurality of objects representing the plurality of applications and arranged to represent the relationships between the applications, the list of candidate executions, and one or more user interface elements allowing user navigation between the first application and the second application, of the list of candidate executions.

Abstract: A method includes receiving, by a computing device, a plurality of data streams from plurality of sources, distributing the data streams to a plurality of sinks on multiple hosts, receiving load information indicating a load on at least one of the plurality of sinks and adjusting the distribution of the data stream accordingly and instructing the plurality of sinks to write the data streams to a distributed data store.

Abstract: A system and method for managing provenance data are disclosed. In accordance with one method, input data elements assessed by a processing element are evaluated. The method further includes determining whether an input window comprising the input data elements includes a sufficient amount of relevant input data. If the input window does not include a sufficient amount of relevant input data, then the input data elements are designated for reference in response to a provenance query.

Abstract: In a method for analyzing social media messages, the method includes one or more processors analyzing social media messages utilizing a set of topics and keywords associated with a brand. The method further includes one or more processors identifying social media messages that include information relating to the brand utilizing the analysis of social media messages utilizing the set of topics and keywords. The method further includes one or more processors determining relevancy scores for the identified social media messages that provide an indication of the percentage likelihood that a social media message is related to the brand. The method further includes one or more processors determining one or more recommendations of social media users associated with social media messages based on the determined relevancy scores.

Abstract: Techniques for securely and adaptively delivering multimedia content are disclosed in which a set of alternate access units for each time slot is obtained. Then, the encryption stream index of each access unit from the set of alternate access units of the previous time slot are obtained. An encryption stream index is then assigned to each access unit in the set of alternate access units in the current time slot, such that the encryption index increases over time. Thus, the invention overcomes the problem of encrypting a multimedia stream that may have multiple access units for each time slot by selecting the encryption index for each access unit such that the encryption index increases, regardless of which access unit the delivery system (e.g., server) selects for transmission.

Abstract: Method, system, and computer program product to enforce application dependencies, by building a data model based on inputs specifying relationships between a plurality of applications, generating, based on the data model, a list of candidate executions for a first application having a specified relationship with a second application, and outputting a visual representation of the data model, the visual representation comprising a plurality of objects representing the plurality of applications and arranged to represent the relationships between the applications, the list of candidate executions, and one or more user interface elements allowing user navigation between the first application and the second application, of the list of candidate executions.

Abstract: A system and method for determining data usage based on provenance information, in a stream-processing system, includes progressively setting usage information for output stream data objects (SDOs), determining input SDOs that an output SDO depends on, based on a provenance dependency function; recursively feeding back the usage information for a subset of SDOs that can be discarded; and discarding the subset of SDOs. A system and method for data retention based on usage information, in a stream-processing system, includes managing retention of SDOs by deleting SDOs that are determined to be of null usage; and enhancing retention characteristics of SDOs that are deemed to have usage.

Abstract: Method, system, and computer program product to enforce application dependencies, by building a data model based on inputs specifying relationships between a plurality of applications, generating, based on the data model, a list of candidate executions for a first application having a specified relationship with a second application, and outputting a visual representation of the data model, the visual representation comprising a plurality of objects representing the plurality of applications and arranged to represent the relationships between the applications, the list of candidate executions, and one or more user interface elements allowing user navigation between the first application and the second application, of the list of candidate executions.

Abstract: A method, system and computer program product of setting up a virtual connection at run time. The method includes the steps of: analyzing a topology of an application to determine at least one port pair where the port pair comprises an output port of a first operator and an input port of a second operator; configuring a first auxiliary operator to receive data from a first operator; configuring a second auxiliary operator to send data to a second operator; deploying the first operator, the second operator, the first auxiliary operator and the second auxiliary operator; receiving a function address of the second auxiliary operator; and sending the function address of the second auxiliary operator to a first destination; where at least one of the steps is carried out using a computer device so that the virtual connection is setup at run time.

Abstract: Fault tolerance is provided in a distributed system. The complexity of replicas and rollback requests are avoided; instead, a local failure in a component of a distributed system is tolerated. The local failure is tolerated by storing state related to a requested operation on the component, persisting that stored state in a data store, such as a relational database, asynchronously processing the operation request, and if a failure occurs, restarting the component using the stored state from the data store.

Abstract: A method includes receiving, by a computing device, a plurality of data streams from plurality of sources, distributing the data streams to a plurality of sinks on multiple hosts, receiving load information indicating a load on at least one of the plurality of sinks and adjusting the distribution of the data stream accordingly and instructing the plurality of sinks to write the data streams to a distributed data store.

Abstract: A system and method for selection of a provenance dependency function in a stream-based data processing infrastructure to optimize backtracing performance in response to a provenance query includes determining performance of a set of dependency functions associated with an analysis component for determining relevancy of each input event received by the analysis component. The relevancy of each input event is determined according to each dependency function and storing a record of relevant events according to a recording method. Relevancy results of the dependency functions are aggregated, and the dependency functions are ordered according to a criterion. Data provenance is provided for a given output event using the input event recording method associated with a best dependency function according to the criterion.

Abstract: A method for allocating a set of components of an application to a set of resource groups includes the following steps performed by a computer system. The set of resource groups is ordered based on respective failure measures and resource capacities associated with the resource groups. An importance value is assigned to each of the components. The importance value is associated with an affect of the component on an output of the application. The components are assigned to the resource groups based on the importance value of each component and the respective failure measures and resource capacities associated with the resource groups. The components with higher importance values are assigned to resource groups with lower failure measures and higher resource capacities. The application may be a partial fault tolerant (PFT) application that comprises PFT application components. The resource groups may comprise a heterogeneous set of resource groups (or clusters).

Abstract: A method, system and computer program product of setting up a virtual connection at run time. The method includes the steps of: analyzing a topology of an application to determine at least one port pair where the port pair comprises an output port of a first operator and an input port of a second operator; configuring a first auxiliary operator to receive data from a first operator; configuring a second auxiliary operator to send data to a second operator; deploying the first operator, the second operator, the first auxiliary operator and the second auxiliary operator; receiving a function address of the second auxiliary operator; and sending the function address of the second auxiliary operator to a first destination; where at least one of the steps is carried out using a computer device so that the virtual connection is setup at run time.

Abstract: Fault tolerance is provided in a distributed system. The complexity of replicas and rollback requests are avoided; instead, a local failure in a component of a distributed system is tolerated. The local failure is tolerated by storing state related to a requested operation on the component, persisting that stored state in a data store, such as a relational database, asynchronously processing the operation request, and if a failure occurs, restarting the component using the stored state from the data store.

Abstract: A system and method for allocating distributed processing systems includes inputting component descriptions in a distributed processing system and determining importance of each component. Capacity and failure characteristics of resource groups representing units of available processing capacity are also input. Components are assigned to a plurality of resource groups based on the capacity. Each resource group includes components where the failure characteristics permit simultaneous failures, such that in the event of such failures, an output value of the application is maximized.

Abstract: Techniques are disclosed for allocation of resources in a distributed computing system. For example, a method for allocating a set of one or more components of an application to a set of one or more resource groups includes the following steps performed by a computer system. The set of one or more resource groups is ordered based on respective failure measures and resource capacities associated with the one or more resource groups. An importance value is assigned to each of the one or more components, wherein the importance value is associated with an affect of the component on an output of the application. The one or more components are assigned to the one or more resource groups based on the importance value of each component and the respective failure measures and resource capacities associated with the one or more resource groups, wherein components with higher importance values are assigned to resource groups with lower failure measures and higher resource capacities.

Abstract: A system and method for dynamically building applications for stream processing includes providing processing elements with a flow specification describing each input and a stream description describing each output such that the flow specification indicates a stream or streams which are to be received based on processing information and the stream descriptions indicate the processing information. Processing elements that can be reused are identified by determining equivalence between the processing elements. Processing elements that are new and are not reusable are instantiated in a flow graph. An application is dynamically composed, using the instantiated processing elements by routing available streams to the instantiated processing elements in accordance with the flow specifications.

Abstract: A publisher-subscriber system includes a broker configured to receive and distribute at least one data stream from publishers to subscribers in accordance with subscriptions. An annotator/classifier is configured to annotate or classify the data stream by employing one or more additional stream-level attributes to create an annotated data stream wherein the annotated data stream applies the stream-level attribute to all messages therein. Subscribers can subscribe to the annotated data stream created by the annotator/classifier.