Abstract: Techniques for transmitting data according to at least one quality of service requirement. A message path is calculated specifying a sequence of broker computers selected from a network of interconnected broker computers. The message path is statistically estimated to fulfill the at least one quality of service requirement. Quality of service metrics are received about the network of interconnected broker computers. If the message path is determined not to fulfill the quality of service requirement, a new message path is calculated specifying a new sequence of broker computers selected from the network of interconnected broker computers. The new message path is statistically estimated to fulfill the at least one quality of service requirement.

Abstract: A cooperative data stream processing system utilizing a plurality of independent, autonomous and heterogeneous sites in a cooperative arrangement process user-defined job requests over dynamic, continuous streams of data. A distributed plan is created that identifies the processing elements that constitute a job that is derived from user-defined inquiries. These processing elements are arranged into subjobs that are mapped to various sites within the system for execution. The jobs are executed across the plurality of distributed sites in accordance with the distributed plan. The distributed plan also includes requirements for monitoring and back-up of the execution sites in the event of a failure on one of those sites. Execution of the jobs in accordance with the distributed plan is facilitated by the identification of an owner site to which the distributed plan is communicated and which is responsible for driving the execution of the distributed plan.

Abstract: Embodiments of the invention relate to message queuing. In one embodiment, a request from an application for retrieving a message from a queue is received. The queue is stored across multiple nodes of a distributed storage system. A preference with respect to message order and message duplication associated with the queue is identified. A message sequence index associated with the queue is sampled based on the preference that has been identified. The message is selected in response to the sampling. The message that has been selected is made unavailable to other applications for a given interval of time, while maintaining the message in the queue. The message is sent to the application.

Abstract: A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to process user-defined job requests over dynamic, continuous streams of data. The sites negotiate peering relationships to share data and processing resources to handle the submitted job requests. These peering relationships can be cooperative or federated and can be expressed using common interest policies. Each site within the system runs an instance of a system architecture for processing job requests and is therefore a self-contained, fully functional instance of the cooperative data stream processing system.

Abstract: A system and method are disclosed for an elastic and scalable publish/subscribe scheme. Subscription information is received at a dispatcher node. A plurality of matching nodes is selected in an overlay network to store the subscription information on a computer readable storage medium. Upon receiving an event at a dispatching node, at least one of the matching nodes with the stored subscription information is selected to process the event.

Abstract: Techniques for guideline-based food purchase management are provided. The techniques include generating a personalized set of nutrition guidelines for a user, generating a user profile for the user, wherein the user profile comprises health information and nutrition preferences, and using the personalized set of nutrition guidelines and the user profile to generate a guideline-based shopping list for the user. Techniques for generating location-based food purchase guidance are also provided. The techniques include obtaining a shopping list of food items and a list of stores to be considered in connection with the food items on the shopping list, using the shopping list, list of stores, a store location map and inter-store routing guidance to generate an inter-store shopping route, and generating an in-store shopping route, for each of the stores in the inter-store shopping route, based on the shopping list, intra-store routing guidance and a floor plan for the store.

Abstract: Techniques for guideline-based food purchase management are provided. The techniques include generating a personalized set of nutrition guidelines for a user, generating a user profile for the user, wherein the user profile comprises health information and nutrition preferences, and using the personalized set of nutrition guidelines and the user profile to generate a guideline-based shopping list for the user. Techniques for generating location-based food purchase guidance are also provided. The techniques include obtaining a shopping list of food items and a list of stores to be considered in connection with the food items on the shopping list, using the shopping list, list of stores, a store location map and inter-store routing guidance to generate an inter-store shopping route, and generating an in-store shopping route, for each of the stores in the inter-store shopping route, based on the shopping list, intra-store routing guidance and a floor plan for the store.

Abstract: A system and method are disclosed for an elastic and scalable publish/subscribe scheme. Subscription information is received at a dispatcher node. A plurality of matching nodes are selected in an overlay network to store the subscription information on a computer readable storage medium. Upon receiving an event at a dispatching node, at least one of the matching nodes with the stored subscription information is selected to process the event.

Abstract: A failure recovery framework to be used in cooperative data stream processing is provided that can be used in a large-scale stream data analysis environment. Failure recovery supports a plurality of independent distributed sites, each having its own local administration and goals. The distributed sites cooperate in an inter-site back-up mechanism to provide for system recovery from a variety of failures within the system. Failure recovery is both automatic and timely through cooperation among sites. Back-up sites associated with a given primary site are identified. These sites are used to identify failures within the primary site including failures of applications running on the nodes of the primary site. The failed applications are reinstated on one or more nodes within the back-up sites using job management instances local to the back-up sites in combination with previously stored state information and data values for the failed applications.

Abstract: A system may include an overlay network linking a plurality of publishers, a plurality of subscribers, and a plurality of brokers. The overlay network may include a first computer network having a first network protocol carried by a second computer network having a second network protocol. The system may also include an agent carried by each of the publishers, the subscribers, and the brokers that adjust the network's topology based upon collected runtime data of condition of each link within the network and/or broker availability to any publisher and subscriber.

Abstract: Methods and systems are provided for optimally trading off replication overhead and consistency levels in distributed data replication where nodes are organized in a hierarchy. The root node has the original data that need to be replicated at all other nodes, and the replicated copies have a freshness threshold that must be satisfied. The data are propagated through periodic updates in the hierarchy. Each node periodically sends data to its child nodes. Given the freshness threshold, an algorithm and its distributed protocol can determine the optimal update period for each link of the hierarchy such that the freshness threshold is satisfied for every node and the overall replication overhead is minimized. The systems and methods can be used in any scenario where replicated data have consistency requirements, such as in a replicate overlay assisted resource discovery system.

Abstract: Techniques for transmitting data according to at least one quality of service requirement. A message path is calculated specifying a sequence of broker computers selected from a network of interconnected broker computers. The message path is statistically estimated to fulfill the at least one quality of service requirement. Quality of service metrics are received about the network of interconnected broker computers. If the message path is determined not to fulfill the quality of service requirement, a new message path is calculated specifying a new sequence of broker computers selected from the network of interconnected broker computers. The new message path is statistically estimated to fulfill the at least one quality of service requirement.

Abstract: Resources disposed within a plurality of distributed and autonomous computing systems, each of which may have its own legacy resource discovery service, are identified and allocated. Resource identification servers disposed within each one of the distributed computing systems communicate resource attribute data to a tree hierarchy of dedicated servers. The resource attribute data are maintained in native formats within the distributed computing systems and are mapped to a common format provided by the dedicated servers. The resource attribute data are aggregated at each node within the tree hierarchy, communicated up through the tree hierarchy to one or more root nodes and replicated down through all of the nodes. Additional system robustness is provided through period resource checks and resource attribute data updates. Resource allocation queries are submitted to any level node within the hierarchy and forwarded to the proper computing system for processing.

Abstract: Techniques are disclosed for optimizing schedules used in implementing plans for performing tasks in data processing systems. For example, an automated method of negotiating for resources in a data processing system, wherein the data processing system comprises multiple sites, comprises a negotiation management component of a computer system at a given one of the sites performing the following steps. One or more tasks from at least one source of one or more plans are obtained. Each plan is annotated with one or more needed resources and one or more potential resource providers at one or more sites in the data processing system. An optimized resource negotiation schedule based on the one or more obtained tasks is computed. The schedule comprises an order in which resources are negotiated.

Abstract: A system and method for protecting streams in a mixed infrastructure includes determining processing elements that are to access a data stream in a stream processing environment and determining a security level for each processing element. Keys are generated per stream per processing element in accordance with the security level. The keys are associated with processing elements in an access control list in a location accessible by producing and consuming processing elements. The stream is decrypted for processing using keys released upon authenticating processing elements in accordance with the access control list. At security boundaries, the stream is re-encrypted in accordance with a next processing element.

Abstract: Methods and systems are provided for optimally trading off replication overhead and consistency levels in distributed data replication where nodes are organized in a hierarchy. The root node has the original data that need to be replicated at all other nodes, and the replicated copies have a freshness threshold that must be satisfied. The data are propagated through periodic updates in the hierarchy. Each node periodically sends data to its child nodes. Given the freshness threshold, an algorithm and its distributed protocol can determine the optimal update period for each link of the hierarchy such that the freshness threshold is satisfied for every node and the overall replication overhead is minimized. The systems and methods can be used in any scenario where replicated data have consistency requirements, such as in a replicate overlay assisted resource discovery system.

Abstract: A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to process user-defined job requests over dynamic, continuous streams of data. A method is provided to organize the distributed sites into a plurality of virtual organizations that can be further combined and virtualized into virtualized virtual organizations. These virtualized virtual organizations can also include additional distributed sites and existing virtualized virtual organizations and all members of a given virtualized virtual organization can share data and processing resources in order to process jobs on either a task-based or goal-based allocation mechanism. The virtualized virtual organization is created dynamically using ad-hoc collaborations among the members and is arranged in either a federated or cooperative architecture. Collaborations between members is either tightly-coupled or loosely coupled.

Abstract: A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to execute jobs derived from user-define inquires over dynamic, continuous streams of data. A method is provided for cooperative planning for the execution of the jobs across the distributed plurality of sites. An identification of the resources available for sharing from each one of the plurality of sites is communicated to one or more planners disposed on the distributed sites. These planners use the resource information to generate planning domains in which the jobs can be processed. Upon receipt of an inquiry at one of the sites, the inquiry is communicated to one of the planners that uses the planning domain to create at least one distributed plan for the inquiry. Processing of the inquiry is conducted in accordance with the distributed plan.

Abstract: A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to process user-defined job requests over dynamic, continuous streams of data. The sites negotiate peering relationships to share data and processing resources to handle the submitted job requests. These peering relationships can be cooperative or federated and can be expressed using common interest policies. Each site within the system runs an instance of a system architecture for processing job requests and is therefore a self-contained, fully functional instance of the cooperative data stream processing system.

Abstract: A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to process user-defined job requests over dynamic, continuous streams of data. A mechanism is provided for orchestrating the execution of distributed jobs across the plurality of distributed sites. A distributed plan is created that identifies the processing elements that constitute a job that is derived form user-defined inquiries. Within the distributed plan, these processing elements are arranged into subjobs that are mapped to various sites within the system for execution. Therefore, the jobs are then executed across the plurality of distributed sites in accordance with the distributed plan. The distributed plan also includes requirements for monitoring of execution sites and providing for the back-up of the execution sites in the event of a failure on one of those sites.