Abstract: A method, computer program product, and computer system for dynamically grouping and un-grouping processing operators and processing elements used by a streaming application. A distributed processing elements utilization of resources may be monitored to identify candidate operators and candidate processing elements for at least one of parallelization and fusion. At runtime, via at least one of parallelization and fusion, the grouping and un-grouping of the identified candidate operators and candidate processing elements may be dynamically adjusted.

Abstract: Disclosed aspects relate to resource usage management in a stream computing environment that includes a set of hosts having a set of processing elements which has a set of stream operators. A first activity indicator may be detected for a first subset of the set of stream operators. A resource action for performance with respect to the first subset of the set of stream operators may be determined based on the first activity indicator. The resource action may be performed with respect to the first subset of the set of stream operators to benefit resource usage in the stream computing environment.

Abstract: Disclosed aspects relate to resource usage management in a stream computing environment that includes a set of hosts having a set of processing elements which has a set of stream operators. A first activity indicator may be detected for a first subset of the set of stream operators. A resource action for performance with respect to the first subset of the set of stream operators may be determined based on the first activity indicator. The resource action may be performed with respect to the first subset of the set of stream operators to benefit resource usage in the stream computing environment.

Abstract: A latency reconfiguration mechanism in a streams manager detects latency mismatches in a streaming application that affect the performance of the streaming application, and reconfigures the streaming application to minimize the latency mismatches. The reconfiguration can include fusing adjacent processing elements into a single processing element, co-locating adjacent processing elements on the same machine, moving processing elements and/or network links, adding parallel processing elements, and rearranging processing elements and/or network links to create more consistent latency paths. By minimizing latency mismatches and providing more consistent latency paths, the performance of a streaming application is increased by not having to perform significant reordering of data tuples.

Abstract: Disclosed aspects relate to cache management in a stream computing environment that uses a set of many-core hardware processors to process a stream of tuples by a plurality of processing elements which operate on the set of many-core hardware processors. The stream of tuples to be processed by the plurality of processing elements which operate on the set of many-core hardware processors may be received. A tuple-processing hardware-route on the set of many-core hardware processors may be determined based on a cache factor associated with the set of many-core hardware processors. The stream of tuples may be routed based on the tuple-processing hardware-route on the set of many-core hardware processors. The stream of tuples may be processed by the plurality of processing elements which operate on the set of many-core hardware processors.

Abstract: Disclosed aspects relate to tuple traffic management in a stream computing environment to process a stream of tuples using a set of control group data with respect to a set of operation-system-level virtualization and resource isolation containers. The set of control group data may be collected with respect to a set of containers in the stream computing environment. A tuple flow model may be determined with respect to the set of containers in the stream computing environment based on the set of control group data. The stream of tuples may be processed using the tuple flow model with respect to the set of containers in the stream computing environment.

Abstract: A streams manager assesses the security risk of streaming communication and dynamically migrates operators of a streaming application based on the security risk. The streams manager may also dynamically change encryption of intranode communication between the operators to increase performance of the computer node. The streams manager includes a stream security module (SSM) with a monitor that monitors risk in the system and changes placement of operators and encryption between operators on a node depending on the risk. The stream security module may use security data and node profile data collected by the monitor or the system to determine the risk. The stream security module may provide recommendations to a customer for the customer to override changes in placement and encryption.

Abstract: A streams manager assesses the security risk of streams communication and when possible turns off encryption of intranode communication between operators of a streaming application on a computer node to increase performance of the computer node. The streams manager includes a stream security module (SSM) with a monitor that monitors risk in the system and changes encryption between operators on a node depending on the risk. The stream security module may use security data and node profile data collected by the monitor or the system to determine the risk. The stream security module may provide recommendations to a customer for the customer to override changes in encryption.

Abstract: A connection can be made to a processing element of a remotely deployed and live streaming application executed by a first data processing system, the processing element containing at least one operator that processes at least one tuple. As the live streaming application is executed, without slowing or modifying data flow of the live streaming application execution to client devices, a copy of the tuple and a memory dump of state data for a state of the operator can be received, and the tuple can be tracked through a call graph. The state data can be loaded into a local instance of the operator loaded into a debugger. At least a portion of the call graph can be presented to a user, and a flow of the tuple through the call graph based on the state data for the operator can be indicated.

Abstract: A method, computer program product, and computer system for dynamically grouping and un-grouping processing operators and processing elements used by a streaming application. A distributed processing elements utilization of resources may be monitored to identify candidate operators and candidate processing elements for at least one of parallelization and fusion. At runtime, via at least one of parallelization and fusion, the grouping and un-grouping of the identified candidate operators and candidate processing elements may be dynamically adjusted.

Abstract: A method, and a system are provided for implementing dynamic switching of network protocols for optimal performance for data being transmitted between nodes. An appropriate transport protocol between nodes is dynamically chosen based upon monitored system and network metrics.

Abstract: A method, and a system are provided for implementing dynamic switching of network protocols for optimal performance for data being transmitted between nodes. An appropriate transport protocol between nodes is dynamically chosen based upon monitored system and network metrics.

Abstract: A latency reconfiguration mechanism in a streams manager detects latency mismatches in a streaming application that affect the performance of the streaming application, and reconfigures the streaming application to minimize the latency mismatches. The reconfiguration can include fusing adjacent processing elements into a single processing element, co-locating adjacent processing elements on the same machine, moving processing elements and/or network links, adding parallel processing elements, and rearranging processing elements and/or network links to create more consistent latency paths. By minimizing latency mismatches and providing more consistent latency paths, the performance of a streaming application is increased by not having to perform significant reordering of data tuples.

Abstract: Sharing data among computing systems can include receiving, using a processor, a data transfer event indicating a transfer of a data file from a source system to a target system, querying, using the processor, the source system for an operating system executed by the source system and a file type of the data file, and querying, using the processor, the target system for an operating system executed by the target system. A target application can be determined using the processor. The target application is adapted for execution on the operating system executed by the target system and is adapted for operating on the file type. Installation of the target application on the target system can be initiated. Further, transfer of the data file from the source system to the target system can be initiated using the processor.

Abstract: Provided are techniques for identification and handling of nested breakpoints during debug session. Program code having functions, nested breakpoints, and an indication of whether each of the breakpoints is likely to be reached is displayed, in a visualization. In response to receiving input that selects a function from the functions, a call graph is displayed for the functions in the program code to enable viewing where each of the nested breakpoints is located in the program code with reference to the functions. An option to disable a nested breakpoint of the nested breakpoints for a current execution of the program code is provided. In response to receiving input that selects the option, the nested breakpoint is disabled.

Abstract: A method, computer program product, and computer system for tracking tuples by ID as the tuples progress through an operator of a chain of operators within a processing element. Dynamic loading may be utilized to load a processing code for the operator in response to invoking fusion for the operator. Ownership of network I/O may be shifted from the operator to a final operator of the chain of operators within the processing element. A tuple ID may be rolled back to a last ID processed by the operator being added into the processing element.

Abstract: Methods for data stream processing in a runtime data processing environment. A stream processing graph that includes several connected operators is received. Source code of the operators is analyzed to identify hints describing whether an operator contains data structures, method parameters or other data that can be applied in a parallelization data processing environment. Performance metrics of the data processing environment within parallel regions is evaluated to determine whether data processing resources can be dynamically scaled up or down. In response to determining that the data processing resources can be dynamically scaled up, one or more operators are split to be processed on two or more parallel processing resources. In response to determining that the data processing resources can be dynamically scaled down, one or more operators are combined to be processed on a single parallel processing resource.

Abstract: Methods and apparatus, including computer program products, implementing and using techniques for data stream processing in a runtime data processing environment. A stream processing graph that includes several connected operators is received. Source code of the operators is analyzed to identify hints describing whether an operator contains data structures, method parameters or other data that can be applied in a parallelization data processing environment. Performance metrics of the data processing environment within parallel regions is evaluated to determine whether data processing resources can be dynamically scaled up or down. In response to determining that the data processing resources can be dynamically scaled up, one or more operators are split to be processed on two or more parallel processing resources. In response to determining that the data processing resources can be dynamically scaled down, one or more operators are combined to be processed on a single parallel processing resource.

Abstract: A method, computer program product, and computer system for consolidating, at a computing device, a number of parallel streams, entering a hardware accelerator in a cluster, into fewer streams. The fewer streams, exiting the hardware accelerator in the cluster, may be returned back into the number of parallel streams.

Abstract: A method, computer program product, and computer system for maintaining, at a computing device, one or more tuples in a software-level queue. The one or more tuples may be transported as a batch of the one or more tuples from the software-level queue to a first queue for processing at a hardware accelerator. After processing the one or more tuples, the one or more tuples may be transported from the first queue to a second queue at the hardware accelerator. The one or more tuples may be transported from the second queue to a next location.