Abstract: A method and system for processing file storage is provided. The method includes receiving a file storage request and determining a file size of a file which is requested to be stored. In response to determining that the file is a small file, a request is sent to an HBase to store the file into the HBase. In response to determining that the file is a large file, a request is sent to a HDFS to store the file into the HDFS. Information associated with the file is stored in a record.

Abstract: In an approach for detecting islanding effect in a power electrical network topology, a computer processor finds, in the power electrical network topology, a point of common coupling between a load and a distributed generation with potential islanding effect as well as a substation supplying power to said point of common coupling; collects operating parameters of the power grid in vicinity of said point of common coupling as well as in vicinity of said substation; determines whether islanding effect exists in said distributed generation and the load in accordance with a variation characteristic of the operating parameters of the power grid in vicinity of said point of common coupling and the operating parameters of the power grid in vicinity of said substation. Higher degree of accuracy and sensitivity of detecting islanding effect is achieved without compromising the energy quality on user side according to embodiments of the present invention.

Abstract: A computer manages a cache for a MapReduce application based on a distributed file system that includes one or more storage medium by receiving a map request and receiving parameters for processing the map request. The parameters include a total data size to be processed, a size of each data record, and a number of map requests executing simultaneously. The computer determines a cache size for processing the map request, wherein the cache size is determined based on the received parameters for processing the map request and a machine learning model for a map request cache size and reads, based on the determined cache size, data from the one or more storage medium of the distributed file system into the cache. The computer processes the map request and writes an intermediate result data of the map request processing into the cache, based on the determined cache size.

Abstract: Aspects of the present invention include a method, which includes updating, by a processor, one or more distributed memory datasets having data stored therein in response to a write data operation, the one or more distributed data memory datasets being located in a database. The method further includes splitting, by the processor, any one of the one or more distributed memory datasets into two distributed memory datasets when a size of the any one of the one or more distributed memory datasets exceeds a threshold value. The method further includes moving, by the processor, the stored data in any one of the one or more distributed memory datasets to regions within the database upon an occurrence of one or more conditions with respect to the one or more distributed memory datasets. Other aspects of the present invention include a system and a computer program product.

Abstract: Technical solutions are described for optimizing operation of a server cluster. An example method includes receiving a job request that executes using a set of data blocks, the job request being associated with an expected completion time. The cluster server is used to identify a set of replica servers, wherein each server from the set of replica servers contains the set of data blocks. In response to each server from the set of replica servers estimating a completion time for the job request that is more than the expected completion time, a new server is initiated, the set of data blocks is relocated from a first server from the set of replica servers to the new server, and the job request is allocated to the new server.

Abstract: An input data set for a model for estimating a missing value of a sensor in a sensor network is determined. The input data set includes one or more sensor readings selected according to a temporal-spatial parameter which is dynamic and specific to the sensor. Then the missing value of the sensor is estimated using the determined input data set as an input to the model.

Abstract: The present invention discloses a method and system for managing a database. According to embodiments of the present invention, there is provided a method for managing a database, each item of data in the database being associated with a timestamp and a data point, the timestamps being used as row keys for rows of a table in the database, the method comprising: obtaining a behavior characteristic of a user based on a previous data access to the database by the user; partitioning columns in the table into column families based on the obtained behavior characteristic and system configuration of the database; and causing data in the database to be stored in respective column families at least in part based on the associated data point. There is further disclosed a corresponding system.

Abstract: A method, system, and computer program product, include determining a first region based on a first point, a second point and a third point associated with temperature indication information that represents association relationship between temperature and pressure, the first and second points being associated with a same temperature value and different pressure values, a temperature inversion ending at the first point, and the temperature inversion starting from the third point, determining, within the first region, a second region associated with the temperature inversion, and predicting atmospheric condition based on the first and second regions.

Abstract: A method, system, and computer program product, include identifying a plurality of pollution process sets and determining pollution sources based on pollution start times of target pollution processes with matched features in the plurality of pollution process sets within a time window.

Abstract: Embodiments of the present disclosure relates to data block processing in a distributed processing system. According to one embodiment of the present disclosure, a computer-implemented method is proposed. A first performance indicator for processing a data block by a first processing module is obtained, where the data block is loaded into the first processing module. Then, a second performance indicator for processing the data block by a second processing module is obtained, where the first and second processing modules being logical instances launched in a distributed processing system for processing data blocks. Next, one processing module is selected from the first and second processing modules for processing the data block based on a relationship between the first and second performance indicators.

Abstract: A computer implemented method provides shuffle optimization in map-reduce processing. The computer implemented method obtains intermediate results from a plurality of mappers for an application on a computing device. The intermediate results are combined from the plurality of mappers and are then partitioned into intermediate results for respective reducers for the application. The intermediate results for the respective reducers are stored into a buffer storage, which is partitioned into buffer sections for the respective reducers. Based on a rate of increase in the intermediate results, the method predicts a time at which a data capture notification shall be sent. The method sends the data capture notification based on a predicted time being reached, to indicate that the intermediate results for respective reducers stored in the buffer storage can be transmitted to the respective reducers, and transmits the intermediate results for the respective reducers to the respective reducers.

Abstract: Disclosed is a novel system, computer program product, and method to compute an air quality forecast. An air quality forecast model, air quality real-time monitoring data, and air quality forecast data is accessed. A deviation in air pollution emission is monitored by classifying a difference between the air quality monitoring data and the air quality forecast data. This monitoring includes classifying any weather differences which are caused by weather, classifying any terrain differences which are caused by a geographic terrain; and, filtering the difference caused by inaccurate pollution emission inventory. The monitoring may repeat in response to a given time period elapsing or a chance in air quality forecast data received.

Abstract: A computer implemented method provides shuffle optimization in map-reduce processing. The computer implemented method obtains intermediate results from a plurality of mappers for an application on a computing device. The intermediate results are combined from the plurality of mappers and are then partitioned into intermediate results for respective reducers for the application. The intermediate results for the respective reducers are stored into a buffer storage, which is partitioned into buffer sections for the respective reducers. Based on a rate of increase in the intermediate results, the method predicts a time at which a data capture notification shall be sent. The method sends the data capture notification based on a predicted time being reached, to indicate that the intermediate results for respective reducers stored in the buffer storage can be transmitted to the respective reducers, and transmits the intermediate results for the respective reducers to the respective reducers.

Abstract: Shuffle optimization in map-reduce processing. The method includes: obtaining intermediate results from a plurality of mappers for an application on a computing device; combining the intermediate results from the plurality of mappers; and partitioning the combined intermediate results into intermediate results for respective reducers for the application based on respective keys of the combined intermediate results.

Abstract: A mechanism is provided for detecting air-pollution anomalies. A historical air-pollution pattern is identified for each of a plurality of air-pollution monitoring stations. For each of the plurality of air-pollution monitoring stations, responsive to receiving real-time data from a particular air-pollution monitoring station, the real-time data is compared to the historical air-pollution pattern associated with the particular air-pollution monitoring station. A density difference value is generated based on the comparison of the real-time data to the historical air-pollution pattern associated with the particular air-pollution monitoring station and a determination is made as to whether the density difference value is greater than a predetermined confidence threshold.

Abstract: The present application discloses a virtualized wireless communication network and an establishing method thereof. The virtualized wireless communication network system includes a plurality of virtual nodes, the plurality of virtual nodes being divided into multiple virtual node cliques according to interference relations among the plurality of virtual nodes, wherein, each virtual node represents a wireless communication node; and a plurality of clique communication management modules that are one-to-one associated with the multiple virtual node cliques, wherein, each clique communication management module is configured to manage data exchange between an associated virtual node clique and other virtual node cliques, so as to emulate communication behavior of the wireless communication node. The present invention has many advantages such as high flexibility, good scalability, low cost and the like.

Abstract: The present invention provides a method and apparatus for data swap in a virtual machine environment. The present invention provides a method for data swap in a virtual machine environment, including: in response to a swap request from a virtual machine, looking up storage space associated with the swap request; and allocating to the virtual machine free physical storage space, in a host, which matches the storage space, so that the free physical storage space logically becomes available storage space to the virtual machine; the virtual machine is running on the host, and the storage space is physical storage space in the host.

Abstract: A computer manages a cache for a MapReduce application based on a distributed file system that includes one or more storage medium by receiving a map request and receiving parameters for processing the map request. The parameters include a total data size to be processed, a size of each data record, and a number of map requests executing simultaneously. The computer determines a cache size for processing the map request, wherein the cache size is determined based on the received parameters for processing the map request and a machine learning model for a map request cache size and reads, based on the determined cache size, data from the one or more storage medium of the distributed file system into the cache. The computer processes the map request and writes an intermediate result data of the map request processing into the cache, based on the determined cache size.

Abstract: A method and system for processing file storage is provided. The method includes receiving a file storage request and determining a file size of a file which is requested to be stored. In response to determining that the file is a small file, a request is sent to an HBase to store the file into the HBase. In response to determining that the file is a large file, a request is sent to a HDFS to store the file into the HDFS. Information associated with the file is stored in a record.

Abstract: In one embodiment, a method for managing a data storage system includes: in response to receiving a data object, sorting data records in the data object on the basis of a first query so as to form a first backup; causing the first backup to be stored in the data storage system; and cause to be stored, in an index of the data storage system, the first query and a first address of the first backup in the data storage system.