Abstract: Techniques are disclosed relating to processing database transactions that include application operations defined by different object relational mapping (ORM) libraries. A transaction router module executing on a computer system receives, from an application, a request for a first database transaction, where the first database transaction includes first and second application operations. The transaction router module then translates the first application operation to one or more first database operations using a first ORM library and the second application operation to one or more second database operations using a second ORM library. Then, the transaction router module determines one or more database connections for the one or more first database operations and the one or more second database operations. Such techniques may advantageously allow applications to switch between using different persistence frameworks and message broker frameworks without substantial adjustments to application code.

Abstract: A distributed database management system comprises a plurality of peers and maintains data on behalf of a plurality of clients. Upon receiving a request to access data, the distributed database management system identifies, from among the plurality of peers, a set of peers associated with the client issuing the request. The distributed database management system provides, to the client, a list of the set of identified peers. Access to data maintained by the distributed database system is based, at least in part, on connections between the client and the set of identified peers.

Abstract: A repartitioning optimizer identifies alternative repartitioning strategies and selects optimal ones, accounting for network transfer utilization and partition sizes in addition to traditional metrics. If prior partitioning was hash-based, the repartitioning optimizer can determine whether a hash-based repartitioning can result in not every computing device providing data to every other computing device. If prior partitioning was range-based, the repartitioning optimizer can determine whether a range-based repartitioning can generate similarly sized output partitions while aligning input and output partition boundaries, increasing the number of computing devices that do not provide data to every other computing device. Individual computing devices, as they are performing a repartitioning, assign a repartitioning index to each individual data element, which represents the computing device to which such a data element is destined.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: Methods, systems, and computer program products are provided for generating application-aware data partitioning to support parallel computing. A label for a user defined data partitioning (UDP) key is generated by a labeling process to configure data partitions of original data. The UDP is labeled by the labeling process to include at least one key property excluded from the original data. The data partitions are evenly distributed to co-locate and balance the data partitions and corresponding computations performed by computational servers. A data record of the data partitions is retrieved by performing an all-node parallel search of the computational servers using the UDP key.

Abstract: An approach is provided for initiating generation of a media compilation based on one or more sampling criteria. A sampling platform determines at least one subset of one or more media items captured of at least one event. The sampling platform also partitions the at least one subset of the one or more media items into one or more bins and generates at least one compilation of the at least one subset of the one or more items based, at least in part, on whether the one or more media items in the one or more bins at least substantially meet one or more sampling criteria.

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of service clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. The system may provide an API through which clients can query tables maintained by the service. Items may be partitioned and indexed in a table according to a simple or composite primary key contained in all items in the table. A composite primary key may include a hash key attribute, and a range key attribute. The range key attribute may be usable to order items having the same hash key attribute value, and to partition them dependent on a range of range key attribute values. A query request may specify a logical or mathematical expression dependent on range key attribute values and may be directed to multiple partitions.

Abstract: An insertion of a record into a table that includes a primary key column and a second column that includes a global uniqueness constraint across all of a plurality of data partitions across which the table is split is initiated without checking that a value of the record in the second column is globally unique by contacting other partitions the one partition to which the record is to be added to. The insertion can be processed, at least in part by implementing a write lock on the one partition but without implementing a read lock on the other partitions. The write lock on the one partition can be released after the insertion is completed, after which the validity of the insertion can be verified, for example by examining the other parts and a delta partition corresponding to the table. The insertion can be undone if the insertion was not valid.

Abstract: A method is provided in one example and includes receiving network traffic associated with a first user and a second user; evaluating keywords in the network traffic in order to identify a topic of discussion involving the first and the second users; determining a first sentiment associated with a first data segment associated with the first user; determining a second sentiment associated with a second data segment associated with the second user; and generating an argument map based on the first data sentiment and the second data sentiment.

Abstract: A full-text search index for a table stored in a set of partitions of a database is refreshed. At a first point in time, table row distribution information and a table partitioning list are captured. At a second point in time, it is determined that an index refresh operation is to be performed. The captured table row distribution information is compared with current table row distribution information to determine whether there is at least one difference. In response to the determining that there is at least one difference, a text index distribution is re-aligned for any table rows that are affected by the change in table data row distribution by: generating a new table partitioning list, and indexing documents associated with table data rows that moved from one partition in the captured table partitioning list to another partition in the new table partitioning list.

Abstract: A system and method for purposing a computing device is disclosed. The computing devices in a network are organized into groups referred to as Pods, each Pod of computing devices further comprising an aggregation node. The aggregation node monitors and controls the computing devices in the Pod. The aggregation node is used to copy a system image from the pristine partition of another computer device or from a library of system images to a target partition of the computer to be repurposed. Because the computing devices in the network are organized into Pods, with each Pod having an aggregation node, there is no single point of failure for the entire management and control of the network. The aggregation nodes for each Pod in the network can communicate with one another to most efficiently allocate resources among the various tasks to be performed by the computing resources in the network. Computing devices in the network can be repurposed and/or reassigned to different Pods based on the various criteria.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of service clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. The system may provide an API through which clients can query tables maintained by the service. Items may be partitioned and indexed in a table according to a simple or composite primary key contained in all items in the table. A composite primary key may include a hash key attribute, and a range key attribute. The range key attribute may be usable to order items having the same hash key attribute value, and to partition them dependent on a range of range key attribute values. A query request may specify a logical or mathematical expression dependent on range key attribute values and may be directed to multiple partitions.

Abstract: Embodiments of the invention relate to data archiving on storage medium such as magnetic tapes based on graph partitioning. One aspect of the invention concerns a method that comprises representing a file system as a graph where each node of the graph represents a file and each edge represents file chunks common to the files corresponding to the nodes connected by the edge. The graph is repeatedly partitioned into disjoint sub-graphs. If the files and duplicate file chunks associated with a sub-graph fit into a data storage medium, then the files and duplicate file chunks are stored in the medium. Otherwise, the method may partition the sub-graph into smaller disjoint sub-graphs taking into consideration of degrees of connectivity of the nodes.

Abstract: A database server provides a partitioned table with a corresponding locally partitioned index. When a request is made to load data into the table by an application, a table partition, along with a corresponding index partition, is created for the application. The corresponding index partition in marked as asynchronously maintained, allowing other index partitions to be used by the query optimizer in formulating query plans for other table partitions. Once the loading of data is complete, a synchronization operation is called with the result of conforming the data in the asynchronously maintained index partition with its corresponding table in which the data was loaded. Once the data in the corresponding table partition and the data in the asynchronously maintained index partition are synchronized, the asynchronously maintained index partition is marked as synchronously maintained.

Abstract: In various embodiments, a system, method, and related computer-related medium to process data transactions are provided. In one example, a system is arranged to process data transactions in a data store including a plurality of databases. The system includes a computer interface module to receive a data transaction request from at least one requesting computer and a data access layer having one or more processors to identify at least one database in the plurality of databases. The data access layer is to define an abstraction layer between a computer interface module and the plurality of databases.

Abstract: A system and method for distributing and accessing files in a distributed storage system uses an ordered list of the storage nodes in the system to determine the storage node on which a file is stored. The distributed storage system includes a cluster of storage nodes and may also include one or more client nodes that participate in the system as storage resources. Each node (client and storage) stores an ordered list of the storage nodes in the system, allowing any of the nodes to access the file. The list is updated whenever a new storage node is added to the system, an existing storage node is removed from the system, or a new storage node is swapped with an existing storage node. Each one of the nodes may independently compute a new mapping of files to the storage nodes when the ordered list is changed.

Abstract: A data repository system and method are provided. A method in accordance with an embodiment includes an operation that can be used to port data from one or more existing database partitions to new database partitions according to a minimally progressive hash. The method can be used to increase the overall size of databases while a system runs hot, with little or no downtime.

Abstract: An input data set is treated as a plurality of grouped sets of key/value pairs, which enhances the utility of the MapReduce programming methodology. By utilizing such a grouping, map processing can be carried out independently on two or more related but possibly heterogeneous datasets (e.g., related by being characterized by a common primary key). The intermediate results of the map processing (key/value pairs) for a particular key can be processed together in a single reduce function by applying a different iterator to intermediate values for each group. Different iterators can be arranged inside reduce functions in ways however desired.

Abstract: A method and system is provided for maintaining customer data in a data store system utilizing a scalable partitioning framework. More specifically, the data store of a customer service system is partitioned (divided) into multiple partitions by a partitionable key of customer data so that each partition owns a subset of the customer data. By maintaining several partitions that store a subset of the customer data exclusive to each other, blackout or brownout problems can be local to one partition and, thus, the availability for the entire system will be increased. Moreover, a set of partitionable keys, a minimal unit to be moved between partitions, is grouped and associated with a partition. By eliminating direct dependencies between the partitions and the partitionable keys, the system can have great flexibility with regard to migrating customer data between partitions and adding a new partition.

Abstract: An apparatus and system are disclosed for reorganizing a set of database partitions based on a ranked set of database partitions. The apparatus includes an evaluation module to determine a fragmentation indicator for each database partition within a set of database partitions. A ranking module ranks the set of database partitions according to a rank criteria. The rank criteria includes, at a minimum, the fragmentation indicator. A reorganization module reorganizes a subset of the database partitions in the ranked set of database partitions. The subset is selected, at least in part, by a rank order for the ranked set of database partitions. The subset may be selected such that ranked database partitions satisfying a dynamic reorganization threshold are included in the subset.

Abstract: A method for determining when a database system query optimizer should employ join skew avoidance steps. The method includes dynamically calculating the worst-case anticipated frequency distribution for a particular relation along a particular set of join column(s) at query execution time. The calculated frequency distribution value is compared to a skew threshold, the skew threshold representing the number of rows on the same distinct value that would lead to avoidable processing inefficiencies. It is then determined that the database system query optimizer should employ join skew avoidance steps if the calculated frequency distribution value exceeds the skew threshold.

Abstract: An information selection method and an information selection apparatus are provided. The information selection method includes continuously selecting arbitrary program information out of a plurality of program information, extracting selection candidate program information to be selected after predetermined program information based on a previous selection order of the plurality of program information, and determining one of the extracted selection candidate program information as program information to be selected after the predetermined program information.

Abstract: In a database system, a query to perforin an ordered analytic function is received. The query contains a clause specifying creation of dynamic partitions, where the clause contains a partitioning condition to specify the dynamic partitions. During processing of the query, the dynamic partitions are identified based on evaluating the partitioning condition.

Abstract: A data repository system and method are provided. A method in accordance with an embodiment includes an operation that can be used to port data from one or more existing database partitions to new database partitions according to a minimally progressive hash. The method can be used to increase the overall size of databases while a system runs hot, with little or no downtime.

Abstract: A method is provided for managing, in a computer system, design of a database system having a set of schemata. The method includes, in a first computer process, extracting dependencies from the database system and identifying the set of schemata. The method further includes, for each specific schema in the set of schemata, creating in a second computer process a partition that, in turn, contains a further partition for each element of the specific schema, so as to establish a hierarchy of partitions in accordance with the structure of the set of schemata. The method also includes storing a representation of the database system including subsystems, dependency relationships among the subsystems, and the hierarchy of partitions.

Abstract: In one embodiment, a method is provided for processing data transactions in a data store that includes databases. In this method, a data transaction request is received from a requesting computer. Different databases are then identified from the databases using a data access layer that defines an abstraction layer. The data transmission request is then split into multiple data transaction requests, and a part of these requests is then communicated to at least one of the identified databases.

Abstract: A method and apparatus for increasing the speed at which a block of data can be partitioning into variable-length subblocks is provided. The method combines a relatively high-speed partitioning algorithm (that can only partition a block into relatively small mean-length subblocks) with a relatively low-speed algorithm (that can partition a block into subblocks of any mean length) to yield a relatively high-speed partitioning algorithm that can partition blocks into subblocks of any mean-length.

Abstract: The systems and methods partition digital data units in a content aware fashion without relying on any ancestry information, which enables one to find duplicate chunks in unrelated units of digital data even across millions of documents spread across thousands of computer systems.

Abstract: A database management system has a plurality of database servers and data can be transferred between them by partitioning a data area into small areas and altering allocation of the small areas to the database servers. After altering the configuration, there occurs degradation in processing speed that accompanies re-creation of the index. If this problem is solved by using conventional techniques, noticeable degradation in processing speed will occur to a specific query at the time of a steady state operation. Accordingly, an index created for each of the small areas and an index to all of the small areas are allocated to the database server and used in combination.

Abstract: Prediction-based compression engines are spoon-fed with sequentially efficiently compressible (SEC) streams of input data that make it possible for the compression engines to more efficiently compress or otherwise compact the incoming data than would be possible with streams of input data accepted on a TV-raster scan basis. Various techniques are disclosed for intentionally forming SEC input data streams. Among these are the tight packing of alike files or fragments into concatenation suitcases and the decomposition of files into substantially predictably consistent (SPC) fragments or segments that are routed to different suitcases according to their type. In a graphics-directed embodiment, image frames are partitioned into segment areas that are internally SPC and multidirectional walks (i.e., U-turning walks) are defined in the segment areas where these defined walks are traced during compression and also during decompression.

Abstract: The invention has as its object to selectively reproduce on an external apparatus apart of digital data with index that a user wants to reproduce out of the digital data being made up of audio data and/or video data without having to use an application program dedicated for the external apparatus. The present invention is characterized in that in a case that a data file linked with a position information file is to be transmitted to the external apparatus, there are provided a data file divider which performs division processing on the data file on the basis of predetermined positions on the time axis of the digital data specified by the position information to thereby create a plurality of divided data files and a data file transmitter for transmitting the divided data files created by the data file divider to the external apparatus.

Abstract: A database management system provides the capability to perform cluster analysis and provides improved performance in model building and data mining, good integration with the various databases throughout the enterprise, and flexible specification and adjustment of the models being built, but which provides data mining functionality that is accessible to users having limited data mining expertise and which provides reductions in development times and costs for data mining projects. A database management system for in-database clustering comprises a first data table and a second data table, each data table including a plurality of rows of data, means for building a clustering model using the first data table using a portion of the first data table, wherein the portion of the first data table is selected by partitioning, density summarization, or active sampling of the first data table, and means for applying the clustering model using the second data table to generate apply output data.