Abstract: Distributed pipeline optimization for data preparation includes: receiving a specification of a set of sequenced operations to be performed on a set of organized data; dividing the set of data into a plurality of work portions based on a cost function that is dependent on at least one dimension of the set of data; and distributing the plurality of work portions to a plurality of processing nodes to be processed according to the specification of operations.

Abstract: A method of operating a telemetry system includes automatically populating a first set of fields in a schema of an event definition using a logging library of the telemetry system, and populating a second set of fields in the schema selected by an event author.

Abstract: A method of operating a telemetry system includes automatically populating a first set of fields in a schema of an event definition using a logging library of the telemetry system, and receiving the set of fields via a request message in an application protocol.

Abstract: Management device for a network of audiovisual information reproduction systems or jukeboxes, including a database with a plurality of sets of arrays, each array containing grouped information either about the composition of a jukebox, or the use of the jukebox, or the payment of fees. The database is managed by a computer server connected with the audiovisual information reproduction systems to receive messages sent by each audiovisual information reproduction device and containing information necessary to update determined sets of arrays in the database, and update data or the program for each audiovisual information reproduction device with information stored in at least one set of arrays in the database and transmitted in this message.

Abstract: A method for determining one or more anomalies associated with a first set of data, said method comprising: processing said first set of data to provide frequency information for a plurality of combinations of data in said first set of data; comparing said frequency information for the first set of data with frequency information for said combinations of data in a reference set of data; and in dependence on said comparison determining one or more anomalies.

Abstract: Methods of the invention include representing biological data in a memory subsystem within a computer system with a data structure that is particular to a location in the memory subsystem and serializing the data structure into a stream of bytes that can be deserialized into a clone of the data structure. In a preferred genomic embodiment, the biological data comprises genomic sequences and the data structure comprises a genomic directed acyclic graph (DAG) in which objects have adjacency lists of pointers that indicate the location of any object adjacent to that object. After serialization and deserialization, the clone genomic DAG has the same structure as the original to represent the same sequences and relationships among them as the original.

Abstract: The present disclosure discloses an online index rebuilding method and apparatus, where the method includes: receiving an online index rebuilding request; adding a schema shared lock to a data table corresponding to an identifier of the data table; obtaining a latch and obtaining a structure control right of the data table; reserving storage space of a shadow index for an original index, allocating an entry address of the shadow index, and adding the entry address of the shadow index to a root page of the data table; releasing the latch; building the shadow index in the storage space of the shadow index; updating an index entry address on the root page of the data table to the entry address of the shadow index after the latch is obtained again; and releasing the latch obtained again and the schema shared lock. Therefore, efficiency of online index rebuilding is improved.

Abstract: Computer implemented methods for concurrent processing of operations on a tree-based data structure include: receiving input at a storage system managing a storage device in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations to be applied to the tree-based data structure; determining one or more nodes of the tree-based data structure to which one or more of the set of heterogeneous operations are to be applied; determining one or more groups of the set of heterogeneous operations according to the one or more nodes to which the set of heterogeneous operations are to be applied; and applying, for each of the one or more groups, the set of heterogeneous operations according to a predefined order. Systems and methods for accomplishing the same are also disclosed.

Abstract: Techniques are described herein for creating a new database table for an exchange operation with a table partition of another table. A database management system (DBMS) is configured to receive a query such as “CREATE TABLE new_table FOR EXCHANGE FROM partition_P1”. The DBMS then identifies properties of the source partition (e.g., partition P1) and creates a new database table based on the properties identified. Some of these properties (e.g., hidden columns, data modeling frameworks, and table statistics) may not otherwise be directly accessible by the external entity that sent the query. The newly created table then has all of the necessary properties to be part of the other table if swapped with the source partition in an exchange operation.

Abstract: Cache optimization for data preparation includes: generating a data traversal program that represents a result of a set of sequenced data preparation operations performed on one or more sets of data, wherein the data traversal program indicates how to assemble one or more affected columns in the one or more sets of data to derive the result; in response to receiving a specification of the set of sequenced operations to be performed on the one or more sets of data, accessing the data traversal program that represents the result or a stored copy of the data traversal program that represents the result; assembling the one or more affected columns in the one or more sets of data according to the data traversal program to re-generate the result; and outputting the result.

Abstract: Signature-based cache optimization for data preparation includes: performing a first set of sequenced data preparation operations on one or more sets of data to generate a plurality of transformation results; caching one or more of the plurality of transformation results and one or more corresponding operation signatures, a cached operation signature being derived based at least in part on a subset of sequenced operations that generated a corresponding result; receiving a specification of a second set of sequenced operations; determining an operation signature associated with the second set of sequenced operations; identifying a cached result among the cached results based at least in part on the determined operation signature; and outputting the cached result.

Abstract: Using a step editor for data preparation includes: receiving an indication of a user input with respect to at least some of a set of sequenced data preparation operations on a set of data; generating, using one or more processors, a signature based at least in part on the set of sequenced data preparation operations, references to the set of data, and the user input; using the generated signature to determine whether there exists a cached result associated with the set of sequenced data preparation operations, the references to the set of data, and the user input; based at least in part on the determination, obtaining a data traversal program representing a result associated with the set of sequenced operations, the references to the set of data, and the user input; and providing output based at least in part on the result represented by the obtained data traversal program.

Abstract: Mechanisms are provided to implement an answer change notification system. The mechanisms receive a change operation to change a portion of a user profile data structure associated with a user and identify a first entry in a question and answer (QA) log data structure, corresponding to the user, having an indicator identifying a previous answer of the first entry as being dependent upon information in the user profile data structure. The mechanisms resubmit a question of the first entry to a question and answer (QA) system to generate a new answer to the question in response to identifying the entry. The mechanisms receive the new answer from the QA system and output, to a client device associated with the user, a notification identifying the new answer to the question in response to receiving the new answer.

Abstract: Injection of metadata into content items. In an embodiment, a content item is received. Data is retrieved from a plurality of metadata sources by, at least, retrieving first data from at least a first one of the plurality of metadata sources, and using the retrieved first data to retrieve second data from at least a second one of the plurality of metadata sources. Descriptive metadata is generated based on at least the retrieved first data and the retrieved second data, and associated with the content item.

Abstract: Data within a database is partitioned into one or more sets of read-only data pages and one or more sets of append-only data pages, wherein each set of read-only data pages is associated to one of the one or more sets of append-only data pages. A repeated map is maintained for at least one of the one or more sets of append-only data pages. An operational position map is maintained for at least one of the one or more sets of append-only data pages. A request is received to update one or more records in a given set of read-only pages, wherein at least one of the one or more records comprises repeated values. The repeated map and operational position map of the append-only pages associated with the given set of read-only pages are updated in response to the request. The one or more sets of append-only pages are automatically merged with their respective set of read-only pages on a periodic basis to create one or more new sets of read-only pages.

Abstract: Disclosed are examples of systems, apparatus, methods and computer program products for automatically matching start criteria and completion criteria of a walkthrough for an application or an online service. A database storing data objects identifying walkthroughs and walkthrough stages can be maintained. An indication can be received that walkthrough stages of a first walkthrough in the database have been added, removed, or modified. It can be determined that a first completion criterion associated with a first walkthrough stage of the first walkthrough and a first start criterion associated with a second walkthrough stage of the first walkthrough do not match. The first start criterion and the first completion criterion can be processed to generate a matched start criterion and a matched completion criterion. The first start criterion and the first completion criterion can be replaced with the matched start criterion and the matched completion criterion to generate a modified walkthrough.

Abstract: Techniques are provided for more efficient multi-row atomic, consistent, isolated and durable (ACID)-compliant transactions with snapshot isolation semantics (or just “multi-row transactions” for short). In some embodiments, the techniques are implemented in a computing system that includes a client application, a lightweight in-memory lease-based lock service, a multi-row transaction orchestrator, and an underlying database system. The transaction orchestrator implements a read protocol and a write protocol that provides support to the client application for carrying out multi-row transactions against the underlying database system irrespective of whether the database system itself supports multi-row transactions. The transaction orchestrator explicitly maintains transaction-level locks obtained from the lease-based lock service as part of the transaction protocol.

Abstract: A method and system for decontaminating raw yield maps by combining filters with spatial outlier detectors is provided. In an embodiment, the method comprises receiving over a computer network electronic digital data comprising first yield data representing crop yields harvested from an agricultural field; applying one or more filters to the first yield data to identify, from the first yield data, first outlier data; generating first filtered data from the first yield data by removing the first outlier data from the first yield data; identifying, in the first filtered data, second outlier data representing outlier values based on one or more outlier characteristics; generating second outlier data from the first filtered data by removing the second outlier data from the first filtered data; generating and causing displaying on a mobile computing device a graphical representation of the crop yields harvested from the agricultural field using only the second outlier data.

Abstract: A data management method includes storing a first event, where a status of the first event includes at least a first state and a second state; storing first data used for a first application, where the first data is associated with the first event; searching, after the status of the first event changes from the first state to the second state, for the first data associated with the first event; and managing the first data and/or the first application according to the status of the first event and the first data. A corresponding terminal is further disclosed.

Abstract: In an approach for calculating a probability of a consistency violation of a transaction in a database management system, a processor receives a plurality of transactions within a predetermined time period. A processor identifies a first pattern in a first transaction of the plurality of transactions based on at least an isolation level of the transaction. A processor identifies a second pattern, wherein the second pattern is the cooperation between the first transaction and the second transaction. A processor determines that the first pattern of the first transaction substantially matches a preexisting pattern, wherein the preexisting pattern corresponds to values for a mathematical model for estimating a percentage of transactions in violation of consistency criteria. A processor extracts values from the first transaction based on the determined preexisting pattern. A processor calculates the mathematical model using the values from the first transaction and the values from the preexisting pattern.

Abstract: According to one embodiment, a translation component is configured to operate on document encoded data to translate the document encoded data into a canonical format comprising a plurality of canonical types that fold together into a byte stream. The translation component is configured to accept any storage format of data (e.g., column store, row store, LSM tree, etc. and/or data from any storage engine, WIREDTIGER, MMAP, AR tree, Radix tree, etc.) and translate that data into a byte stream to enable efficient comparison. When executing searches and using the translated data to provide comparisons there is necessarily a trade-off based on the cost of translating the data and how much the translated data can be leveraged to increase comparison efficiency.

Abstract: According to one embodiment, a translation component is configured to operate on document encoded data to translate the document encoded data into a canonical format comprising a plurality of canonical types that fold together into a byte stream. The translation component is configured to accept any storage format of data (e.g., column store, row store, LSM tree, etc. and/or data from any storage engine, WIREDTIGER, MMAP, AR tree, Radix tree, etc.) and translate that data into a byte stream to enable efficient comparison. When executing searches and using the translated data to provide comparisons there is necessarily a trade-off based on the cost of translating the data and how much the translated data can be leveraged to increase comparison efficiency.

Abstract: In various example embodiments, a system and method for structuring search results for attribute comparison are presented. A product selection from a user device is received. The product selection has a plurality of attributes associated with it. A plurality of adjustable sliders configured to allow the user to refine a search is generated and caused to be displayed in a user interface of the user device. One or more adjustment of the plurality of adjustable sliders is received. In response to the user to an adjustment made to a first adjustable slider of the plurality of adjustable sliders, an analysis of a networked database for search results that match values indicated by the plurality of adjustable sliders including the adjustment made to the first adjustable slider is performed. The search results are caused to be presented on the user interface displayed on the user device. The search results can be in example form of a multi-dimensional graph or list.

Abstract: Embodiments of the present invention disclose a method, computer program product, and system for searching a database using a user entered search query. A search query for a database is received by the computer and the computer applies condition-action rules based on natural language processing rules to identify one or more phrases within the search query that is associated an entity identifier. The computer further identifies any taxonomy variants that have been established for the identified phrases. The computer creates a search string that includes search query and the entity identifiers. The database search is conducted by the computer and the results are displayed for the user.

Abstract: Automated join detection includes: identifying a set of one or more candidate joins of a first table and a second table; evaluating a set of one or more quality measures corresponding to the set of one or more candidate joins; obtaining a set of one or more selected joins among the set of one or more candidate joins, the set of one or more selected joins being selected based at least in part on one or more corresponding quality measures; and generating a joined table, including by joining the first table and the second table according to a selected join.

Abstract: This disclosure addresses the need for pre-caching of database query results which in general is the caching of the results of query evaluation before they are needed. In certain embodiments of query evaluation, the computing time required to evaluate a query is dominated by the reverse query, the evaluation of which determines the available remaining selectors and the number of items associated with each. The time to compute this reverse query is on average approximately proportional to the number of matching items. This result is used as a very convenient method of determining, before evaluation, which conjunctive queries will take the most time to evaluate—the long queries. These long queries can then be pre-cached, making them available for faster responses when they are needed. Methods of using cached conjunctive query results to more quickly evaluate certain queries comprising the disjunctive Boolean operators are also described.

Abstract: An approach for joining operations on document-oriented databases. The approach consists of receiving database identifiers, common attributes and results attributes for core and target databases being joined. Common attributes are searched for in the databases. The searches performed include string, expansive, character and nested. Common attribute join conflicts are identified and input is received to resolve attribute join conflicts. Resolved join conflicts are updated in a join substitution database for subsequent use and joined data results are output to virtual table(s).

Abstract: An approach for joining operations on document-oriented databases. The approach consists of receiving database identifiers, common attributes and results attributes for core and target databases being joined. Common attributes are searched for in the databases. The searches performed include string, expansive, character and nested. Common attribute join conflicts are identified and input is received to resolve attribute join conflicts. Resolved join conflicts are updated in a join substitution database for subsequent use and joined data results are output to virtual table(s).

Abstract: Techniques are described herein for sharing a dictionary across multiple in-memory compression units (IMCUs). After a dictionary is used to encode a first column vector in a first IMCU, the same dictionary is used to encode a second column vector in a second IMCU. The entries in the dictionary are in sort order to facilitate binary searching when performing value-to-code look-ups. If, during the encoding of the second column vector, values are encountered for which the dictionary does not already have codes, then a “sort-order-boundary” is established after the last entry in the dictionary, and entries for the newly encountered values are added to the dictionary, after the sort-order-boundary. To facilitate value-to-code look-ups, the new entries are also sorted relative to each other, creating a second “sort order set”. A new version of the dictionary may be created when the number of sort order sets in the first version of the dictionary reaches a configurable threshold.

Abstract: An approach for joining operations on document-oriented databases. The approach consists of receiving database identifiers, common attributes and results attributes for core and target databases being joined. Common attributes are searched for in the databases. The searches performed include string, expansive, character and nested. Common attribute join conflicts are identified and input is received to resolve attribute join conflicts. Resolved join conflicts are updated in a join substitution database for subsequent use and joined data results are output to virtual table(s).

Abstract: In an approach to presenting a topic to a user, a computer acquires and records a related person who is connected to the user, and acquires conversation data, wherein the data is at least one of a comment of the user, a comment of each related person who is connected to the user, and a conversation between the related person and the user. The computer extracts a topic from the conversation data, and records the topic. The computer calculates a degree of balance of a topic with a related person, a degree of the user's liking to the related person, a degree of the user's liking to the topic, and a degree of the related person's liking to the topic, and a degree of balance of the topic with the related person is calculated. The computer then presents a topic based on the degree of balance.

Abstract: Systems and methods can support log synchronization associated with a complex event handler. An event log can support storing prior event indicators. A new event indicator may be received. Temporal indicators associated with the new event indicator may be analyzed to establish consistent relative ordering with respect to prior event indicators. A synchronized timestamp may be generated in response to analyzing the temporal indicators. The new event indicator may be translated into a canonical form. The translated event indicator may be stored into the event log according to the synchronized timestamp. A rule associated with the complex event handler may be evaluated to identify a stored event dependency within a rule condition of the rule. The event log may be searched to match the translated event indicator against the stored event dependency. The rule may be triggered, and its action executed, in response to satisfying the rule condition.

Abstract: A method of adapting a GUI that comprises selecting a group of multivariable objects which comply with a search query from a plurality of multivariable objects, each the plurality of multivariable object is defined by a plurality of object variable values each of one of a plurality of object variables, for each one of the plurality of object variables, calculating a plurality of difference values each defines a difference between a first object variable value of one of the group of multivariable objects and a second object variable value of another of the group of multivariable objects, identifying a set of differentiating object variables from the plurality of object variables according to an analysis of the plurality of difference values of each one of the plurality of object variables, and updating a GUI accordingly.

Abstract: System and techniques for autonomously adaptive performance monitoring are described herein. A text input field may be presented on a graphical user interface. A flow-selector may be presented in contact with the text input field in response to receiving user input at the text input field. Here, the flow-selector includes a set of flow choices selected based on the user input. A user selection of a flow choice may be received. Next-steps flow elements may be presented in response to the user selection. User query choices may be collected from the next-steps flow elements to populate a query template corresponding to the flow choice. The query template is executed to produce search results.

Abstract: System and techniques for facet expansion are described herein. A user interface element may be presented on facet selection portion of a search result display including search results. Here, the user interface element is arranged to accept user input of a facet. Partial user input for a facet may be received. A peer entity to an entity corresponding to the facet may be obtained. A peer facet may be presented in a suggestion element in the facet selection portion in response to receiving the partial user input.

Abstract: A system is provided and includes search, analytics acquisition, CTR, and scoring modules. The search module: receives query requests from one or more user devices for respective queries; and based on the query requests and a CTR-based scoring model, conducts searches to provide search results for the queries. The analytics acquisition module acquires analytics data corresponding to the queries. The analytics data includes query files for the queries and selection files for the queries for which a selection event occurred. At least some of the selection events occur when a user of the one or more user devices selects a search result item in the search results provided for the queries. The CTR module determines a normalized CTR based on the analytics data. The scoring module updates the CTR-based scoring model based on the normalized CTR. The search module conducts a search based on the updated CTR-based scoring model.

Abstract: An individual user graphical user interface (GUI) that provides functionality for an individual user to upload content items, identify a category for each content item (each category identifying to an attribute of the individual user), create a portfolio by selecting content items, and share the portfolio such that the portfolio may be remotely viewed via a communications network.

Abstract: A system deploys visualization tools, business analytics software, and big data software in a multi-instance mode on a large, coherent shared memory many-core computing system. The single machine solution provides or high performance and scalability and may be implemented remotely as a large capacity server (i.e., in the cloud) or locally to a user. Most big data software running in a single instance mode has limitations in scalability when running on a many-core and large coherent shared memory system. A configuration and deployment technique using a multi-instance approach, which also includes visualization tools and business analytics software, maximizes system performance and resource utilization, reduces latency and provides scalability as needed, for end-user applications in the cloud.

Abstract: A data storage system includes a column store and a row store. The data storage system may generate a query summary. The data storage system determines query summary fields comprised of a subset of fields from a schema used to store data in the row store. The data storage system searches the column store or the row store for query results in the query summary fields. A query summary field is generated from the query results in the query summary fields.

Abstract: Systems and methods are provided for limiting cardinality of large datasets. A query is received from a client device. In response to the query, a query dataset comprised of a plurality of data records is retrieved from a remote data store. A fragmentation relation, representation relation, visualization relation, and result relation of the query dataset are determined. The result relation is returned to a client device. Based a selection from the result relation via graphical user interface of the client device, an inverse function is determined for mapping the selection back to the original data relation. A subset of data records corresponding to the selection are retrieved and returned to the client device for rendering in a graphical user interface.

Abstract: A mechanism is provide for preparing a high-quality data repository. Data and related metadata from a set of data sources are ingested thereby forming a set of unprepared data. The set of unprepared data is transformed based on a set of functions into a set of transformed data. A set of semantic text descriptions that detail the transformation of the set of unprepared data to the set of transformed data is generated using a first set of semantic associations, a second set of semantic associations, and a set of semantic transformation associations. The set of transformed data is tested against one or more governance policies that tracks data lineage to ultimately show that prepared data is in compliance. Responsive to the set of transformed data adhering to the one or more governance policies, a high-quality data repository is automatically built using the transformed data.

Abstract: There are provided systems and methods for performing metadata extraction and management. Such a system includes a computing platform having a hardware processor, a system memory, and metadata extraction and management unit stored in the system memory. The system is configured to extract multiple metadata types from a media asset, and to aggregate the multiple metadata types to produce an aggregated metadata describing the media asset. The system is further configured to transform the aggregated metadata into at least one database entry identifying the media asset, and to map the at least one database entry into a graphical database so as to relate the media asset to at least one other media asset represented in the graphical database.

Abstract: A system, method, and computer-readable storage medium configured to facilitate the parallel transfer of Structured Query Language (SQL) data to a software framework.

Abstract: According to one embodiment, a translation component is configured to operate on document encoded data to translate the document encoded data into a canonical format comprising a plurality of canonical types that fold together into a byte stream. The translation component is configured to accept any storage format of data (e.g., column store, row store, LSM tree, etc. and/or data from any storage engine, WIREDTIGER, MMAP, AR tree, Radix tree, etc.) and translate that data into a byte stream to enable efficient comparison. When executing searches and using the translated data to provide comparisons there is necessarily a trade-off based on the cost of translating the data and how much the translated data can be leveraged to increase comparison efficiency.

Abstract: The claimed invention relates to a system and method for generating actionable intelligence and information by utilizing a multi-sensor, multi-temporal; multi-spatial, multi-format data (mSTSFA) architecture stored in a NoSQL data architecture to qualify spatial (accuracy) and contextual information integrated into a real time Engineering Grade location based analysis and predictive analytics engine returning users based queries in a 3D visualization including Virtual Reality (VR)/Augmented Reality functionality. The present invention is a systemized platform for handling geospatial, geophysical, financial, temporal and attribute data input directly to analyze the datasets to serve the operational and business needs of the industries such as transportation, water, environmental, engineering, telecommunication, finance, energy, natural resources, defense and security.

Abstract: Systems and methods are provided for fragmentation. A dataset including data records is received. Fragments corresponding to the data records are determined. A fragment identifier table is determined which includes a plurality of fragment codes. Each fragment code uniquely corresponds to each fragment. A fragment having a fragment attributes is selected. Based on the fragment attributes, a fragment boundary set is determined. Each fragment boundary set is unique to each fragment code. An encoded limited cardinality dataset is generated based on the fragment boundary sets for visualization on a client device.

Abstract: A computer implemented method for classifying at least one source dataset of a computer system. The method may include providing a plurality of associated reference tables organized and associated in accordance with a reference storage model in the computer system. The method may also include calculating, by a data classifier application of the computer system, a first similarity score between the source dataset and a first reference table of the reference tables based on common attributes in the source dataset and a join of the first reference table with at least one further reference table of the reference tables having a relationship with the first reference table. The method may further include classifying, by the data classifier application, the source dataset by determining using at least the calculated first similarity score whether the source dataset is organized as the first reference table in accordance to the reference storage model.

Abstract: Methods and arrangements for associating physical activities and digital activities of a user. A plurality of physical attributes of the user are collected, via mining sensor data from at least one physical state associated with the user and a plurality of digital attributes relating to at least one digital activity of the user via mining the data from a digital presence of the user. A plurality of physical attributes and digital attributes are grouped based on a similarity measure. There are stored a grouping of the physical and digital attributes at a data storage location. Other variants and embodiments are broadly contemplated herein.

Abstract: A method, non-transitory computer readable medium and apparatus for transforming a knowledge base into a machine readable format for an automated system are disclosed. For example, the method includes clustering two or more documents of a plurality of documents in the knowledge base that are similar based upon a domain specific shingling function, identifying a pattern from each document of the plurality of documents that are clustered, wherein the pattern comprises a sequence of steps, aggregating the pattern of the each document of the plurality of documents that are clustered into a trie data-structure that is machine readable, receiving a request and applying the trie data-structure to provide a solution to the request.

Abstract: An apparatus includes an extraction unit configured to extract a feature amount from each of a plurality of pieces of input data, a calculation unit configured to calculate, based on an identification model for identifying to which one of a plurality of labels each of the plurality of pieces of input data belongs, which is generated using the feature amount, a likelihood indicating how likely each of the plurality of pieces of input data belongs to the labels, and a presenting unit configured to present attribute information about the input data based on the feature amount and the likelihood.