Abstract: A system and method for pruning data based on metadata. The method may include receiving a query that includes a plurality of predicates and identifying one or more applicable files including database data satisfying at least one of the plurality of predicates. The identifying the one or more applicable files including reading metadata stored in a metadata store that is separate from the database data. The method further includes pruning inapplicable files including database data that does not satisfy at least one of the plurality of predicates to create a reduced set of files and reading the reduced set of files to execute the query.

Abstract: In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can provide support for auto-mapping of complex data structures, datasets or entities, between one or more sources or targets of data, referred to herein in some embodiments as HUBs. The auto-mapping can be driven by a metadata, schema, and statistical profiling of a dataset; and used to map a source dataset or entity associated with an input HUB, to a target dataset or entity or vice versa, to produce an output data prepared in a format or organization (projection) for use with one or more output HUBs.

Abstract: A method of securely deploying a software package comprises storing validity data describing restricted use of a restricted use token; receiving a command including a URL to deploy a software package; testing validity of the URL based on the validity data; and downloading, when the testing is successful, the software package via a secure channel, wherein the method is performed using one or more processors.

Abstract: Machine-learning-aided automatic taxonomy for web data. In an embodiment, a training dataset of annotated features is used to train a model to predict a class in a taxonomy of web-based activities. The features may be derived from a uniform resource locator (URL) of an online resource and associated metadata. During operation, the features may be extracted from the URL and metadata of each activity record in web data. The trained model may be applied to the extracted features for each activity record to predict a class within the taxonomy. The predicted taxonomic class may be stored in association with the URL that was extracted from the activity record to produce a taxonomized URL.

Abstract: Provided is a computer-implemented method for monitoring and improving data quality of transaction data that may include receiving transaction data associated with a plurality of payment transactions from an acquirer system. The transaction data may include a transaction record associated with each payment transaction of the plurality of payment transactions. Each transaction record may include a plurality of data fields. Each respective data field of the plurality of data fields may be categorized into a respective type of a plurality of types. A data quality score for each respective data field of the plurality of data fields may be determined based on the respective type of the respective data field. A system and computer program product are also provided.

Abstract: Currently, changes to third-party software services, such as the deprecation of certain components, must be manually monitored. Accordingly, embodiments are disclosed to automatically monitor changes to such services in a scalable manner. In particular, monitors may be configured to scrape webpages at URLs associated with deprecation or other information about the services, and manage review and resolution of each substantive change. The monitors may be used to facilitate up-to-date adapters for interfacing with the services in an integration platform as a service (iPaaS) platform.

Abstract: In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can provide data governance functionality such as, for example, provenance (where a particular data came from), lineage (how the data was acquired/processed), security (who was responsible for the data), classification (what is the data about), impact (how impactful is the data to a business), retention (how long should the data live), and validity (whether the data should be excluded/included for analysis/processing), for each slice of data pertinent to a particular snapshot in time; which can then be used in making lifecycle decisions and dataflow recommendations.

Abstract: Some embodiments provide a program that receives a set of data for a first record in a first table. The set of data includes a set of values for a set of attributes. In a data loading process configured to load a subset of the set of data into a subset of a first set of columns in the first table, the program determines that a first column in a first set of columns does not belong in the subset of the first set of columns. The program generates the first record in the first table. The program generates a value for the first column in the first set of columns that refers to a second record in the second table configured to represent a defined type of record. The program stores the value in the first column in the first set of columns of the first record.

Abstract: Techniques are provided for determining the semantic-type of a target column based on “fingerprints” that are created based on the values in the target column. The fingerprint set for the target column is only generated once, not once per semantic-type. Thus, the target column only needs to be scanned once, and resource usage is minimized. Once generated, the fingerprint set of the column is compared against fingerprint set that corresponds to each semantic-type to generate a “similarity measure”. The semantic-type whose fingerprint set produces the highest similarity measure relative to the target column's fingerprint set is determined to be the semantic-type of the target column.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include generating, with a processing device, a query plan for a query, the query comprising a join operator expression for a disjunctive predicate, wherein the join operator expression includes a conjunctive predicate and a disjunctive operator. The method may further include generating a bloom filter for the disjunctive operator. Additionally, the method may include generating a result set as a result of evaluating the join operator expression using the disjunctive operator and bloom filter for the disjunctive predicate.

Abstract: Systems and methods involving data structures for efficient management of paragraph vectors for textual searching are described. A database may contain records, each associated with an identifier and including a text string and timestamp. A look-up table may contain entries for text strings from the records, each entry associating: a paragraph vector for a respective unique text string, a hash of the respective unique text string, and a set of identifiers of records containing the respective unique text string. A server may receive from a client device an input string, compute a hash of the input string, and determine matching table entries, each containing a hash identical to that of the input string, or a paragraph vector similar to one calculated for the input string. A prioritized list of identifiers from the matching entries may be determined based on timestamps, and the prioritized list may be returned to the client.

Abstract: An RDBMS specifies a graph algorithm function (GAF) that takes a graph object as input and returns a logical graph object as output. GAFs are used within graph queries to compute temporary and output properties (“GAF-computed properties”), which are live for the duration of the query cursor execution. GAF-computed output properties are accessible in the enclosing graph pattern matching query as though they were part of the input graph object of the GAF. Temporary cursor-duration tables are generated for the query cursor during compilation of a graph query that includes a GAF, and are used to store the GAF-computed properties. Each temporary table corresponds to one of the primary tables of the input graph, and includes, as a foreign key, primary key information from the corresponding primary table.

Abstract: A computer server system includes a communications module; a processor coupled with the communications module; and a memory coupled to the processor and storing processor-executable instructions which, when executed by the processor, configure the processor to obtain, from at least one big data source, a big data dataset that includes transaction data; categorize the transaction data into a number of data buckets; analyze the transaction data from at least one of the data buckets to generate at least one graphical user interface to display at least some of the transaction data from the at least one of the data buckets, the at least one graphical user interface including at least one selectable interface element to adjust a display of the graphical user interface; and send, via the communications module and to a computing device, the at least one graphical user interface for display.

Abstract: A system for multipass sort includes a communication interface and a processor. The communication interface is configured to receive from a client device a request to sort a dataset that includes a plurality of rows. The processor is configured to perform a first sort pass on the dataset in part by: extracting prefixes associated with a first schema element associated with the dataset for the plurality of rows; and sorting the extracted prefixes utilizing an integer sort algorithm based on a sort order included in the request to sort the dataset, where sorting the extracted prefixes includes utilizing NULL values to resolve a tied range that includes at least two rows of the plurality of rows having a same extracted prefix.

Abstract: A system and method include providing a user interface for provisioning of the database using a selectable one of a plurality of database engines supported by the database management system, each of the supported ones of the plurality of database engines having a software profile, receiving a request via the user interface to create a new database for a first database engine not supported by the database management system, requesting and receiving a first binary for a database software for the first database engine and a second binary for an operating system software for the first database engine, and creating a first software profile from the first binary and the second binary, said first software profile usable by the database management system to create the new database using the first software profile.

Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Abstract: A display control device includes processing circuitry configured to store index setting information indicating combinations of a plurality of indexes and importance of each of the indexes, and visual variable setting information indicating a visual variable used when a value of the index of the importance is displayed for each importance on a heatmap, calculate values of the indexes collectively on a display basis on the heatmap, specify the combinations of the plurality of indexes and the importance of each of the indexes with reference to the index setting information, and determine a visual variable which is allocated to each of the indexes based on the visual variable setting information and the importance of each of the indexes specified.

Abstract: In an example, a first set of text may be received from a client device. A set of content items may be selected from among content items based upon the first set of text and a plurality of sets of content item text associated with the content items. A set of terms may be determined based upon the first set of text and the set of content items. A similarity profile associated with the set of terms may be generated. The similarity profile is indicative of similarity scores associated with similarities between terms of the set of terms. Relevance scores associated with the set of terms may be determined based upon the similarity profile. One or more search terms may be selected from among the set of terms based upon the relevance scores. A search may be performed based upon the one or more search terms.

Abstract: Implementations include actions of obtaining a set of entities based on one or more terms of a query, obtaining one or more entities associated with each live event of a plurality of live events, identifying a live event that is responsive to the query based on comparing at least one entity in the set of entities to one or more entities associated with each live event of a plurality of live events, determining that an event search result corresponding to the live event is to be displayed in search results, and in response: providing the event search result for display, the event search result including information associated with the live event, the information including an indicator of an occurrence of the live event.

Abstract: A method and system for a website building system (WBS) integrating at least one third party application includes detecting the presence of a spider associated with a search engine; parsing the output of said least one third party application to determine non-text components according to a request from said search engine spider, said non-text components having encoded or interleaved text, extracting said encoded or interleaved text from said non-text components; creating a search engine friendly page containing said extracted text; and providing said search engine friendly page to said spider.