Abstract: A computer-implemented method for responding to user behaviors includes storing category specifications for a plurality of categories configured to characterize users, storing categories for users in a computer network system, detecting behaviors of a user in real time, and determining in real time if the behaviors of the user is within a first category specification associated with a first category that the user is tagged with. If the behaviors of the user exceed the first category specification, the method assigns a second category to the user in real time in response to the detected user behaviors.

Abstract: In response to receiving a test suite specification, a processor of a testing platform determines a schedule of execution of a test suite to test a system under test (SUT). The SUT has a hardware resource set including at least one of a set including a processor system and a data storage system, and the test suite includes a plurality of tests, each including a respective set of one or more testcases. The processor initiates execution of the test suite on the SUT in accordance with the schedule. In response to failure of a hardware resource during execution of the test suite, the processor automatically and dynamically reallocating a test in the test suite to at least one different hardware resource in the hardware resource set.

Abstract: This invention relates to a system, method and computer program product for a data warehouse model validation system, said data warehouse model validation system having an ETL model and a corresponding data warehouse model, said data ETL system comprising: an element group locator for locating an element group across the ETL model and the data warehouse model, whereby the element group comprises ETL elements and related data warehouse elements; an inconsistency determiner for determining inconsistencies between the ETL elements and data warehouse elements, whereby one or more elements are missing from the data warehouse model or one or more elements in the data warehouse model do not correspond to expected elements or features of elements; and an inconsistency recorder for recording any located missing elements or unexpected elements from the located element group.

Abstract: This disclosure provides a method and system to perform data integrity checks in a data warehouse (DWH) feed using machine-learning (ML) processes. According to an exemplary method, a ML integrity check is performed on received data which has been extracted from a plurality of source files, and after ML processes validate the extracted data, the validated data is transformed and loaded to a DWH.

Abstract: Performance data generated according to a first schema is read. From the first schema, object descriptors having common primitive types are identified. A second schema is then created. The second schema defines a plurality of rows and at least one column. The rows include a record corresponding to an identified object descriptor. The at least one column corresponds to a primitive type in common with the identified object descriptors.

Abstract: A method of migrating data from one or more source databases to one or more target databases may include generating a pre-migration analysis for a plurality of objects stored in the one or more source databases, and generating a plurality of migration scripts that transfer the plurality of objects from the one or more source databases to the one or more target databases. The method may also include generating a migration plan that defines an execution order for the plurality of migration scripts, and migrating the plurality of objects from the one or more source databases to one or more target databases according to the migration plan. The method may further include validating the plurality of objects on the one or more target databases.

Abstract: A computer-executed method includes storing in a data store data attributes, data objects, and a data analysis tool (DAT). Each data object has an attribute set and an identifier set. The method includes identifying each data object that has an attribute set comprising a data attribute matching each reference data attribute associated with the DAT, and selecting an identified data object as an input data object for the DAT. The DAT generates a new data object as a function of the input data object, which includes analyzing the input data object with reference to an auxiliary data object and creating an identifier set for the new data object that includes an identifier of the new data object for distinguishing the new data object from each other data object in the data store, and the identifier of the auxiliary data object. The new data object is stored in the data store.

Abstract: A corpus of information describing queries used to access a transactional data store may be used to identify analytical relationships that are not explicitly defined in a schema or supplied by a user. Join relationships may be identified based on field coincidence in elements of queries in the corpus. Join relationships may be indicative of dimensions and attributes of a dimension. Hierarchy levels for a dimension may be identified based on factors including data type, reference in an aggregating clause, and reference in a grouping clause.

Abstract: A cloud-based ETL system provides error detection, error correction and reporting of data integration flows hosted by cloud services. Categories of errors are identified using one or more checks at different points of a data integration flow and one or more actions selected based at least in part on the error category. A determination can be made whether the error category is fault tolerant and one or more actions can be selected based at least in part on the error fault tolerance to correct the error, restart a flow, or generate a notification assisting a user to correct the error.

Abstract: An automated method and computer program product are provided for synchronizing a column-oriented target database with a row-oriented source database. Change data are replicated from a change log of the row-oriented source database via a staging database to the column-oriented target database. The change data including inserts and deletes. Change data of the change log is read into the staging database and is consolidated and grouped into a consolidated grouping of inserts, and a consolidated grouping of deletes. The consolidated grouping of inserts from the staging database are applied to the target database in a batched manner, and the consolidated grouping of deletes from the staging database are applied to the target database in a batched manner.

Abstract: End user data partitioning can include receiving a number of data queries for a data source from a user, developing a dimension relation graph based on attributes of the number of data queries, and partitioning the data source based on the dimension relation graph.

Abstract: A system derives a first schema that is specific to a first log entry type associated with a log code, a second schema that is specific to a second log entry type associated with the log code, and a common schema for the first log entry type and the second log entry type. The system stores the first schema and the common schema in a container for the first log entry type, and the second schema and the common schema in a container for the second log entry type. The system identifies a schema identifier in a log entry corresponding to a system user event. The schema identifier corresponds to a schema in the container for the first log entry type or the container for the second log entry type. The system identifies log data by applying the corresponding schema to the log entry, and outputs the log data.

Abstract: According to certain aspects, a method can include creating a backup copy of data associated with a virtual machine (VM) on one or more secondary storage devices, wherein the backup copy includes corresponding secondary copies of a plurality of files associated with the VM; analyzing metadata associated with the secondary copies to determine which of the plurality files are eligible to be removed from the primary storage device; in response determining that one or more files are eligible to be removed from the primary storage device, for respective file of the one or more files: determining whether the respective file has been changed since a first time at which the backup copy of the data associated with the VM was created; in response to determining that the respective file has not changed since the first time, removing the respective file; and adding a file placeholder for the removed file.

Abstract: One or more techniques and/or systems are provided for interactively associating a semantic concept with a unique term that is input by a user. As the user is creating a document and/or once the user has completed a draft of the document, the document is parsed to identify unique terms (e.g., persons, places, things, services, etc.) in the document. When a unique term is identified, a query is generated to locate one or more semantic concepts (e.g., URLs, URNs, or other identifiers, for example) that are associated with the identified unique term and a notification indicative of the results is generated. From this notification, the user can select whether to associate the unique term with any and/or all of the located semantic concepts. In this way, supplemental content may be added to a document that the user is creating, for example.

Abstract: Disclosed is a network of systems that includes plural disparate storage systems that store user data, the disparate storage systems including NoSQL server databases that provide storage and retrieval of data modeled in forms besides tabular relations used in relational databases, and index storage system, a relational graph storage system and one or more data storage query platforms in communication with the plural disparate storage that have queries produced in a modeling language that abstracts application programmer functionality from network functionality.

Abstract: Example implementations are directed to a system and method to reduce deployment cost of data analytics application by designing both an application deployment plan and data integration plan, implementing the plans into an application template automatically and deploying application components and data in accordance with the desired implementation. Through example implementations, the need for separate terminals for a data engineer and an application engineer can be eliminated.

Abstract: Extract, transform, and load application (ETL) complexity management framework systems and methods are described herein. The present disclosure describes systems and methods that reduce the complexity in managing ETL flow and correcting errant data that is subsequently identified. One or more methods include defining an ETL job definition, defining a data asset definition, defining a data asset dependency definition, receiving an ETL flow to provide execution of one or more ETL flow steps, providing retrieval of data from a source data asset, applying a data control to the source asset data, and producing an ETL job registration, a data asset status, a latest asset available date, a data asset consumer identifier, and a target data asset based on at least one of the ETL job definition, the data asset definition, the data dependency definition, and the source asset data.

Abstract: A computer system architecture and method for providing compliance with data regulations, by: (a) collecting a data input stream with a data collection terminal; (b) using a compliance device driver resident in the data collection terminal to: (1) select data corresponding to pre-identified data compliance fields, and (2) apply a compliance markup language parser to generate pseudonymized data; and (c) using an automated compliance network appliance and an automated compliance server to: (1) transmit the pseudonymized data into immutable audit ledger, wherein the immutable audit ledger is assembled and verified by blockchain, and (2) transmit the data input stream into a data lake; (d) hosting access portals for accessing data: (1) stored in the data lake, and (2) stored in the immutable audit ledger.

Abstract: Methods, systems, and computer-readable media for a transformation specification format for multiple execution engines are disclosed. A transformation specification is expressed according to a transformation specification format. The transformation specification represents a polytree or graph linking one or more data producer nodes, one or more data transformation nodes, and one or more data consumer nodes. An execution engine is selected from among a plurality of available execution engines for execution of the transformation specification. The execution engine is used to acquire data from one or more data producers corresponding to the one or more data producer nodes, perform one or more transformations of the data corresponding to the one or more data transformation nodes, and output one or more results of the one or more transformations to one or more data consumers corresponding to the one or more data consumer nodes.

Abstract: A system and method for creating an instance model is provided. The system provides an information extraction and modeling framework from wide spectrum of document types such as PDF, Text, HTML, LOG, CSV, images, audio/video files and DOCX. In this framework information is extracted and mapped on a domain conceptual model like ER model and the instance model is created. Initially a template model is created using the existing ER model and the plurality of data sources. The template model, the existing ER model and the information extracted from the plurality of data sources are then provided as input to generate the instance model. The system or method is not limited to extract information from log files. This can be useful for different types of files type if the structures and formats of data are different. The system can also be used with unstructured type of data sources.

Abstract: Systems and methods for applicant tracking system (ATS) integration with a deduplicator are disclosed. A recruiting company computer system accesses a first entity record external to an ATS. The recruiting company computer system determines that the first entity record corresponds to a second entity record within the ATS based on at least first information of the first entity record and second information of the second entity record. The first information is different from the second information. The recruiting company computer system imports, into the second entity record within the ATS, information from the first entity record external to the ATS in response to the first entity record corresponding to the second entity record. The recruiting company computer system provides, in response a request to access information about an entity associated with the second entity record, the information from the first entity record.

Abstract: A system and method for forming a search query. Key-word search terms that include a homonym are received. One icon is selected to represent an intended meaning of the homonym. A first row of unique icons pertaining to an entity associated with a search query is displayed. Notification is received that a single unique object represented by a single icon of the unique icons in the first row is modified by a specific attribute and in response, a second row of the single icon modified by the specific attribute is displayed. Acceptance of the displayed single icon modified by the specific attribute is received for inclusion in the search query. The one icon and the single icon are displayed. In response to a user indicating that the displayed icons correctly represent a key-word search as intended by the user, the search based on meanings of the displayed icons is initiated.

Abstract: A method and system for forming a search query. Key-word search terms that include a homonym are received. One icon is selected to represent an intended meaning of the homonym. A first row of unique icons pertaining to an entity associated with a search query is displayed. Notification is received that a single unique object represented by a single icon of the unique icons in the first row is modified by a specific attribute and in response, a second row of the single icon modified by the specific attribute is displayed. Acceptance of the displayed single icon modified by the specific attribute is received for inclusion in the search query. The one icon and the single icon are displayed. In response to a user indicating that the displayed icons correctly represent a key-word search as intended by the user, the search based on meanings of the displayed icons is initiated.

Abstract: Aspects described herein generally improve the quality, efficiency, and speed of data processing systems by generating staging data independently from the execution of control scripts which process the staging data. The staging data can be independently loaded, validated, and utilized across multiple control scripts, reducing redundancy in the loading of data and the overhead of executing separate data processing for each control script. The control scripts can be automatically validated, such as by verifying expected output data ranges. Additionally, the complexity of the control scripts can be reduced as the loading of data is not performed by the control scripts. The controls scripts can generate a variety of output data, such as an indication of impacted accounts, and provide notifications based on the output data. A variety of machine learning classifiers can be used to automatically generate the staging data and validate the staging data and/or output data.

Abstract: Disclosed are apparatuses, methods, and computer readable media for improved message presentation to a user with respect to correlation of messages in a multi-participant message thread. Conversational awareness may be determined by analyzing contents of individual messages and assigning them to an existing context or creating a new context. Association of messages to contexts allows for grouping related messages related to their subject matter. Further, analysis of individual users within a multi-party communication stream (e.g., a thread with a group of participants) can be used to report previous and predict future user activity of a specific user. Groups of different sizes have been determined to sometimes have different participation dynamics. For example, people communicate differently in small groups versus large groups and within a given group, individual participation dynamics can be further analyzed. Disclosed systems learn and leverage this communication dynamic.

Abstract: This invention relates to a system, method and computer program product for a data warehouse model validation system, said data warehouse model validation system having an ETL model and a corresponding data warehouse model, said data ETL system comprising: an element group locator for locating an element group across the ETL model and the data warehouse model, whereby the element group comprises ETL elements and related data warehouse elements; an inconsistency determiner for determining inconsistencies between the ETL elements and data warehouse elements, whereby one or more elements are missing from the data warehouse model or one or more elements in the data warehouse model do not correspond to expected elements or features of elements; and an inconsistency recorder for recording any located missing elements or unexpected elements from the located element group.

Abstract: A method for testing an extract, transform, load job (ETL job) is provided. The method may include receiving an uploaded ETL job. The method may also include executing a functionality test of the uploaded ETL job using a test bucket. The method may further include determining the executed functionality test produces a failure result. The method may also include creating at least one test sub-job corresponding to at least one stage within on the uploaded ETL job. The method may further include generating a plurality of test data for each of the at least one created test sub-job. The method may also include executing the at least one created test sub-job using the plurality of generated test data. The method may further include creating a report based on a plurality of results associated with the at least one executed test sub-job.

Abstract: An optimization system receives a first information item to be placed within a web page that is displayed on a graphical user interface. The optimization system retrieves stored user data about the first information item from a user history database. The optimization system calculates a redundancy value for the first information item, which is a measure of the frequency with which the user has interacted with the first information item in the past. The optimization system compares the redundancy value for the first information item to a predetermined threshold value. When the redundancy value of the information item is greater than or equal to the predetermined threshold value, the optimization system requests a second information item from the information source server. The optimization system receives the second information item and places the second information item on the web page that is displayed on the graphical user interface.

Abstract: A declarative user interface representation in one declarative format can be converted to a declarative user interface representation in a different declarative format. Templates can rely on template helpers for mapping and programmatic extensions to support a wide variety of declarative formats. Complex scenarios such as composition and decomposition of properties across declarative representations can be supported. A hierarchical approach facilitates development and maintenance of solutions.

Abstract: A system and method for extracting a skeleton topology structure for an electric power grid, the method comprising: receiving a description of a topology sub-structure corresponding with user's need and a description of skeleton topology sub-structure extracted from the topology sub-structure; generating a first incidence matrix based on the description of the topology sub-structure and a second incidence matrix based on the description of the skeleton topology sub-structure; generating a third incidence matrix based on a primary topology structure of electric power grid; searching from the third incidence matrix a sub-matrix that matches the first incidence matrix; obtaining a fourth incidence matrix by using the second incidence matrix to transform the matching sub-matrix; and generating a skeleton topology structure corresponding to the primary topology structure based on the fourth incidence matrix.

Abstract: A method of network monitoring includes, acquiring a log including site information for identifying a web site that is previously accessed by an access source and content information for identifying a content that is requested by the access source, generating a first relationship information that includes site information for identifying the web site which provides the content, generating a second relationship information that includes content information for identifying another content that is requested by the access source requesting the content, generating a third relationship information that includes a second combination of the access source and the web site which provides both the content and the other content for the access source, and determining a group that includes the first combinations corresponding to the second combinations whose degree of similarity is no less than a threshold, based on the third relationship information.

Abstract: A data profiling server extracts data from multiple data sources and stores the extracted data in a target data store. Validation functions are performed on the extracted to determine whether the data conforms to validation rules associated with the target data store and/or business logic definitions. For data that does not conform to the validation rules, the data profiling server identifies recommended transformations to be applied to the data to correct any errors that cause the data to not conform with the validation rules. A data profiling application presents to a user the data extracted from the multiple data sources in a grid structure. The grid structure also includes error indicators identifying the data that does not conform to the validation rules and specifies the recommended transformations identified by the data profiling server.

Abstract: One embodiment provides a method comprising storing a first set of explicitly enumerated data values in a first window data structure associated with a first time span. The method further comprises storing one or more sets of summary statistics in a second set of window data structures. Each window data structure of the second set of window data structures is associated with a time span older than the first time span. The one or more sets of summary statistics are based on a second set of data values. The method further comprises storing a third set of explicitly enumerated data values in a third set of window data structures, wherein each data value of the third set of explicitly enumerated data values is annotated.

Abstract: A data preprocessing system builds transformation scripts for preprocessing datasets for processing by a data analysis system. The data preprocessing system presents various representations of data of a dataset including visual representations, textual representations, or structural representations. The data preprocessing system receives selections of attributes or values based on these representations. The data preprocessing system generates recommendations of transformations based on the attributes or values selected. The data preprocessing system builds a transformation script based on the recommendations of the transformations. The transformation script can be used for preprocessing the dataset for analysis by a data analysis system.

Abstract: A data processing apparatus is deployed at a cloud network, and includes a plurality of data nodes, a receiver circuit, and an aggregation circuit. The receiver circuit is coupled to the plurality of data nodes and is configured to receive downtime records from a plurality of industrial machines, and to selectively route the downtime records to selected ones of the plurality of data nodes based upon a predetermined criteria. Each of the plurality of data nodes comprises a control circuit and a memory. Each control circuit is configured to, in parallel with the other control circuits, to further populate the downtime records with other data related to the operation of the machines, determine one or more time windows for each of the records, and divide the records according to the time windows.

Abstract: The migration of a database may be accomplished, at least in part, by careful preparation and analysis of the database objects prior to migration. Migration may be performed more timely, efficiently, and reliably if the source database and its objects are first analyzed to determine the properties of the data, relationships, dependencies, and the like. The length of time of the copying and installation is reduced may selecting appropriate operations for different object types.

Abstract: Certain embodiments of the present disclosure relate to systems and methods that control access to system resources, such as interfaces, access rights to events, query systems, and other suitable system resources. Further, certain embodiments of the present disclosure relate to a collision detection technique that is implemented to control which and/or a number of queue positions within a queue that are processed. In some implementations, a collision may be detected when two or more users request the same access right within a defined time period.

Abstract: An analyzer system may include a computer-apparatus to collect traces from a pool of business traces, and to assign an unique vector value to each trace. The system may also include an assembler to create a tree based upon the unique vector value of each trace. The system may further include an analyzer to detect sub-trees within the tree to identify similarities among the traces based upon the traces inclusion within a given sub-tree.

Abstract: A building energy management system includes building equipment, a data collector, an analytics service, a timeseries database, and an energy management application. The building equipment monitor and control one or more variables in the building energy management system and provide data samples of the one or more variables. The data collector collects the data samples from the building equipment and generates a data timeseries including a plurality of the data samples. The analytics service performs one or more analytics using the data timeseries and generates a results timeseries including a plurality of result samples indicating results of the analytics. The timeseries database stores the data timeseries and the results timeseries. The energy management application retrieves the data timeseries and the results timeseries from the timeseries database in response to a request for timeseries data associated with the one or more variables.

Abstract: Utilizing social media and articles to suggest item attribute importance to improve site level searching for a user.

Abstract: The technology disclosed relates to creating and frequently updating multiple online analytic processing (OLAF) analytic databases from an online transaction processing (OLTP) transaction updatable system that includes transaction commit, rollback, and field level security capabilities. It also relates to transparently decoupling extraction from rebuilding of frequently updated OLAP analytic databases from the OLTP transaction updatable system.

Abstract: Embodiments are disclosed for utilizing an efficient database architecture while facilitating query flexibility. An example method includes generating, by data modeling circuitry of a database system, a database comprising a set of dimension tables and a hybrid fact table, wherein each hybrid fact table record identifies corresponding records in the set of dimension tables that specify (i) current values of the one or more attributes corresponding to the hybrid fact table record and (ii) records comprising historical values of the one or more attributes corresponding to the hybrid fact table record that were current at the time of creation of the hybrid fact table record. The example method further includes receiving a query, and generating and transmitting a response to the received query. Corresponding apparatuses and computer program products are also provided.

Abstract: A method includes receiving, by a real-time Data Warehouse (rDW) from a first task, a first dataset and spreading the first dataset to produce a first plurality of objects, where the first plurality of objects includes a first object and a second object. The method also includes storing the first object in a first location in an rDW data repository and storing the second object in a second location in the rDW data repository.

Abstract: In one embodiment, a method receives a query for analyzing data in a database. The query includes a first level calculation that is performed before a second level calculation. When an operator associated with the first level calculation is present in the query, the method performs: calculating the first level calculation by removing group by information for a reference characteristic for the first level calculation to determine a first level calculation result and calculating the second level calculation using the first level calculation result using the group by information for the reference characteristic to determine a second level calculation result, wherein the group by in the second level calculation combines groups of rows in a database table of the database that have the same content in their specified columns.

Abstract: A method, executed by a computer, for de-duplicating data includes receiving a dataset, pivoting the dataset along a set of columns that have a common domain to provide a pivoted dataset, de-duplicating the pivoted dataset to provide a de-duplicated dataset, and using the de-duplicated dataset. De-duplicating the pivoted dataset may include computing similarity scores for records that have different primary keys and merging records that have a similarity score that exceeds a selected threshold value. The method may include determining the set of columns having a common domain by referencing a business catalog and/or conducting a data classification operation on some or all of the columns of the dataset. The method may also include pivoting the dataset along another set of columns that have a different common domain. A computer system and computer program product corresponding to the method are also disclosed herein.

Abstract: A technique for repairing damage to a file system stores compression headers in association with respective extents of in-line compressed data. A repair utility reads contents from the compression headers and applies the contents to repair damaged metadata structures that the file system uses to access the associated extents. In this fashion, the repair utility restores access to compressed data, which would otherwise be lost.

Abstract: Various embodiments provide for integrating process cloud services (PCS) with intelligence cloud service (ICS). PCS analytics data is obtained using one or more PCS analytics data export application programming interfaces (APIs). A semantic model is provided to a predefined ICS system. The PCS analytics data is transferred to an ICS integration service. The PCS analytics data is converted into a format for a target intelligence system. The converted PCS analytics data is transferred to a service for later distribution as star schema data, for use by Representational State Transfer (REST) Application Programming Interfaces (APIs). The PCS analytics data is transferred from the ICS integration service to the predefined ICS system, for use by the ICS integration service to integrate PCS with the ICS system.

Abstract: System parses fields in a log entry, corresponding to a system user event, to identify a log code for a first and second log entry types. System identifies a common schema for the first and second log entry types. System identifies a schema identifier by applying the common schema to the log entry. The schema identifier corresponds to a schema in a container for the first log entry type or a container for the second log entry type. The container for the first log entry type includes the common schema and a first schema that is specific to the first log entry type. The container for the second log entry type includes the common schema and a second schema that is specific to the second log entry type. System identifies log data by using the corresponding schema to parse fields in the log entry, and outputs the log data.

Abstract: A process for electronically detecting a criminal organization out-of-boundary condition includes first maintaining a stored first geofence definition defining a geographic area associated with a first criminal organization. One or more audio and/or video streams or RF identifiers from recording devices located outside of the first geofence definition are received. An out-of-boundary condition is then detected in the streams or identifiers in which an individual determined to be associated with the first criminal organization is detected outside of the first geofence definition. Responsive to detecting the out-of-boundary condition, causing an electronic notification of the out-of-boundary condition to be one of output and transmitted to another computing device notifying the another computing device of the out-of-boundary condition.

Abstract: A method for providing composite cognitive insights which includes receiving streams of data from a plurality of data sources; processing the streams of data from the plurality of data sources, the processing the streams of data from the plurality of data sources performing data enriching and generating a sub-graph for incorporation into a cognitive graph; processing the cognitive graph, the processing the cognitive graph providing a plurality of individual cognitive insights; and, generating a composite cognitive insight, the composite cognitive insight being composed of the plurality of individual cognitive insights.