Abstract: Techniques for configuring and processing control path commands may include: partitioning control path components of a control path into a plurality of portions; performing first processing that configures a federation of a plurality of appliances, wherein each of the plurality of appliances includes a plurality of processing nodes, and wherein the first processing includes: for each of the plurality of appliances, configuring each of the plurality of processing nodes of said each appliance to run one portion of the plurality of portions of control path components; and selecting one of the plurality of appliances as a primary appliance of the federation; receiving a first management command at the primary appliance of the federation; and servicing the first management command by one or more of the plurality of appliances of the federation.

Abstract: In accordance with an embodiment, described herein is a system and method for providing extensibility in an analytic applications environment, including a semantic layer that enables the use of custom semantic extensions to extend a semantic data model (semantic model). In accordance with an embodiment, customizations to the out-of-the-box semantic model are performed using a layered approach, wherein the factory code for the semantic model remains intact, with changes/delta editable by the customer layered on top of that model, such that the changes can be patched/reversed if necessary.

Abstract: A novel distributed method for machine learning is described, where the algorithm operates on a plurality of data silos, such that the privacy of the data in each silo is maintained. In some embodiments, the attributes of the data and the features themselves are kept private within the data silos. The method includes a distributed learning algorithm whereby a plurality of data spaces are co-populated with artificial, evenly distributed data, and then the data spaces are carved into smaller portions whereupon the number of real and artificial data points are compared. Through an iterative process, clusters having less than evenly distributed real data are discarded. A plurality of final quality control measurements are used to merge clusters that are too similar to be meaningful. These distributed quality control measures are then combined from each of the data silos to derive an overall quality control metric.

Abstract: The present application is directed towards systems and methods for automated analysis and transformation of databases and related applications and functions from a row-oriented syntax to a column-oriented syntax. An analyzer client executed by a processor of a client device may analyze a first database of a business management system comprising a plurality of arrays in a row-oriented syntax. A transformer executed by the processor may execute a first array of a second database having a column-oriented syntax. Iteratively, for each array of the plurality of arrays of the first database, the transformer may read an element at a first position of said array, and write the element to a next position of the first array of the second database.

Abstract: A device retrieves historical data and new data each a respective hardware component identifier and a respective associated value. The device creates a synthesized set of data by having subsets for anomalous data, data that is associated with an attenuation signal, and other data. The device discards the anomalous data and weights the data associated with an attenuation signal. The device generates a searchable database, the searchable database including each hardware component named by an entry of the synthesized set of data, along with an associated value determined based on the weighted value of the entry. The device receives user input of a search query, and outputs search results based on a comparison of the user input of the search query to entries of the searchable database.

Abstract: A device automatically extracts a data file from an upstream source based on ingestion parameters. The data file is in a first format that is not readable by a downstream data analysis utility and includes a plurality of data records. Each record includes one or more entries. The ingestion parameters include a file identifier, transformation instructions, and storage instructions for the extracted data file. The device generates table(s) based on the extracted data file by, determining, based on the transformation instructions, a table in which to store each entry and a row and column of the determined table in which to store the entry. Entries are input into the determined tables, rows, and columns. The table(s) are stored in a second format that is readable by the downstream data analysis utility.

Abstract: Embodiments are directed to organizing conversation information. A tracker vocabulary may be provided to a universal model to predict a generalized vocabulary associated with the tracker vocabulary. A tracker model may be generated based on the portions of the universal model activated by the tracker vocabulary such that a remainder of the universal model may be excluded from the tracker model. Portions of a conversation stream may be provided to the tracker model. A match score may be generated based on the track model and the portions of the conversation stream such that the match score predicts if the portions of the conversation stream may be in the generalized vocabulary predicted for the tracker vocabulary. Tracker metrics may be collected based on the portions of the conversation and the match scores such that the tracker metrics may be included in reports or notifications.

Abstract: Embodiments of the present disclosure may provide a streamlined process for performing operations, such as data sharing and data replication, using multiple accounts. A global identity (also referred to as an organization user) may be employed, where the global identity may have access to multiple accounts across the same or different deployments. The global identity may switch between accounts from its login session and perform various tasks in the context of different accounts without undergoing further authentication.

Abstract: A message system may use publish-subscribe techniques to process messages for several applications. In situations where an application requires an output, it may need to wait for a response message back from a service via the message system. However, in certain implementations, the message system may not be able to determine that a particular message involves request-response communication. The efficient response communication techniques described herein receive the response message having a routing key and a payload including a correlation identifier and then generate a second response having a routing key that includes the correlation identifier. This message may then be efficiently routed to an application based on the correlation identifier.

Abstract: A method and system may be implemented for automatically analyzing data in a database. A method for use in a low-latency database analysis system may include generating a schema. The schema may be based on a portion of an external database. The method may include storing the schema in an in-memory database. The method may include receiving a data-query. The method may include generating a resolved-request. The resolved-request may be based on the data-query and the stored schema. The stored schema may be used for executing the query on the external database. The method may include receiving results data responsive to the data-query from the external database. The method may include outputting the results data for display on a user interface.

Abstract: A transaction processing method and system, and a server. The method includes: after a first node obtains at least one statement corresponding to a first transaction, the first node may classify each statement, and send the at least one statement to at least one second node according to the type of each statement, respectively, so that the at least one second node generates an execution plan corresponding to the received statement according to the statement; then the first node may process the first transaction according to the received execution plan which corresponds to the at least one statement of the first transaction and is sent by the at least one second node. In this way, embodiments can process statements of multiple types at the same time, thereby effectively improving the processing efficiency of the system, reducing the operating burden of the system, and ensuring the transactionality of the system.

Abstract: The method receives a visual specification, which specifies a data source, visual variables, and data fields from the data source. Each visual variable is associated with data fields and each data field is a dimension or a measure. The method forms dimension tuples comprising distinct ordered combinations of data values for the dimensions D. For each measure, the method: forms a set S of the dimensions D plus dimensions from a primary key corresponding to the measure; retrieves intermediate tuples containing the fields in S and the measure, without aggregation; and aggregates the intermediate tuples according to the dimensions D. For each dimension tuple, the method forms an extended tuple by appending the aggregated data values corresponding to each measure field. The method then builds and displays a data visualization according to the extended tuples and the visual variables to which the data fields are associated.

Abstract: Systems and methods for supporting rollback of changes made to target systems via an integration platform. Systems and methods can support reverting changes made to target applications via an integration platform (e.g., integration platform as a service (iPaaS)) with the help of a rollback activity. In order to aid the development and design of integration orchestrations, the integration platform can provide suggested rollback actions. In case of a failure in integration runtime, this rollback activity will revert changes made to the target applications prior to the point of fault. This helps ensure that systems remain in consistent state.

Abstract: A big data analysis system may include a big data repository communicatively coupled to a data accumulation server and a predictive graph processing system. The data accumulation server may be configured to receive information from a plurality of data sources, the information corresponding to user interaction with one or more computing devices associated with an organization via a networked computing system, store the information received from the plurality of sources in the big data repository; and monitor the plurality of data sources to update the data stored in the big data repository. The predictive graph processing system is configured to receive information stored in the big data repository, transform the information received from the big data repository into a predictive graph data set based on a predictive model, and store the predictive graph data set to a visualization data repository.

Abstract: A computer-implemented system and method process insertions into a target table of a target database system. A single query language INSERT statement is received that is related to a source partition of a source table in a source database system that identifies a block of data (BOD), having a partition identifier (PID), to be inserted into the target table. The INSERT statement contains no literal PID. Responsive to receiving the INSERT statement, a processor receives the BOD from the source table via a data stream, and inserts the BOD into the target table. This reduces the overhead associated with using an INSERT statement to insert multiple rows of data into a target table.

Abstract: Techniques are disclosed for implementing, by a computer infrastructure orchestration service (CIOS) changes to a first service. The CIOS may manage a declarative infrastructure provisioner to deploy infrastructure resources and artifacts based at least in part on declarative instructions. CIOS may detect a first modification to a first service (e.g., a sidecar application). CIOS may identify that configuration data of a second service references (e.g., via a git submodule) the first service. In response to identifying a second modification to the second service, CIOS may present, via a graphical interface, changes comprising the first modification to the first service. User input may be received indicating acceptance or rejection of the changes of the first modification. CIOS may then implement the changes to the first service according to the user input. For example, CIOS may ensure that the first service, as modified, is deployed alongside the second service in subsequent deployments.

Abstract: Techniques for efficiently storing client data blocks on a distributed-computing system are provided. The system includes a fast performance tier and a large capacity tier. The capacity tier stores the client data blocks in erasure encoded data stripes. The performance tier stores logical map data including an address map indicating a correspondence between logical addresses associated with a first layer of the system and physical addresses associated with a second layer. A method includes receiving a request to include additional client data blocks in the client blocks. The request indicates logical addresses for additional blocks. Corresponding physical addresses for additional block are determined. Each additional block is stored at the physical address. Additional logical map data is stored in the performance tier. Storing the additional logical map data includes updating the address map to indicate the correspondence between the logical addresses and the physical addresses for the additional blocks.

Abstract: In a computer-implemented method for adapting time series database schema of a time series database, time series data ingested into a time series database according to a time series database schema is accessed over a time period, wherein time series data comprises a plurality of dimensions. The time series data of the time period is analyzed to determine a data shape of the time series data of the time period. It is determined whether to adapt the time series database schema based at least in part on the data shape of the time series data of the time period. In some embodiments, the time series database schema is adapted based at least in part on the data shape of the time series data of the time period. Time series data is then ingested into the time series database according to the adapted time series database schema.

Abstract: A computer-implemented method for transferring changes made to data from a data source to a data destination may be provided. The method involves using a message broker with streaming channels for transferring data changes from a data source to a data consumer. To determine if data has changed, a portion of data labeled with a key is sent through a hash function. A hash value with the corresponding key that has already been sent to the data consumer is retrieved from a streaming channel and compared to the newly created hash value. If the newly created hash value is different the data is sent to data consumers by the message broker via one streaming channel, while the new hash value is stored in the other streaming channel.

Abstract: Disclosed are various embodiments for searching encrypted data stores. A first computing device can send a message authentication code (MAC) to a second computing device the MAC representing a tuple of a name of a data table, an identifier of a column of the data table, and a valid value for the column. Then, an encrypted bitmap can be received from the second computing device and decrypted to generate a decrypted bitmap. Bitwise operations can be performed on the decrypted bitmap to identify a plurality of encrypted rows of the data table containing a cell that comprises the valid value. Then, at least the plurality of at least partially encrypted rows of the data table can be requested from the second computing device and decrypted to generate a respective plurality of decrypted rows.

Abstract: Embodiments provide systems and methods for learning extract, transform, and load mappings by example. A plurality of features can be extracted from a source schema and a target schema. Example ETL mappings can be provided to a machine learning algorithm that comprise definitions for extracting data from source tables and loading the extracted data into target tables. Using the machine learning algorithm and based on the source schema, target schema, and extracted features, one or more ETL rules can be predicted that define logic for extracting data from the source schema and loading the extracted data into the target schema. Additional ETL mappings can be generated based on the predicted ETL rules, the additional ETL mappings providing additional definitions for extracting data from the source schema and loading the extracted data into the target schema.

Abstract: A data movement system is provided for moving data using a data-to-file-to-data movement path. The data movement system includes a source database, a target database, a configuration database, and a data movement server. The data movement is in communication with the source database, the target database, and the configuration database. The processor is configured to receive a configuration record including source details and target details. The processor is also configured to define an extraction query based on the source details and to apply the extraction query to the source database to obtain an extraction load. The processor is further configured to generate a load file based on the extraction load, to define a load script based on the target details, and to apply the load script to the load file to obtain a load query. The processor is also configured to update the target database with the load query.

Abstract: Managing ETL (extract-transform-load) operation execution by receiving an updated ETL (extract-transform-load) job flow, determining affected stages of a current ETL job flow, sending a message to stop data flow for all stages of the current ETL job flow, sending a switch message along the current ETL job flow through the affected stages, receiving notice of receipt of the switch message downstream from the affected stages of the current ETL job flow, moving an affected stage of the current ETL job flow from a first ETL section to a second ETL section; and resuming data flow according to the updated ETL job flow.

Abstract: Embodiments of the present disclosure provide methods and devices for formatting a database. The method can include clustering, into each block, a set of sequence of elements from each column of a database; generating a file storing one or more blocks having the clustered set of sequences of elements; providing a first table storing file schema information that associates the elements with the file; and providing a second table for storing block schema information that associates the columns with the one or more blocks.

Abstract: In an approach, a processor obtains a plurality of security policies for managing security of a database, the security policies defining respective sets of items to be monitored. A processor stores, respectively, the sets of items defined by the plurality of security policies in one or more item tables based on a predetermined criterion, the predetermined criterion comprising security policy relevance, security policy structure, and available space in the one or more item tables. A processor creates, for each of the one or more item tables, one or more respective index tables associated with the items in the respective item table. A processor creates, for each of the one or more item tables, a fast traverse block (FTB) associated with the created one or more respective index tables.

Abstract: The subject technology receives raw input data from a source table, the raw input data including data comprising input features for a machine learning model, the raw input data being in a first format including at least multiple rows with each row including multiple columns of values. Based at least in part on the source table, the subject technology generates table metadata corresponding to the source table. Based at least in part on the received raw input data, the subject technology generates column metadata corresponding to values from the source table. The subject technology generates cell data for a feature store table based at least in part on the values from the source table. The subject technology performs at least one database operation to generate the feature store table including at least the generated table metadata, the generated column metadata, and the generated cell data.

Abstract: Alternative portions of a query plan to perform a query may be selected for processing partial results of a query. A query engine may generate a plan to perform a query that includes an operation that selects between different portions of the plan according to input data. The input data may be a partial result of the query generated by the data store instead of the query engine and provided to the query engine. The input data may be an individual item of the data set that is modified to a different version within a consistent view of the data set for the query.

Abstract: Various methods, apparatuses/systems, and media for integrating data are provided. A processor implements a data processing framework configured to run native on a big data platform and abstracts data processing constructs to a user friendly template, thereby eliminating necessity of user initiated tasks of instantiating language level objects. The processor also implements a core set of data pipeline configurations on the template configured to initiate a chain of user defined data transformations. A receiver operatively connected with the processor via a communication network receives input of the chain of the user defined data transformations. The processor tests each transformation independently of each other and outputs data integration solutions on the big data platform based on a positive test result.

Abstract: A method includes generating a table in a relational database, the table comprising an empty column reserved for a custom tag. The column has a default column name. The method also includes, in response to an instruction to create the custom tag, generating a column mapping that maps a name of the custom tag to the column name. The column mapping is stored outside the relational database and is used by a parser to translate between the name of the custom tag and the column name.

Abstract: A building energy management includes building equipment, one or more data platform services, a timeseries database, and an energy management application. The building equipment operate to monitor and control a variable and provide raw data samples of a data point associated with the variable. The timeseries database stores a plurality of timeseries associated with the data point. The plurality of timeseries include a timeseries of the raw data samples and the one or more optimized data timeseries generated by the data platform services based on the raw data timeseries. The energy management application generates an ad hoc dashboard including a widget and associates the widget with the data point. The widget displays a graphical visualization of the plurality of timeseries associated with the data point and includes interactive user interface options for switching between the plurality of timeseries associated with the data point.

Abstract: In an approach to determine performance information of a target item operating under a particular set of context information, a method, in response to receiving a request for performance information of a target item, and operating with a first computing device, identifies context information of the first computing device. The method determines whether a knowledge base includes a response that correlates to the request for performance information of the target item operating within context information similar to the first computing device. The method, in response to determining that the knowledge base includes the response that correlates to the request for performance information of the target item, sends the performance information to the first computing device, and initiates a communication channel between the first computing device and a second computing device operating the target item and having similar context information of the first computing device.

Abstract: A novel distributed method for machine learning is described, where the algorithm operates on a plurality of data silos, such that the privacy of the data in each silo is maintained. In some embodiments, the attributes of the data and the features themselves are kept private within the data silos. The method includes a distributed learning algorithm whereby a plurality of data spaces are co-populated with artificial, evenly distributed data, and then the data spaces are carved into smaller portions whereupon the number of real and artificial data points are compared. Through an iterative process, clusters having less than evenly distributed real data are discarded. A plurality of final quality control measurements are used to merge clusters that are too similar to be meaningful. These distributed quality control measures are then combined from each of the data silos to derive an overall quality control metric.

Abstract: A data processing framework suitable for use with data stores is described. The data processing framework identifies which data stores are storing corrupt or invalid data that may cause end user software applications to malfunction or crash altogether. Data parsing, validation and transformation may be performed as parts of a single operation to increase speed and reduce computing overhead.

Abstract: Systems and methods for processing data are provided. The system may include at least a processor and a non-transient data memory storage, the data memory storage containing machine-readable instructions for execution by the processor, the machine-readable instructions configured to, when executed by the processor, provide an information delivery platform configured to: extract raw data from a plurality of source systems; load and store the raw data at a non-transient data store; receive a request to generate data for consumption for a specific purpose; in response to the request, select a set of data from the raw data based on a data map; transform the selected set of data into a curated set of data based on the data map; and transmit the curated set of data to a channel for consumption.

Abstract: A computer implemented method includes establishing, by a data intake and query system, a network connection between the data intake and query system and an application and infrastructure monitoring platform. The data intake and query system receives a data stream from the application and infrastructure monitoring platform. The computer implemented method further includes transforming the data stream while receiving the data stream to obtain a transformed data stream. Further, the transformed data stream is analyzed while receiving the data stream to generate analysis results, which are presented.

Abstract: Technologies are described for performing automated data integration, reconciliation, and/or self-healing using machine learning. For example, data integration can be checked using a reconciliation procedure. The number of times that the reconciliation is performed can be determined dynamically by a machine learning model. For each iteration, reconciliation can be performed to check integrated data against source data. If any reconciliation errors are found, then self-healing operations can be performed. Results of the reconciliation can be output. The reconciliation results can be used to update the machine learning model so that the machine learning model can dynamically adjust the number of iterations to perform based at least in part on reconciliation results.

Abstract: A method for calculating a size of a processing unit in a system, the system being configured to perform a reflection process that includes reflecting information on an operation performed in a row-type database on a column-type database in a predetermined processing unit, the method includes: executing a first calculation process for calculating a reference value based on processing costs of multi-version concurrency control performed in the processing unit by the database management system and information on processing costs of a process of converting the information on the operation performed in the row-type database into a form of the column-type database after the reflection process is executed in a previous time; and executing a second calculation process for calculating a size of a processing unit based on the reference value calculated by the first calculation process and information on a deviation of the number of valid rows.

Abstract: An aspect of the present disclosure facilitates validation of responses received from server systems. In one embodiment, data values are maintained in a data source. Upon sending a request containing a query criteria to a server system, the server system retrieves data values satisfying the query criteria from the data source. A (actual) response is received from the server system, the actual response containing data values retrieved by the server system. Data values satisfying the query criteria are also retrieved directly from the data source and an expected response is constructed by incorporating the directly retrieved data values. The data values contained in the expected and the actual responses are then compared to determine a difference between the expected and actual responses as a basis for validating the actual response received from the server system for the request.

Abstract: Computer-implemented methods and systems for migrating business data from source business system to target business system are described. The method performed by server system includes receiving request from user device to migrate source data from source business system to target business system and accessing source metadata associated with source business system and target metadata associated with target business system. The method includes generating one or more source staging tables based on source metadata. The one or more source staging tables include information of a plurality of source columns that is to be mapped. The method includes creating mapping sets for mapping source data to the target metadata, transforming the source data stored in the one or more source staging tables into a target data structure based on the target metadata, and loading a target staging table associated with target business system based, at least in part, on the transforming step.

Abstract: Aspects of the present disclosure address systems, methods, and devices for tracking object dependencies in a cloud database system. An object dependency created between a referencing object and a referenced object is detected. Based on detecting the object dependency, a dependency record is generated. The dependency record includes dependency information describing the object dependency between the reference object and the referenced object. The dependency record is stored in a database of dependency records.

Abstract: Systems and methods are described for scheduling a query for execution. The system receives and parses a query to identify one or more portions of the query. The system determines a resource allocation for each portion of the query, and determines an availability of compute resources for the different portions of the query. Based on the resource allocation and the availability of compute resources, the system schedules the query.

Abstract: Enhanced database query handling is provided using pre-calculated association pagination ranking. Such pre-calculated rankings can be generated using various approaches including by calculating rank along with an intersection, addition, and deletion of association rows corresponding to a parent row being modified or by a stored procedure which uses the ranking operations. Related apparatus, systems, techniques and articles are also described.

Abstract: Techniques are provided for creating a unified data model for entities in components of an access management system. A method can include identifying an entity of a component, identifying a first attribute metadata for the entity, creating a data type for the first attribute metadata of the entity, defining the entity in accordance with the created data type, and storing the defined entity in a unified storage.

Abstract: There is disclosed a method of processing a data packet received by a packet sniffer, the packet containing an associated identifier, the method including transmitting the packet to a recipient, determining if the identifier corresponds to a particular network, wherein if the identifier is determined to correspond to the particular network, the identifier is provided to the recipient; and if the identifier is determined to not correspond to the particular network, the identifier is withheld from the recipient.

Abstract: Aspects of the present disclosure involve systems, methods, devices, and the like for creating a unified analytics platform for big data analytics, visualization and data store. In one embodiment the unified analytics platform can include at least a computer platform, core data platform, and integration platform. The unified analytics platform is designed to provide visibility into datasets, provide operational metrics, and provide the integration of notebooks and tools for big data analysis.

Abstract: Described herein are systems and methods of transforming data models, for example, creating a data warehouse. A directives model may be loaded based upon a parsed directives file. An entity model may be loaded, and tables, attributes, and foreign keys of a data warehouse model may be created based upon the directives model and the entity model. Mappings may be created between tables, columns, and foreign keys of the data warehouse model and entities, attributes, and relationships, respectively, of the entity model. Code to define a data warehouse may be generated based upon the tables, attributes, and foreign keys of the data warehouse model. Code to transfer data from the source data source can be generated based upon the created mappings. A lineage report can be generated that provides information identifying a corresponding source for each table and column in the data warehouse.

Abstract: Described herein is a computer implemented method. The method comprises processing clauses included in an original SQL query to add each clause to either a set of caching clauses or a set of non-caching clauses and determining whether a subscription should be created. In response to determining the subscription should be created, the method comprises creating the subscription by generating a first stage SQL query based on the set of caching clauses, causing execution of the first stage SQL query to obtain a set of first stage query results from a database; and storing the set of first stage query results in a cache.

Abstract: According to an embodiment of the present invention, a probability to achieve a purpose is increased in a case where information is intermittently allocated to a plurality of elements. An information processing apparatus according to an embodiment of the present invention is an information processing apparatus that allocates information to respective elements included in a first set and includes an allocator. The allocator uses a first subset and a second subset, and allocates the information to at least one element included in the first subset. The first subset is constituted by elements which are included in the first set and in which information allocation may be performed at a present time. The second subset is constituted by elements which are included in the first set and in which the information allocation is not performed at the present time.

Abstract: A multistage data sniffer instance can include a first stage that scans a given file for a set of data fields based on a configuration file for a selected format of the given file. The multistage data sniffer instance can also include a second stage that evaluates a value in each data filed in the set of data fields for the selected format to determine a validity of values in the set of data fields. The multistage data sniffer instance can further include a third stage that extracts data within the plurality of fields of the given file, aggregates the data based on a predetermined set of rules defined in the configuration file and outputs a data to a data mart database characterizing the aggregated data.

Abstract: Replication and failover of database data is disclosed. A method includes replicating database data stored in a primary deployment such that the database data is further stored in a secondary deployment. The method includes executing one or more updates to the database data at the secondary deployment when the primary deployment is unavailable and propagating the one or more updates to the primary deployment when the primary deployment becomes available again. The method includes executing queries on the database data at the primary deployment when the primary deployment is available.