Abstract: A method of identifying relationships between data collections is disclosed. Each data collection comprises a plurality of data records made up of data fields. The method comprises performing a relationship search process based on a first seed value and a second seed value. A first set of records from the data collections is identified based on the first seed value. A second set of records from the data collections is identified based on the second seed value. The process then searches for a common value across the first and second record sets, wherein the common value is a value which appears in a first field in a first record of the first record set and in a second field in a second record of the second record set, wherein the first record is from a first data collection and the second record is from a second data collection.

Abstract: A system and method for determining a relationship among data sets. The method includes selecting a first data set from a first table, and a second data set from a second table, forming an inclusion dependency pair of data based on the selected first data set and the selected second data set, determining a resultant of the inclusion dependency pair, and determining a primary key-foreign key relationship between the first data set and the second data set based on the determined resultant.

Abstract: A computer-implemented method of importing data from a data source to a target storage system is disclosed. The method involves executing an automated data import process to retrieve data from the data source and update the target storage system based on the retrieved data. The automated data import process operates in dependence on a source data schema of the data source. The method involves detecting a change in the source data schema, and, in response to detecting the change in the source data schema, modifying the automated data import process. The modified automated data import process can subsequently be used to retrieve further data from the data source and update the target storage system based on the further retrieved data.

Abstract: Example implementations described herein are directed to systems and methods for managing internet protocol (IP) address assignment to servers on rack(s) based on their physical locations within the rack(s). Through the example implementations, the physical location of a server within a data center rack can be determined based on the IP address. Example implementations can involve issuing a ping local to determine a plurality of servers; retrieving power on time, current system time, and operating system (OS) uptime for each of the plurality of servers; determining a power on order for the plurality of servers; determining physical locations of the plurality of servers within one or more racks based on installation instructions and power on order; and assigning IP addresses to the plurality of servers based on the physical locations.

Abstract: In some examples, a client device may send, to a service computing device over a network, a request for information related to a change to a file system associated with the client device. In response, the client device may receive, from the service computing device, a list of one or more changes to the file system. The client device may add and/or update metadata for a stub file on the client device based at least partially on the received list of one or more changes. In some cases, the stub file may include a stub data structure stored on the client device, while the file content of the file corresponding to the stub file may be stored at a storage over the network.

Abstract: A system and method for continuous processing of data streams residing in distributed data sources. The method includes: receiving a plurality of data streams from a plurality of distributed data sources; processing each of the plurality of data streams using a plurality commands, the plurality commands are executed in parallel using a graph execution engine; and transporting, through a transport layer, each of the plurality of data streams using the plurality commands to at least one data sink.

Abstract: In some examples, a computer system may receive a data object including object data. For example, the system may be one of three or more systems and each system may be located at a different respective geographic location. The system may determine a value representative of the object data content. The system may further determine a plurality of chunks by dividing the object data into a plurality of data chunks based on a total number of the systems and determining a parity chunk for the plurality of data chunks. The system may determine a respective role value corresponding to each of the systems, and may send individual ones of the chunks to respective ones of the systems based on the role value and the value representative of the object data content. In some examples, each system may perform data deduplication independently of the other systems based on received chunks.

Abstract: A computer-implemented method of managing data in a data repository is disclosed. The method comprises maintaining a data repository, the data repository storing data imported from one or more data sources. A database entity added to the data repository is identified and a metadata object for storing metadata relating to the database entity is created and stored in a metadata repository. The metadata object is also added to a documentation queue. Metadata for the metadata object is received from user via a metadata management user interface and the received metadata is stored in the metadata repository and associated with the metadata object.

Abstract: Example implementations described herein involve a system that manages a dispatch of data within an Internet of Things (IoT) system that can involve a first process for intaking new data and conducting one of dispatching the new data or queuing the new data; a second process executed at lower priority than the first process involving determining if queued data exceeds a retry count; forwarding the queued data to a third process if the retry count does not exceed the threshold; and popping the queued data into an error process if the queued data exceeds the retry count; and the third process executed after receiving the queued data from the second process, involving attempting to dispatch the queued data.

Abstract: Example implementations described herein are directed to connecting and configuring a device to be configured into an Internet of Things (IoT) system through utilizing a mobile device. The mobile device is provided with an application that will provision and commission a device to be configured into the IoT system. Through example implementations described herein, the mobile device allows for a device to be configured to be connected to an IoT system even if no network connectivity is available for the device to be configured. The mobile device also provides sensor data in real time to determine if the device to be configured is properly configured.

Abstract: A system and method for data entries deduplication are provided. The method includes indexing an input data set, wherein the input data set is in a tabular formant and the indexing includes providing a unique Row identifier (RowID), wherein rows are the data entries; computing attribute similarity for each column across each pair of rows; computing, for each pair of rows, row-to-row similarity as a weighted sum of attribute similarities; clustering pairs of rows based on their row-to-row similarities; and providing an output data set including at least the clustered pairs of rows.

Abstract: A system and method for performing a hash bucketing process on data in motion are presented. The method includes applying a first hash function on an input dataset to map the input dataset to a bucket, wherein the first hash function results with a first hash value; applying a second hash function on the first hash value to map the input dataset to a record in the bucket; generating metadata based on the input dataset, wherein the metadata at least points to the original location of the input dataset; and storing the generated metadata in the record in the bucket.

Abstract: A method and software tool for identifying relationships between columns of one or more data tables are disclosed. In the disclosed method, a relationship indicator is computed for each of a plurality of column pairs, each column pair comprising respective first and second columns selected from the one or more data tables. The relationship indicator comprises a measure of a relationship (e.g. indicating a strength or likelihood of a relationship) between data of the first column and data of the second column. Relationships between columns of the data tables are then identified in dependence on the computed relationship indicators. The identified relationships may be used to create and execute data queries.

Abstract: In some examples, a computer system may receive an indication to present a user interface presenting data for a plurality of metrics related to at least one of a status or a performance of a storage system. The computer system may determine a status of a workflow that processes messages received from the storage system. Further, the computer system may determine an end time for a plurality of metric visualizations to present in the user interface based on the status of the workflow. The computer system may render the plurality of metric visualizations, with each metric visualization having the determined end time. Further, the computer system may provide the plurality of rendered metric visualizations for presentation in the user interface on a display.

Abstract: A system and computer implemented method for cataloging database metadata using a signature matching process are provided. The method includes receiving an input name to be matched to a key in a seed table; generating a first fingerprint by decomposing the received input name into a first set n-grams; generating, based on the received input name, a second fingerprint using a predetermined pronunciation schema, wherein the second fingerprint is a phonetic fingerprint; generating a third fingerprint by decomposing the second fingerprint into a second set of n-grams; identifying a matching key by matching any combination of the first fingerprint, the second fingerprint, and the third fingerprint against keys in the seed table; and cataloging the metadata with the matching key as a tag.

Abstract: In some examples, a computer system may receive a data object including object data. For example, the system may be one of three or more systems located at different respective geographic locations. The system may determine a plurality of chunks by dividing the object data into a plurality of data chunks based on a total number of the systems and determining a parity chunk. The system may send some of the chunks to the other systems, while maintaining a complete instance of the object data at the first system. The system may associate a time with the object data based on receiving the data object. Additionally, the system may associate a time threshold with the complete instance of the object data. For example, the time threshold may indicate a time at which the complete instance of the object data is to be deleted from the system.

Abstract: In system for analyzing large data sets, document/file format can be discovered by attempting to parse the file using several parsers to generate a schema, assigning a score to each parsing, and selecting a parser based on the assigned scores. Schema element attributes, such as statistical parameters, can be derived and used in identifying schema elements associated with other files. Attributes of identified schema elements can be used to substitute missing data values with values based on such attributes. Data values corresponding schema elements can be selected and highlighted, and schema elements and/or attributes thereof can be highlighted based on selected data values. From a cluster of files, a lineage relationship between file pairs, indicating whether one file is derived from another, can be determined for several files. In reducing/compacting data, utilization of all available reducers can be optimized according to current utilization of one or more reducers.

Abstract: A computer program product comprising computer-readable instructions that, when executed in a computer system including one or more computers, cause the computer system to generate or update a user interface of a software application, the computer program product including programmably interconnected objects, said objects including one or more model objects, one or more view objects, and one or more controller objects, wherein each model object is interconnected for data exchange with one or more view objects and/or with one or more controller objects; and each model object includes one or more sub-model objects including hierarchically structured data and representing a state of the user interface; and each view object is associated with at least one model object or at least one sub-model object and configured to generate the user interface or update the user interface in accordance with a change of the state.

Abstract: In some examples, a system may use content classes to manage indexing of object data. A content class may include a set of one or more content properties. Each content property may include a name, an expression for extracting data, and an expression type. When object data is received, such as for indexing, the expression type of each content property may be compared with the data type of the received data. Based at least in part on determining that the expression type matches the data type, the system may extract a data value from the received data in accordance with the expression. The system may save the extracted data value to a data structure in association with the name of the content property, and may subsequently use the data value and the name of the content property when creating an index for the object data.

Abstract: A method and system for tagging database columns are presented. The method includes receiving an input column name of at least one column in a database; performing signature matching of the input column name to contents of a seed table; determining a first confidence score for the signature matching; and tagging a matching value in the seed table as a tag for the input column name, when a first confidence score exceeds a first threshold value.