Abstract: In some examples, a computing device is able to communicate with a plurality of data clusters. For example, a first data cluster may be configured with a first version of data cluster software including a first library version, and a second data cluster may be configured with a second version of data cluster software including a second library version that is different from the first. The computing device may execute a single instance of an application to send, based at least on information in the first library version, to the first data cluster, a request for first data, and may receive the first data from the first cluster. Further, the computing device may send, based at least on information in the second library version, second data that is based on the first data to the second data cluster to store the second data with the second data cluster.

Abstract: In some examples, a first computing device of a plurality of computing devices may be a member of a first partition group. For example, the first partition group may correspond to a first partition of multiple partitions of a metadata database. The first partition group may include at least one other computing device. Furthermore, each of the plurality of computing devices may maintain at least one partition of the metadata database so that the metadata database is partitioned into multiple partitions distributed across the computing devices. The first computing device may determine a first time threshold for a heartbeat communication to be sent to the other computing device(s) in the first partition group, and may select a random time before the first time threshold. The first computing device may send, based on the selected random time, the heartbeat communication to the other computing device(s) in the first partition group.

Abstract: In some examples, a data node may provide a file system executed in a user space. The data node may invoke a device check process configured to perform a test for failure of a storage device associated with a target volume by sending a file system level request for obtaining metadata of data stored to the target volume. For instance, the target volume may be a virtual storage volume that represents storage capacity on a network storage. The user-space file system may receive the request for the metadata and may retrieve preconfigured metadata of the target volume that is stored locally in advance of the invoking of the device check process. The preconfigured metadata may be sent to the device check process in response to the request. For example, the preconfigured metadata may at least partially cause the target volume to pass the test.

Abstract: Example implementations described herein are directed to a novel Automated Machine Learning (AutoML) framework that is generated on an AutoML library so as to facilitate functionality to incorporate multiple machine learning model libraries within the same framework through a solution configuration file. The example implementations further involve a solution generator that identifies solution candidates and parameters for machine learning models to be applied to a dataset specified by the solution configuration file.

Abstract: In some examples, a first computing device may receive, from a second computing device, partition mapping information indicating partitions of a metadata database. The first computing device may be able to communicate with a plurality of metadata nodes, each metadata node maintaining a portion of the metadata database based on the partitioning of the metadata database to distribute the metadata database across the plurality of metadata nodes. The first computing device may determine to send a request to the metadata database based at least on key information. The first computing device may determine, based on the partition mapping information, a first metadata node of the plurality of metadata nodes indicated to maintain a partition of the metadata database corresponding to the key information. The first computing device, may send, to the first metadata node, based on the partition mapping information, a request to perform a database operation.

Abstract: Example implementations described herein are directed to systems and methods for managing internet protocol (IP) address assignment to servers on rack(s) based on their physical locations within the rack(s). Through the example implementations, the physical location of a server within a data center rack can be determined based on the IP address. Example implementations can involve issuing a ping local to determine a plurality of servers; retrieving power on time, current system time, and operating system (OS) uptime for each of the plurality of servers; determining a power on order for the plurality of servers; determining physical locations of the plurality of servers within one or more racks based on installation instructions and power on order; and assigning IP addresses to the plurality of servers based on the physical locations.

Abstract: Systems and methods described herein are directed to upgrading one or more of add-on firmware and disk firmware for a server, which can involve connecting a port of the server to an isolated network, the isolated network dedicated to firmware upgrades for the server; caching onto cache memory of the server, an operating system received through the isolated network; booting the operating system on the server from the cache memory; conducting an Network File System (NFS) mount on the server to determine hardware information associated with the upgrading of the one or more of the add-on firmware and the disk firmware; and upgrading the one or more of the add-on firmware and the disk firmware based on the hardware information.

Abstract: In some examples, a first computing device may receive, from a server, an indication that the server has recovered data. For instance, the first computing device may store metadata including a mapping for one or more file systems accessed by one or more client devices. Furthermore, a second computing device may store a copy of the mapping stored on the first computing device. The first computing device may receive, from the server, a mapping of the one or more file systems determined by the server based on the recovered data. The first computing device may compare the mapping from the server with the mapping of the one or more file systems on the first computing device, and may send, to the server, information about changes determined between the two mappings to enable the server to update the mapping on the server based on the changes.

Abstract: In some examples, a system may determine a mountpoint included in a first filesystem for mounting a userspace filesystem. For instance, the userspace filesystem may be executable in a userspace provided by an operating system. The system may determine a bind mount path for a mount path corresponding to the mountpoint. The system may bind mount, to the determined bind mount path, at least a portion of a second filesystem associated with the mount path corresponding to the mountpoint. In addition, the system may mount the userspace filesystem at the mountpoint, and may incorporate data from the second filesystem into the userspace filesystem through the bind mount path.

Abstract: A method of identifying relationships between data collections is disclosed. Each data collection comprises a plurality of data records made up of data fields. The method comprises performing a relationship search process based on a first seed value and a second seed value. A first set of records from the data collections is identified based on the first seed value. A second set of records from the data collections is identified based on the second seed value. The process then searches for a common value across the first and second record sets, wherein the common value is a value which appears in a first field in a first record of the first record set and in a second field in a second record of the second record set, wherein the first record is from a first data collection and the second record is from a second data collection.

Abstract: A system and method for determining a relationship among data sets. The method includes selecting a first data set from a first table, and a second data set from a second table, forming an inclusion dependency pair of data based on the selected first data set and the selected second data set, determining a resultant of the inclusion dependency pair, and determining a primary key-foreign key relationship between the first data set and the second data set based on the determined resultant.

Abstract: A computer-implemented method of importing data from a data source to a target storage system is disclosed. The method involves executing an automated data import process to retrieve data from the data source and update the target storage system based on the retrieved data. The automated data import process operates in dependence on a source data schema of the data source. The method involves detecting a change in the source data schema, and, in response to detecting the change in the source data schema, modifying the automated data import process. The modified automated data import process can subsequently be used to retrieve further data from the data source and update the target storage system based on the further retrieved data.

Abstract: Example implementations described herein are directed to systems and methods for managing internet protocol (IP) address assignment to servers on rack(s) based on their physical locations within the rack(s). Through the example implementations, the physical location of a server within a data center rack can be determined based on the IP address. Example implementations can involve issuing a ping local to determine a plurality of servers; retrieving power on time, current system time, and operating system (OS) uptime for each of the plurality of servers; determining a power on order for the plurality of servers; determining physical locations of the plurality of servers within one or more racks based on installation instructions and power on order; and assigning IP addresses to the plurality of servers based on the physical locations.

Abstract: In some examples, a client device may send, to a service computing device over a network, a request for information related to a change to a file system associated with the client device. In response, the client device may receive, from the service computing device, a list of one or more changes to the file system. The client device may add and/or update metadata for a stub file on the client device based at least partially on the received list of one or more changes. In some cases, the stub file may include a stub data structure stored on the client device, while the file content of the file corresponding to the stub file may be stored at a storage over the network.

Abstract: A system and method for continuous processing of data streams residing in distributed data sources. The method includes: receiving a plurality of data streams from a plurality of distributed data sources; processing each of the plurality of data streams using a plurality commands, the plurality commands are executed in parallel using a graph execution engine; and transporting, through a transport layer, each of the plurality of data streams using the plurality commands to at least one data sink.

Abstract: In some examples, a computer system may receive a data object including object data. For example, the system may be one of three or more systems and each system may be located at a different respective geographic location. The system may determine a value representative of the object data content. The system may further determine a plurality of chunks by dividing the object data into a plurality of data chunks based on a total number of the systems and determining a parity chunk for the plurality of data chunks. The system may determine a respective role value corresponding to each of the systems, and may send individual ones of the chunks to respective ones of the systems based on the role value and the value representative of the object data content. In some examples, each system may perform data deduplication independently of the other systems based on received chunks.

Abstract: A computer-implemented method of managing data in a data repository is disclosed. The method comprises maintaining a data repository, the data repository storing data imported from one or more data sources. A database entity added to the data repository is identified and a metadata object for storing metadata relating to the database entity is created and stored in a metadata repository. The metadata object is also added to a documentation queue. Metadata for the metadata object is received from user via a metadata management user interface and the received metadata is stored in the metadata repository and associated with the metadata object.

Abstract: Example implementations described herein are directed to connecting and configuring a device to be configured into an Internet of Things (IoT) system through utilizing a mobile device. The mobile device is provided with an application that will provision and commission a device to be configured into the IoT system. Through example implementations described herein, the mobile device allows for a device to be configured to be connected to an IoT system even if no network connectivity is available for the device to be configured. The mobile device also provides sensor data in real time to determine if the device to be configured is properly configured.

Abstract: Example implementations described herein involve a system that manages a dispatch of data within an Internet of Things (IoT) system that can involve a first process for intaking new data and conducting one of dispatching the new data or queuing the new data; a second process executed at lower priority than the first process involving determining if queued data exceeds a retry count; forwarding the queued data to a third process if the retry count does not exceed the threshold; and popping the queued data into an error process if the queued data exceeds the retry count; and the third process executed after receiving the queued data from the second process, involving attempting to dispatch the queued data.

Abstract: A method and software tool for identifying relationships between columns of one or more data tables are disclosed. In the disclosed method, a relationship indicator is computed for each of a plurality of column pairs, each column pair comprising respective first and second columns selected from the one or more data tables. The relationship indicator comprises a measure of a relationship (e.g. indicating a strength or likelihood of a relationship) between data of the first column and data of the second column. Relationships between columns of the data tables are then identified in dependence on the computed relationship indicators. The identified relationships may be used to create and execute data queries.