Abstract: The present disclosure provides a high-performance data lake system and a data storage method. The data storage method includes the following steps: S1: converting a file into a file stream; S2: converting the file stream into an array in which multiple subarrays are nested; and S3: converting the array into a resilient distributed dataset (RDD), and storing the RDD to a storage layer of a data lake. The present disclosure provides a nested field structure, which lays the foundation for parallel processing in reading, and effectively improves read performance. Furthermore, the present disclosure flexibly generates a number of nested subarrays according to hardware cores, such that the data lake achieves better extension performance, and can keep optimal writing efficiency for different users.

Abstract: A system may include a server, which may receive a request from a customer device. The server may retrieve data records associated with the request from data sources. The server may process and present a standardized data record on analyst devices. The server may remove data gaps from the standardized data record, in response to receiving inputs from at least one analyst device. The server may generate and update status indicators on a user interface of each analyst device when any analyst device may operate on the standardized data record. The server may use a completed data record to generate a dynamic electronic document. The server may present the dynamic electronic document on a user interface of the customer device. The server may update values within the dynamic electronic document when there is a change in information within the data records.

Abstract: A computer-implemented method is provided of finding one or more data items that match one or more defined criteria in a dataset. The method comprises identifying data snippets of the dataset using a set of one or more attention rules; categorizing the identified data snippets using fuzzy matching by assigning them to buckets such that each bucket contains data snippets that are similar to another according to a similarity measure; classifying buckets containing data snippets having more than a threshold number of the true positive data items as true positive buckets and remaining buckets as false positive buckets; calculating culling rules based on the true positive buckets and the false positive buckets, and using the culling rules to remove the false positive data items from the true positive buckets.

Abstract: A method for extending data lifetime for reference in deduplication is provided. The method includes determining that a quantity of user data has at least a threshold amount of data that is re-created in a storage system. The method includes protecting at least portions of the quantity of user data from erasure by garbage collection in the storage system during a predetermined time interval, wherein the protected at least portions are available for data deduplication of further user data in the storage system during the predetermined time interval.

Abstract: Snapsets containing snapshots from a group of storage volumes are assigned snapset IDs. The same snapset ID is applied to each snapshot of the snapset, and is a globally unique value within the storage system. The snapset ID is assigned to a snapshot upon creation, and remains the same regardless of creation or deletion of other snapshots on the storage volume. By assigning a snapset ID to each snapshot of the snapset, and maintaining the snapset ID as a constant value as long as the snapshot is maintained on the storage system, it is possible to easily determine which snapshots form a given snapset. Control operations on particular storage groups or across storage groups are implemented by specifying particular actions to be taken on snapsets or snapshots, which are identified using the snapset IDs.

Abstract: Systems and methods electronically generate sample dots, produce resources associated with the dots and estimate a resource for a target point from known resources of dots near the target point based on client side version of digital rules, cataloged data and coarse values previously received from the online service platform. A client receives cataloged data of a cataloged domain in which the cataloged data includes data representing a plurality of dots and a respective computed resource value for each of the dots and each dot of the plurality of dots represents a point in the cataloged domain. In response to confirming a target point is in the cataloged domain, the system discovers a closest one or more dots to the target point based on the cataloged data, estimates a statistic for a resource for the target point based on the respective computed resource values of the closest one or more dots, stores the estimated statistic in a memory, and produces the local estimate based on the estimated statistic.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that dynamically capture, organize, and utilize digital media clips. For example, in one or more implementations, the disclosed systems can capture and generate digital media clips of content items that include both content metadata of the content items as well as contextual metadata of contextual signals surrounding the content item. Additionally, in some implementations, the disclosed systems analyze contextual metadata to search, retrieve, discover, and organize new and existing digital media clips. Further, in various implementations, the disclosed systems facilitate generating digital media clip libraries as well as the creation of digital media collections, where different types of digital media clips can be combined in a cohesive interactive graphical user interface.

Abstract: A method for table data processing includes retrieving, by at least one hardware processor, first table data associated with a first table version from a plurality of partitions of a storage device. The method further includes detecting, by the at least one hardware processor, an executed transaction causing a modification to the first table data resulting in a second table data associated with a second table version. The second table data is modified to include partition metadata. The partition metadata includes table version information with a mapping of the first table version and the second table version to a plurality of transactions associated with generating the first table data and the second table data.

Abstract: Technology described herein can perform deletion of a snapshot or portion thereof. In an embodiment, a system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, to delete a snapshot, or a portion of a snapshot, of a real filesystem, reading an inode mapping file (IMF) of the snapshot that indexes a virtual inode number (VIN) corresponding to a real inode. The operations further can comprise identifying the real inode of the snapshot referenced by the VIN, identifying a file object corresponding to the real inode, and deleting the file object from the snapshot.

Abstract: A method for the reverse deletion of a plurality of snapshots in a chain of snapshots is provided. The method includes in reverse order, starting from a latest snapshot in time to an earliest snapshot in time of the plurality of snapshots: identifying at least one of a first set of one or more data blocks of a snapshot that are shared with an earlier snapshot in time in the chain of snapshots or a second set of one or more data blocks of the snapshot that are owned by the snapshot and processing the second set of one or more data blocks and skipping processing the first set of one or more data blocks, wherein processing the second set of one or more data blocks comprises performing one or more actions to maintain and/or delete data blocks of the second set of one or more data blocks.

Abstract: Technology described herein can globally perform management of virtual inode numbers (VINs) of all snapshots of a real filesystem. In an embodiment, a system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, in response to a request to generate a primary snapshot of a real filesystem, allocating a primary instance of an individual VIN for a virtual inode corresponding to the primary snapshot, resulting in an allocated VIN having associated therewith a weight expressed as a value. The operations further can comprise storing the weight for the allocated VIN at a corresponding VIN metadata object exclusive to the VIN.

Abstract: A method for verifying structured data includes receiving structured data, deconstructing the structured data into corresponding elements and obtaining standard structured data having corresponding standard elements. The method also includes comparing the elements of the structured data with the standard elements of the standard structured data to identify any element differences. For each element difference, the method includes comparing the element difference against a registry of element comparisons, determining whether the element difference is expected or unexpected based on a heuristic or at least one rule, and when the element difference is unexpected, generating a signal indicating the presence of an unexpected element in the structured data.

Abstract: The disclosure is directed to, among other things, systems and methods for remote storage of information associated with a node of a distributed ledger network. In some embodiments, the systems and methods described herein may involve providing, by a first node of a distributed ledger network of nodes and to a first server, first information associated with the first node, wherein the network of nodes includes a network of vehicles. The systems and methods may also involve receiving, by a ledger of the first node and from the first server, a pointer to the first information that is stored at the first server, wherein the pointer is stored in the ledger, and wherein the ledger is a distributed ledger included in all of the nodes of the network of nodes.

Abstract: A method for managing a plurality of computing machines includes accessing a catalogue memory structure storing a plurality of component signatures, discovering one or more of the software components being instantiated in a software image of each computing machine according to corresponding ones of the component signatures, detecting one or more connections each being established between at least two instantiated software components of different computing machines, receiving a restore command for restoring a target recovery point selected among a plurality of recovery points, and restoring the target recovery point in response to the restore command by restoring the snapshots associated with the target recovery point on corresponding computing machines.

Abstract: Techniques are provided for tiering snapshots to archival storage in remote object stores. A restore time metric, indicating that objects comprising snapshot data of snapshots created within a threshold timespan are to be available within a storage tier of a remote object store for performing restore operations, may be identified. A scanner may be executed to evaluate snapshots using the restore time metric to identify a set of candidate snapshots for archival from the storage tier to an archival storage tier of the remote object store. For each candidate snapshot within the set of candidate snapshots, the scanner may evaluate metadata associated with the candidate snapshot to identity one or more objects eligible for archival from the storage tier to the archival storage tier, and may archive the one or more objects from the storage tier to the archival storage tier.

Abstract: A method for active learning using sparse training data can include training a machine learning model using less than ten first training data points to generate a candidate machine learning model. The method can include performing a Monte Carlo process to sample one or more first outputs of the candidate machine learning model. The method can include testing the one or more first outputs to determine if each of the one or more first outputs satisfy a respective convergence condition. The method can include, responsive to at least one first output not satisfying the respective convergence condition, training the candidate machine learning model using at least one second training data point corresponding to the at least one first output. The method can include, responsive to the one or more first outputs each satisfying the respective convergence condition, outputting the candidate machine learning model.

Abstract: A memory system includes a memory device including memory blocks, and a controller configured to generate a result indicative of whether a number of free memory blocks satisfies a reference after beginning of garbage collection for the memory device, selectively perform a journaling operation for a request based on the result, and program data, collected by the garbage collection, in the memory device.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for creating, exporting, viewing and testing, and importing custom applications in a multitenant database environment. These mechanisms and methods can enable embodiments to provide a vehicle for sharing applications across organizational boundaries. The ability to share applications across organizational boundaries can enable tenants in a multi-tenant database system, for example, to easily and efficiently import and export, and thus share, applications with other tenants in the multi-tenant environment.

Abstract: Embodiments of the present disclosure provide methods, electronic devices and computer program products for accessing data. A method comprises receiving, at a first device, a file system operation request for accessing target data, the target data being stored at a second device after being pre-processed, and the first device providing a file system interface for data stored at the second device; forwarding the file system operation request to the second device, such that the target data is restored at the second device; receiving the restored target data from the second device; and providing the target data as a response to the file system operation request. Embodiments of the present disclosure allow users to access backup data stored after being pre-processed through normal file system operations and can achieve high data access performance.

Abstract: Systems, methods, and computer readable media related to generating query variants for a submitted query. In many implementations, the query variants are generated utilizing a generative model. A generative model is productive, in that it can be utilized to actively generate a variant of a query based on application of tokens of the query to the generative model, and optionally based on application of additional input features to the generative model.