Abstract: The technologies described herein are generally directed toward writing multiple data objects to object storage. An example method can include receiving an aggregated data object comprising a set of data objects and metadata corresponding to respective data objects of the set of data objects. The method can further include updating the metadata to comprise the respective storage locations. Further, the method can include updating the metadata to include the respective storage locations. The method can further include allocating the metadata to be stored by metadata processing equipment, resulting in the metadata being stored in metadata storage equipment.

Abstract: The technologies described herein are generally directed toward retrieving data from streaming storage. In an embodiment, a method can include receiving an application data request that identifies application data to be retrieved from a sequence of stored data chunks that correspond to a stored stream of data. The method can further include, based on the application data request, estimating a first estimated location of the application data, with the first estimated location including an identified chunk of a sequence of chunks. Further, the method can include, based on the application data request and a characteristic of the identified chunk, retrieving, by the system, a first data block that is estimated to comprise the application data, resulting in a first retrieved data block.

Abstract: The technologies described herein are generally directed toward using chunks of data to store streaming data at a cloud service provider. In an embodiment, a method can include, based on first application data, initiating a cloud storage transaction, corresponding to a chunk of data. The method can further include communicating a first data part of the chunk to the cloud storage server. Further, the method can include, based on second application data, communicating a second data part of the chunk to the cloud storage server, with the first data part and the second data part being, as generated, stored in a commit buffer, and with a first chunk offset and a second chunk offset being stored in metadata. The method can further include, based on a size of the chunk, communicating a commit signal to commit the cloud storage transaction.

Abstract: The technologies described herein are generally directed toward retrieving data from streaming storage. In an embodiment, a method can include receiving an application data request that identifies application data to be retrieved from a sequence of stored data chunks that correspond to a stored stream of data. The method can further include, based on the application data request, estimating a first estimated location of the application data, with the first estimated location including an identified chunk of a sequence of chunks. Further, the method can include, based on the application data request and a characteristic of the identified chunk, retrieving, by the system, a first data block that is estimated to comprise the application data, resulting in a first retrieved data block.

Abstract: Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for generating images. The method includes acquiring a descriptive text for describing image content of a target image, determining position prior information, and generating the target image based on the descriptive text and position prior information. According to the method of the embodiments of the present disclosure, a target image that can be used for rare data simulation in rare scenarios can be generated by means of an input descriptive text. In addition, the method allows for position perception editing and operation, and can control, based on position prior information, a direction and a position of an object generated in the target image, thereby effectively and diversely generating images. Moreover, the method provided in the present disclosure is based on object types in each subdivided image block, thus making position perception more accurate.

Abstract: A system of managing data, comprising a streaming data storage system operating on a processor that causes the processor to receive streaming data and to process the streaming data in response to an append command, a long-term storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a chunk of data, an object storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate an object of data and a tables system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a table of data.

Abstract: Techniques described herein relate to a method for storing data in columnar storage. The method includes obtaining a columnar storage write request associated with a file, wherein the file comprises rows and columns of file data; in response to obtaining the columnar storage write request: writing the file data to storage using column-based caches; generating file metadata based on the writing of the file data to the storage; and assigning a key to the file metadata; and storing the file metadata using a key-value service.

Abstract: A system of managing data, comprising a streaming data storage system operating on a processor that causes the processor to receive streaming data and to process the streaming data in response to an append command, a long-term storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a chunk of data, an object storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate an object of data and a tables system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a table of data.

Abstract: Support for dynamic scaling is provided. An example method includes: acquiring a current value of a metric associated with a plurality of components of an application; determining a target number of replicas of at least one component of the plurality of components based on the current value of the metric and a scaling policy for the plurality of components; and updating a configuration manifest of the application based on the target number of replicas of the at least one component. In this manner, the configuration manifest of the application can be automatically updated based on metric values of the components at runtime and a custom scaling policy, thereby achieving dynamic scaling of various components of the application without additional user effort to implement separate dynamic scaling mechanisms.

Abstract: A system of managing data, comprising a streaming data storage system operating on a processor that causes the processor to receive streaming data and to process the streaming data in response to an append command, a long-term storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a chunk of data, an object storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate an object of data and a tables system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a table of data.

Abstract: Indices of storage systems are managed. An example method includes: receiving a flush cycle for flushing expired events in a storage system including multiple events in a data stream, determining, based on the flush cycle, a time slice for managing the index of the storage system, creating a slice index node for the time slice in response to determining that the occurrence time of a first event that enters the storage system in the data stream is within the time slice, wherein the slice index node includes an index node of the first event, and adding the slice index node to the index. A corresponding device and a corresponding computer program product are provided. Thus, the index of a large number of events entering the storage system is managed according to the occurrence time of the events, and the storage system can be searched and updated accurately and effectively.

Abstract: Transmission of data for object storage, such as stream transmission for object storage, is disclosed. For instance, a group of objects acquired from an object layer is stored in a storage space for storing an object stream and serves as a first part of the object stream; and, in response to that an event related to at least one object in the group of objects occurs at a client terminal, the event-related information is stored in the storage space and is used as a second part of the object stream, the event-related information including at least one of the following: an identifier of the at least one object, a type of the event, and metadata of the event.

Abstract: Embodiments of the present disclosure relate to processing data. An example method includes acquiring data related to a first moment in streaming data of an object to be processed. The method further includes storing the data in a first entry of a data table based on an identification of the object to be processed, wherein the data table further includes a second entry before the first entry, and the second entry stores data related to a second moment before the first moment in the streaming data. The method further includes updating an index related to the object to be processed based on the first entry. Thus, a solution to the problem of performing search in data at different moments is provided, and it is unnecessary for a user to participate in the solution, thus improving the user experience and reducing the use of storage resources.

Abstract: Techniques for processing an access request and updating a storage system are provided. For instance, a method comprises: receiving an access request for an object associated with a storage system, the storage system including a plurality of physical nodes, each of the plurality of physical nodes including at least one set of virtual units, each set of virtual units including at least one virtual unit; determining, from a plurality of sets of virtual units included in the plurality of physical nodes of the storage system, a target set of virtual units associated with the object; and determining, from the target set of virtual units, a target virtual unit corresponding to the object. With the technical solution of the present disclosure, not only a set of virtual units on a physical node may be easily split and merged, but also huge computing resources that need to be allocated may be saved, so better user experience may be brought about at a lower cost.

Abstract: Transmission of data for object storage, such as stream transmission for object storage, is disclosed. For instance, a group of objects acquired from an object layer is stored in a storage space for storing an object stream and serves as a first part of the object stream; and, in response to that an event related to at least one object in the group of objects occurs at a client terminal, the event-related information is stored in the storage space and is used as a second part of the object stream, the event-related information including at least one of the following: an identifier of the at least one object, a type of the event, and metadata of the event.

Abstract: Retention of events of an ordered event stream is disclosed. Expiration of events stored in a segment of an ordered event stream (OES) can be desirable. New events are added to a head of an OES segment, and pruning events from a tail of the OES segment can be valuable. Processing applications can register a processing scheme for a segment, e.g., at-least-once processing, exactly-once processing, etc., and can generate checkpoints indicating a degree of advancement in processing events of the segment. The ordered event stream can determine a cut point indicative of a progress point, that before which, events of an OES can be marked as ready for expiration. However, events that are marked for expiration can be retained to allow processing based on a checkpoint, e.g., expiration of the event can be refused until there is an assurance the event was read by the processing application.

Abstract: Data stream processing is performed. For instance, a method includes: in response to a query request on an input data stream, determining data labels for input data items in the input data stream, the data labels at least indicating a first matching relationship between the input data items and a first query condition of the query request; determining an output data stream corresponding to the input data stream, the output data stream including output data items corresponding to the input data items, and each output data item having a data label; and determining query information corresponding to the query request based on the output data stream. In this way, it is possible to respond to real-time query requests for data streams.

Abstract: Embodiments of the present disclosure relate to processing data. An example method includes acquiring data related to a first moment in streaming data of an object to be processed. The method further includes storing the data in a first entry of a data table based on an identification of the object to be processed, wherein the data table further includes a second entry before the first entry, and the second entry stores data related to a second moment before the first moment in the streaming data. The method further includes updating an index related to the object to be processed based on the first entry. Thus, a solution to the problem of performing search in data at different moments is provided, and it is unnecessary for a user to participate in the solution, thus improving the user experience and reducing the use of storage resources.

Abstract: Indices of storage systems are managed. An example method includes: receiving a flush cycle for flushing expired events in a storage system including multiple events in a data stream, determining, based on the flush cycle, a time slice for managing the index of the storage system, creating a slice index node for the time slice in response to determining that the occurrence time of a first event that enters the storage system in the data stream is within the time slice, wherein the slice index node includes an index node of the first event, and adding the slice index node to the index. A corresponding device and a corresponding computer program product are provided. Thus, the index of a large number of events entering the storage system is managed according to the occurrence time of the events, and the storage system can be searched and updated accurately and effectively.

Abstract: Embodiments of the present disclosure provide a method and device for storing data. The method comprises: generating a data block corresponding to data to be stored; aligning the data block to a boundary of a tracking unit of a predefined size for validating the data; and storing the aligned data block in at least one storage unit of a storage space, the at least one storage unit having an identical size. The method according to embodiments of the present disclosure can align the data block so as to minimize the waste of storage space and avoid the situation where the rest data cannot be validated due to disappearance of partial data.