Abstract: Small objects are efficiently stored with erasure codes by combining a small object with other small objects and/or large objects to form a single large object for chunking, and providing early notification of permanent storage to the sources of the objects to prevent small objects from becoming stale while waiting for additional objects to be combined.

Abstract: Metadata area legends are stored at a first location, such as a dedicated partition of a persistent data storage device (PDSD). The metadata area legends have a number of descriptors that describe a number of reserved metadata areas that lie at a second location that is logically separate from the first location, such as a regular partition of the PDSD. Requests to delete the metadata area legends, as well as requests to add new metadata area legends, can be verified to prevent the metadata area legends from being accidentally or maliciously changed.

Abstract: Techniques are provided for storing files in a parallel computing system based on a user-specified parser function. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a parser from the distributed application for processing the plurality of files prior to storage; and storing one or more of the plurality of files in one or more storage nodes of the parallel computing system based on the processing by the parser. The plurality of files comprise one or more of a plurality of complete files and a plurality of sub-files. The parser can optionally store only those files that satisfy one or more semantic requirements of the parser. The parser can also extract metadata from one or more of the files and the extracted metadata can be stored with one or more of the plurality of files and used for searching for files.