Abstract: A method for synchronous replication of stream data includes receiving a stream of data blocks for storage at a first storage location associated with a first geographical region and at a second storage location associated with a second geographical region. The method also includes synchronously writing the stream of data blocks to the first storage location and to the second storage location. While synchronously writing the stream of data blocks, the method includes determining an unrecoverable failure at the second storage location. The method also includes determining a failure point in the writing of the stream of data blocks that demarcates data blocks that were successfully written and not successfully written to the second storage location. The method also includes synchronously writing, starting at the failure point, the stream of data blocks to the first storage location and to a third storage location associated with a third geographical region.

Abstract: A replicated block storage service provides durable and high performance network-attached storage replicated in two or more zones of a single region, and remains available despite a single zone failure. A probe file is generated to determine a health state of a replicated disk. When a disk is degraded, a lease is created indicating which replica is trusted and providing visibility to backend jobs to facilitate reconciliation of data between the first replica and the second replica. Moreover, degraded file markers are generated for use by the backend jobs in quickly identifying the data to be copied.

Abstract: A method for managing data processing includes receiving, from a user of a data query system, a data query for data stored in a data store in communication with the data query system. The method also includes receiving a staleness parameter indicating an upper time boundary for the data query. The upper time boundary limits a query response to data within the data store that is older than the upper time boundary. The method further includes determining whether the data stored within the data store satisfies the staleness parameter. When a portion of the data within the data store fails to satisfy the staleness parameter, the method includes generating the query response that excludes the portion of the data that fails to satisfy the staleness parameter.

Abstract: Embodiments of the present invention relate to systems, methods, and computer-storage media for affinitizing datasets based on efficient query processing. In one embodiment, a plurality of datasets within a data stream is received. The data stream is partitioned based on efficient query processing. Once the data stream is partitioned, an affinity identifier is assigned to datasets based on the partitioning of the dataset. Further, when datasets are broken into extents, the affinity identifier of the parent dataset is retained in the resulting extent. The affinity identifier of each extent is then referenced to preferentially store extents having common affinity identifiers within close proximity of one other across a data center.

Abstract: Embodiments of the present invention relate to systems, methods, and computer-storage media for affinitizing datasets based on efficient query processing. In one embodiment, a plurality of datasets within a data stream is received. The data stream is partitioned based on efficient query processing. Once the data stream is partitioned, an affinity identifier is assigned to datasets based on the partitioning of the dataset. Further, when datasets are broken into extents, the affinity identifier of the parent dataset is retained in the resulting extent. The affinity identifier of each extent is then referenced to preferentially store extents having common affinity identifiers within close proximity of one other across a data center.

Abstract: A system and method facilitating incremental web crawl(s) using chunk(s) is provided. The system can be employed, for example, to facilitate a web-crawling system that crawls (e.g., continuously) the Internet for information (e.g., data) and indexes the information so that it can be used as part of a web search engine. The system facilitates incremental re-crawls and/or selective updating of information (e.g., documents) using a structure called a chunk to simplify the process of an incremental crawl. A chunk is a set of documents that can be manipulated as a set (e.g., of up to 65,536 (64K) documents). “Document” refers to a corpus of data that is stored at a particular URL (e.g., HTML, PDF, PS, PPT, XLS, and/or DOC Files etc.) A chunk is created by an indexer. The indexer can place into a chunk documents that have similar property(ies). These property(ies) include but are not limited to: average time between change and average importance. These property(ies) can be stored at the chunk level in a chunk map.

Abstract: A system and method are presented for monitoring user browsing information. Such information can include, but is not limited to, the web pages visited by users, search queries submitted by users, the manner in which users browse the Internet and search for content, as well as any demographic information and interests of the corresponding users. Once a particular type of user browsing information has reached a certain threshold of activity by users, the invention can be configured to detect activity that reaches the threshold and then can increase the monitoring of the information.

Abstract: A system and method are presented for monitoring user browsing information. Such information can include, but is not limited to, the web pages visited by users, search queries submitted by users, the manner in which users browse the Internet and search for content, as well as any demographic information and interests of the corresponding users. Once a particular type of user browsing information has reached a certain threshold of activity by users, the invention can be configured to detect activity that reaches the threshold and then can increase the monitoring of the information.

Abstract: A software package manager uses a distribution unit containing components for a software package and a manifest file that describes the distribution unit to manage the installation, execution, and uninstallation of software packages on a computer. Information in the manifest file pertaining to a software package is stored in a code store data structure upon installation of the package. The manifest file also contains information that permits the software package manager to resolve any software dependencies upon installation. The software package manager uses the code store data structure to locate the required components when the software is executed and to remove the components appropriately when the software is uninstalled.

Abstract: Rather than comparing an old file with a new file to generate a set of patching instructions, and then compressing the patching instructions to generate a compact patch file for transmission to a user, a patch file is generated in a single operation. A compressor is pre-initialized in accordance with the old version of the file (e.g. in an LZ77 compressor, the history window is pre-loaded with the file). The pre-initialized compressor then compresses the new file, producing a patch file from which the new file can be generated. At the user's computer, a parallel process is performed, with the user's copy of the old file being used to pre-initialize a decompressor to which the patch file is then input. The output of the decompressor is the new file. The patch files generated and used in these processes are of significantly reduced size when compared to the prior art.

Abstract: A verification module evaluates intermediate language code to verify that the code is type safe before passing the code for execution. Code that is not deemed type safe can cause execution errors, corrupt the runtime environment in which the code is executing, or cause unauthorized access to protected system resources. As such, the verification module protects a computer system from intentional or inadvertent breaches of the computer system's defenses. The verification module verifies the type safety of intermediate language code that supports significant high-level language constructs prohibited by verification modules in the prior art. One exemplary verification module verifies intermediate language code that includes one or more reference-based parameter, such as a pointer (or reference), a value class containing pointers, a RefAny parameter, or a nested pointer.

Abstract: Rather than comparing an old file with a new file to generate a set of patching instructions, and then compressing the patching instructions to generate a compact patch file for transmission to a user, a patch file is generated in a single operation. A compressor is pre-initialized in accordance with the old version of the file (e.g. in an LZ77 compressor, the history window is pre-loaded with the file). The pre-initialized compressor then compresses the old file, producing a patch file from which the new file can be generated, as a single process. At the user's computer, a parallel process is performed, with the user's copy of the old file being used to pre-initialize a decompressor to which the patch file is then input. The output of the decompressor is the new file, again produced as a single process. The patch files generated and used in these processes are of significantly reduced size when compared to the prior art.

Abstract: A software package manager uses a distribution unit containing components for a software package and a manifest file that describes the distribution unit to manage the installation, execution, and uninstallation of software packages on a computer. Information in the manifest file pertaining to a software package is stored in a code store data structure upon installation of the package. The manifest file also contains information that permits the software package manager to resolve any software dependencies upon installation. The software package manager uses the code store data structure to locate the required components when the software is executed and to remove the components appropriately when the software is uninstalled.

Abstract: Rather than comparing an old file with a new file to generate a set of patching instructions, and then compressing the patching instructions to generate a compact patch file for transmission to a user, a patch file is generated in a single operation. A compressor is pre-initialized in accordance with the old version of the file (e.g. in an LZ77 compressor, the history window is pre-loaded with the file), or other reference file available on the user's computer for purpose other than serving as a decompression aid. The pre-initialized compressor then compresses the old file, producing a patch file from which the new file can be generated. At the user's computer, a parallel process is performed, with the user's copy of the old file being used to pre-initialize a decompressor to which the patch file is then input. The output of the decompressor is the new file. The patch files generated and used in these processes are of significantly reduced size when compared to the prior art.

Abstract: A software package manager uses a distribution unit containing components for a software package and a manifest file that describes the distribution unit to manage the installation, execution, and uninstallation of software packages on a computer. Information in the manifest file pertaining to a software package is stored in a code store data structure upon installation of the package. The manifest file also contains information that permits the software package manager to resolve any software dependencies upon installation. The software package manager uses the code store data structure to locate the required components when the software is executed and to remove the components appropriately when the software is uninstalled.

Abstract: A software package manager uses a distribution unit containing components for a software package and a manifest file that describes the distribution unit to manage the installation, execution, and uninstallation of software packages on a computer. Information in the manifest file pertaining to a software package is stored in a code store data structure upon installation of the package. The manifest file also contains information that permits the software package manager to resolve any software dependencies upon installation. The software package manager uses the code store data structure to locate the required components when the software is executed and to remove the components appropriately when the software is uninstalled.

Abstract: Rather than comparing an old file with a new file to generate a set of patching instructions, and then compressing the patching instructions to generate a compact patch file for transmission to a user, a patch file is generated in a single operation. A compressor is pre-initialized in accordance with the old version of the file (e.g. in an LZ77 compressor, the history window is pre-loaded with the file). The pre-initialized compressor then compresses the old file, producing a patch file from which the new file can be generated. At the user's computer, a parallel process is performed, with the user's copy of the old file being used to pre-initialize a decompressor to which the patch file is then input. The output of the decompressor is the new file. The patch files generated and used in these processes are of significantly reduced size when compared to the prior art.

Abstract: A method of providing data files includes compressing the files using a cross-file compression technique. The technique makes use of ancillary files that are stored along with the data files. The ancillary files include lookup tables and indexes. A lookup table for a data file indicates the position of the last occurrence of individual data values within the data file. Each displacement index for a data file indicates displacements from respective data elements to prior strings of a particular match length that match strings of the particular match length begun by the respective data elements. Indexes corresponding to different match lengths are provided. In response to client requests for subsets of available data files, a server compresses each subset of data files using a pattern-matching compression scheme that attempts to represent given strings by referencing prior matching strings across file boundaries.

Abstract: Described above is a two-pass method of compressing a data input stream to create a compressed data output stream. The method includes a step of sequentially examining data elements and strings of data elements of the data input stream in a first compression pass using a sliding window compression scheme to find literal data elements and pairs of length and displacement values. The first compression pass includes: coding the literal data elements and the length values with a first minimum redundancy coding tree; coding the displacement values with a second minimum redundancy coding tree; and adding the coded literal data elements, length values, and displacement values to a single data buffer. The method also includes a second compression pass that is performed whenever the single data buffer reaches a predefined capacity.