Abstract: A web-scale data processing system and method are provided herein. More particularly, a web-scale data processing system and method for crawling, storing, processing, encoding, and/or serving web-scale data are disclosed.
Type:
Application
Filed:
April 24, 2012
Publication date:
August 16, 2012
Applicant:
SEOmoz, Inc.
Inventors:
Benjamin Cappel HENDRICKSON, Nicholas Stefan GERNER
Abstract: A system and method for obtaining and processing web-scale data are provided herein. More particularly, a web-scale data processing system and method for crawling, storing, processing, encoding, and/or serving web-scale data are disclosed.
Type:
Grant
Filed:
October 2, 2009
Date of Patent:
May 15, 2012
Assignee:
SEOmoz, Inc.
Inventors:
Benjamin Cappel Hendrickson, Nicholas Stefan Gerner
Abstract: Large integers may be stored according to byte-stable variable-length encoding systems and methods, eliminating the need to store many leading-zero bits in large integers. Such a byte-stable variable-length integer encoding scheme may represent identical sequences of numbers in a consistent byte pattern within a byte stream, preserving the redundancy of the data and allowing for improved compression rates.