Abstract: A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.

Abstract: Isolation types may be determined for resources that execute portions of code. Code may be received via a network-based interface from a client for execution. An execution plan for the code may be generated and evaluated to determine one or more isolation types for computing resources that execute the code. The computing resources that are configured to provide the determined isolation types may then be identified and execution of the code initiated at the identified computing resources.

Abstract: A distributed database system may implement fast crash recovery. Upon recovery from a database head node failure, a connection with one or more storage nodes of a distributed storage system storing data for a database implemented by the database head node may be established. Upon establishment of the connection with the storage nodes, that database may be made available for access, such as for various access requests. In various embodiments, redo log records may not be replayed in order to provide access to the database. In at least some embodiments, the storage nodes may provide a current state of data stored for the database in response to requests.

Abstract: Version controlled Extract, Transform, Load (ETL) code may be hosted for developing or executing the ETL job in an ETL system. A version of ETL code may be obtained from version controlled code store and maintained in a data store. Development or execution clients may submit access requests for the version of ETL code which may be serviced from the version stored in the data store. Updates to the version of the ETL code may be eventually committed to the version controlled code store. The latest version of ETL code may also be obtained from the version controlled code store when providing the ETL code in response to a request to retrieve the ETL code.

Abstract: A data store may implement on-demand data schema modifications. Request to change a schema for a data set in a data store may be received and a description of the change recorded as part of a schema history for the data set. The request to change the schema may then be acknowledged. When access requests directed to the data set are received at the data store, the schema history for the data set may be evaluated. If the schema history indicates that data that is to be accessed in order to service the data store needs to include one or more changes in the schema history, then a version of the schema that includes the one or more changes may be applied to the data.

Abstract: Data may be efficiently analyzed and compressed as part of a data compression service. A data compression request may be received from a client indicating data to be compressed. An analysis of the data or metadata associated with the data may be performed. In at least some embodiments, this analysis may be a rules-based analysis. Some embodiments may employ one or more machine learning techniques to historical compression data to update the rules-based analysis. One or more compression techniques may be selected out of a plurality of compression techniques to be applied to the data. Data compression candidates may then be generated according to the selected compression techniques. In some embodiments, a compression service restriction may be enforced. One of the data compression candidates may be selected and sent in a response.

Abstract: A database system may implement asynchronous processing for synchronous requests received at the database. A pool of request processing threads may be maintained. As access requests for the database are received from clients, an available request processing thread in the pool may process the access request. The access request may be dependent on a persistent storage I/O operation, such that processing of the access request waits at least until the persistent storage I/O operation is complete. The request processing thread may perform processing operations for the access request so that persistent storage I/O operation is performed and become available to process other access requests. A response processing thread may determine that the persistent storage I/O operation for the access request is complete, and send a response to the client.

Abstract: Dynamic generation of data catalogs may be implemented for accessing data sets in different storage locations. Data sets may be accessed in order to extract portions of data. Structure recognition techniques may be applied to the extracted data in order to determine structural information for the data sets. The structural information may then be stored as part of a data catalog for the data sets. Requests to access the data catalog from different clients may be received and the requested structural data supplied so that the clients may access different data sets utilizing the supplied structural data. Data catalogs may be updated as changes to data sets are made.

Abstract: Nodes of a database service may receive a read request to perform a read of a record stored by the database service and a transaction request to perform a transaction to the record. First and second indications of time may be associated with the read and transaction, respectively. A potential read anomaly (e.g., fuzzy read, read skew, etc.) may be detected based, at least in part, on a determination that the first indication of time is within a threshold value of the second indication of time. In response to detecting the potential read anomaly, the read may be performed after the transaction specified by the transaction request, regardless of whether the first indication of time is indicative of an earlier point in time than the second indication of time.

Abstract: A data store may implement dynamic query predicates for processing queries. Query predicate indexes may be maintained for individual storage locations in a data store. The query predicate indexes may indicate data values not stored in the storage location that satisfy a query predicate in the query predicate index. The query predicate index may be populated when queries are received at the data store which include new query predicates. When queries are received that include query predicates that correspond to query predicates in the query predicate index, the index values for each of the storage locations may be evaluated. Based, at least in part, on the evaluation, particular storage locations may be identified to read for servicing the queries.

Abstract: A probabilistic counting structure such as a hyperloglog may be formed during a table scan for each of a selected set of columns. The columns may be selected based on an initial estimate of relatedness, which may be based on data types of the respective columns. An estimated cardinality of an intersection or union of columns may be formed based on an intersection of the probabilistic data structures. A join path may be determined based on the estimated cardinality of an intersection or union of the columns.

Abstract: A database may provide a visualization of restoration operation granularity. Information may be obtained describing possible restoration points within a given period of time for a database. The information may include a respective state of the database associated with a point in time and a respective restoration operation type for the possible restoration points. A graphical representation may be generated illustrating the possible restoration points ordered within the given time period according to the points in time associated with the respective states of the database. The graphical representation may be provided via network-based interface to a client of the database. The graphical representation of the possible restoration points may be updated in response to changes to the given time period or database volume events that modify the possible restoration points.

Abstract: A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

Abstract: A data store may implement variable data replication and quorum-based durability schemes. Write requests may be maintained in complete views of the data or incomplete views of the data at storage nodes in a quorum set in the data store. Write requests may be received at different storage nodes sufficient to establish a write quorum, and stored in the replicas of the data at the storage nodes where they are received. Storage nodes maintaining incomplete views of the data may determine that each of the storage nodes maintaining a complete view of the data in the quorum set stores certain write requests, and in response, may reclaim storage space for those write requests included in the incomplete views of the data in order to store other data at the storage node.

Abstract: Cardinality values can be derived from an approximate quantile summary. An approximate quantile summary can be generated for a column of a database table at data ingestion, data update, upon request, in response to a query, and in various other scenarios. When a query is received that includes a predicate directed to the column of the approximate quantile summary, a cardinality value may be derived from the boundary values of one or more quantiles that include the predicate. The cardinality value may then be used to select a query plan. The query may be performed according to the selected query plan.

Abstract: Tiered storage may be implemented for processing data. Data processors may maintain some of a data set, including user data and metadata describing the user data, locally. The data set is also maintained a data store remote to the data processor. When processing requests are received, a determination is made as to whether the local portions of the data set can execute the processing request or one or more additional portions of the data set are needed from the remote data store. If additional portions of the data set are needed, then a request may be sent to the data store for the additional portions. Once received, the data processor may execute the processing request utilizing the additional portions. Portions of the data set maintained locally at the data processor may be selected and flushed from local storage to the remote data store.

Abstract: A database system may receive a write request that specifies a modification to be made to a particular data record stored by the database system. A log record representing the modification to be made to the particular data record may be sent to a storage service of the database system. An indication (e.g., log record or other indication) that indicates a cached version of the particular data record stored in a read replica's cache is stale may be sent to a read replica. For a subsequent read of the particular data record received by the read replica, the read replica may request the particular data record from the storage service.

Abstract: Recognizing unknown data objects may be implemented for data objects stored in a data store. Data objects that are identified as unknown may be accessed to retrieve a portion of the data object. Different representations of the data object may be generated for recognizing different data schemas. An analysis of the representations may be performed to identify a data schema for the unknown data object. The data schema may be stored in a metadata store for the unknown data object.

Abstract: A storage engine may selectively maintain a spatial index for accessing spatial data. A spatial query may be received and the portions of the spatial index to evaluate may be determined by replacing those portions associated with regions identified for the query that do not exist in the spatial index with portions that do exist in the spatial index and are associated with a region that includes the identified regions. When inserting spatial objects into the spatial index, a determination may be made whether to create a new portion in the index if none currently exist that match a spatial index value for a new spatial object or to modify the spatial index value of the spatial object to insert the spatial object into an existing portion of the spatial index.

Abstract: Dynamic generation of data catalogs may be implemented for accessing data sets in different storage locations. Data sets may be accessed in order to extract portions of data. Structure recognition techniques may be applied to the extracted data in order to determine structural information for the data sets. The structural information may then be stored as part of a data catalog for the data sets. Requests to access the data catalog from different clients may be received and the requested structural data supplied so that the clients may access different data sets utilizing the supplied structural data. Data catalogs may be updated as changes to data sets are made.