Abstract: A distributed storage system may store data object instances in persistent storage and may store keymap information for those data object instances in a distributed hash table on multiple computing nodes. Each data object instance may include a composite key containing a user key. The keymap information for each data object instance may map the user key to a locator and the locator to the data object instance. A request to store or retrieve keymap information for a data object instance may be routed to a particular computing node based on a consistent hashing scheme in which a hash function is applied to a portion of the composite key of the data object instance. Thus, related entries may be clustered on the same computing nodes. The portion of the key to which the hash function is applied may include a pre-determined number of bits or be identified using a delimiter.

Abstract: A resource manager of a virtualized computing service indicates to a client that FPGA-enabled compute instances are supported at the service. From a set of virtualization hosts of the service, a particular host from which an FPGA is accessible is selected for the client based on an indication of computation objectives of the client. Configuration operations are performed to prepare the host for the application, and an FPGA-enabled compute instance is launched at the host for the client.

Abstract: Transaction requests may be ordered in a distributed database according to an independently assigned sequence. Different distributed system nodes, such as a transaction coordinator and a storage node may independently assign sequence numbers to requests to access a distributed database. A storage node may receive the request from a transaction coordinator with an assigned sequence number and another request to which the storage node may assign a sequence number. The storage node can then order performance of the requests based on the sequence numbers.

Abstract: Managed materialized views may be generated from across heterogeneous data sources. A request to create a materialized view may be received and performed by a materialized view management platform, which may obtain and generate the materialized view from different data sources and store the materialized view in a target system. Changes to the data sources may be obtained at the materialized view management platform and updates to the materialized view may be determined. The materialized view in the target system may be updated.

Abstract: A system is disclosed allowing a customer to directly build a shopping cart on a supplier's website without logging into a third-party eProcurement system for which the customer is making purchases. Various push and pull models are used for allowing the customer to request content of shopping cart. After the customer logs into the supplier's website, the customer can be identified as associated with the third-party eProcurement system. In such a case, the user is displayed a control allowing the shopping cart to be transmitted to the third-party system for approval.

Abstract: A service implemented at a first isolated virtual network of a provider network is added to a database of privately-accessible services. Configuration changes that enable network packets to flow between the first isolated virtual network and a second isolated virtual network without utilizing a network address accessible from the public Internet are implemented. Service requests originating at the second isolated virtual network are transmitted to the first isolated virtual network via private pathways of the provider network. Metrics corresponding to service requests directed from the second isolated network to the service are collected and provided to the respective owners of one or both isolated virtual networks.

Abstract: A data storage system includes multiple head nodes and data storage sleds. Volume data is replicated between a primary and one or more secondary head nodes for a volume partition and is further flushed to a set of mass storage devices of the data storage sleds. Volume metadata is maintained in a primary and one or more secondary head nodes for a volume partition and is updated in response to volume data being flushed to the data storage sleds. Also, the primary and secondary head nodes store check-points of volume metadata to the data storage sleds, wherein in response to a failure of a primary or secondary head node for a volume partition, a replacement secondary head node for the volume partition recreates a secondary replica for the volume partition based, at least in part, on a stored volume metadata checkpoint.

Abstract: The following description is directed to a logic repository service. In one example, a method of a logic repository service can include receiving a first request to generate configuration data for configurable hardware using a specification for application logic of the configurable hardware. The method can include generating the configuration data for the configurable hardware. The configuration data can include data for implementing the application logic. The method can include encrypting the configuration data to generate encrypted configuration data. The method can include signing the encrypted configuration data using a private key. The method can include transmitting the signed encrypted configuration data in response to the request.

Abstract: A network function virtualization service includes an action implementation layer and an action decisions layer. On a flow of network traffic received at the service, the action implementation layer performs a packet processing action determined at the action decisions layer.

Abstract: Source code of a graph analysis program expressed in a platform-independent language which supports linear algebra primitives is obtained. An executable version of the program is generated, which includes an invocation of a function of a parallel programming library optimized for a particular hardware platform. A result of executing the program is stored.

Abstract: Techniques are described for providing logical networking functionality for managed computer networks, such as for virtual computer networks provided on behalf of users or other entities. In some situations, a user may configure or otherwise specify a network topology for a virtual computer network, such as a logical network topology that separates multiple computing nodes of the virtual computer network into multiple logical sub-networks and/or that specifies one or more logical networking devices for the virtual computer network. After a network topology is specified for a virtual computer network, logical networking functionality corresponding to the network topology may be provided in various manners, such as without physically implementing the network topology for the virtual computer network. In some situations, the computing nodes may include virtual machine nodes hosted on one or more physical computing machines or systems, such as by or on behalf of one or more users.

Abstract: A tiering service enables a client to custom specify service level agreements for data items to be tiered and automatically promotes and demotes the data items amongst a warm tier, a plurality of intermediate tiers, and a cold tier to ensure the service level agreement commitments are met. In some embodiments, a client specifies segmentation criteria for defining multiple segments of data items included in a data scope or table and assigns latency targets to the segments in order to define the service level agreement. Also, in some embodiments, a plurality of intermediate tiers are implemented on common underlying hardware by varying metadata management to implement intermediate tiers that have progressively increasing latencies.

Abstract: A trained model and/or an edge client running on an edge device may obtain data from a data source (e.g., a security video camera) and determine, based on a result of processing the data using the model, whether to send an indication of an upcoming data/video stream to the provider network (e.g., indicating a bearer modification). The received indication may be used by the provider network to send a request to a serving wireless infrastructure (e.g., telco operator/wireless mobile core) for configuration of one or more resources on behalf of the edge device to process the upcoming data stream. The received indication may be used by the provider network in order to configure one or more resources at the provider network to process the upcoming data stream. The edge device initiates transmission of the data stream from the data source to the provider network via the serving wireless infrastructure.

Abstract: A distributed data store may maintain versioned hierarchical data structures. Different versions of a hierarchical data structure may be maintained consistent with a transaction log for the hierarchical data structure. When access requests directed to the hierarchical data structure are received, a version of the hierarchical data structure may be identified for processing an access request. For access requests with snapshot isolation, the identified version alone may be sufficient to consistently process the access request. For access requests with higher isolation requirements, such as serializable isolation, transactions based on the access request may be submitted to the transaction log so that access requests resulting in committed transactions may be allowed, whereas access requests resulting in conflicting transactions may be denied.

Abstract: Systems and processes are described for establishing and using a secure channel. A shared secret may be used for authentication of session initiation messages as well as for generation of a private/public key pair for the session. A number of ways of agreeing on the shared secret are described and include pre-sharing the keys, reliance on a key management system, or via a token mechanism that uses a third entity such as a hub to manage authentication, for example. In some instances, the third party may also perform endpoint selection (e.g., load balancing) by providing a particular endpoint along with the token.

Abstract: Methods and apparatus for coordinating inter-region operations in provider networks. An inter-region coordinator (IRC) operates asynchronously to the control planes of regional networks to coordinate inter-region operations. The IRC in a region may include one or more IRC servers. To perform inter-region operations, the servers may implement a local-remote-local method in which a server invokes an API in the local region to get work, sends the work to a control plane of a remote region, receives a response from the remote region, and informs the control plane in the local region of the status of the work.

Abstract: Methods, systems, and computer-readable media for virtualizing graphics processing in a provider network are disclosed. A virtual compute instance is provisioned from a provider network. The provider network comprises a plurality of computing devices configured to implement a plurality of virtual compute instances with multi-tenancy. A virtual GPU is attached to the virtual compute instance. The virtual GPU is implemented using a physical GPU, and the physical GPU is accessible to the virtual compute instance over a network. An application is executed using the virtual GPU on the virtual compute instance. Executing the application generates virtual GPU output that is provided to a client device.

Abstract: An interface for requesting, and technique for generation of, a backup of a past state of a database table are provided. Changes made to a database table are accumulated, in durable storage, and snapshots of partitions of the table are obtained. The accumulated changes and the successive partition snapshots are used to generate a past state of the database at any point in time across a continuum between successive snapshots. Although each partition of the table may have a snapshot that was generated at a time different from when other partition snapshots were generated, changes from respective change logs may be selectively log-applied to distinct partitions of a table to generate backup in the past of the entire table at common point-in-time across partitions.

Abstract: At a managed control plane service, end-user application programming interfaces (APIs) of an application to be implemented at a provider network are determined. A set of common operational requirements of the application, to be fulfilled without obtaining program code for the requirements, are identified. In response to an invocation of an end-user API of the application, computations are performed at a resource selected by the managed control plane service, and one or more tasks to satisfy a common operational requirement are initiated by the managed control plane service.

Abstract: A spatial search may be performed using representations of geometric shapes stored in a key-value store. A request to perform a spatial search may be received, the request including a geometric shape composed of one or more points. The points of the geometrical shape may be translated into one or more spatial indexes representing spatial cells using a space-filling curve. A key-value store may then be incrementally searched for each spatial index to identify spatial cells intersecting the geometric shape for which other known geometric shapes exist. The key-value store may then be searched to identify the known geometric shapes intersecting the geometric shape included in the search.

Abstract: In operating an ecommerce marketplace, pre-generated metadata is used to transform images after a request is received, in real-time, and with low latency. To accomplish the high-speed image transformation, an offline annotation of relevant image features is obtained using machine learning and metadata is stored based upon the results. The metadata is then used to perform the high-speed image transformation at request time. The transformations can include image cropping, adjustment in saturation, contrast, brightness, extracting portions of the image, etc. In one example, the metadata includes a bounding box giving coordinates of how to crop an image so that the image can be cropped without analyzing content or context of the image. Instead, to obtain the coordinates of the bounding box, content and context of the image are analyzed in pre-request processing. In this way, when the request is received, the more difficult and time-consuming processing is already completed.

Abstract: A system that implements a data storage service may store data on behalf of clients in multiple replicas on respective computing nodes. The system may employ an external service to select a master replica for a replica group. The master replica may service consistent read operations and/or write operations that are directed to the replica group (or to a data partition stored by the replica group). The master replica may employ a quorum based mechanism for performing replicated write operations, and a local lease mechanism for determining the replica authorized to perform consistent reads, even when the external service is unavailable. The master replica may propagate local leases to replica group members as replicated writes. If another replica assumes mastership for the replica group, it may not begin servicing consistent read operations that are directed to the replica group until the lease period for a current local lease expires.

Abstract: Disclosed are system architectures and techniques for multiple access points for a data container. Control plane and data plane APIs are disclosed for generating access points and associating the access points with data containers, as well as for generating access point policies that specify permissions for the access points. More than one access point may be associated with a single data container. A storage service receives requests directed to the access points, and grants or denies the requests based on the permissions specified in the corresponding policies. Various types of use cases for using access policies are contemplated such as default, regional, or network-based (e.g., VPC-based) use-cases or the like. A system may implement layers of policies such as identity and access management policies, access point policies and data container policies.

Abstract: At a managed control plane service, constituent services and operational requirements of an application are identified. In response to an end-user request directed to the application, contents of an inter-service request are generated at a resource selected by the managed control plane service for a first constituent service, and a response to the message is generated at another resource selected for a second constituent service. Tasks to be performed for the operational requirements are initiated by the managed control plane service.

Abstract: Methods and apparatus are disclosed for programming reconfigurable logic devices such as FPGAs in a networked server environment. In one example, a system hosting a network service providing field programmable gate array (FPGA) services includes a network service provider configured to receive a request to implement application logic in a plurality of FPGAs, allocate a computing instance comprising the FPGAs in responses to receiving the request, produce configuration information for programming the FPGAs, and send the configuration information to an allocated computing instance. The system further includes a computing host that is allocated by the network service provider as a computing instance which includes memory, processors configured to execute computer-executable instructions stored in the memory, and the programmed FPGAs.

Abstract: Devices, systems, and methods are provided for performing activity evaluations. A method may include determining, by a device, a heart rate. The method may include determining, based on an activity template, a first biometric and a threshold associated with the first biometric. The method may include determining first data associated with the first biometric, the first data indicative of a first quantity. The method may include comparing, by the at least one processor, the first quantity to the threshold, and determining, based on the first data, second data associated with a second biometric, the first biometric different than the second biometric, the second data indicative of a second quantity. The method may include determining that the first quantity is associated with the second quantity. The method may include sending a message to a second device for presentation, the message associated with the first biometric.

Abstract: Technologies are provided for dynamically changing a size of a cache region of a storage device. A storage device controller writes data to the cache region of the storage device using a particular storage format. The storage device controller then migrates the cached data to a storage region of the device, where the data is written using a different storage format. A dynamic cache manager monitors input and output activity for the storage device and dynamically adjusts a size of the cache region to adapt to changes in the input and/or output activity. The dynamic cache manager can also adjust a size of the storage region. The storage device controller can automatically detect that the storage device has dynamic cache support and configure the storage device by creating the cache region and the storage region on the device.

Abstract: A shippable data transfer device includes a data storage medium encased in a chamber surrounded by an anti-tamper casing. The anti-tamper casing includes an anti-tamper layer with identifying elements arranged in a unique or otherwise identifiable pattern. The anti-tamper layer is configured to actively re-arrange, alter, or obscure the identifying elements in response to a breach of the anti-tamper casing.

Abstract: Methods, apparatus, and computer-accessible storage media for providing redundant storage gateways. A client may create a storage gateway group and add storage gateways to the group. The client may assign one or more volumes on a remote data store to each the storage gateways in the group. Volume data for each storage gateway in the group may be replicated to at least one other storage gateway in the group. If one of the gateways in the group becomes unavailable, one or more other gateways in the group may take over volumes previously assigned to the unavailable gateway, using the replicated data in the group to seamlessly resume gateway operations for the respective volumes. Client processes that previously communicated with the unavailable gateway may be manually or automatically directed to the gateway(s) that are taking over the unavailable gateway's volumes.

Abstract: Devices, systems, and methods are provided for performing volume and intensity-based activity evaluations. A method may include determining, by a device, a heart rate. The method may include determining, based on the heart rate, a motion threshold amount for a time period. The method may include determining motion data. The method may include comparing the motion data to the motion threshold for the time period. The method may include determining, based on the comparison of the motion data to the motion threshold for the time period, an activity intensity level associated with the heart rate and the motion data. The method may include determining, based on the activity intensity level, an activity score. The method may include sending data indicating the activity score for presentation at a second device.

Abstract: A service provider may apply customer-selected or customer-defined auto-scaling policies to a cluster of resources (e.g., virtualized computing resource instances or storage resource instances in a MapReduce cluster). Different policies may be applied to different subsets of cluster resources (e.g., different instance groups containing nodes of different types or having different roles). Each policy may define an expression to be evaluated during execution of a distributed application, a scaling action to take if the expression evaluates true, and an amount by which capacity should be increased or decreased. The expression may be dependent on metrics emitted by the application, cluster, or resource instances by default, metrics defined by the client and emitted by the application, or metrics created through aggregation. Metric collection, aggregation and rules evaluation may be performed by a separate service or by cluster components. An API may support auto-scaling policy definition.

Abstract: Methods, systems, and computer-readable media for automated packetless network reachability analysis are disclosed. An analysis is performed of network configuration data for a network comprising a host computer. Based at least in part on the analysis, one or more ports at the host computer that are reachable from another computer are determined. Based at least in part on the analysis, one or more routes to the one or more ports are determined. A report is generated that is descriptive of the one or more ports and the one or more routes.

Abstract: A write lock request for a data object on behalf of a first data accessor is received at a lock manager. The data object is currently locked on behalf of a second data accessor. The lock manager modifies lock metadata associated with the data object to indicate the first data accessor as the primary lock owner, and designates the second data accessor as a non-primary owner.

Abstract: Methods, systems, and computer-readable media for augmenting storage functionality using emulation of storage characteristics are disclosed. An access request for a data set is received. The access request is formatted according to a first protocol associated with a first data store, and the first data store is associated with first storage characteristics. The access request is translated into a translated access request. The translated access request is formatted according to a second protocol associated with a second data store, and the second data store is associated with second storage characteristics differing at least in part from the first storage characteristics. The translated access request is sent to the second data store. The translated access request is performed by the second data store on the data set using emulation of one or more of the first storage characteristics not included in the second storage characteristics.

Abstract: At a machine learning service, a determination is made that an analysis to detect whether at least a portion of contents of one or more observation records of a first data set are duplicated in a second set of observation records is to be performed. A duplication metric is obtained, indicative of a non-zero probability that one or more observation records of the second set are duplicates of respective observation records of the first set. In response to determining that the duplication metric meets a threshold criterion, one or more responsive actions are initiated, such as the transmission of a notification to a client of the service.

Abstract: A virtual network verification service for provider networks that leverages a declarative logic programming language to allow clients to pose queries about their virtual networks as constraint problems; the queries may be resolved using a constraint solver engine. Semantics and logic for networking primitives of virtual networks in the provider network environment may be encoded as a set of rules according to the logic programming language; networking security standards and/or client-defined rules may also be encoded in the rules. A description of a virtual network may be obtained and encoded. A constraint problem expressed by a query may then be resolved for the encoded description according to the encoded rules using the constraint solver engine; the results may be provided to the client.

Abstract: In response to a determination that a first logical table is to be created in a data sheet, a respective row identifier is generated and stored for individual rows of the first logical table. To indicate a relationship between a particular cell of another logical table and a particular row of the first logical table, the row identifier of the particular row is stored. After a modification of a value stored in the particular row, the row identifier is used to determine an updated value to be displayed in the particular cell of the other logical table.

Abstract: Respective labels indicative of compression-related quality degradation for a set of media object tuples which meet a divergence criterion are obtained; each tuple comprises a reference media object and a pair of corresponding compressed media object versions. Pairs of training records for a machine learning model are generated using the labeled media object tuples and multiple perceptual quality algorithms, with each training record comprising respective perceived quality degradation scores generated by each of the multiple algorithms for a given compressed media object of a tuple. A machine learning model is trained, using the record pairs, to predict quality degradation scores for compressed media objects.

Abstract: One or more multi-stage optimization iterations are performed with respect to a compression algorithm. A given iteration comprises a first stage in which hyper-parameters of a perceptual quality algorithm are tuned independently of the compression algorithm. A second stage of the iteration comprises tuning hyper-parameters of the compression algorithm using a set of perceptual quality scores generated by the tuned perceptual quality algorithm. The final stage of the iteration comprises performing a compression quality evaluation test on the tuned compression algorithm.

Abstract: Systems and methods for video distribution synchronization are described herein. An example method to distribute a media stream over a distribution network to a number of devices may include determining, by one or more computer processors coupled to memory, a common master-client shared time reference Tref. Example methods may include determining a mean intermediate arrival time for first packets of a first type of the media stream, determining respective first playout times for the first packets based on the mean intermediate arrival time and the time reference Tref, associating the first packets with the respective first playout times, and distributing the media stream to the plurality of devices.

Abstract: Methods, systems, and computer-readable media for record-by-record live migration using a lock store are disclosed. Migration of a data set comprises acquiring a migration lock from a lock store for an individual record being migrated. An instance of the record is then stored in the destination, and the record is deleted from the source. Subsequently, the migration lock is deleted from the lock store or otherwise released. During the migration, a rate at which insert requests are directed to the destination is dialed up. During the process of updating an insert treatment configuration across a distributed set of migration router instances, an insert lock is acquired from the lock store for each record being inserted. Also metadata is stored indicating a determined location for the record being inserted. Any subsequent requests to insert a record with a same key are inserted at the indicated determined location.

Abstract: Various embodiments of apparatuses and methods for distributed threat sensor data collection and data export of a malware threat intelligence system are described. In some embodiments, the system comprises a plurality of threat sensors, deployed at different network addresses and physically located in different geographic regions in a provider network, which detect interactions from sources. In some embodiments, a distributed threat sensor data collection and data export service receives a stream of sensor logs from the plurality of threat sensors. The stream of sensor logs has information about interactions with the threat sensors, including an identifier of the source. The service aggregates the information in the sensor logs by the source, computes significance scores for each source where a significance score quantifies a likelihood that the source is engaging in threatening network communications, and provides the significance scores to other destinations.

Abstract: A satellite antenna ground station service includes a plurality of ground stations and associated data centers, wherein the data centers are part of a provider network. Clients may reserve satellite antenna access time-slots via a user interface of the satellite antenna ground station service and store data directly to a data center of the provider network or to the client's premises via a direct connection between the client and the provider network. In some embodiments, the provider network may offer a plurality of network-based services, such as a compute service, a data storage service, a machine learning service, or a data analytics service, and a client may utilize one or more of these services to analyze and process downlinked data received from a satellite of the client via a satellite antenna ground station of the satellite antenna ground station service of the provider network.

Abstract: Various embodiments of apparatuses and methods for distributed threat sensor analysis and correlation of a malware threat intelligence system are described. In some embodiments, the system comprises a plurality of threat sensors, deployed at different network addresses and physically located in different geographic regions in a provider network, which detect interactions from sources. In some embodiments, a distributed threat sensor analysis and correlation service obtains significance scores for different sources of the interactions with the plurality of threat sensors. The service determines which of the sources are malicious actors based on the significance scores. The service receives identifiers of known actors such as compute instances in the provider network, client devices in a client network, or deployed IoT devices in a remote network, and correlates the malicious actors with the known actors to identify which known actors might be infected by malware.

Abstract: Various embodiments of apparatuses and methods for threat sensor deployment and management in a malware threat intelligence system are described. In some embodiments, the system comprises a plurality of threat sensors, deployed at different network addresses and physically located in different geographic regions in a provider network, which detect interactions from sources. In some embodiments, a threat sensor deployment and management service determines a deployment plan for the plurality of threat sensors, including each threat sensor's associated threat data collectors. The threat data collectors can be of different types such as utilizing different communication protocols or ports, or providing different kinds of responses to inbound communications. The different threat sensors can have different lifetimes.

Abstract: A storage gateway serves as an interface between processes on a customer network and a service provider. The storage gateway is located on-premise with the customer processes. To customer processes, it appears that data is stored locally. However, the storage gateway interfaces with a remote storage service to store the data. For cached gateways, the primary data store is a remote data store, while frequently accessed data may be locally cached by the gateway. Reads may be satisfied from the local cache or from virtual data storage; writes are handled so as to appropriately update data blocks in the local cache and/or in virtual data storage. For shadowing gateways, the primary data store is the local data store; reads are passed through to a local data store, and writes are shadowed to virtual data storage as well as being sent to local data store.

Abstract: A peripheral device includes one or more processors and a memory storing program instructions that when executed implement an extension manager of a virtualized computing service. The extension manager establishes a secure network channel for communications between the peripheral device, which is located at a premise external to a provider network, and a data center of the provider network. The extension manager assigns a network address of the substrate network of the service to a hardware server at the external premise. The substrate address is also assigned to an extension traffic intermediary at the data center. In response to a command directed to the virtualized computing service, one or more compute instance configuration operations are performed at the hardware server.

Abstract: A number of cells of a control plane of a virtualized computing service are set up, including a first cell with one or more request processing nodes, a local instance of a data store, and metadata indicating a set of virtualization hosts. A request processer transmits a request for a virtual machine to the first cell. A request processor of the cell initiates a workflow to launch the virtual machine using a virtualization host; the workflow includes storing a record of the request in the local instance of the data store.

Abstract: A system that provides services to clients may receive and service requests, various ones of which may require different amounts of work. The system may determine whether it is operating in an overloaded or underloaded state based on a current work throughput rate, a target work throughput rate, a maximum request rate, or an actual request rate, and may dynamically adjust the maximum request rate in response. For example, if the maximum request rate is being exceeded, the maximum request rate may be raised or lowered, dependent on the current work throughput rate. If the target or committed work throughput rate is being exceeded, but the maximum request rate is not being exceeded, a lower maximum request rate may be proposed. Adjustments to the maximum request rate may be made using multiple incremental adjustments. Service request tokens may be added to a leaky token bucket at the maximum request rate.

Abstract: The following description is directed to a logic repository service. In one example, a method of a logic repository service can include receiving a first request to generate configuration data for configurable hardware using a specification for application logic of the configurable hardware. The method can include generating the configuration data for the configurable hardware. The configuration data can include data for implementing the application logic. The method can include receiving a second request to download the configuration data to a host server computer comprising the configurable hardware. The method can include transmitting the configuration data to the host server computer in response to the second request so that the configurable hardware is configured with the host logic and the application logic.