Abstract: Techniques for improving data security and access control at the distributed execution level of distributed computing systems are provided. The techniques can include receiving a data access request from a data processing application to access data, directing the data access request to a security data application, modifying the data access request, executing the modified data access request to obtain data that is responsive to the modified data access request, and providing the obtained data to the data processing application.

Abstract: A policy system enforces data security policies for requests from accessing data stored on a distributed data storage system received from a client device. The policy enforcement system can determine user credentials from the requests. The enforcement system then determines whether the user credentials allow the request to retrieve the data and if yes, whether the user credentials allow the request to retrieve the data without obligations. Upon determining that user credentials allow the request to retrieve the data without obligations, the policy enforcement system directs the client device to communicate directly with a name node of the data storage system, short-circuiting additional data retrieval and filtering of the policy system.

Abstract: Systems, computer program products and methods implementing policy enforcement for search engines are described. A policy engine receives a user identifier associated with a search query including one or more query terms. The policy engine receives, from a preprocessor of a search engine, an intermediate representation of the search query. The intermediate representation includes one or more index terms corresponding to the one or more query terms. The policy engine determines, based on a particular policy, if the user is prohibited from accessing data associated with a particular index term. In response, the policy engine modifies the intermediate representation, including negating the particular index term. The policy engine then submits the modified intermediate representation to a query processing module of the search engine, causing the query processing module to exclude content corresponding to the particular index term from search results.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining, by a policy enforcement system in a first compute node, a plurality of policies and data associating a plurality of user credentials with the plurality of policies. A request is received from a compute process for data from a file system in the first compute node. The request includes user credentials. The request for data is sent to the file system, and the data is received from the file system. Based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials is selected from the plurality of policies. The policy enforcement system filters the data from the file system based on the one or more policies, and sends the filtered data to the compute process.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.

Abstract: Systems, computer program products and methods implementing access control for compound structures including subfields are described. A policy system receives a database schema and a data access policy. The database schema defines multiple subfields of a data column. The policy includes one or more rules limiting access to the subfields. A policy analyzer of the policy system creates an access control metadata that stores correspondence between the subfields and the rules. The policy analyzer represents the subfields in the access control metadata using relations between subfields and other components of the database. The policy analyzer provides the access control metadata to a policy enforcer for enforcing the policy on the subfields.

Abstract: Systems, computer program products and methods implementing access control on a distributed file system are described. A file system enforcement point protects an HDFS from unauthorized access by authenticating a declared identity of a task submitting a request from a client. Upon receiving the request, the file system enforcement point submits a challenge to the client, requesting the task to provide credentials of the declared identity. The task submits credentials. On the client, each task has access to credentials of a true identity of the task. Accordingly, in case a task submits a claimed identity that is different from the true identity of the task, the task cannot submit correct credentials in response to the challenge. The file system enforcement point authenticates the declared identity using the submitted credentials. The file system enforcement point allows the client to access the HDFS only upon successful authentication.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.

Abstract: Systems, computer program products and methods implementing access control for compound structures including subfields are described. A policy system receives a database schema and a data access policy. The database schema defines multiple subfields of a data column. The policy includes one or more rules limiting access to the subfields. A policy analyzer of the policy system creates an access control metadata that stores correspondence between the subfields and the rules. The policy analyzer represents the subfields in the access control metadata using relations between subfields and other components of the database. The policy analyzer provides the access control metadata to a policy enforcer for enforcing the policy on the subfields.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.

Abstract: Systems, computer program products and methods implementing access control for compound structures including subfields are described. A policy system receives a database schema and a data access policy. The database schema defines multiple subfields of a data column. The policy includes one or more rules limiting access to the subfields. A policy analyzer of the policy system creates an access control metadata that stores correspondence between the subfields and the rules. The policy analyzer represents the subfields in the access control metadata using relations between subfields and other components of the database. The policy analyzer provides the access control metadata to a policy enforcer for enforcing the policy on the subfields.

Abstract: Systems, computer program products and methods implementing access control on a distributed file system are described. A file system enforcement point protects an HDFS from unauthorized access by authenticating a declared identity of a task submitting a request from a client. Upon receiving the request, the file system enforcement point submits a challenge to the client, requesting the task to provide credentials of the declared identity. The task submits credentials. On the client, each task has access to credentials of a true identity of the task. Accordingly, in case a task submits a claimed identity that is different from the true identity of the task, the task cannot submit correct credentials in response to the challenge. The file system enforcement point authenticates the declared identity using the submitted credentials. The file system enforcement point allows the client to access the HDFS only upon successful authentication.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining, by a policy enforcement system in a first compute node, a plurality of policies and data associating a plurality of user credentials with the plurality of policies. A request is received from a compute process for data from a file system in the first compute node. The request includes user credentials. The request for data is sent to the file system, and the data is received from the file system. Based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials is selected from the plurality of policies. The policy enforcement system filters the data from the file system based on the one or more policies, and sends the filtered data to the compute process.

Abstract: Systems, computer program products and methods implementing access control for compound structures including subfields are described. A policy system receives a database schema and a data access policy. The database schema defines multiple subfields of a data column. The policy includes one or more rules limiting access to the subfields. A policy analyzer of the policy system creates an access control metadata that stores correspondence between the subfields and the rules. The policy analyzer represents the subfields in the access control metadata using relations between subfields and other components of the database. The policy analyzer provides the access control metadata to a policy enforcer for enforcing the policy on the subfields.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining, by a policy enforcement system in a first compute node, a plurality of policies and data associating a plurality of user credentials with the plurality of policies. A request is received from a compute process for data from a file system in the first compute node. The request includes user credentials. The request for data is sent to the file system, and the data is received from the file system. Based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials is selected from the plurality of policies. The policy enforcement system filters the data from the file system based on the one or more policies, and sends the filtered data to the compute process.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.

Abstract: A policy system enforces data security policies for requests from accessing data stored on a distributed data storage system received from a client device. The policy enforcement system can determine user credentials from the requests. The enforcement system then determines whether the user credentials allow the request to retrieve the data and if yes, whether the user credentials allow the request to retrieve the data without obligations. Upon determining that user credentials allow the request to retrieve the data without obligations, the policy enforcement system directs the client device to communicate directly with a name node of the data storage system, short-circuiting additional data retrieval and filtering of the policy system.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining, by a policy enforcement system in a first compute node, a plurality of policies and data associating a plurality of user credentials with the plurality of policies. A request is received from a compute process for data from a file system in the first compute node. The request includes user credentials. The request for data is sent to the file system, and the data is received from the file system. Based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials is selected from the plurality of policies. The policy enforcement system filters the data from the file system based on the one or more policies, and sends the filtered data to the compute process.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for storing, by a policy enforcement system, a plurality of policies and data associating a plurality of user credentials with the plurality of policies; receiving, from a client device, a request for data from a file system, the request further comprising user credentials; forwarding the request for data to a second node that stores the data from the file system; receiving, from the node, the data from the file system; selecting from the plurality of policies, based on the received user credentials and the data associating the plurality of user credentials with the plurality of policies, one or more policies that correspond to the received user credentials; filtering, by the policy enforcement system, the data from the file system based on the one or more policies; and sending the filtered data to the client device.