Abstract: Systems and methods for data usage monitoring in multi-tenancy enabled HADOOP clusters are disclosed. According to one embodiment, a method for monitoring data usage in multi-tenancy enabled HADOOP clusters may include: (1) receiving metadata related to a dataset in one or more multi-tenant clusters; (2) receiving entitlement data for a plurality of users to the dataset; (3) receiving group membership data for the plurality of users; (4) receiving access permissions for the plurality of users to the dataset; (5) receiving audit logs comprising access history for the plurality of users to the dataset; (6) joining the metadata, entitlement data, group membership data, access permissions, and audit logs into a searchable database; (7) receiving a query comprising at least one of a date range, a file, a directory, a user, and a group of users; (8) applying the query to the searchable database; and (9) returning results to the query.

Abstract: Security design and architecture for a multi-tenant Hadoop cluster are disclosed. In one embodiment, in a multi-tenant Hadoop cluster comprising a plurality of tenants and a plurality of applications, a method for identifying, naming, and creating a multi-tenant directory structure in a multi-tenant Hadoop cluster may include (1) identifying a plurality of groups for a directory structure selected from the group consisting of a superuser group, a plurality of tenant groups, and at least one application group; (2) creating an active directory for each of the groups; (3) adding each of a plurality of users to one of the plurality of tenant groups and the application group; (4) creating tenant directories and home directories for the users; and (5) assigning owners, group owners, default permissions, and extended access control lists to the tenant directories and the home directories.

Abstract: Systems and methods for optimizing cluster resource utilization are disclosed. Systems and methods for optimizing cluster resource utilization are disclosed. In one embodiment, in an information processing apparatus comprising at least one computer processor, a method for optimizing cluster resource utilization may include: (1) retrieving cluster usage information for at least one cluster resource in a multi-tenant environment; (2) determining tenant usage for the cluster resource for each of a plurality of tenants; (3) determining a tenant resource commitment for the cluster resource for each tenant; and (4) presenting tenant usage and tenant resource commitment for each resource.

Abstract: Systems and methods for optimizing cluster resource utilization are disclosed. Systems and methods for optimizing cluster resource utilization are disclosed. In one embodiment, in an information processing apparatus comprising at least one computer processor, a method for optimizing cluster resource utilization may include: (1) retrieving cluster usage information for at least one cluster resource in a multi-tenant environment; (2) determining tenant usage for the cluster resource for each of a plurality of tenants; (3) determining a tenant resource commitment for the cluster resource for each tenant; and (4) presenting tenant usage and tenant resource commitment for each resource.

Abstract: Systems and methods for optimizing cluster resource utilization are disclosed. Systems and methods for optimizing cluster resource utilization are disclosed. In one embodiment, in an information processing apparatus comprising at least one computer processor, a method for optimizing cluster resource utilization may include: (1) retrieving cluster usage information for at least one cluster resource in a multi-tenant environment; (2) determining tenant usage for the cluster resource for each of a plurality of tenants; (3) determining a tenant resource commitment for the cluster resource for each tenant; and (4) presenting tenant usage and tenant resource commitment for each resource.

Abstract: Security design and architecture for a multi-tenant Hadoop cluster are disclosed. In one embodiment, in a multi-tenant Hadoop cluster comprising a plurality of tenants and a plurality of applications, a method for identifying, naming, and creating a multi-tenant directory structure in a multi-tenant Hadoop cluster may include (1) identifying a plurality of groups for a directory structure selected from the group consisting of a superuser group, a plurality of tenant groups, and at least one application group; (2) creating an active directory for each of the groups; (3) adding each of a plurality of users to one of the plurality of tenant groups and the application group; (4) creating tenant directories and home directories for the users; and (5) assigning owners, group owners, default permissions, and extended access control lists to the tenant directories and the home directories.

Abstract: Security design and architecture for a multi-tenant Hadoop cluster are disclosed. In one embodiment, in a multi-tenant Hadoop cluster comprising a plurality of tenants and a plurality of applications, a method for identifying, naming, and creating a multi-tenant directory structure in a multi-tenant Hadoop cluster may include (1) identifying a plurality of groups for a directory structure selected from the group consisting of a superuser group, a plurality of tenant groups, and at least one application group; (2) creating an active directory for each of the groups; (3) adding each of a plurality of users to one of the plurality of tenant groups and the application group; (4) creating tenant directories and home directories for the users; and (5) assigning owners, group owners, default permissions, and extended access control lists to the tenant directories and the home directories.

Abstract: Systems and methods for resource management for multi-tenant applications in a Hadoop cluster are disclosed. In one embodiment, in an information processing device comprising at least one computer processor, a method for resource management for multi-tenant applications in a Hadoop cluster may include: (1) allocating an initial allocation of a resource in a resource pool to a plurality of tenants, each tenant having a workload; (2) determining a memory requirement for each of the plurality of tenants; (3) determining a maximum number of concurrent queries or jobs for each of the plurality of tenants; (4) determining a memory and vcore requirement for each of the plurality of tenants based on the memory requirement and maximum number of concurrent queries or jobs; and (5) allocating the resources to each of the plurality of tenants.

Abstract: Systems and methods for data usage monitoring in multi-tenancy enabled HADOOP clusters are disclosed. According to one embodiment, a method for monitoring data usage in multi-tenancy enabled HADOOP clusters may include: (1) receiving metadata related to a dataset in one or more multi-tenant clusters; (2) receiving entitlement data for a plurality of users to the dataset; (3) receiving group membership data for the plurality of users; (4) receiving access permissions for the plurality of users to the dataset; (5) receiving audit logs comprising access history for the plurality of users to the dataset; (6) joining the metadata, entitlement data, group membership data, access permissions, and audit logs into a searchable database; (7) receiving a query comprising at least one of a date range, a file, a directory, a user, and a group of users; (8) applying the query to the searchable database; and (9) returning results to the query.

Abstract: Systems and methods for resource management for multi-tenant applications in a Hadoop cluster are disclosed. In one embodiment, in an information processing device comprising at least one computer processor, a method for resource management for multi-tenant applications in a Hadoop cluster may include: (1) allocating an initial allocation of a resource in a resource pool to a plurality of tenants, each tenant having a workload; (2) determining a memory requirement for each of the plurality of tenants; (3) determining a maximum number of concurrent queries or jobs for each of the plurality of tenants; (4) determining a memory and vcore requirement for each of the plurality of tenants based on the memory requirement and maximum number of concurrent queries or jobs; and (5) allocating the resources to each of the plurality of tenants.

Abstract: Systems and methods for optimizing cluster resource utilization are disclosed. Systems and methods for optimizing cluster resource utilization are disclosed. In one embodiment, in an information processing apparatus comprising at least one computer processor, a method for optimizing cluster resource utilization may include: (1) retrieving cluster usage information for at least one cluster resource in a multi-tenant environment; (2) determining tenant usage for the cluster resource for each of a plurality of tenants; (3) determining a tenant resource commitment for the cluster resource for each tenant; and (4) presenting tenant usage and tenant resource commitment for each resource.

Abstract: Security design and architecture for a multi-tenant Hadoop cluster are disclosed. In one embodiment, in a multi-tenant Hadoop cluster comprising a plurality of tenants and a plurality of applications, a method for identifying, naming, and creating a multi-tenant directory structure in a multi-tenant Hadoop cluster may include (1) identifying a plurality of groups for a directory structure selected from the group consisting of a superuser group, a plurality of tenant groups, and at least one application group; (2) creating an active directory for each of the groups; (3) adding each of a plurality of users to one of the plurality of tenant groups and the application group; (4) creating tenant directories and home directories for the users; and (5) assigning owners, group owners, default permissions, and extended access control lists to the tenant directories and the home directories.