DYNAMIC DIRECTORY SERVICE
In a network of computer nodes, a directory service provides both the physical location of directory information around the network and the directory information itself in a single data structure. This single data structure is distributed throughout the network, and continuously redistributed, so as to create a directory service that is both more flexible, and more robust, than prior art directory services.
Latest MANGOSOFT CORPORATION Patents:
This application is a continuation-in-part of co-pending U.S. patent applications Ser. No. 08/754,481, filed Nov. 22, 1996, and Ser. No. ______ filed Mar. 28, 1997 and bearing attorney docket number CLC-002. The entirety of both of these applications is incorporated herein by reference.
TECHNICAL FIELDThe present invention relates in general to directory services and, more specifically, to a dynamic directory service that maintains a directory in which is stored both (1) directory information and (2) information identifying the physical layout or structure of the directory.
BACKGROUND INFORMATIONComputer-based structured storage systems, such as computer file systems and database systems, have been remarkably successful at providing users with quick and facile access to enormous amounts of data Structured storage systems have allowed businesses to generate and maintain enormous stores of persistent data that the company can modify and update over the course of years. For many companies, this persistent data is a valuable capital asset that is employed each day to perform the company's core operations. The data can be, for example, computer files (e.g., source code, wordprocessing documents, etc.), database records and information (e.g., information on employees, customers, and/or products), and/or Web pages.
A typical computer based structured storage system includes a central server that provides centralized control over the structured store of data. The structured store of data is the information that is being maintained by the system, such as information in the files and directories of a file system or within the records of a database system. The central server provides system services to a plurality of interconnected network client nodes, and each of the client nodes employs the central server to access and manipulate the structured store of data.
It is common to use the central network server to provide a directory service, i.e., a specialized hierarchical database of network, user, and other computer system configuration information. This information typically includes operating system configuration information, application program configuration information, network configuration information, network-accessible resources, exported devices and services, network printers, and user account records. Network user account records are what allow network-wide, unified “log on,” and they allow sophisticated application programs to obtain broader, coherent name spaces (e.g., electronic mail system application programs). Novell's Directory Server (NDS), available from Novell, Inc. of Provo, Utah, is one example of software that implements a directory service and that runs on a central server to allow maintenance of and access to directory information.
Although central server directory services such as NDS generally work, problems arise from relying on centralized control. For example, the operation of the network is dependent upon the proper functioning of the central server. Any failure of the server to maintain proper operation, such as a power failure, hardware failure, or other such system failure, will disable the entire network and generally prevent users from obtaining access to the network and its resources. Additionally, a flood of client requests (e.g., access of user account records) can overload the central server and slow down or crash the network. Accordingly, reliance on a centralized, server-based directory service can result in slow operation or total network failure during periods of heavy use.
An additional problem with a client-server network system is that it provides a static operating environment that is set for optimal performance at a certain level of network activity. Consequently, the network fails to exploit available resources to improve system performance. In particular, as the system activity rises above or drops below the expected level of network activity, the static operating environment lacks any ability to reconfigure dynamically the allocation of network resources to one providing better performance for the present level of activity.
Technology has been developed to improve the reliability and operation of a centralized server directory service. This technology involves employing a plurality of central servers. Each of the servers provides a directory service. Whenever the directory information changes, all of the redundant servers are updated so that requests can be made to any of the centralized servers without impacting the correctness of the response.
While this statically replicated technology improves upon the single server arrangement, it generally performs poorly or fails as the size of the network increases. As client nodes are added to the network and more directory servers are added to the network to handle the corresponding increased network load, the sustainable update rate of information managed by the servers decreases because the overhead and complexity of propagating updates (i.e., changes to the data stored in and replicated across the directory servers) increases with the increased network size. Thus, known directory services are inherently read-mostly repositories, and that is the way they are designed and used. That is, known directory services are used only for infrequently-updated directory information.
Examples of this centralized, statically replicated, hierarchical directory service technology include Microsoft's Active Directory (NT 5.0), NDS, Banyan's Streetalk, and X.500 directory services. Lightweight Directory Access Protocol (LDAP) is a “common” protocol that can be used to access data from any compatible directory server such as NDS.
SUMMARY OF THE INVENTIONIt is an object of the invention to provide a dynamic directory service that is an improvement over centralized, statically replicated, hierarchical database directory services.
It is also an object of the invention to provide a dynamic directory service that maintains a directory in which is stored both (1) directory information and (2) the physical layout of the directory itself. That is, the invention involves the use of a directory having both information about where to find the directory information of interest as well as the actual directory information.
It is a further object of the invention to provide a dynamic directory service that is more reliable and provides improved fault tolerant operation over existing directory services, and that has the ability to replicate and move data dynamically in response to network activity levels and access patterns. This ability optimizes performance and minimizes the time required, to provide directory information (e.g., network configuration information, organizational information, user configuration information, and network-accessible resources such as exported devices and services, network printers, and user account records) to requesting network nodes.
It is yet another object of the invention to provide a dynamic directory service that provides distributed control over a structured store of directory information and that allows that information to be changed and/or updated with higher frequency than generally possible with existing directory services without adversely effecting network performance or network node access times.
It is still another object of the invention to provide a dynamic directory service that maintains and allows access to frequently-changing, as well as infrequently-changed, directory information.
The directory service of the invention maintains both the physical location information and the directory information itself in the same data structure. That data structure is distributed around the network, and all of the directory information is homeless (except, usually, a root record). The distributed directory service of the invention allows network nodes to locate copies of records to which the nodes want access. A globally unique identifier (GUID) is associated with each record as a unique index key that can be used to identify the record on the network. It contains no location information, and a given record can be physically located anywhere on the network. Each of the records also can migrate from physical location (node) to physical location (node) around the network. The invention provides automatic migration and/or replication of directory information among the network nodes without reconfiguring the network nodes and in a manner transparent to users at the nodes. An example of a mechanism that can be used to keep replicated records consistent is a single-writer, multiple-reader, write-invalidate protocol.
The directory service of the invention can, in some specific embodiments, employ a globally-addressable unstructured memory system to maintain simultaneously both the directory information and layout information in accordance with the invention. For example, the directory service can employ the distributed shared memory (DSM) system described in the above-identified, incorporated-by-reference patent applications, which DSM system distributes the storage of data across some or all of the storage devices connected to a network. Storage devices that may be connected to the network and accessible to the network nodes by address via the DSM system include, for example, RAM, hard disk drives, tape drives, floppy disk drives, and CD-ROM drives. In some embodiments, the dynamic directory service is a computer program that interfaces to a DSM system to operate the DSM system as a memory device that provides storage of and access to the directory information. The directory service program can direct the DSM system to map directory information into the shared memory space. The DSM system can include functionality to share, migrate, and replicate data coherently. In one embodiment, the DSM system provides memory device services to the directory service program. These services can include read, write, allocate, flush, or any other similar or additional service suitable for providing low level control of a storage device. The directory service program employs these DSM system services to allocate and access portions of the shared memory space for creating and manipulating the directory information. In connection with these embodiments, a system and related method for accessing directory information includes a computer network, a globally addressable data storage system, and a plurality of computers coupled to the network and the data storage system. The globally addressable data storage system provides persistent storage of data and contains directory information. The plurality of computers access the data storage system to obtain directory information. Based on the access patterns by the computers and/or the available network resources, the data storage system replicates and migrates directory information among two or more of the computers.
In one aspect, the invention relates to a method comprising the steps of providing a plurality of nodes interconnected by a network and storing on one or more of the nodes a directory containing both the directory information and information about the layout of the directory (i.e., where to find the directory information).
In another aspect, the invention relates to a method comprising the steps of providing a plurality of nodes interconnected by a network, providing a directory service on the network by installing on each of the nodes a directory service program that allows directory information to be accessed by each of the nodes and that stores on one or more of the nodes a directory including both the directory information and information about the location of the directory information on the network, and obtaining both the physical location of directory information of interest and the directory information of interest itself by accessing the directory service.
The foregoing and other objects, aspects, features, and advantages of the invention will become more apparent from the following description and from the claims.
In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
According to the invention, a computer network system 10 includes a plurality of network nodes that access a common directory service. The directory service is a specialized hierarchical structured database. Examples of the types of information that can be stored and maintained by the directory service include operating system configuration information, application program configuration information, network configuration information, network-accessible resources, exported devices and services, network printers, and network user account records. Each of the nodes on the network includes at least a directory service program that accesses and manages the directory service. The directory service may be stored in an addressable shared memory or it may be stored in a more traditional fashion. For example, each node may be responsible for storing a particular element or elements of the directory service. In such an embodiment, the directory service program can access a desired portion of the structured store using a globally-unique tag or identifier. The underlying system would translate the tag or identifier into one or more commands for accessing the desired data, including network transfer commands. In another embodiment, the directory service is stored in an addressable shared memory space, which allows the network nodes transparently to access portions of the structured store using standard memory access commands. In a preferred embodiment, the tag or identifier is an address into an addressable memory space such as a 128-bit address space.
The system 10 can be a file system, a database system a Web server, an object repository system, or any other structured storage system that maintains an organized set of data. In the disclosed embodiment, the system 10 is a directory service that maintains various directory information.
Referring to
A system 10 according to the invention can provide, among other things, each network node 12a-12d with shared control over the directory service 22 and, therefore, the system 10 can distribute control of the directory information across the nodes of the network. To this end, each node of the system 10, such as node 12a, includes a directory service program 32a that operates as a structured directory service adapted to maintain directory information and to employ all of the network nodes for storing and allowing access to the directory information. These cooperating elements provide a structured storage system that has a distributed architecture and thereby achieves greater fault tolerance, reliability, and flexibility than known directory services that rely on centralized control and one or more centralized servers. Accordingly, the invention provides computer networks with distributively controlled and readily scaled directory services.
Still referring to
Each of the directory service programs 32a-32d is a software module that couples to the directory service. The directory service program 32a can stream data to, and collect data from, the directory service subsystem. Each of the directory service programs 32a-32d can be a peer incarnation (i.e., an instance) residing on a different one of the network nodes 12a-12d.
One or more of the directory service programs 32a-32d can provide a graphical user interface 42 that graphically depicts the directory service 22. The graphical user interface 42 could allow a user at a node, for example at node 12a, to insert directory information graphically within the directory service 22. To this end, the directory service program 32a can generate a set of commands that will present a stream of data that will result in directory information being stored within the directory service 22. As shown in
A structured dynamic directory service according to the invention looks to all network nodes like a coherent, single directory service system when in fact it spans all participating nodes coupled to the network. The directory service of the invention differs from known directory services in a variety of ways. For example, the directory service of the invention: maintains data coherence among network nodes; automatically replicates directory information for redundancy and fault tolerance; automatically and dynamically migrates directory information to account for varying network usage and traffic patterns; and provides a variety of other advantages and advances, some of which are disclosed in the above-identified, incorporated-by-reference applications. The replication and migration can be done on the basis of node accesses and/or on the basis of the availability of network resources.
Referring to
The directory service of the invention can, as an option, employ more than one set. A benefit of breaking up the directory service 60 into a plurality of sets is that it may provide more flexible management for users of the directory service. As the directory service grows into very large sizes (e.g., hundreds of nodes with thousands of gigabits of storage), it may be desirable to have the directory information organized into groups of management entities such that management actions can be independently applied to individual groups without affecting the operation of the others.
In a set, the root (e.g., the root 80) provides the starting point to locate the directory information maintained by the directory service of the invention. The root can be, and preferably is, stored in a static and well-known location on the network (e.g., at a particular address or on one or more network nodes). When a node is accessing a set for the first time, it first looks up the root to determine the key associated with the set. Once it has determined the key, the node can access the root of the set. From the root, it then can traverse the set's entire tree to locate the desired piece of directory information.
As an example, in
Referring to
In one particular embodiment, directory record 320 can be a page of a global address space that spans both persistent (e.g., hard disks) and volatile (e.g., RAM) storage devices. For example, the page can be a 4 kilobyte portion of the shared address space described in the above-identified, incorporated-by-reference applications. In this embodiment, the GUIDs are unique addresses of the global address space such as 128-bit addresses in a huge 2128 address space.
Referring still to
In accordance with the invention, directory records are indexed by globally unique identifiers (GUIDs), such as 128-bit values. In the directory records, these GUIDs can appear in the data fields or the key fields of various of the directory entries. As described in more detail below, a GUID subtree within the directory provides a map or assignment of GUIDs to network nodes that have a copy of certain directory information. It is the leaf nodes of the GUID subtree that identify the one or more nodes that have a copy of desired directory information. Once these nodes are identified by the directory service, the directory service accesses one or more of them (typically just one of the identified nodes) to obtain the directory information and then pass it to the requesting node. Optionally, the directory system can choose which node to contact to obtain the directory information based on the state of the global system such as the current load on the various nodes or the quality of the network service between the local node and the node(s) that have copies of the desired directory record.
Referring to
It is important to note that, while traversing the GUID tree, the directory service first always checks to see if entries indexed by a GUID are cached locally before the directory service invokes recursively the GUID tree to locate a copy of the record remotely.
At this point, the directory service can store a copy of (replicate) this directory information (e.g., user Jones' password) on the requesting node. Actually, if replication is performed by the directory service, all or a portion of the record that contains this directory information is replicated on the requesting node. If the directory service decides to replicate this record on the requesting node, the directory service adds the requesting node's identifier (e.g., a number) to the leaf node in the GUID tree that stores the list of nodes caching the record associated with the GUID, and the directory service then updates or invalidates the local caches of all of the other network nodes that also have that directory record cached locally.
In one embodiment, the directory service makes the decision on whether or not to replicate based on node access patterns and/or on the availability of certain network resources. For example, if a node has requested a certain record a certain number of times, the directory service will replicate that record on that node. The directory service monitors and records a variety of network information, including node access patterns and network resource availability, and some or all of this network information is used to determine what records should be replicated on what nodes. In accordance with the invention, this replication feature of the directory service results in certain records being replicated on the nodes that most often access those records and not being replicated on (and/or removed from) nodes that access those records less often or not at all. The directory service of the invention provides this replication feature dynamically during normal operation, and thus, as access patterns change, the records and the copies of the records move or migrate among the various network nodes to accommodate the changing patterns. This replication/migration feature of the invention points up an important aspect of the invention, and that is that all of the directory records are homeless (except possibly the root directory record which, while it typically will be replicated, typically is placed at fixed locations on the network).
Because the directory records contain, according to the invention, both the GUID tree structure (i.e., the information about where directory information is located) and the actual directory information, the directory structure (i.e., the GUID tree) is dynamically moved and replicated among the network nodes just as is the directory information itself.
These dynamic replication and migration features provide tremendous benefits to networks utilizing the directory service of the invention. For example networks utilizing the directory service of the invention can be expanded (i.e., additional nodes can be added to the network and/or other networks can be interconnected to the network) without substantially impacting the performance and speed realized by the nodes as they access directory information via the directory service. That is, the directory service scales very well. This is because, once the network is up and running, all nodes generally will have cached locally the records they most often access, and access times for these nodes for the directory information that is cached locally will be the same (very fast) regardless of the size of or the traffic on the network.
Referring to
Referring now to
If the entry in the root directory record does not contain the requested directory information in the entry's data field (step 616 of
As shown in
The loop defined by steps 628, 630, and 634 in
It is important to note that the cache of locally stored directory entries can be indexed so that at any stage of the GUID tree walk operation it is possible to obtain the desired record from the local cache instead of the tree structure. For example, if record 398 is cached on the node attempting to lookup “/USERS/JONES/PASSWORD,” the directory service will be able to find the locally cached record via a hash lookup for either /USERS/JONES/PAS SWORD or the corresponding GUID. Similarly, while traversing the GUID subtree, the directory service first always checks to see if entries indexed by a GUID are cached locally before the directory service invokes recursively the GUID tree to locate a copy of the record remotely.
In the course of the recursion, the various records that the directory service accesses lead the directory service to the leaf record 412. This happens by the directory service beginning at the GUID tree root directory record 404 and examining the key field of each of the directory entries in the record 404 (or one of the copies 405, 407 of the record 405 on other nodes in the network) to determine the range in which the GUID logical link from the record 400 falls. If the GUID logical link from the record 400 falls within the range identified by the key field of directory entry 409, the GUID in the data field of the entry 409 is used as the logical link or pointer to the next record. This process continues until the ranges in the key fields of the directory entries of the subsequent records reduce to a single GUID such as the GUID “12540” in the key field of a directory entry 411 in the GUID leaf record 412.
Again, while traversing the GUID subtree, the directory service first always checks to see if entries indexed by a GUID are cached locally before the directory service invokes recursively the GUID tree to locate a copy of the record remotely. In referencing the GUID tree, the directory service uses the responsible node information in the data fields of the GUID tree records in the location process.
It is important to realize that all of these records that the directory service accesses in this recursive loop (and, in general, even when it is not in the recursive loop) can, and in many instances will, be located on different network nodes. This is because the directory service of the invention is distributed, and in general the records that make up the directory and that contain the various pieces of the directory information maintained by the directory service are located on different nodes all over the network. As the GUID tree is walked in the manner described herein, the directory service can replicate the accessed GUID tree records and make them local on the requesting node. This replication is optional and is accomplished as described hereinabove.
It also is important to realize that the records that describe the structure of the directory layout (i.e., the GUID tree records) are stored by the directory service in the same manner as it stores the records having the actual directory information and also maintained by the directory service in the same way that it maintains the records with the actual directory information.
Referring to step 622 of
Instead of going from the root directory record 400 (or any one of its copies 401, 403, 405) to walking the GUID tree to obtain the requested directory information, it is possible that the directory service of the invention will find an entry in the root directory record 400 that provides a logical link (e.g., GUID “12540”) in its data field directly to an entry 399 in a record 398 that has the requested directory information in its data field. It also is possible, as mentioned hereinabove, that an entry 402 in the root directory record 400 itself will have the requested directory information in its data field. A typical situation, however, is when the GUID tree is walked by the directory service via the GUID tree records in order to locate and return the requested directory information to the requesting network node.
In accordance with the invention, the directory service tracks ownership and responsibility for directory information thereby providing a level of indirection between the actual directory information itself and the physical location of that directory information on the network.
It should be appreciated that the directory service of the invention comprises a hierarchical structured storage mechanism for directory information. To this end, the directory service of the invention provides a structure that continually subdivides itself into smaller and smaller sections. Further, each section is represented by directory records of the same structure, but each section indexes different amounts (sizes) of directory information.
In accordance with the invention, more frequently accessed directory information is copied and distributed among various network nodes, and rarely used directory information generally will appear on only a few network nodes. Also, directory records will migrate to those nodes that access them most, providing a degree of self-organization that reduces network traffic.
In general, all of the functionality of the directory service of the invention can be implemented in software. In one embodiment, an instance of a directory service program resides and executes on each of the network nodes and provides all of the directory service features and functionality described herein. It is possible to perform one or more of the various functions of the directory service with dedicated electronics or a combination of hardware and software, and this hardware might be added to a general purpose computer to implement the directory service of the invention.
Variations, modifications, and other implementations of what is described herein will occur to those of ordinary skill in the art without departing from the spirit and the scope of the invention as claimed. Accordingly, the invention is to be defined not by the preceding illustrative description but instead by the spirit and scope of the following claims.
Claims
1-10. (canceled)
11. At least one computer-readable medium containing a set of executable instructions for causing programmable apparatus to perform a method of obtaining directory information, said method being operable on a plurality of nodes interconnected by a network, said method comprising the steps of:
- providing a distributed directory service on the network by installing on each of the nodes a directory service program that allows directory parts to be selectively and dynamically migrated between ones of said plurality of nodes, without requiring restructuring the directory, storing on said plurality of nodes the directory that includes both the directory information and information about the location of the directory information on the network, such that said directory is not statically replicated to all nodes of said plurality of nodes; and
- obtaining both the location information and the directory information by accessing the distributed directory service.
12. The at least one computer-readable medium of claim 11 wherein the directory service providing step further comprises the step of utilizing the directory service program on each of the plurality of nodes to replicate said directory parts.
13. The at least one computer-readable medium of claim 12 wherein the directory service providing step further comprises the step of utilizing the directory service program on each of the plurality of nodes to selectively and dynamically replicate said directory parts based on the number of accesses of the directory service by each of said nodes.
14. The at least one computer-readable medium of claim 12 wherein the directory service providing step further comprises the step of utilizing the directory service program on each of the plurality of nodes to selectively and dynamically replicate said directory parts based on available resources on the network.
15. The at least one computer-readable medium of claim 11 wherein the distributed directory service providing step further comprises the step of utilizing the directory service program on each of the plurality of nodes to selectively and dynamically migrate said directory parts based on the number of accesses of the distributed directory service by the nodes.
16. The at least one computer-readable medium of claim 11 wherein the distributed directory service providing step further comprises the step of utilizing the directory service program on each of the plurality of nodes to selectively and dynamically migrate said directory parts based on available resources on the network.
Type: Application
Filed: Jun 23, 2008
Publication Date: Oct 16, 2008
Applicant: MANGOSOFT CORPORATION (Westborough, MA)
Inventors: Daniel J. Dietterich (Acton, MA), Robert S. Phillips (Brookfield, MA), John B. Carter (Salt Lake City, UT), Scott H. Davis (Groton, MA), Steven J. Frank (Hopkinton, MA), William Abraham (Windham, NV)
Application Number: 12/144,508
International Classification: G06F 17/30 (20060101);