ARTIFICIAL PLAYER CHARACTER FOR MASSIVE MULTI-PLAYER ON-LINE GAME
An artificial intelligence system for autonomous artificial player characters in an on-line game is provided. The artificial player character (APC) makes use of an adaptive neural net wherein the function of the artificial player character at any given time is determined as a composition of other functions defined by one or more of the following: the game designer, the personality preference filters of the artificial player character, the current statistical levels of the artificial player character, the database of historical interactions of the artificial player character, a master database of historical interactions by other artificial player characters, and interactions of the artificial player character with other characters and/or the game environment.
This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application Ser. No. 60/903,381, filed on Feb. 26, 2007, and incorporated herein by reference. This application is related to U.S. Non-Provisional patent application Ser. No. ______, filed on even date herewith, having attorney docket number B575.101.103, and incorporated herein by reference.
FIELD OF THE INVENTIONThe invention relates generally to a form of an autonomous artificially intelligent software entity for a computer game. More specifically the invention relates to a form of an autonomous artificially intelligent player character for a massive multi-player on-line game.
BACKGROUND OF THE INVENTIONMassive multi-player on-line (MMO) games are computer games capable of supporting a number of players (e.g., hundreds or thousands of players) simultaneously. In one arrangement, players log onto a central server over the Internet and participate in a gaming experience shared by numerous remotely connected users. As such, MMO games are usually played via remote clients accessing one of a number of central servers. Typically, MMO games are played in a large persistent virtual environment that enables players to compete with and against each other on a grand scale and to interact cooperatively with people around the world.
In one application, MMO games provide a form of electronic entertainment which allows numerous individuals to simultaneously take part in an ongoing virtual world. These games are customarily set in a fantasy role-playing environment, although they can theoretically be set in any conceivable fictional genre, historical setting, or other environment. The usual way in which the player experiences the MMO game is to create an avatar, or Player Character, which then travels throughout the virtual world and interacts with a wide variety of characters which are either controlled by other players or by the game server.
The Player Character customarily spends most of its time improving itself through the collection of experience points, which are earned, for example, through killing monsters and completing adventures assigned to the Player Character by server-controlled Non-Player Characters (NPC) scattered throughout the virtual world in strategic locations.
The decision-making utilized in this process naturally requires human involvement in order to make the Player Character's decision to embark on the adventure in search of the promised rewards or to reject it. This is a somewhat more complicated process than it might look at first, since it requires balancing a number of variables which include, for example, the perceived ability of the Player Character to find Monster Y, the amount of time required to do so, the risks of travel to Monster Y's location, the ability of the Player Character to kill x number of Monster Y, the likelihood that the Player Character will survive the encounter, the trustworthiness of the NPC to make good on its end of the bargain, and finally, the value of the reward on offer to the Player Character.
A similar process is required for a wide variety of interactions between Player Characters and NPCs, Monsters, and the environment itself. These include, but are not limited to, travel from point A to point B, economic production, economic transactions, character allegiances and alliances, and combat. With the sole exception of combat, wherein an attack on a non-autonomous character type will inevitably be responded to by an attack in self-defense, these processes inevitably require at least one of the two parties involved in the interaction to be an autonomous human-controlled character.
As outlined in the Table of
1. Player Characters are human-controlled characters who take active part in the on-line game. Also known as “avatars” they are autonomous, travel freely about the game environment subject only to the limits set by the game designers, and their attributes evolve over time. Their statistics, appearance, skill sets, and possessions are dynamic, constantly changing over the course of the game. Player Characters are only present within the game when the human player controlling them is on-line.
2. Administrative Characters are human-controlled characters who only enter the on-line game in order to deal with specific problems, usually those brought to the attention of the game's managers by the users. Also known as “DMs” or “Game Masters”, these Administrative Characters are autonomous and serve police and technical support functions for the participating users, however their attributes are static as they exist primarily for specific game management purposes. They are very seldom encountered.
3. Non-Player Characters are server-controlled characters who primarily exist in order to provide information, supplies, and rewards for the Player-Characters. Usually called “NPCs”, they are non-autonomous, seldom move of their own volition outside of a strictly limited area (if at all), and their attributes do not evolve over time. Their statistics, appearance and skill sets are static, their possessions may or may not be dynamic; if their possessions are variable, the variable alters solely on the basis of actions on the part of the Player Characters. Their presence is typically constant within the game although their appearance may occasionally be made variable by the use of a trigger based on a Player Character's action.
4. Monster Characters are server-controlled characters who primarily serve as the generic opposition for the Player Characters. Often described generically as “enemies” they are non-autonomous, their movement is strictly circumscribed and their basic attributes do not vary over time although their statistics and state will invariably alter radically in the course of a combat encounter with a Player Character or a Pet. Their presence and location are usually constant and unchanging within the game, subject only to their temporary elimination by a Player Character or the failure of a Player Character to trigger their presence. Their statistics, appearance, and skill sets are essentially static, and their decision-making capabilities are crude and essentially limited to a binary option of fight or flight.
5. Pets are a special form of Non-Player Character which occasionally accompany certain types of Player Characters. They are server-controlled partially-autonomous characters which unlike other NPCs, possess dynamic statistics and skill sets as well as the ability to move about the game environment by following around the Player Character to whom they belong.
6. Environmental Characters are server-controlled characters who primarily serve a decorative purpose within the game. They are non-autonomous, their movement is typically limited, and they seldom have significant interaction with the Player Characters.
In the field of Massive Multi-player On-line games, the division between the activities of the Player Characters and the other five character forms is distinct and easily defined. Player Characters voluntarily engage in goal-oriented adventures, economic activity, and combat, develop skills, abilities, talents, and occupations along elective paths, and obtain experience, reputation, money, and physical possessions. In contrast, none of the five other character forms are able to do any of these things as without the direction of a human mind providing purpose and personal preferences for the character, they simply do not have the capacity.
Even the sole other dynamic character type, the Pet, is wholly dependent upon the ability of the owning Player Character's human mind to make decisions for it as its movements are based upon the Player Character to whom they belong. For example, the Pet's dynamic statistics and skills are chosen by the owning Player Character, and even its semi-autonomous ability to initiate combat is completely at the discretion of the Player Character.
And while MMO games are rightly very popular in the gaming community, they are already running into a fundamental design limitation due to their heavy reliance upon Player Characters to provide the dynamic aspect of the entertainment. Since MMO game designers have limited control over the decision-making processes of the participating Player Characters and are forced to leave the vast majority of these decisions to the discretion of human beings controlling the Player Character, they are forced to spend an inordinate amount of time limiting an individual Player Character's ability to destroy the gaming experience for himself and others. Every aspect of the design, from the storyline to the physics model, is forced to take into account the possibility that any player may, at any time, elect to behave in a manner detrimental to the designer's purpose for the game.
This discretion is often exercised to the disadvantage of other Player Characters and to the detriment of the game experience. For example, it is not uncommon for a Player Character to find himself suddenly abandoned in the midst of a horde of enemies because the other Player Characters with whom he was cooperating have departed the scene, either because it was necessary for them to go off-line or simply because they had already achieved their goals and were uninterested in continuing to help the Player Character. This occurs frequently in current MMO games and severely detracts from the game's entertainment value.
Furthermore, the wide variety in maturity levels and sophistication among the players of an MMO game means that Player Characters are often interacting with each other under the handicap of possessing mutually incompatible entertainment goals.
For these and other reasons, there is a need for providing MMO game designers with a mechanism for greater control over Player Characters in a massive multi-player on-line game.
SUMMARYOne aspect of the present invention provides an artificial player character for a computer game as illustrated and described herein.
One aspect of the present invention provides a method of defining an artificial player character for a computer game as illustrated and described herein.
One aspect of the present invention provides a computer-readable medium including computer-executable instructions for performing a method of defining an artificial player character for a computer game as illustrated and described herein.
One aspect of the present invention provides a computer game including an artificial player character as illustrated and described herein.
One aspect of the present invention provides a method of playing a computer game including an artificial player character as illustrated and described herein.
One aspect of the present invention provides a computer-readable medium including computer-executable instructions for performing a method of playing a computer game including an artificial player character as illustrated and described herein.
An Artificial Player Character (APC) is a client-side or server-side software entity which possesses a specific identity within a Massive Multi-player On-line (MMO) game and is capable of fully autonomous action within the virtual gaming world. In one embodiment, the Artificial Player Character is an artificially intelligent (AI) player character capable of simulating learned behavior and of propagating itself through the creation of distinct entities which share aspects of its core elements.
When a plurality of players play a Massive Multi-player On-line game, they encounter two types of characters, player characters and non-player characters. Player characters are controlled by humans and therefore possess a wide variety of motivations and objectives, while non-player characters possess only response-based motivations and static objectives assigned to each individual non-player character by the programmers of the MMO game. A process for creating proactive motivations and dynamic objectives for non-player characters allows non-player characters to independently determine their immediate and long-term objectives based on unique preferences derived from their virtual personality, then act proactively on the basis of those objectives for the duration of their existence within the MMO game environment.
In one embodiment, an APC is comprised of a set of game or character statistics for the APC, a set of graphical images for the APC and a dynamic list of “possessions” of the APC which may be utilized within the game, and a set of two or more probability-related values used for the purpose of determining a personality trait and the corresponding behavior or activity of the APC. The APC may also possess a database, such as an experience database, containing the record of historical encounters and the results of those encounters (see, e.g.,
The set of two or more probability-related values used for the purpose of determining the activity of the APC are referred to as Preference Filters. A Preference Filter is defined as the APC-specific probability-based preferences related to a specific group of potential game interactions or encounters. One example of a Preference Filter, such as Preference Filters 20 of
As illustrated in
As illustrated in
Character Statistics, such as Character Statistics 40 of
While Character Statistics describe what an APC can do, Graphics and Possessions, such as Graphics and Possessions 50 of
In one embodiment, Possessions contain both Attributes and dynamic Values, such as Attributes 54 and Values 56 of
In one embodiment, both Character Statistics and Possessions are individually tracked and monitored, and taken in combination with various environmental and inter-player encounters, represent potential Inputs for the APC in certain situations. For example, an encounter with an armor vendor may represent a potential Input for an APC if the vendor is (a) selling the type of armor worn by the APC, (b) is selling armor that would increase the APC's Character Statistics (in this case, Defense Rating), and (c) selling the armor for less money than the APC possesses.
As illustrated in
As an example of the decision-making process of
The Filter values 24 for an APC's Preference Filters 20 may be assigned in various ways. In one embodiment, static values for Preference Filters of an APC are determined by the game designer or other individual. In another embodiment, values for Preference Filters of an APC are assigned by a random process. Assigning values by a random process would allow for the widest variability of APC behavior, although some of the behaviors may prove to be undesirable for the gaming experience.
In one embodiment, values for Preference Filters of an APC are assigned by a “learned” process. In the “learned” process, the APC's Preference Filters are permitted to evolve by means of access to a database containing the record of historical encounters and the results of those encounters. In one embodiment, additional weight is given to Preference Filters whose use is “effective” as defined by the game designer, and less weight is given to Preference Filters whose use proves to be “ineffective” as defined by the game designer.
In one embodiment, assigning APC Preference Filter Values for an APC is by an “evolutionary” method wherein two or more APC databases and Preference Filter sets are combined to produce a “child” APC with its own distinct Preference Filter values. This method can produce Artificial Player Characters that evolve into a more interesting and effective opposition over time instead of simply respawning as perfect clones of Artificial Player Characters that were eliminated from the game on a previous occasion. In addition, this method can produce Artificial Player Characters as Non-Player Characters that will interact with the Player-Characters in a more complex and entertaining manner.
The Preference Filters provide the APC with a simulated and unique set of motivations that can be applied to every game encounter, thus providing a wide range of potential variability within the context of the encounters. Therefore, the game designer may want to define every variant of encounter in terms of the Preference Filters to which it is relevant. For example, an encounter between a Race X class 1 (Dwarf Warrior) and a traveling Race X class 6 (Dwarf Merchant) may be tagged with specific relevance IDs for Loyalty, Attachment, Conversation, and Trading, whereas a subsequent encounter by the same Race X class 1 with a Race Y class 1 who specializes in warrior training may be tagged with specific relevance IDs for Ambition and Aggression.
In one embodiment, when the game queries the APC regarding its prevailing attitude towards the imminent encounter, (which will usually be triggered via a basic proximity setting), a first step is to determine which of the relevance IDs are relevant to the situation at hand (see, e.g.,
In one embodiment, once the relevant Preference Filters have been identified, the game will then compare the relative strength of the competing interests based on their assigned values and calculate a decision regarding the APC's action (see, e.g.,
Using the comparison method for the encounter described above, and positing, for example, a value of 99 for Conversation and a value of 33 for all the other relevant Preference Filters, the game will ascertain that the APC has a propensity to speak with the traveling Dwarf Merchant and will attempt to strike up a conversation with him. This APC action may lead to a new encounter, “Conversation with a Dwarf Merchant,” and inspire the game to engage in a second round of relevance ID checks. In one embodiment, this second round of relevance ID checks may include an inquiry into Negotiation as well as an inquiry into the APC's present material requirements as indicated by the state of its current possessions.
As illustrated in
In the embodiment of
The database containing the record of historical encounters and the results of those encounters is referred to as the APC's Experience Database. In one embodiment, an experience database, such as Experience Database 60 (
In one embodiment, the Experience Database functions by assigning a value expressing the results of the encounter and storing it according to the appropriate encounter type. For example, a combat encounter between a level 12 APC and three level 15 Orcs in which the APC killed two Orcs before being forced to run away from the third will be given an encounter ID, summarized in the database as C3L3, and may be assigned a result value of 2 (defeat, hit/damage ratio<0.5) on a hypothetical scale of 0-5. This results scale, of course, may be set to any level of detail, for example, from binary to near infinite.
Accordingly, with this example, each time the APC engages in a similar encounter, regardless of whether it is a level 2 character fighting three level 5 Farm Pigs or a level 50 character fighting three level 53 Iron Hellknights, the results would be recorded and averaged with the results from previous encounters of the same type. The greater the number of encounters stored in the Experience Database, the more weight it would be provided as a modifier applied to the Preference Filters.
In one embodiment, based on the encounters stored in the Experience Database, the Preference Filter value itself may be modified. For example, an APC which regularly records results of 0 (death, hit/damage ratio<0.25) for a specific encounter ID will eventually “learn” to exert a powerful modifier to avoid engaging in that encounter type regardless of how high its Preference Filter value for Risk-taking or Aggression might be. The extent to which this modifier influences the Preference Filters and, therefore, the APC's actions may be defined by the game designer.
In one embodiment, as the quest continues, the APC Manager monitors PCs looking for a group, as illustrated at 914. If a player is looking for the type of group already existing and including one or more APCs, the APC Manager will offer the player character the choice to join the existing group in mid-quest, as illustrated at 916. If the player character wishes to join the existing group, the APC Manager will remove the APC most similar to the new player character, and replace the APC with the player character, as illustrated at 918. Accordingly, the quest can be completed, as illustrated at 920, with the player character taking the place of the previously assigned APC.
In one embodiment, the APC Manager allows multiplayer groups to complete multiplayer quests that would otherwise be abandoned when members drop out in mid-quest. This is accomplished by substituting appropriate APCs for the missing player characters whenever a group member exits the game or the group, thus allowing the remaining player characters to continue with their quest in the company of the APC(s). Because the APC Manager actively monitors PCs looking for groups, it is aware when an individual player character is in search of a similar group or quest, at which point the APC Manager will offer the player character the choice to join the multiplayer group in mid-quest. In one embodiment, if the player character wishes to join the group, the APC Manager will remove the APC most similar to the new player character, replacing it with the player character.
In addition to simulating intelligent decision-making by the APC regarding its actions when presented with encounters within the game, the present invention also provides a means of simulating more intelligent tactical decision-making with regards to the APC's use of various items and possessions within the context of those encounters. For example, in a conventional MMO game, a wounded monster will usually attempt to heal itself to the full extent of its capacity regardless of how much time it possesses. With the more sophisticated possibilities of the present invention, an APC would have the capability of determining the optimal balance of health and time required to give it the greatest possibility of surviving the encounter.
The present invention enhances the ability of the game designer to exert influence over the game events, encounters, and interactions experienced by the participating users. It also provides the game designer with an effective means of implementing the continuous evolution of artificial intelligence within a massive multi-player on-line game and, in one embodiment, offers the prospect of eliminating the human element entirely.
The present invention provides a system for the introduction of autonomous, dynamic artificial player characters which do not require human direction and are not subject to human control to the field of massive multi-player on-line games. These Artificial Player Characters serve as a substitute for human-controlled Player Characters which are less subject to the negative aspects of human behavior and decision-making that detract from the game's ethos, and provide the MMO game designer with an enhanced ability to create an interesting and enjoyable game experience for the human players as they encounter the virtual world.
The autonomous Artificial Player Character of the present invention is more complex than the automated combat scripts (“bots”) which are used in many 3D shooters and is applicable to a much broader range of human/AI and AI/AI interactions than mere combat. It is also substantially different from the utilities illegally used in some MMO games which automate tasks for a Player Character, because all of the activities performed by the utilities require human decision-making and initiation.
The present invention also makes technologically feasible the possibility of a massive multi-player on-line game which requires neither multiple players nor on-line access. For example, with application of the APC to an MMO game, it becomes theoretically possible to create a dynamic massive multi-player on-line game without requiring either human players or on-line access. This has interesting utility in non-gaming applications wherein it may be considered desirable to have access to an evolutionary process from inside the process itself.
One embodiment of the present invention provides a system for the implementation of an artificial player character in a massive multi-player on-line game. The system may include a software entity with a unique, artificially intelligent capacity for decision-making regarding every form of encounter it experiences in the virtual game world, including but not limited to, adventuring, combat, conversation, exploration, economic activity, questing, and trading. As such, the entity may possess complete autonomy within the virtual game world. In one embodiment, this autonomy is subject to the discretion of the game designer or a human player to whom ownership of the artificial player character has been granted. In addition, the entity may possess a status equivalent to human-controlled Player Characters within the virtual game world.
One embodiment of the present invention provides a system for the implementation of learned behavior by artificial player characters, non-player characters, and monsters in a massive multi-player on-line game. The system may include a set of values that dictate the preferred behaviors of an individual artificial player character, non-player character, or monster, and a database containing a record of past encounters and the results of those encounters. The results of the actions dictated by the implementation of the set of values are stored in the database such that the interpretation of the historical results are implemented to modify the set of values for future encounters. In one embodiment, the database may contain provisions for overriding the set of values based upon historical results.
One embodiment of the present invention provides an automated system for creating artificial player characters, non-player characters, and monsters for a massive multi-player on-line game. The system may include the combination of two or more sets of values that dictate the preferred behaviors of the specified artificial player character, non-player character, or monster into a single set of values dictating the preferred behaviors of a new entity.
One embodiment of the present invention provides a system for creating a zero-player off-line version of a massive multi-player on-line game. The system may include a three-dimensional graphical virtual environment entirely populated with artificial player characters which may be visited by non-participatory human-controlled avatars or a limited number of participatory human-controlled player characters.
One embodiment of the present invention provides a process for monitoring the requirements of individual player characters attempting to form multiplayer groups and providing Artificial Player Characters to substitute for player characters when insufficient player characters are available for group formation.
One embodiment of the present invention provides a process for monitoring the requirements of multiplayer groups and substituting Artificial Player Characters for player characters as needed in order to prevent premature dissolution of those groups. This embodiment also encompasses a system of removing APCs from the group and replacing them with human-controlled player characters when interested player characters become available.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims.
Claims
1. An artificial player character for a computer game, comprising:
- a set of at least two probability-related values which determine an activity of the artificial player character,
- each of the probability-related values comprising a preference filter including a filter identification and a filter value for the filter identification, wherein the filter value is an integer value and defines how relevant the preference filter is to an encounter of the artificial player character during the computer game.
2. The artificial player character of claim 1, further comprising:
- a set of character statistics for the artificial player character;
- a set of graphics for the artificial player character; and
- a list of possessions of the artificial player character.
3. The artificial player character of claim 1, further comprising:
- an experience database including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.
4. The artificial player character of claim 1, wherein the set of at least two probability-related values determine a personality trait of the artificial player character and a corresponding behavior of the artificial player character.
5. The artificial player character of claim 1, wherein the preference filter defines a probability-based preference of the artificial player character to a potential interaction during the computer game.
6. The artificial player character of claim 1, wherein the filter value of the preference filter is assigned by an individual.
7. The artificial player character of claim 1, wherein the filter value of the preference filter is assigned by a random process.
8. The artificial player character of claim 1, wherein the filter value of the preference filter is assigned by a learned process including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.
9. The artificial player character of claim 1, wherein the filter value of the preference filter is assigned by an evolutionary process combining at least two preference filters.
10. The artificial player character of claim 1, wherein the preference filter includes one of a loyalty filter, an ambition filter, an exploration filter, an adventure filter, a risk-taking filter, an aggression filter, a conversation filter, an attachment filter, a persistence filter, a trading filter, an anchorage filter, an altruism filter, a sanity filter, a greed filter, a social filter, a romance filter, a negotiation filter, an occupation filter, an economy filter, and a learning filter.
11. The artificial player character of claim 1, wherein the computer game is a massive multi-player on-line game.
12. A method of defining an artificial player character for a computer game, the method comprising:
- identifying a set of at least two probability-related values which determine an activity of the artificial player character, including, for each of the probability-related values, assigning a filter value to a preference filter,
- wherein the filter value is an integer value and defines how relevant the preference filter is to an encounter of the artificial player character during the computer game.
13. The method of claim 12, further comprising:
- identifying a set of character statistics for the artificial player character;
- identifying a set of graphics for the artificial player character; and
- identifying a list of possessions of the artificial player character.
14. The method of claim 12, further comprising:
- generating an experience database including historical encounters of the artificial player character and results of the historical encounters of the artificial player character.
15. The method of claim 12, wherein assigning the filter value to the preference filter includes defining a probability-based preference of the artificial player character to a potential interaction during the computer game.
16. The method of claim 12, wherein assigning the filter value to the preference filter includes assigning the filter value by an individual.
17. The method of claim 12, wherein assigning the filter value to the preference filter includes assigning the filter value by a random process.
18. The method of claim 12, wherein assigning the filter value to the preference filter includes assigning the filter value by a learned process including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.
19. The method of claim 12, wherein assigning the filter value to the preference filter includes assigning the filter value by an evolutionary process combining at least two preference filters.
20. The method of claim 12, wherein the preference filter includes one of a loyalty filter, an ambition filter, an exploration filter, an adventure filter, a risk-taking filter, an aggression filter, a conversation filter, an attachment filter, a persistence filter, a trading filter, an anchorage filter, an altruism filter, a sanity filter, a greed filter, a social filter, a romance filter, a negotiation filter, an occupation filter, an economy filter, and a learning filter.
21. The method of claim 12, wherein the computer game is a massive multi-player on-line game.
22. A computer-readable medium including computer-executable instructions for performing a method of defining an artificial player character for a computer game, the method comprising:
- identifying a set of at least two probability-related values which determine an activity of the artificial player character, including, for each of the probability-related values, assigning a filter value to a preference filter,
- wherein the filter value is an integer value and defines how relevant the preference filter is to an encounter of the artificial player character during the computer game.
Type: Application
Filed: Feb 26, 2008
Publication Date: Aug 28, 2008
Inventor: THEODORE BEALE (Zug)
Application Number: 12/037,917
International Classification: A63F 9/24 (20060101);