ARTIFICIAL PLAYER CHARACTER FOR MASSIVE MULTI-PLAYER ON-LINE GAME

Abstract

An artificial intelligence system for autonomous artificial player characters in an on-line game is provided. The artificial player character (APC) makes use of an adaptive neural net wherein the function of the artificial player character at any given time is determined as a composition of other functions defined by one or more of the following: the game designer, the personality preference filters of the artificial player character, the current statistical levels of the artificial player character, the database of historical interactions of the artificial player character, a master database of historical interactions by other artificial player characters, and interactions of the artificial player character with other characters and/or the game environment.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application Ser. No. 60/903,381, filed on Feb. 26, 2007, and incorporated herein by reference. This application is related to U.S. Non-Provisional patent application Ser. No. ______, filed on even date herewith, having attorney docket number B575.101.102, and incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates generally to a form of an autonomous artificially intelligent software entity for a computer game. More specifically the invention relates to a form of an autonomous artificially intelligent player character for a massive multi-player on-line game.

BACKGROUND OF THE INVENTION

Massive multi-player on-line (MMO) games are computer games capable of supporting a number of players (e.g., hundreds or thousands of players) simultaneously. In one arrangement, players log onto a central server over the Internet and participate in a gaming experience shared by numerous remotely connected users. As such, MMO games are usually played via remote clients accessing one of a number of central servers. Typically, MMO games are played in a large persistent virtual environment that enables players to compete with and against each other on a grand scale and to interact cooperatively with people around the world.

In one application, MMO games provide a form of electronic entertainment which allows numerous individuals to simultaneously take part in an ongoing virtual world. These games are customarily set in a fantasy role-playing environment, although they can theoretically be set in any conceivable fictional genre, historical setting, or other environment. The usual way in which the player experiences the MMO game is to create an avatar, or Player Character, which then travels throughout the virtual world and interacts with a wide variety of characters which are either controlled by other players or by the game server.

The Player Character customarily spends most of its time improving itself through the collection of experience points, which are earned, for example, through killing monsters and completing adventures assigned to the Player Character by server-controlled Non-Player Characters (NPC) scattered throughout the virtual world in strategic locations.

FIG. 1 demonstrates the usual decision-making process for a human-controlled Player Character presented with the opportunity to engage in an adventure of an MMO game. The adventure structure and the basic form of the interaction between Player Characters and Non-Player Characters is generally a simple form of offer-and-reward which is contingent, for example, upon the player killing a certain number of monsters, collecting a certain number of items, or traveling from one point to another.

The decision-making utilized in this process naturally requires human involvement in order to make the Player Character's decision to embark on the adventure in search of the promised rewards or to reject it. This is a somewhat more complicated process than it might look at first, since it requires balancing a number of variables which include, for example, the perceived ability of the Player Character to find Monster Y, the amount of time required to do so, the risks of travel to Monster Y's location, the ability of the Player Character to kill x number of Monster Y, the likelihood that the Player Character will survive the encounter, the trustworthiness of the NPC to make good on its end of the bargain, and finally, the value of the reward on offer to the Player Character.

A similar process is required for a wide variety of interactions between Player Characters and NPCs, Monsters, and the environment itself. These include, but are not limited to, travel from point A to point B, economic production, economic transactions, character allegiances and alliances, and combat. With the sole exception of combat, wherein an attack on a non-autonomous character type will inevitably be responded to by an attack in self-defense, these processes inevitably require at least one of the two parties involved in the interaction to be an autonomous human-controlled character.

As outlined in the Table of FIG. 2, MMO games often contain six distinct variants of game characters, which can be divided into two categories, autonomous and non-autonomous. These variants can be further differentiated on the basis of their control mechanism and state.

1. Player Characters are human-controlled characters who take active part in the on-line game. Also known as “avatars” they are autonomous, travel freely about the game environment subject only to the limits set by the game designers, and their attributes evolve over time. Their statistics, appearance, skill sets, and possessions are dynamic, constantly changing over the course of the game. Player Characters are only present within the game when the human player controlling them is on-line.

2. Administrative Characters are human-controlled characters who only enter the on-line game in order to deal with specific problems, usually those brought to the attention of the game's managers by the users. Also known as “DMs” or “Game Masters”, these Administrative Characters are autonomous and serve police and technical support functions for the participating users, however their attributes are static as they exist primarily for specific game management purposes. They are very seldom encountered.

3. Non-Player Characters are server-controlled characters who primarily exist in order to provide information, supplies, and rewards for the Player-Characters. Usually called “NPCs”, they are non-autonomous, seldom move of their own volition outside of a strictly limited area (if at all), and their attributes do not evolve over time. Their statistics, appearance and skill sets are static, their possessions may or may not be dynamic; if their possessions are variable, the variable alters solely on the basis of actions on the part of the Player Characters. Their presence is typically constant within the game although their appearance may occasionally be made variable by the use of a trigger based on a Player Character's action.

4. Monster Characters are server-controlled characters who primarily serve as the generic opposition for the Player Characters. Often described generically as “enemies” they are non-autonomous, their movement is strictly circumscribed and their basic attributes do not vary over time although their statistics and state will invariably alter radically in the course of a combat encounter with a Player Character or a Pet. Their presence and location are usually constant and unchanging within the game, subject only to their temporary elimination by a Player Character or the failure of a Player Character to trigger their presence. Their statistics, appearance, and skill sets are essentially static, and their decision-making capabilities are crude and essentially limited to a binary option of fight or flight.

5. Pets are a special form of Non-Player Character which occasionally accompany certain types of Player Characters. They are server-controlled partially-autonomous characters which unlike other NPCs, possess dynamic statistics and skill sets as well as the ability to move about the game environment by following around the Player Character to whom they belong.

6. Environmental Characters are server-controlled characters who primarily serve a decorative purpose within the game. They are non-autonomous, their movement is typically limited, and they seldom have significant interaction with the Player Characters.

In the field of Massive Multi-player On-line games, the division between the activities of the Player Characters and the other five character forms is distinct and easily defined. Player Characters voluntarily engage in goal-oriented adventures, economic activity, and combat, develop skills, abilities, talents, and occupations along elective paths, and obtain experience, reputation, money, and physical possessions. In contrast, none of the five other character forms are able to do any of these things as without the direction of a human mind providing purpose and personal preferences for the character, they simply do not have the capacity.

Even the sole other dynamic character type, the Pet, is wholly dependent upon the ability of the owning Player Character's human mind to make decisions for it as its movements are based upon the Player Character to whom they belong. For example, the Pet's dynamic statistics and skills are chosen by the owning Player Character, and even its semi-autonomous ability to initiate combat is completely at the discretion of the Player Character.

And while MMO games are rightly very popular in the gaming community, they are already running into a fundamental design limitation due to their heavy reliance upon Player Characters to provide the dynamic aspect of the entertainment. Since MMO game designers have limited control over the decision-making processes of the participating Player Characters and are forced to leave the vast majority of these decisions to the discretion of human beings controlling the Player Character, they are forced to spend an inordinate amount of time limiting an individual Player Character's ability to destroy the gaming experience for himself and others. Every aspect of the design, from the storyline to the physics model, is forced to take into account the possibility that any player may, at any time, elect to behave in a manner detrimental to the designer's purpose for the game.

This discretion is often exercised to the disadvantage of other Player Characters and to the detriment of the game experience. For example, it is not uncommon for a Player Character to find himself suddenly abandoned in the midst of a horde of enemies because the other Player Characters with whom he was cooperating have departed the scene, either because it was necessary for them to go off-line or simply because they had already achieved their goals and were uninterested in continuing to help the Player Character. This occurs frequently in current MMO games and severely detracts from the game's entertainment value.

Furthermore, the wide variety in maturity levels and sophistication among the players of an MMO game means that Player Characters are often interacting with each other under the handicap of possessing mutually incompatible entertainment goals.

For these and other reasons, there is a need for providing MMO game designers with a mechanism for greater control over Player Characters in a massive multi-player on-line game.

SUMMARY

One aspect of the present invention provides an artificial player character for a computer game as illustrated and described herein.

One aspect of the present invention provides a method of defining an artificial player character for a computer game as illustrated and described herein.

One aspect of the present invention provides a computer-readable medium including computer-executable instructions for performing a method of defining an artificial player character for a computer game as illustrated and described herein.

One aspect of the present invention provides a computer game including an artificial player character as illustrated and described herein.

One aspect of the present invention provides a method of playing a computer game including an artificial player character as illustrated and described herein.

One aspect of the present invention provides a computer-readable medium including computer-executable instructions for performing a method of playing a computer game including an artificial player character as illustrated and described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a process diagram illustrating an existing decision-making process for a human-controlled Player Character in an MMO game.

FIG. 2 is a table outlining character variants for existing game characters in an MMO game.

FIG. 3 is a diagram illustrating one embodiment of a neural net illustrating one embodiment of AI-based decision-making for an Artificial Player Character.

FIG. 4 is a diagram illustrating one embodiment of a model of an Artificial Player Character.

FIG. 5 is a diagram illustrating another embodiment of a model of an Artificial Player Character.

FIG. 6 is a process diagram illustrating one embodiment of decision making for interaction of an Artificial Player Character.

FIG. 7 is a process diagram illustrating one embodiment of determining which Preference Filters of an Artificial Player Character are relevant and how they are subsequently applied.

FIG. 8 is a diagram illustrating one embodiment of determining which Preference Filters of an Artificial Player Character have priority.

FIG. 9 is a table illustrating one embodiment of Preference Filters for an Artificial Player Character.

FIG. 10 is a table illustrating one embodiment of a portion of a database of exemplary encounters of an Artificial Player Character.

FIG. 11 is a process diagram illustrating one embodiment of updating a database of encounters of an Artificial Player Character.

FIG. 12 is a process diagram illustrating one embodiment of an exemplary encounter by an exemplary Artificial Player Character.

FIG. 13 is a process diagram illustrating one embodiment of decision-making for actions of an Artificial Player Character.

FIG. 14 is a process diagram illustrating one embodiment of triggering actions for a character in an MMO game.

FIG. 15 is a process diagram illustrating one embodiment of incorporating an Artificial Player Character in an MMO game.

FIG. 16 is a process diagram illustrating another embodiment of incorporating an Artificial Player Character in an MMO game.

DETAILED DESCRIPTION

An Artificial Player Character (APC) is a client-side or server-side software entity which possesses a specific identity within a Massive Multi-player On-line (MMO) game and is capable of fully autonomous action within the virtual gaming world. In one embodiment, the Artificial Player Character is an artificially intelligent (AI) player character capable of simulating learned behavior and of propagating itself through the creation of distinct entities which share aspects of its core elements.

When a plurality of players play a Massive Multi-player On-line game, they encounter two types of characters, player characters and non-player characters. Player characters are controlled by humans and therefore possess a wide variety of motivations and objectives, while non-player characters possess only response-based motivations and static objectives assigned to each individual non-player character by the programmers of the MMO game. A process for creating proactive motivations and dynamic objectives for non-player characters allows non-player characters to independently determine their immediate and long-term objectives based on unique preferences derived from their virtual personality, then act proactively on the basis of those objectives for the duration of their existence within the MMO game environment.

FIG. 3 is a diagram of a neural net illustrating one method of AI-based decision-making for an action of an APC. For example, from inputs “X” and “Y”, filters F1, F2, and F3 produce an APC Action 10. In a game environment, an APC frequently faces multiple simultaneous inputs, wherein, for example, X represents the opportunity of the APC to have a conversation with a nearby blacksmith NPC, Y represents the proximity of an auction house, and Z represents the state of the player-character's armor. The filters are values standing for the APC's unique preferences, wherein, for example, F1 represents the APC's aggressiveness, F2 represents its spending preference, and F3 represents its racial tolerance. Facing the same inputs, in one embodiment, an APC with an F1 higher than F2 and F3 will elect to talk to the blacksmith to get its armor repaired, whereas an APC with a high F3 will not talk to the blacksmith, if the blacksmith is of a different racial type than the APC.

In one embodiment, an APC is comprised of a set of game or character statistics for the APC, a set of graphical images for the APC and a dynamic list of “possessions” of the APC which may be utilized within the game, and a set of two or more probability-related values used for the purpose of determining a personality trait and the corresponding behavior or activity of the APC. The APC may also possess a database, such as an experience database, containing the record of historical encounters and the results of those encounters (see, e.g., FIG. 10).

The set of two or more probability-related values used for the purpose of determining the activity of the APC are referred to as Preference Filters. A Preference Filter is defined as the APC-specific probability-based preferences related to a specific group of potential game interactions or encounters. One example of a Preference Filter, such as Preference Filters 20 of FIGS. 4 and 5, includes a relevance or Filter ID 22 paired with a variable integer or Filter Value 24 and related to a specified behavioral trait. In this example, the relevance or Filter ID 22 and the corresponding variable integer or Filter Value 24 may be used by the game designer to determine how relevant the Preference Filter 20 is to any potential game interaction or encounter of the APC. The variable integer can be of any value, and in one exemplary embodiment will range between one and 99. Accordingly, in an MMO game making use of twenty Preference Filters, a range of variable integers from one to 99 would allow for a near infinite (20 to the 99th power, specifically) variety of APCs with uniquely individual personality traits and corresponding behaviors or activities. Examples of Preference Filters for an APC are described below with reference to FIG. 9.

FIGS. 4 and 5 illustrate two possible models of an Artificial Player Character. The Artificial Player Character model of FIG. 4 provides one embodiment of a model of an Artificial Player Character 30 without a learning capacity, and the Artificial Player Character model of FIG. 5 provides one embodiment of a model of an Artificial Player Character 30′ with a learning capacity.

As illustrated in FIG. 4, one embodiment of a model of an Artificial Player Character 30 contains the basic elements of Preference Filters 20, Character Statistics 40, and Graphics and Possessions 50. In one embodiment, Preference Filters 20 include Filter IDs 22 and corresponding Filter Values 24, Character Statistics 40 include statistics 42 and corresponding values 44, and Graphics and Possessions 50 include items 52, and corresponding attributes 54 and values 56. As illustrated in FIG. 5, in addition to the basic elements of Preference Filters 20, Character Statistics 40, and Graphics and Possessions 50, another embodiment of a model of an Artificial Player Character 30′ contains the additional element of an Experience Database 60. In one embodiment, Experience Database 60 includes Experience IDs 62, and corresponding descriptions 64 of the experiences and average values 66 of the experiences. The Experience Database, as described below, gives the APC a capacity for learned behavior based on past encounters (i.e., a simulated learning capacity).

Character Statistics, such as Character Statistics 40 of FIGS. 4 and 5, describe the APC's capabilities within the game environment, and may include attributes such as height, weight, experience level, strength, speed, intelligence, charisma, stamina, constitution, attack power, defense rating, hit points, experience points, magical mana and so forth of the APC. This allows for a wide range of unique capabilities from one APC to the next as well as a measure to compare the relative abilities of one APC to another.

While Character Statistics describe what an APC can do, Graphics and Possessions, such as Graphics and Possessions 50 of FIGS. 4 and 5, indicate what an APC looks like and what an APC owns. Graphics include a visual representation of the APC as it is viewed within the game environment, while Possessions may include money as well as the various weapons, armor, trinkets, consumables, quest materials, and other items that may be of use or economic value to the APC.

In one embodiment, Possessions contain both Attributes and dynamic Values, such as Attributes 54 and Values 56 of FIGS. 4 and 5. A specific possession, such as a shield, for example, may possess several Attributes such as increasing various Character Statistics or conveying other benefits, and may also possess both a monetary value as well as a use value to the specific APC. For example, a shield possessed by an APC that is of the wrong class to make use of it may possess zero use value, but may not be valueless to the APC due to its monetary value.

In one embodiment, both Character Statistics and Possessions are individually tracked and monitored, and taken in combination with various environmental and inter-player encounters, represent potential Inputs for the APC in certain situations. For example, an encounter with an armor vendor may represent a potential Input for an APC if the vendor is (a) selling the type of armor worn by the APC, (b) is selling armor that would increase the APC's Character Statistics (in this case, Defense Rating), and (c) selling the armor for less money than the APC possesses.

FIG. 6 illustrates one way in which the APC makes a decision regarding an encounter or interaction presented by a Player Character, an NPC, another APC, a Monster Character or the environment of the game world. For example, after the presented encounter or interaction, as illustrated at 102, applicable preference filters are determined by relevance, as illustrated at 104. After passing through the relevant Preference Filters (e.g., F2, F3, F4), Preference Filter priorities are calculated and applied, as illustrated at 106. As such, from the Preference Filter having priority (e.g., F2), an APC action is determined, as illustrated at 108.

FIG. 7 illustrates one way in which the APC can determine which of its Preference Filters are relevant to the decision-making process and how they are subsequently applied, and FIG. 8 illustrates one way in which the APC can determine which of the Preference Filters that are applicable to the current decision have priority in the decision-making process.

As illustrated in FIG. 7, in one embodiment, when an APC receives an alert, which can be based on the APCs proximity to another character or environmental object, a time-based alarm, or a statistically-based warning, as illustrated at 202, the specific type of encounter is identified by its Encounter ID Tag, as illustrated at 204. In one embodiment, checking this Encounter ID tag against the Experience Database allows the APC to save computational cycles by overriding the Filter system and automatically determining an action if an override is relevant, as illustrated at 206 and 208. If not, the Encounter ID tag will dictate the specific Filters which are relevant to the situation, as illustrated at 210 and 212. In one embodiment, for simple encounters and more predictable action, the relevant filter with the largest value is selected to determine the APC action, as illustrated at 214 and 216. In one embodiment, for more complex encounters, the values of the relevant filters are calculated as relative probabilities, as illustrated at 218, any relevant modes from the Experience Database are applied, as illustrated at 220, and a random number is generated, as illustrated at 222, to select between the various probabilities to determine the APC action, as illustrated at 224.

As an example of the decision-making process of FIG. 7, due to a fall from a height, for example, an APCs health might drop below 20 when the APC is not under attack, thus triggering a warning that the APC is about to die. The Encounter ID tag indicates that the Experience Database should check for overrides, and the override indicates that the APC should drink a health potion if it has one, thus dictating the APC's action. In a more complicated encounter, if the APC's health drops below 20 due to an unexpected attack from a powerful enemy nearby, for example, the Encounter ID's lack of an override may indicate that the relevant Filters in this Encounter are Risk-Taking, Aggression, and Sanity. High values for the first two filters cause the APC to recklessly attack despite the low probability of survival, whereas low values or a high value for Sanity, except in the case of an unlikely random probability determination, cause the APC to take action to increase its chance of survival by running away.

FIG. 8 illustrates one way in which the APC can determine which of the Preference Filters that are applicable to the current decision have priority in the decision-making process. When an Encounter takes place, as illustrated at 302, the appropriate Encounter ID tag provides the relevant Filters to the APC, as illustrated at 304 and 306. The process then ignores all other Filters and returns the numerical values only for the relevant Filters in order to determine the APC's action in the situation, as illustrated at 308 and 310. The result may be computed either by a straightforward comparison or by converting the values to relative probabilities and selected by a random probability determination, which may be further altered by applicable modifiers from the Experience Database.

FIG. 9 illustrates one embodiment of a potential set of Preference Filters 20, their function 26, their relevance or Filter IDs 22, and their Filter values 24 for an exemplary APC. In one embodiment, the potential set of Preference Filters 20 include a loyalty filter, an ambition filter, an exploration filter, an adventure filter, a risk-taking filter, an aggression filter, a conversation filter, an attachment filter, a persistence filter, a trading filter, an anchorage filter, an altruism filter, a sanity filter, a greed filter, a social filter, a romance filter, a negotiation filter, an occupation filter, an economy filter, and a learning filter. This structure of Preference Filters allows for a vast range of divergent behaviors. For example, an APC with high values assigned to Preference Filters for Aggression, Attachment, and Altruism is likely to immediately launch an unprovoked attack on anything that might be interpreted as posing a threat to an individual in the group with which it is identified, whereas an APC with low values assigned to Preference Filters such as Risk-Taking, Attachment, and Social would not be inclined to join a group in the first place, nor risk its own safety in defending other members of it. The disclosed set of Preference Filters is one example of one potential set of Preference Filters, and may include or encompass other filter criteria.

The Filter values 24 for an APC's Preference Filters 20 may be assigned in various ways. In one embodiment, static values for Preference Filters of an APC are determined by the game designer or other individual. In another embodiment, values for Preference Filters of an APC are assigned by a random process. Assigning values by a random process would allow for the widest variability of APC behavior, although some of the behaviors may prove to be undesirable for the gaming experience.

In one embodiment, values for Preference Filters of an APC are assigned by a “learned” process. In the “learned” process, the APC's Preference Filters are permitted to evolve by means of access to a database containing the record of historical encounters and the results of those encounters. In one embodiment, additional weight is given to Preference Filters whose use is “effective” as defined by the game designer, and less weight is given to Preference Filters whose use proves to be “ineffective” as defined by the game designer.

In one embodiment, assigning APC Preference Filter Values for an APC is by an “evolutionary” method wherein two or more APC databases and Preference Filter sets are combined to produce a “child” APC with its own distinct Preference Filter values. This method can produce Artificial Player Characters that evolve into a more interesting and effective opposition over time instead of simply respawning as perfect clones of Artificial Player Characters that were eliminated from the game on a previous occasion. In addition, this method can produce Artificial Player Characters as Non-Player Characters that will interact with the Player-Characters in a more complex and entertaining manner.

The Preference Filters provide the APC with a simulated and unique set of motivations that can be applied to every game encounter, thus providing a wide range of potential variability within the context of the encounters. Therefore, the game designer may want to define every variant of encounter in terms of the Preference Filters to which it is relevant. For example, an encounter between a Race X class 1 (Dwarf Warrior) and a traveling Race X class 6 (Dwarf Merchant) may be tagged with specific relevance IDs for Loyalty, Attachment, Conversation, and Trading, whereas a subsequent encounter by the same Race X class 1 with a Race Y class 1 who specializes in warrior training may be tagged with specific relevance IDs for Ambition and Aggression.

In one embodiment, when the game queries the APC regarding its prevailing attitude towards the imminent encounter, (which will usually be triggered via a basic proximity setting), a first step is to determine which of the relevance IDs are relevant to the situation at hand (see, e.g., FIG. 6). In the case of the encounter described above, for example, which might be titled “Proximity of a traveling Dwarf Merchant,” the specific IDs mentioned above (i.e., Loyalty, Attachment, Conversation, and Trading) would be selected. In addition, the general IDs deemed relevant to all APC-NPC encounters, in this case perhaps Aggression, Sanity, and Social, may also be selected.

In one embodiment, once the relevant Preference Filters have been identified, the game will then compare the relative strength of the competing interests based on their assigned values and calculate a decision regarding the APC's action (see, e.g., FIG. 6). This calculation may include anything from a simple comparison of the values followed by application of whichever value happens to be the largest to a complex fuzzy logic application which introduces a random element into the process and combines the various values of the Preference Filters to produce a variable result.

Using the comparison method for the encounter described above, and positing, for example, a value of 99 for Conversation and a value of 33 for all the other relevant Preference Filters, the game will ascertain that the APC has a propensity to speak with the traveling Dwarf Merchant and will attempt to strike up a conversation with him. This APC action may lead to a new encounter, “Conversation with a Dwarf Merchant,” and inspire the game to engage in a second round of relevance ID checks. In one embodiment, this second round of relevance ID checks may include an inquiry into Negotiation as well as an inquiry into the APC's present material requirements as indicated by the state of its current possessions.

FIG. 10 is an example of a database, such as Experience Database 60, which records the results of an APC encounter, catalogues the APC encounter according to an encounter ID, and averages the results for use in evolved APC behavior, and FIG. 11 illustrates one embodiment of updating the database of encounters for the APC.

As illustrated in FIG. 10, one embodiment of Experience Database 60 includes Experience IDs 62, corresponding descriptions 64 of the experiences, a number 63 of such experiences, a previous result 65 of such experiences, and an average value 66 of such experiences, and an over/under value 67 for the experience, and an under action 69 of the APC for the experience. As such, Experience Database 60 records the results of an APC encounter, catalogues the APC encounter according to an encounter ID, and averages the results for use in future APC encounters to simulate learning behavior, with each encounter type having a specific ID as well as a coded description.

In the embodiment of FIG. 10, CILO indicates Combat against 1 opponent of the same level, whereas 13L6 indicates an Invitation to join a group that contains characters an average of 6 levels higher than the APC. The # sign indicates the number of such encounters the APC has already experienced, Previous Result rates the previous encounter on a positive scale from 0 to 5, and AVG is the average result for all of the previous encounters of that type. In the embodiment of FIG. 10, the 425 combat encounters with a single character (4.2) have turned out much better than the two previous invitations to join a group with higher level characters (1.0). O/U represents the Over/Under for the average of previous encounters that trigger the override when the next encounter takes place, and U indicates the override-dictated action. Because the APC does not have complete control over whether an encounter type will take place or not, the overrides will not prevent additional information being added to the Experience Database.

FIG. 11 illustrates one embodiment of the way in which the Experience Database of the APC's encounters is updated. In one embodiment, when the encounter is complete, as illustrated at 402, a scored result for the encounter is assigned a numerical value according to a predefined metric for the encounter type, as illustrated at 404. This result is recorded and averaged with the previous results, as illustrated at 406, and the database is updated to reflect these modified results, as illustrated at 408. For example, if an APC encountered a Dragon that was 10 levels higher, but was unable to flee even though the override indicated that he should and wound up killing the Dragon, the 5 result might be enough to change the average encounter score above the O/U value. The next time the APC encounters a monster that is 10 levels higher, instead of the Experience Database override telling him to flee, the APC would consult its preference filters. This thus allows for dynamic “learned” behavior that varies depending upon past results.

The database containing the record of historical encounters and the results of those encounters is referred to as the APC's Experience Database. In one embodiment, an experience database, such as Experience Database 60 (FIG. 10), is maintained for only those APCs which exhibit “learned” behavior that modifies the particular application of the Preference Filters over time according to the results of past applications. The results of each encounter by the APC are stored in its Experience Database for future reference by the game in modifying the application of the Preference Filters or modifying the Preference Filters themselves in the case of an evolutionary “learned behavior” approach, for the individual APC or for use in creating more effective future APCs. In one embodiment, an override trigger capable of overriding the action dictated by the Preference Filters may be provided.

In one embodiment, the Experience Database functions by assigning a value expressing the results of the encounter and storing it according to the appropriate encounter type. For example, a combat encounter between a level 12 APC and three level 15 Orcs in which the APC killed two Orcs before being forced to run away from the third will be given an encounter ID, summarized in the database as C3L3, and may be assigned a result value of 2 (defeat, hit/damage ratio<0.5) on a hypothetical scale of 0-5. This results scale, of course, may be set to any level of detail, for example, from binary to near infinite.

Accordingly, with this example, each time the APC engages in a similar encounter, regardless of whether it is a level 2 character fighting three level 5 Farm Pigs or a level 50 character fighting three level 53 Iron Hellknights, the results would be recorded and averaged with the results from previous encounters of the same type. The greater the number of encounters stored in the Experience Database, the more weight it would be provided as a modifier applied to the Preference Filters.

In one embodiment, based on the encounters stored in the Experience Database, the Preference Filter value itself may be modified. For example, an APC which regularly records results of 0 (death, hit/damage ratio<0.25) for a specific encounter ID will eventually “learn” to exert a powerful modifier to avoid engaging in that encounter type regardless of how high its Preference Filter value for Risk-taking or Aggression might be. The extent to which this modifier influences the Preference Filters and, therefore, the APC's actions may be defined by the game designer.

FIG. 12 illustrates an example of a specific encounter by an exemplary APC. If an APC dwarf happened to be engaged in mining gold and was encountered by a normally hostile orc of a similar level that did not attack, as illustrated at 502, the APC would be free to consider its next action. Since the mere presence of a hostile character would not be enough to trigger an override, the Preference Filters are queried and consulted, as illustrated at 504, and those determined to be most relevant to the Encounter type based on the ID tag (i.e., Aggression, Conversation and Persistence) are returned, as illustrated at 506 and 508. The competing Filter Values are then calculated, as illustrated at 510. In one embodiment, the APC's level of Conversation (90), is more than three times higher than either Aggression or Persistence, thus leading to a 15 percent probability of a response dictated by Aggression, a 67 percent probability of a response dictated by Conversation, and an 18 percent probability of a response dictated by Persistence, as illustrated at 512. At this point, any situational mods are applied, as illustrated at 514, and a random probability determination produces a result of 87, as illustrated at 516. As this result falls into the top 18 percent, the action of Persistence is applied such that the APC ignores the orc and continues mining, as illustrated at 518. The largely positive results of the encounter are then computed and stored in the APC's Experience Database, as illustrated at 520.

FIG. 13 is a process diagram demonstrating one way in which an APC can make decisions about its actions regarding changes to its dynamic statistics using a combination of pre-set triggers and Preference Filters. A character statistic can change for a variety of positive and negative reasons, from simple time passing to receiving damage in battle, collecting items of economic value, and successfully accomplishing a task. When a character statistic changes, as illustrated at 602, the APC will check to see if there is a pre-set override attached to that statistical change, as illustrated at 604. Statistics may change for combat or non-combat reasons. For example, an APC with an alchemy skill improves that skill by successfully brewing a potion. If, for example, the threshold required to learn a new potion deemed sufficiently important by the game's designers has been crossed, the APC will take action to go to the nearest Alchemy trainer and learn that new potion. If the change causes the level to fall below the pre-set trigger, as illustrated at 606, the process continues, as illustrated at 608 and described below with reference to FIG. 14, for example, based on whether the change is for combat or non-combat reasons. If there is no pre-set threshold, the APC may or may not take action depending on its unique Preference Filters, as illustrated at 610, 612, 614, 616, 618, and 620, similar to that described above with reference to FIG. 12.

FIG. 14 is a process diagram demonstrating one way to determine if a change causes the level to fall below a pre-set trigger. In one embodiment, following a character statistics change, as illustrated at 702, the process is divided, for example, into combat and non-combat responses, as illustrated at 704 and 706, because combat situations present unique danger to the APC and may require a different response. In one embodiment, if an APC is damaged by an enemy attack to such an extent that its hit points fall below a pre-set level, as illustrated at 708, the APC will check to see if it possesses any means of restoring that statistic, as illustrated at 710, such as a health potion or a restorative spell. If the APC possesses the means, it will perform the action that will restore the statistic to the highest possible level, as illustrated at 712. For example, if the APC has a minor health potion and a major health potion, it will drink the major one, and then respond as determined by its Preference Filters. If, on the other hand, the APC possesses no means to restore the statistic, the APC, for example, will flee from the attacker, as illustrated at 714. The process for statistical changes that are non-combat-related, as illustrated at 706 and 716, is similar, except that the APC will not flee if it has no means of improving the statistic, instead it will simply continue with what it was doing, as illustrated at 718.

FIG. 15 is a process diagram illustrating one embodiment of monitoring the formation of multiplayer groups and utilizing APCs as player-character substitutes in order to form functional multiplayer groups in the absence of sufficient player characters. In one embodiment, an APC Manager monitors the formation of multiplayer groups, as illustrated at 802. In one embodiment, a player announces formation of a multiplayer group, as illustrated at 804. If other players join the group within the specified time, as illustrated at 806, the group embarks on and completes the multiplayer quest, as illustrated at 808. If, after a pre-determined amount of time passes, an insufficient number of players required to complete the group quest join the group, as illustrated at 810, the APC Manager determines the appropriate APC types and levels required to complete the group, as illustrated at 812, and assigns the correct number of APCs of a suitable level to the group, as illustrated at 814, in order to allow one or more player-characters to engage in group quests regardless of the number of interested and available human players. Thereafter, the group, including the APC(s), embarks on and completes the multiplayer quest, as illustrated at 816.

FIG. 16 is a process diagram illustrating the current result of multiplayer group breakdown and comparing this with one embodiment of monitoring multiplayer group rosters and allowing multiplayer groups to avoid breakdown due to the loss of one or more members by substituting Artificial Player Characters when player characters exit the group, and removing Artificial Player Characters when player characters subsequently become available. In one embodiment, an APC Manager monitors the multiplayer groups, as illustrated at 902. In one embodiment, a group of human players embark on a multiplayer quest, as illustrated at 904. Traditionally, if one or more player characters drop out in mid-quest, as illustrated at 906, the quest cannot be completed and is abandoned, as illustrated at 908. By monitoring the multiplayer groups, however, the APC Manager determines the appropriate APC types and levels required to complete the group, as illustrated at 910, and assigns the APCs to replace the drop-outs, as illustrated at 912, if one or more player characters drop out in mid-quest. Accordingly, the quest can be completed with the APC(s) taking the place of the player character(s) that dropped out.

In one embodiment, as the quest continues, the APC Manager monitors PCs looking for a group, as illustrated at 914. If a player is looking for the type of group already existing and including one or more APCs, the APC Manager will offer the player character the choice to join the existing group in mid-quest, as illustrated at 916. If the player character wishes to join the existing group, the APC Manager will remove the APC most similar to the new player character, and replace the APC with the player character, as illustrated at 918. Accordingly, the quest can be completed, as illustrated at 920, with the player character taking the place of the previously assigned APC.

In one embodiment, the APC Manager allows multiplayer groups to complete multiplayer quests that would otherwise be abandoned when members drop out in mid-quest. This is accomplished by substituting appropriate APCs for the missing player characters whenever a group member exits the game or the group, thus allowing the remaining player characters to continue with their quest in the company of the APC(s). Because the APC Manager actively monitors PCs looking for groups, it is aware when an individual player character is in search of a similar group or quest, at which point the APC Manager will offer the player character the choice to join the multiplayer group in mid-quest. In one embodiment, if the player character wishes to join the group, the APC Manager will remove the APC most similar to the new player character, replacing it with the player character.

In addition to simulating intelligent decision-making by the APC regarding its actions when presented with encounters within the game, the present invention also provides a means of simulating more intelligent tactical decision-making with regards to the APC's use of various items and possessions within the context of those encounters. For example, in a conventional MMO game, a wounded monster will usually attempt to heal itself to the full extent of its capacity regardless of how much time it possesses. With the more sophisticated possibilities of the present invention, an APC would have the capability of determining the optimal balance of health and time required to give it the greatest possibility of surviving the encounter.

The present invention enhances the ability of the game designer to exert influence over the game events, encounters, and interactions experienced by the participating users. It also provides the game designer with an effective means of implementing the continuous evolution of artificial intelligence within a massive multi-player on-line game and, in one embodiment, offers the prospect of eliminating the human element entirely.

The present invention provides a system for the introduction of autonomous, dynamic artificial player characters which do not require human direction and are not subject to human control to the field of massive multi-player on-line games. These Artificial Player Characters serve as a substitute for human-controlled Player Characters which are less subject to the negative aspects of human behavior and decision-making that detract from the game's ethos, and provide the MMO game designer with an enhanced ability to create an interesting and enjoyable game experience for the human players as they encounter the virtual world.

The autonomous Artificial Player Character of the present invention is more complex than the automated combat scripts (“bots”) which are used in many 3D shooters and is applicable to a much broader range of human/AI and AI/AI interactions than mere combat. It is also substantially different from the utilities illegally used in some MMO games which automate tasks for a Player Character, because all of the activities performed by the utilities require human decision-making and initiation.

The present invention also makes technologically feasible the possibility of a massive multi-player on-line game which requires neither multiple players nor on-line access. For example, with application of the APC to an MMO game, it becomes theoretically possible to create a dynamic massive multi-player on-line game without requiring either human players or on-line access. This has interesting utility in non-gaming applications wherein it may be considered desirable to have access to an evolutionary process from inside the process itself.

One embodiment of the present invention provides a system for the implementation of an artificial player character in a massive multi-player on-line game. The system may include a software entity with a unique, artificially intelligent capacity for decision-making regarding every form of encounter it experiences in the virtual game world, including but not limited to, adventuring, combat, conversation, exploration, economic activity, questing, and trading. As such, the entity may possess complete autonomy within the virtual game world. In one embodiment, this autonomy is subject to the discretion of the game designer or a human player to whom ownership of the artificial player character has been granted. In addition, the entity may possess a status equivalent to human-controlled Player Characters within the virtual game world.

One embodiment of the present invention provides a system for the implementation of learned behavior by artificial player characters, non-player characters, and monsters in a massive multi-player on-line game. The system may include a set of values that dictate the preferred behaviors of an individual artificial player character, non-player character, or monster, and a database containing a record of past encounters and the results of those encounters. The results of the actions dictated by the implementation of the set of values are stored in the database such that the interpretation of the historical results are implemented to modify the set of values for future encounters. In one embodiment, the database may contain provisions for overriding the set of values based upon historical results.

One embodiment of the present invention provides an automated system for creating artificial player characters, non-player characters, and monsters for a massive multi-player on-line game. The system may include the combination of two or more sets of values that dictate the preferred behaviors of the specified artificial player character, non-player character, or monster into a single set of values dictating the preferred behaviors of a new entity.

One embodiment of the present invention provides a system for creating a zero-player off-line version of a massive multi-player on-line game. The system may include a three-dimensional graphical virtual environment entirely populated with artificial player characters which may be visited by non-participatory human-controlled avatars or a limited number of participatory human-controlled player characters.

One embodiment of the present invention provides a process for monitoring the requirements of individual player characters attempting to form multiplayer groups and providing Artificial Player Characters to substitute for player characters when insufficient player characters are available for group formation.

One embodiment of the present invention provides a process for monitoring the requirements of multiplayer groups and substituting Artificial Player Characters for player characters as needed in order to prevent premature dissolution of those groups. This embodiment also encompasses a system of removing APCs from the group and replacing them with human-controlled player characters when interested player characters become available.

Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A computer game, comprising:

an artificial player character comprising a set of at least two probability-related values which determine an activity of the artificial player character,

each of the probability-related values comprising a preference filter including a filter identification and a filter value for the filter identification, wherein the filter value is an integer value and defines how relevant the preference filter is to an encounter of the artificial player character during the computer game.

2. The computer game of claim 1, further comprising:

a player character; and

one or more of an administrative character, a non-player character, a monster character, a pet character, and an environmental character.

3. The computer game of claim 1, wherein the artificial player character further comprises a set of character statistics for the artificial player character, a set of graphics for the artificial player character, and a list of possessions of the artificial player character.

4. The computer game of claim 1, wherein the artificial player character further comprises an experience database including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.

5. The computer game of claim 1, wherein the filter value of the preference filter for the artificial player character is assigned by an individual.

6. The computer game of claim 1, wherein the filter value of the preference filter for the artificial player character is assigned by a random process.

7. The computer game of claim 1, wherein the filter value of the preference filter for the artificial player character is assigned by a learned process including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.

8. The computer game of claim 1, wherein the filter value of the preference filter for the artificial player character is assigned by an evolutionary process combining at least two preference filters.

9. The computer game of claim 1, wherein the preference filter includes one of a loyalty filter, an ambition filter, an exploration filter, an adventure filter, a risk-taking filter, an aggression filter, a conversation filter, an attachment filter, a persistence filter, a trading filter, an anchorage filter, an altruism filter, a sanity filter, a greed filter, a social filter, a romance filter, a negotiation filter, an occupation filter, an economy filter, and a learning filter.

10. The computer game of claim 1, wherein the computer game is a massive multi-player on-line game.

11. The computer game of claim 1, wherein the encounter is presented by one of a player character, a non-player character, another artificial player character, a monster character, and an environment of the computer game.

12. The computer game of claim 1, wherein the artificial player character is introduced into the computer game in place of a player character when the player character leaves the computer game.

13. The computer game of claim 1, wherein the artificial player character is introduced into the computer game when less than a predetermined number of player characters are available to play the computer game.

14. A method of playing a computer game including an artificial player character, the method comprising:

presenting the artificial player character with an interaction;

determining which of a plurality of preference filters for the artificial player character are applicable to the interaction, including comparing a filter value of each of the preference filters to the interaction and determining a relevance of each of the preference filters to the interaction, the filter value defining how relevant a corresponding preference filter is to the interaction of the artificial player character during the computer game;

calculating a priority of the preference filters relevant to the interaction, and applying the relevant preference filters to the interaction;

determining an action of the artificial player character to the interaction based on the relevant preference filters; and

performing the action by the artificial player character.

15. The method of claim 14, further comprising:

identifying a set of at least two probability-related values which determine an activity of the artificial player character, including, for each of the probability-related values, assigning the filter value to the corresponding preference filter,

wherein the filter value is an integer value and defines how relevant the preference filter is to the interaction of the artificial player character during the computer game.

16. The method of claim 15, wherein assigning the filter value to the corresponding preference filter includes assigning the filter value by an individual.

17. The method of claim 15, wherein assigning the filter value to the corresponding preference filter includes assigning the filter value by a random process.

18. The method of claim 15, wherein assigning the filter value to the corresponding preference filter includes assigning the filter value by a learned process including a record of historical encounters of the artificial player character and results of the historical encounters of the artificial player character.

19. The method of claim 15, wherein assigning the filter value to the corresponding preference filter includes assigning the filter value by an evolutionary process combining at least two preference filters.

20. The method of claim 14, wherein each of the preference filters include one of a loyalty filter, an ambition filter, an exploration filter, an adventure filter, a risk-taking filter, an aggression filter, a conversation filter, an attachment filter, a persistence filter, a trading filter, an anchorage filter, an altruism filter, a sanity filter, a greed filter, a social filter, a romance filter, a negotiation filter, an occupation filter, an economy filter, and a learning filter.

21. The method of claim 14, wherein the computer game is a massive multi-player on-line game.

22. The method of claim 14, wherein the interaction is presented by one a player character, a non-player character, another artificial player character, a monster character, and an environment of the computer game.

23. The method of claim 14, further comprising:

introducing the artificial player character into the computer game in place of a player character when the player character leaves the computer game.

24. The method of claim 14, further comprising:

introducing the artificial player character into the computer game when less than a predetermined number of player characters are available to play the computer game.

25. A computer-readable medium including instructions for performing a method of playing a computer game including an artificial player character, the method comprising:

presenting the artificial player character with an interaction;

determining which of a plurality of preference filters for the artificial player character are applicable to the interaction, including comparing a filter value of each of the preference filters to the interaction and determining a relevance of each of the preference filters to the interaction, the filter value defining how relevant a corresponding preference filter is to the interaction of the artificial player character during the computer game;

calculating a priority of the preference filters relevant to the interaction, and applying the relevant preference filters to the interaction;

determining an action of the artificial player character to the interaction based on the relevant preference filters; and

performing the action by the artificial player character.

26. A computer-readable medium including instructions for performing a method of playing a computer game including an artificial player character, the method comprising:

monitoring a multiplayer group of the computer game; and

introducing the artificial player character into the computer game if one of (a) a player character of the multiplayer group of the computer game leaves the computer game and (b) less than a predetermined number of player characters are available to form the multiplayer group of the computer game,

wherein introducing the artificial player character into the computer game includes determining a type and a level of the artificial player character and assigning the artificial player character to the multiplayer group of the computer game.

27. The computer-readable medium including instructions for performing a method of playing a computer game including an artificial player character of claim 26, the method further comprising:

after introducing the artificial player character into the computer game, removing the artificial player character from the computer game and replacing the artificial player character with a new player character if the new player character is looking for a group similar to the multiplayer group of the computer game including the artificial player character.