SYSTEM FOR MULTIDIMENSIONAL ANALYSIS, MODULATION AND OPTIMIZATION OF FACIAL EXPRESSIONS
Apparatus and associated methods relate to method of determining a treatment plan with Botulinum Toxin (BT) and other procedures able to modify facial expressions, the method includes using a series of facial images of a patient eliciting various expressions to select individual expressive features as candidates for modification, the selected expressive features mapping to muscles that are used to create such expressive features, the muscles mapping into related expressive features of facial images of the patient eliciting different expressions, the related expressive features being weighted so as to provide further information used in determining an optimum treatment dosing for each treatment location. In an illustrative example, the method may include mapping the facial image of the patient to a model image using facial markers. In an exemplary embodiment, the method may advantageously provide a treatment that results in optimum expressive features as revealed in many different expressions.
In today's society, the face is essential for human communication, social interactions, facial feedback processes, signaling emotion and the perception of attractiveness and youthfulness. Scientists have long recognized the importance of facial expressions to communicate states of mind—this may include not only emotions, sensations, feelings, moods, personality traits and mental conditions, but also motivational, behavioral and cognitive processes.
Most people want to have pleasant and functional facial expressions in a modern social and communicative environment. Factors such as looking good for one's age, looking relaxed and natural, avoiding frozen looks, and maintaining a functional degree of facial animation are among those deemed desirable. Facial expressions play a big role in perceived physical attractiveness of a person, and people who are considered physically attractive receive preferential treatment in education, employment, medical care, legal proceedings, and romantic encounters that may result in greater happiness and success. The facial expressions of emotions are transmitted normally using static, dynamic, transient and permanent facial components.
However, both hyperactive (hyperfunctional) and hypoactive (hypofunctional) features of facial expressions can also transmit miscues about age, emotions, intentions and other mental states. In other words, facial expressions are central to human communication and the wrong facial expression can convey the wrong social message.
SUMMARY OF THE DISCLOSED TECHNOLOGYApparatus and associated methods relate to method of determining a treatment plan with Botulinum Toxin (BT) and other procedures able to modify facial expressions, the method includes using a series of facial images of a patient eliciting various expressions to select individual expressive features as candidates for modification, the selected expressive features mapping to muscles that are used to create such expressive features, the muscles mapping into related expressive features of facial images of the patient eliciting different expressions, the related expressive features being weighted so as to provide further information used in determining an optimum treatment dosing for each treatment location. In an illustrative example, the method may include mapping the facial image of the patient to a model image using facial markers. In an exemplary embodiment, the method may advantageously provide a treatment that results in optimum expressive features as revealed in many different expressions.
Various embodiments may achieve one or more advantages. For example, some embodiments may automate or semi-automate the generation of a BT treatment plan, alone or in combination with other agents and procedures able to modify facial features. In some embodiments, the generated treatment plan may optimally produce expressive features in response to a wide array of expressions. In an exemplary embodiment, the method may permit a physician of low-skill or modest experience to perform high-quality BT treatment on patients. For example, a physician with modest experience may treat a patient that might otherwise be referred to another more experienced physician. In some embodiments, the treatment dosages may be reduced due to the competing demands of the related expressive features. For example, a patient may desire a certain feature to be reduced while expressing fear, but may not desire to reduce a related expressive feature while expressing delight. An exemplary method may educate the patient, so that the patient feels that the treatment program is directed in part by the patient's own inputs.
Like reference symbols in the various drawings indicate like elements.
DETAILED DESCRIPTIONAs used herein, the term “psycho-physiological state” means a condition experienced by a living being (whether communicated and/or felt) and which may result in a particular facial expression; including but not limited to worry, anxiety, fear, rage, anger, irritation, contempt, disgust, pain, despair, distress, sadness, shame, boredom, fatigue/somnolence, calmness, serenity, friendliness, amusement, relief, delight, pleasure, happiness, love, pride, elated joy, awe, interest, surprise, eyebrow flash greeting, novelty, expectation, goal attainment, power/control, external/internal standards disturbance or intrinsic pleasantness. Such meaningful conditions may be experienced by any animal, including, without limitation, mammals or humans. Thus, the methods herein may be, in certain embodiments, applied to any animal including, but not limited to, mammals or humans.
As discussed herein, a psycho-physiological state is considered “basic expressive” if it has characteristic static, dynamic or temporal activation patterns of Facial Expression Activation Markers (FEAMs) and/or muscles.
Facial Expression Activation Markers (FEAMs)As used herein, the term “Facial Expression Activation Marker (FEAM)” means a basic static or dynamic topographic facial feature that cannot be further divided into smaller functionally meaningful components of facial expressions. A FEAM is an action element —an elementary facial feature whose intensity can be measured, but is otherwise a functionally indivisible anatomic and neuromotor facial component that can help an observer detect and form a hypothesis regarding the identity of the activated muscle(s) and mechanisms responsible for a particular facial expression.
FEAMs have been found herein to be the essential components of modern facial expressions that provide a common ground for multidisciplinary approaches to properly study and modulate human facial function, including communication of aesthetic mimetic and perceptual signals during the coding and decoding of human faces. That is, they are a “common denominator” that can be changed by various agents, procedures, and methods that can be used to modulate facial expressions—for example, laser surfacing can modulate FEAMs by remodeling and altering the properties of the facial skin, fillers can be used to decrease facial depressions or enhance elevations, while selective denervations of muscles (e.g., botulinum toxin, nerve ablations and surgical reconstructive methods) can target distinct sets of topographic FEAMs, but also alter distances between facial landmarks.
In certain embodiments herein, FEAMs can be grouped into functional “sets” based on muscle specificity (Table 1). In certain embodiments herein, each functional set comprises one or more FEAMs, and each functional set corresponds to a muscle activity characterizing a desired facial expression (Table 1;
In certain embodiments, the methods herein are concerned with measuring the degree of expression or activation intensities of a FEAM. That is, a FEAM's degree of expression or activation intensity can be assessed (for example, visually and/or by palpation) according to criteria that are discussed herein. Automatic facial recognition programs able to quantitatively measure some FEAMs can also be used to determine the degree of FEAMs expression. In certain embodiments, a clinician will be able to assess the activation intensities of FEAMs and facial expressions of an individual. The facial expressions of basic expressive psycho physiological states (
Thus, in certain embodiments, the present technology is directed to a method for achieving a desired facial expression on the face of a subject, the method comprising the steps of:
(a) measuring the activation intensity or intensities of one or more facial expression activation markers (FEAMs) of the subject for a desired facial expression;
(b) determining the activation intensity or intensities of one or more optimal FEAMs for the desired facial expression;
(c) grouping the one or more optimal FEAMs into one or more functional sets, each set comprising a desired optimal pattern of FEAMs and each set corresponding to a muscle activity characterizing the desired expression; and
(d) applying an agent or procedure to the face, head or neck of the subject to implement the desired optimal pattern of FEAMs and corresponding muscle activity for the desired facial expression.
In certain embodiments, steps (a), (b) and (c) may include the production, evaluation, and optimization of full face displays and associated perceptual spaces characterizing multiple dimensions of psycho-physiological states, as will be discussed further herein.
This evaluation is further discussed in
In
Thus, in various embodiments, the determination of one or more optimal FEAMs for the desired facial expression comprises analyzing and testing the effects of FEAM changes on the facial expressions of two or more relevant psycho-physiological states.
SREDIC Intensity ScaleIn certain embodiments, the methods herein include the application of the SREDIC Intensity Scale, which has herein been developed to determine and report the functional activation intensities of FEAMs organized by facial muscle specificity. As used herein, “SREDIC” refers to Specific feature, Relative distance, Elevations, Depressions, Interacting features and Coactivated distances.
As used in the methods herein, the “primary” FEAMs may help to formulate the functional hypothesis, and include the S (Specific features) and R (Relative distance) FEAMs; the “secondary” FEAMs E (Elevations) and D (Depressions) further confirm and refine the functional hypothesis; and the “tertiary” FEAMs I (Interacting features) and C (Coactivated distances) further expand or reformulate the functional hypothesis.
Primary FEAMs formulate the functional hypothesis. The primary topographic FEAMs are defined in certain embodiments of the methods herein as:
(i) Specific (S) or the most specific and unique topographic change or deformation resulting from a specified muscle activity, and
(ii) Relative specific or unique distance (R) changes between specified facial landmarks.
The presence of any FEAMs that are specific and in general are not significantly shared or explained better by other co-occurring muscle activation can provide an important clue that a certain muscle activation is relevant to the examined facial expression. For example, vertical glabellar lines and decreased distance between eyebrows are produced most specifically by the corrugator muscles. Together with basic knowledge of functional anatomy, a clinician can formulate a muscle activity hypothesis as well as determine the possible types of muscle variants that might be present, for example, suggesting likely locations for the individual origins (bone attachments) of the activated muscle.
Functional splitting of the facial activations into surface topographic components (depression/elevation of surface facial features changing topographic shapes and forms) and into distance components (landmarks moving relative to each other in terms of relative distances, vector direction and angles) as described herein may be particularly helpful, since certain aesthetic procedures like laser ablations can erase surface topographic components while leaving the distance components and muscle activity largely intact. Similar natural variations in facial skin can also occur between different individuals. Topographical facial features may include, in various embodiments, elevations, depressions, shapes and relative distances between facial landmarks.
Examples of FEAMs can be seen in Table 1, where FEAMs associated with a muscle activity are presented in a sequential, modular, hierarchical fashion based on muscle specificity and organized by a functional evaluation of the individual face that starts in the middle of the upper face and moves laterally and then back to the middle of face and going down, and so on.
Secondary FEAMs further confirm and refine the functional hypothesis. Functionally relevant deformations in facial features overlying the trajectory of potential muscle activities are defined as secondary FEAMs, being the topographic facial elevations and depressions characteristic of a specific muscle activity and therefore confirming the functional hypothesis that a particular muscle or variant was activated during a particular facial expression. In certain embodiments, the secondary topographic FEAMs are presented in the current methods as surface deformations that can be observed visually and/or if possible by palpation (p). The secondary topographic features include:
(i) Elevations (E) that are surface deformations that are caused directly by the contracting muscle mass (e.g., superficial silhouettes or deeper muscle bulges) or their actions (bunches, etc.). Elevations can often be detected visually (or by palpation of contracted muscles whenever applicable in the area of the face (or neck) overlying the muscle perimeter and trajectory of muscle activities, and
(ii) Depressions (D) that are surface deformations characteristically observed during particular muscle activities, such as lines, furrows, insertions in the skin tissue and other depressions overlaying the muscle perimeter and trajectory of muscle activities. Knowledge of the depressions derived from the insertions of the muscles in a particular facial location of a person can guide the locations of the treatments in that particular individual.
The secondary topographic deformations can occur in a characteristic form during isolated muscle activity or sometimes in an altered form in conjunction with other co-occurring topographic deformations from co-activated muscles. For example, deformation or decreased appearance of the corrugator depression may be due to co-activation of the frontalis during certain facial expressions like fear. Unusual deformations or presentations of the primary- and secondary topographic features can therefore be due to co-occurring muscle activities or can also be a clue to natural or artificially-induced variation.
Tertiary FEAMs further expand or reformulate the functional hypothesis. Many FEAMs are shared or distorted by other co-activated muscles, not being specific to a single muscle. Any such FEAMs related by anatomic proximity or functional interactions during a particular expression are defined in the current methods as tertiary topographic features. They include:
(i) Interacting (I) co-activated features such as shared topographic features with altered shapes and activation intensities due to co-activated muscles, and
(ii) Coactivated distances (C) such as changes in distances between selected landmarks with alterations (deviations or distortions) in intensity activations due to interactions with functionally related co-activated muscles.
In certain embodiments, knowledge of common anatomic and neuromuscular variations can narrow down the mechanistic possibilities. Unusual deformations or presentations of the topographic features can be a clue to natural or artificially induced variation, or to other concurrent facial muscle co activations that further expand and refine the functional hypothesis. While the approximate location for common muscle and FEAM variants on the face is known based on functional anatomy knowledge including superficial insertions in the skin tissue and deeper attachments (e.g., to bones, etc.), more precise localization and identifications of variants in any given individual will also have to take into account the activation pattern of primary, secondary and tertiary features during particular facial expressions.
Quaternary facial features (FEAMs) are defined as individual permanent features that do not depend on muscle activations and often are not even in the areas adjacent to active muscles during facial expressions, including but not limited to scars, lesions, fat herniations, landmark shapes, hair color, eye color, etc. In certain embodiments, such characteristics related to individual appearance (for example, scars, growths, hair color and style) could be considered quaternary facial features and be added as descriptive codes at the end of scoring when they are visible for a particular facial expression, since they could also be hotspots of facial perception for observers. So, depending on purposes, such FEAMs might be specified individually at the end of coding for facial expressions and would provide more complete facial descriptions, but in general the reporting and modulation of facial expressions is more importantly connected to the activation patterns of the muscles and primary, secondary and/or tertiary FEAMs.
The SREDIC classification scheme discussed herein therefore organizes FEAMs according to muscle specificity based on Specific feature, Relative distance, Elevations, Depressions, Interacting features, and Coactivated distances (known herein interchangeably as the “SREDIC classification” or the “SREDIC Intensity Scale” as shown in Table 1), and in certain embodiments provides the basis for reporting differential patterns of activation intensities for FEAMs and associated muscles during facial expressions, using the Intensity Scale criteria shown in
In certain embodiments, the SREDIC Intensity Scale (as shown in
0—no activation or neutral, with expression activation closer to 0%.
1—trace, barely noticeable or subtle activation, with expression activation closer to, in various embodiments, about 10 to about 25%, or about 15% or about 20%.
2—marked activation or clearly visible with expression activation closer to, in various embodiments, about 25% to about 45%, or about 30% or about 40%.
3—strong or pronounced activation, with expression activation closer to, in various embodiments, about 45% to about 70%, or about 50% or about 60%.
4—extreme activation with expression activation closer to, in various embodiments, about 70% to about 90%, or about 75%, or about 80% or about 90%.
5—maximum natural activation or natural apex activation, with facial expression activation close to, in various embodiments, about 90% to about 100%, or about 95% or about 100%.
In certain embodiments, a seventh gradation (6) can be useful to signal hyperactivation (hyperexpression) or stimulated activation that is clearly noticeable beyond the natural maximum expression, sometimes possibly due to a condition or procedure that has altered the facial expression activation beyond the natural maximum, with facial expression activation closer to, in various embodiments, about 110%, about 115%, about 120%, about 130% or about 150%.
In certain embodiments, “U” can be added as a descriptive code to signal Uncertainty or to note justified disputes when separate evaluators disagree on the presence or absence of very subtle, transient, uncertain or hard to detect features. With the SREDIC system, the observers (or, in certain embodiments, automated system(s)) that use the current scale are presumed to be able to detect activation intensities of markers within a certain margin of error. The exact margin of error would vary based on the actual ability of observers or automated systems to detect activated features. “U” therefore may be used to signal uncertainty that a FEAM or muscle was in fact activated.
In certain embodiments, “X” can also be added if necessary to the of coding for part of a facial expression to signify that activation intensities for specified FEAMs or muscles could not be evaluated or examined, due to shadows, poor visibility, etc.
In certain embodiments, tertiary FEAMs can be reported using the SREDIC Classification by specifying the co-activated muscles for a particular FEAM, but in certain embodiments they would have no intensity. In certain embodiments, quaternary FEAMS or other descriptive features can also be added at the end of the coding, but in certain embodiments they would also have no intensity. The SREDIC classification and assessment therefore allows a functionally meaningful, modular reporting of detailed personal facial features that is based on muscle specificity.
For example, to illustrate the modular reporting of the SREDIC classification and assessment (Table 1,
(L)E1:LLSAN-S(80%),R(80%),E(a80%,b20%,c60%,d60%,e60%,p60%),
D(a20%,b40%,c40%,d80%,e60%,f80%,g20%,h80%,i80%,j60%),
I(a20%PR+40%CT+20%OOSM+20%DE,b20%PR+20%DE+20%OOSM,c40%O O,d20%NA+,e40%OOP+40%OOL+
40%ZMA,f20%NA,g20%NA+40%OOL+40%ZMA,h0%,i40%ZMA)
C(a20%PR+40%CT+20%OOSM+20%DE,b40%OOP+40%OOL+40%ZMA,c0%,d40%ZMA+e0%+f0%+g40%ZMA); and
(L)E2:LLSAN-S(60%),R(60%),E(a40%,b40%,c40%,d40%,e40%,p40%),
D(a20%,b20%,c40%,d40%,e20%,f40%,g20%,h60%,i80%,j60%),
I(a40%PR+40%CT+40%OOSM+20%DE,b40%PR+20%DE+40%OOSM,c40%O O,d40%NA,e40%OOP,f40%NA,g40%NA+40%LLS, h0%, i40%LLS),
C(a40%PR+40%CT+40%OOSM+20%DE,b40%OOP+40%LLS,c40%LLS,d40% LLS+e0%+f0%+g40%LLS);
This particular type of modular and hierarchical facial analysis is unique in terms of both the level of detail and usefulness for anybody trying to analyze, optimize and modulate the FEAMs of individual facial expressions.
The present technology relates, in certain embodiments, to the systematic and comprehensive assessment, measurement, analysis, production, modulation and/or optimization of individual facial expression patterns in terms of intensity activation patterns of individual FEAMs and corresponding muscles. The current technology combines a dimensional model of Core Affect Circumplex with further cognitive maturation and differentiation into discrete emotion schemes, emotional dimensions and individual functional facial expression patterns of psycho-physiological states, providing the basis for a novel comprehensive approach to analyzing, producing, optimizing and/or selectively modulating the individual facial components and displays of expressive psycho-physiological states.
In certain embodiments, the technology herein provides methods and systems that better do any one or more of the following: detect, analyze, extract, classify, measure, compare, interpret, report, produce, selectively modulate or optimize facial expression markers during inter- and intra-personal signal communication of basic expressive psycho-physiological states. In certain embodiments, the methods and systems herein also provide reproducible means to detect and reveal in sufficient detail the key differential, interacting and shared functional anatomic elements and patterns of whole face expressions necessary to guide and/or assist interested clinicians to safely assess, optimize and/or change facial expressions in a balanced manner that is faithful to individual human anatomic, functional and aesthetic principles, and that optimizes human communication and wellbeing.
The methods herein contemplate applications incorporating any approach, procedure or agent able to change facial expressions; for example, applying any agent capable of modifying an FEAM or applying any procedure that includes the step of modifying an FEAM. Any such agent may be applied to the subject's body, for example, to the face, head or neck topically or by injection.
As an example, in certain embodiments, the methods herein contemplate the use of botulinum toxin and hyaluronic acid modulation and optimization of facial displays during basic emotion and other psycho-physiological states. Certain case studies herein discuss optimized selective targeting with botulinum toxin (BT) injections of the activation intensity patterns of FEAMs and muscles characteristic of the individual prototypic expression of emotions such as, among other conditions, fear. That is, in certain embodiments, the methods herein contemplate the implementation of desired muscle activity in a way that includes partial or complete paralysis of a muscle located in the subject's body, for example, within the subject's face, head or neck.
As used herein, the term “BT” means botulinum toxin in any commercially available form for cosmetic and medical purposes, such as, for example, the product owned and sold by Allergan, Inc. under the trademark Botox®. In certain embodiments, the methods herein can be applied to optimize various medical, surgical, reconstructive, aesthetic and other selective approaches (including but not limited to procedures like BT and filler injections).
The methods herein allow comprehensive quantitative analysis, optimization and selective balanced changes of human facial features and their perceptions in a functionally relevant way, during facial expressions of multiple basic emotions and other meaningful normal expressive psycho-physiological states. Quantitative methods are discussed; these are based on indivisible functional anatomic elements of facial expressions.
The methods described herein also include, in various embodiments, the elicitation, collection, measuring, optimization or modulation of the Individual Prototypical Facial Expressions of basic expressive psycho-physiological states in terms of activation intensity patterns of FEAMs and muscles in the whole face as maturing and differentiating mimetic signals branching along the corresponding psycho-physiological dimensions of an Individual Facial Expression Circumplex (IFEC). The IFEC has been developed herein, and is based on Individual Maximal Facial Prototypes of Psycho-Physiological States, also described herein, that are organized around the affect circumplex. It has been found herein, that unique to the art, the Individual Facial Prototypes of emotions for any given individual usually falls within the muscle activation sets described in
The perceived mimetic signal values of modulated faces can then be further evaluated and optimized in terms of reported shifts in the multidimensional perceptual spaces based on the measurements of perceived signal coordinates along the dimensions of the Facial Expression Circumplex of psycho-physiological states.
Functional SetsIn certain methods herein, the activation intensities of the FEAMs may be reported in a hierarchical and modular fashion into the one or more functional sets based on muscle specificity. This may be done to facilitate the determination of both unique FEAMs present during a particular muscle activation and FEAMs shared with other co-activated muscles and facial expressions of the IFEC. Specifically, in certain embodiments the activation intensities of the FEAMs are grouped into one or more functional sets based on muscle specificity of changes in topographic facial features. These functional sets may facilitate the determination of both unique FEAMs present during a particular muscle activation and FEAMs shared with other co-activated muscles- and facial expressions, which help to determine and optimize which personal facial expressions of the IFEC will be affected by changes in specific FEAMs and muscle activity.
Thus, as an example, a clinician can report FEAM activation intensity measurements and SREDIC analysis for a facial expression of an individual to assess and report activation intensity patterns for the facial features of a patient, either by observing the face, photographs or other digital recordings. The facial expressions of basic expressive psycho physiological states can show intra individual asymmetry and inter individual variation in the activation intensity patterns of FEAMs during the facial expression of, e.g., disgust. This illustrates the need for the more precise and personalized method of SREDIC to describe, analyze, optimize and/or modulate the intensity of elementary components (FEAMs) of facial expressions.
Turning now to the Figures,
In
Variant configurations, intensities, sequences and partial displays of modal facial expressions of basic psycho-physiological states may be produced during personal differentiation and contexts, suggesting a need for the elicitation, elucidation and SREDIC analysis of Individual Maximal Facial Prototypes of Psycho Physiological States that are individually meaningful modal facial expressions and intensity activation patterns of FEAMs characteristic for each selected emotion, mental and psycho-physiological state.
In certain embodiments, the changes in the facial expressions of the relevant psycho-physiological states are differentiated along the dimensions of a graph having an x axis and a y axis. In certain embodiments, the x axis represents valence (pleasantness) and the y axis represents arousal (excitement).
Table 1 shows an illustration of the SREDIC Classification System, which has herein been developed. As discussed herein, the SREDIC system is a modular hierarchical and functional classification system that can be used to partially or comprehensively report and analyze the activation intensity of FEAMs grouped under corresponding muscles. The x-axis of the table includes FEAMs presented hierarchically based in muscle specificity such as, e.g., specific or unique changes, relative distances, elevations, depressions, interacting coactivated features and coactivated distances. For each muscle, Table 1 presents examples of corresponding FEAMs organized such as, e.g., Specific or unique change, Relative Distance, Elevations, Depressions, Interacting Coactivated Features and Coactivated Distances.
By allowing a balance of personal mimetic differentiation with perceptual and aesthetic goals, in certain embodiments the present methods facilitate facial modulating approaches to beneficially and optimally change facial expressions of psycho-physiological expressive states and related intra-and inter-personal signals by any one or more of the following:
-
- (1) evaluating, determining and facilitating the production of optimal individual activation intensity patterns of FEAMs and muscles based on targeting of the prototypical facial features and expressions characteristic of basic expressive psycho-physiological states;
- (2) predicting and measuring selective relative shifts in the perception of modulated facial features and expressions along the circumflex coordinates of valence (x axis, pleasure-displeasure continuum), arousal (y axis, inactivity-tension continuum) and other dimensions of psycho-physiological states to determine the optimal corresponding activation patterns of FEAMs and muscles; and
- (3) selectively and/or quantitatively modulating specific intensity activation patterns of static, dynamic or temporal FEAM(s) and muscle(s) to facilitate production of novel optimal facial expressions, as well as future identification of associated functional changes in perception, aesthetics, mimetic differentiation, psycho-physiology, psycho-pathology, cognition, neuroanatomic plasticity or other important signals associated with human communication and nature.
The multidimensional effects on modulated facial expressions and observers' perceptual spaces are further illustrated in a case study through optimized selective targeting with BT injections of the activation intensity patterns of FEAMs and muscles characteristic of the individual prototypic expression of fear (
A 35 year old Caucasian woman with no significant past medical history complained of undesirable facial features during facial expressions that she believed conveyed negative emotions to others, including unpleasant features and overexpression of lines in the forehead, glabella and mouth regions. The patient reported these bothersome features to be barely noticeable at rest, but becoming significant during certain facial expressions Like many patients seeking treatment with BT for hyperkinetic features, the patient wished to enhance her facial appearance while being able to maintain some functional ability to communicate meaningful facial expressions of basic emotions, wanting to avoid a “frozen” facial appearance after the treatment.
Multidimensional testing for functional facial expressions was conducted as described in the current disclosure, by eliciting the Maximal Individual Prototypic Facial Expressions along the circumplex dimensions of 16 psycho-physiological states. More specifically, the patient was asked to maximally pose with and without modal microscenarios (and with and without cognitive enhancement including mirror and guiding instructions) the maximal facial expressions for the following psycho-physiological states: disgust, anger, fear, tension, surprise, interest, joy/happiness, pleasure, love, friendly, content, calm, fatigue, boredom, sadness, and pain/suffering.
For each of 8 selected expressive psycho-physiological states, the photographs of four Maximal Individual Prototypic Facial Expressions were selected together with the patient to best fit the FEAMs and corresponding muscular activation patterns that commonly occur during the corresponding prototypic facial expressions for the relevant psycho-physiological states. After examining these expressions, the patient pointed out that she was most bothered by the unpleasant overexpression of the features displayed during the facial expressions of fear, but was also bothered by her features during facial expressions of anger, fatigue, and boredom. In general she stated that she wanted, if possible, a shift from more “negative” and “unpleasant” facial expressions to more “positive” and “pleasant” facial expressions.
Visual and tactile examination of the specific locations and activation intensities for the expressed facial features (FEAMs) and muscles during the elicitation of the Maximal Individual Prototypic Facial Expressions was done selectively along multiple dimensions, and the key findings were explained to the patient as necessary. Multidimensional evaluations of facial expressions done together with the patient revealed that her ideal balance of aesthetic and mimetic goals would involve BT targeting of selective features characteristic of the Maximal Individual Prototypic Facial Expression of “fear.” These considerations guided the BT injections of specific muscles (Table 1,
BT (BOTOX® from Allergan) was injected to selectively decrease key components of the muscle activity characteristic of the “fear” facial expression by using previously published injection locations for relevant individual muscles and relatively lower doses than usually used for aesthetic treatments to preserve acceptable mimetic differentiation. It should be noted that BOTOX® from Allergan is provided by the manufacturer in units. In the upper face, the BT (dilution 4 units BT per 0.1 mL saline) injections were: 2 units (u) in the procerus, 4 u per side in the medial corrugator above the medial brow (above the inner canthus), and 2 u per side in the lateral corrugator in the corrugator depression ˜1 cm above the superior orbital rim. Muscles were also injected in the upper forehead with 2 units/side in the medial frontalis, another 2 units/side more lateral and superior on the forehead to target the uppermost transverse lines produced by the frontalis, and 1 unit/side targeting the lateral frontalis.
Platysma and Risorius are commonly observed to work together in fear to stretch the mouth inferiorly and horizontally, so two BT injections (1 unit per injection, 2 injections per side) were injected into the platysma along the mandibular margin, lateral to the depressor anguli oris (DAO) based on palpation and known anatomy. The treatment also took into consideration the mild melomental asymmetry with downturned corners of the mouth being more pronounced on the right (particularly hyperexpressed during the expression of fear), and an additional 0.8 u of BT was injected in the playsma more laterally on the right side.
The patient was asked to precisely mimic (without a mirror) three times each of the quadruplicate Maximal Individual Facial Expressions for the 8 selected psycho physiological states (plus the baseline facial expression) before and 3 weeks after the BT injections. Triplicate mimicking facial expressions of each Maximal Individual Facial Expression were recorded, and the best corresponding mimicking expressions before and after procedure were chosen to be selectively and functionally compared in terms of intensity activation patterns for FEAMs and corresponding muscles. The same photographs were also shown to 3 observers to evaluate (in increments of about 10% in the degree of corresponding facial expression) the relative perceptive “signal value” along the dimensions of 8 psycho-physiological states and compared to the baseline (0%) and each corresponding Maximal Individual Prototypic Facial Expression (100%). The positive shifts and average perceptual values for mimicked facial expressions along 8 psycho physiological dimensions before and after BT are presented in
The patient was extremely satisfied with the results of the multidimensional analysis and modulation of her facial expressions (
According to the functional interactions within the Facial Expression Circumplex, LPS was not targeted even though it is a component of the fear prototype, to avoid the possibility of increasing the facial expression of somnolence/fatigue and decreasing the expressions of surprise, interest and awe (
The case illustrates for the first time the use of a multidimensional method to functionally analyze, measure, target, modulate, enhance and/or optimize selectively the key components of the individual prototypic facial expression patterns and perceptual spaces characteristic of whole facial expressions associated with basic expressive psychophysiological states.
EXAMPLE 2 Exemplary Step-by-Step ProcessThe following is a non-limiting illustration of an exemplary step by step process to measure, modulate, and optimize prototypic facial expressions and associated perceptual processes in terms of activation intensity patterns of FEAMs and muscles:
(1) Collection of photographic or video baseline (resting) and apex (maximal) facial expressions (elicited, mimicked, posed, spontaneous, felt or unfelt) providing a wide range of personal, functional and meaningful facial movements characteristic of basic expressive psycho-physiological states.
(2) The facial expressions from Step 1 judged to be most similar to the activated prototypic facial displays of psycho-physiological states are selected together with the patient to obtain functionally and personally relevant Maximal Individual Prototypic Facial Expressions.
(3) Organization of the Maximal Individual Prototypic Facial Expressions around a Maximal Individual Facial Expression Circumplex of facial displays characterizing psycho-physiological states, therefore establishing a functional and meaningful maximal personal range of facial activation that can, in certain embodiments, be reproduced in a reliable manner through elicitation or mimicking before and/or after any procedures used to modulate facial expressions.
(4) Analysis of the patterns of whole (or selected parts) of facial expressions in the Maximal Individual Facial Expression Circumplex by measuring and reporting relevant personal activation intensity patterns of FEAMs and corresponding muscles that occur and co-occur during Maximal Individual Prototypic Facial Expressions for each psycho-physiological state.
(5) Review of the functional goals (aesthetic, mimetic differentiation, associated processes) for the different Maximal Individual Prototypic Facial Expressions of psycho-physiological states of interest with the patient or subject.
(6) Patients are asked to mimic (or in certain embodiments elicit) the relevant facial expressions of interest in the Maximal Individual Facial Expression Circumplex “before” and “after” the procedure is used to modulate facial expressions.
(7) Record the facial displays before the facial modulating procedure to obtain the “Pre-modulated” Facial Expressions of Psycho-physiological States by mimicking (or in certain embodiments eliciting) the Maximal Individual Prototypic Facial Expressions to obtain reproducible personal differentiation along the psycho-physiological dimensions of the “Pre-modulated” Facial Expression Circumplex.
(8) Analysis of the patterns of whole (or selected parts) of facial expressions in the Pre-modulated Facial Expression Circumplex by measuring and reporting the personal activation intensity patterns of FEAMs and corresponding muscles.
(9) Selective targeting of relevant facial features (balance points) based on the expressed components of Individual Maximal Prototypic Facial Expressions of psycho-physiological states in terms of measured and ideal intensity activation patterns of FEAM(s) and corresponding muscles.
(10) Optimization of modulation targets (intensity activations of FEAMs and corresponding muscles) by examining within the Maximal Individual Facial Expression Circumplex the functional interactions, limits and/or consequences of modulating selected FEAMs and corresponding muscles with approaches able to change facial aesthetic and mimetic function (animation) of faces, e.g., cosmetic, surgical, reconstructive or medical approaches.
(11) Individualizing the degree (e.g., adjust the BOTOX® doses) and manner (e.g., establish the optimal activation intensity of specific individual FEAMs that will guide the specific sites of treatment with hyaluronic acid) of intervention with modulating approaches to enhance faces in a predictable manner meant to integrate the objective optimization of functional target goals (desired aesthetic, mimetic differentiation goals and associated processes) of individual patients.
(12) Record the facial displays after the facial modulating procedure(s) to obtain the Modulated Facial Expressions of Psycho-physiological States, by mimicking (or in certain embodiments eliciting) the facial expressions organized along the Maximal Individual Facial Expression dimensions of psycho-physiological states to obtain reproducible novel personal differentiation along the Modulated Facial Expression Circumplex.
(13) Following perceptual evaluation of the recorded facial expressions by observers along multiple dimensions (categories) of psycho-physiological states, functional multi-dimensional analysis of the reported perceived mimetic differentiation and aesthetic “signal values” of the “pre-modulated” and “modulated” facial expressions is done by establishing the dimensional coordinates in perceptual judgment spaces and resulting shifts, taking the following embodiments in the current invention:
(i) after establishing the baseline facial expression as 0% perceptual “signal value” and each recorded Maximal Individual Prototypic Facial Expression of selected psycho-physiological states as being 100% (the radius for the corresponding state in the circumplex circle), relative dimensional shifts in measured perceived basic emotions and other psycho-physiological states for the newly modulated facial expressions are visualized as deviations from this circle along the corresponding multiple dimensional states of the Facial Expression Circumplex (“spider” or “radar-type” graphs can be used to visualize the results); and
(ii) measuring the perceptual “signal value” coordinates of the newly obtained facial expressions along bipolar dimensional continuums using an 11 point scale (−5 to +5 or −100%, −80%, −60%, −40%, −20%, 0, +20%, +40%, +60%, +80%, +100%, with 0% being the baseline facial expression and (+/−)100% being internal maximum standards along the bipolar continuums) including for, but not limited to, the following perceptual dimensions:
(a) functional mimetic dimension axis signals: x-axis (pleasure, displeasure); y-axis (unresponsiveness, arousal);
(b) functional bi-polar mimetic dimension signals: lethargic, alert; apathic, surprised; tired, energized; fatigue, awe; bored, interested; resigned, hopeful; shame, pride; insecure, confident; miserable, elated joy; sad, happy; pain, comfort; suffering, ecstasy; displeasure, pleasure; disgust, love; contempt, respect; upset, satisfied; angry, friendly; rage, amusement; hostile, peaceful; fear, serene (content); agitated (anxious), calm; tense, relaxed; unresponsive, aroused;
(c) functional mimetic aesthetic signals: pleasantness; perceived relative age (relative to specified time range); etc.
(14) Compare the facial expressions, muscles and FEAMs measurements of psycho-physiological states from the Pre-modulated Facial Expression Circumplex with the corresponding facial expressions and measurements from the Modulated Facial Expression Circumplex to review and optimize the results of the facial modulating approach, as well as to further refine and balance any new functional goals (aesthetic, mimetic differentiation of facial expressions, associated processes and perceptual spaces) with the patient or subject.
In further embodiments of the technology, other applications or individual steps may be present:
(15) Compare psychological (behavioral, intention, communicative, etc) and FEAMs (and corresponding muscles) measurements from the Pre-modulated Facial Expression Circumplex with the corresponding measurements from the Modulated Facial Expression Circumplex to determine the optimal psychological correlates for selective lesional models of facial expressions based on targeting the individual prototypic facial expression component(s) of basic emotions and psycho-physiological states.
(16) Compare psycho-pathological (including but not limited to phobias, post-traumatic stress syndrome, anxiety disorders, performance anxiety, public speech anxiety, anorexia, addiction disorders, etc.) and FEAMs (and corresponding muscles) measurements from the Pre-modulated Facial Expression Circumplex with the corresponding measurements from the Modulated Facial Expression Circumplex to determine the optimal psycho-pathological correlates for selective lesional models of facial expressions based on targeting individual prototypic facial expression component(s) of basic emotions and psycho-physiological states.
In conclusion, the combination of selective quantitative measurements, analysis, modulation and optimization of the activation intensities of FEAMs and corresponding muscles to characterize and elicit the optimal Individual Prototypes of Facial Expression, combined with determinations of the perceptual signal values of newly modulated faces along multiple coordinates of psycho-physiological dimensions, provide a strong and unique foundation that allows the current technology to properly balance and optimize facial expressions and associated perceptual processes in a safer, more comprehensive, reliable, reproducible, and efficient way than previously possible.
All embodiments described herein are illustrative and in no way limit the scope of the technology and the technology may be embodied in other forms not explicitly described here, without departing from the spirit thereof.
After facial images have been collected, each of which being associated with a psycho-physiological state, the patient and doctor may preview the images. In some embodiments, the patient and doctor may select from among the images a subset to use in the generation of a treatment program. After selecting images to be used in determining a BT treatment plan, the doctor and patient may identify one or more FEAMs that are considered to be a candidate for modification. In some embodiments, the selection of FEAM modification candidates may be performed by selecting an image for display on a display device 550. A selection cursor may be positioned above the display device to select the FEAM modification candidate.
After one or more FEAMs have been selected as candidates for modification, the muscles that control such FEAMs may be determined, and a determination may be made whether to target the FEAMs with muscle modulating procedures like BT or in combination with other agents and procedures that would alter only the skin surface and texture FEAMs (S, E, D, I) without significantly affecting the FEAMs characterizing distances (R, C). In some embodiments, a mapping 535 between the FEAM and the muscle or muscles that create such a FEAM may be used to create a list of muscles candidates for treatment. Each muscle or muscle group that controls a particular FEAM may also control a related FEAM elicited by the same or a different psycho-physiological state. In an exemplary embodiment, a mapping 540 between the muscle or muscle groups and the FEAMs that are controlled by such muscles may be employed. For example, if selected FEAM is from an image that was elicited with an expression of fear, the same muscle or muscles that create the selected FEAM may also create a related FEAM elicited with an expression of excitement. A patient may desire to reduce the intensity of the FEAM for the expression of fear, but may also have no desire to reduce the related FEAM for the expression of excitement. These two competing desires must be weighed and reconciled in the determination of treatment.
In some embodiments, after the one or more FEAMs have been selected and the FEAM creating muscles identified, the images corresponding the related FEAMS may be presented to the patient and doctor. As each image is displayed, for example, the patient and doctor may assign a weight corresponding to the amount of FEAM modification desired for that psycho-physiological state's expression. A single muscle of muscle group may have one or more weights assigned to it, each one corresponding to one of the related FEAMs. In some embodiments, a treatment dose may be determined using an average of the various assigned weighting values. In some embodiments, each of the selected images may be sequentially presented. As each image is presented, the FEAMs related to previously selected FEAMs may be annotated. The patient and doctor may select new FEAMs as well as assigning weights for modification of the annotated related FEAMs.
After all of the selected images have been presented, and all of the FEAMs candidates and related FEAMs have been assigned weights for all of the images, a treatment plan may be determined. In some embodiments, the images may have to be cycled through more than one time to ensure that all related FEAMs have been presented to be assigned a weight. In an exemplary embodiment, after the treatment program has been determined, the treatment plan may be presented in various manners. For example, an image may be displayed with treatment locations annotated. In some embodiments, an associated dosage annotation may be presented for each treatment location. In some embodiments, the treatment plan may be transmitted to an automated dosimeter for preparing the syringes for actual treatment. In the figure, the transmission of the treatment plan may be via a cloud 545 or wire-less network, for example.
For more than 200 years, studies have gradually revealed the main components of facial expressions and their muscular basis (Hjortsjo, 1970). These studies culminated with the development of the “Facial Action Coding System” (FACS) by Ekman and Friesen in the 1970s. FACS is a method of analyzing and measuring facial expressions in terms of intensity patterns of facial “Action Units” (AUs) derived from the actions of one or more muscles. The FACS has been used in psychology, neuroscience, artificial intelligence, criminal law and medicine.
Nevertheless, progress has been slow in understanding the personal functional, perceptual and communication effects and implications of facial modulation by modern aesthetic and reconstructive approaches on perceived functional “signal value” (aesthetic, mimetic, and associated processes) during the expression and transmission of whole facial gestalts or individual components. These effects are particularly poorly characterized, studied and understood in the whole face, in great measure due to a lack of sufficiently detailed functional anatomic methods that are able to properly measure and approach the complexity and components of individual human facial expression in sufficient detail.
For many people, facial wrinkles and unsightly facial expressions often occur due to overactivity of the underlying facial musculature. While the current aesthetic approaches are mostly directed at decreasing the appearance of isolated facial features of aging such as wrinkles, facial expressions can also be modulated by surgical, reconstructive and medical aesthetic procedures just for the specific purpose of changing facial expressions and associated processes. This can be done by modulating the movement of facial tissue, changing the shapes and relative positions of topographic facial components, including expression lines. Most efforts to modulate human faces and their perceptions focus mostly on decreasing isolated unpleasant aesthetic features (such as localized hyperdynamic lines and wrinkles) in facial expressions at rest or during elicitation of simple muscle movements. These simple, isolated muscle movements are generally not fully meaningful on a personal and functional level, being not fully representative of the complex expressions characteristic of emotional states in the whole face. Due to the complexity of comprehensively analyzing interacting facial expression features and difficulty to reliably elicit them in a fully differentiated and functional fashion during the expression of various mental states, facial procedures and agents are often used for purely cosmetic or gross reconstructive purposes guided mostly by isolated or localized aesthetic considerations. This is often done without a more subtle comprehensive understanding of the important effects that modulating procedures can have on the function (aesthetic, mimetic, associated perceptual processes, signal communication, facial feedback, etc.) of full facial displays during the expressions of multiple basic emotions and other psychophysiological states.
To date, no efforts have been made to try to modulate facial expressions based on a more functional, multidimensional, comprehensive “full face” approach that takes into account the shared, interacting and differential components of facial prototypes expressed during various expressive psycho-physiological states. Furthermore, currently there is no functional method to selectively, reproducibly and systematically produce, measure, modulate or optimize the individual facial expressions and features that are characteristic of basic emotions and other expressive psycho-physiological states in a reliable and balanced manner, in a way that preserves acceptable mimetic facial differentiation and takes into consideration the complex differential interactions and activations of functional anatomic expression markers and muscles that occur as communication signals during different psycho-physiological states.
Facial expressions of psycho-physiological states can be adaptive or maladaptive, depending on personal circumstances, context and goals. For example, failure to display or recognize a differentiated facial expression of fear in combat has different adaptive value (and very different consequences) than overexpressing fear during posttraumatic shock syndrome or phobias in social contexts such as conversation or speech. The optimization of facial expressions is complex also because depending on context and individual, a simple facial movement like, e.g., eyebrows raising caused by the frontalis muscle can communicate or be a part of various facial expressions of psycho-physiological states. Similarly, depending on natural and artificially-induced individual variations in functional anatomy, tissue characteristics and neuromotor supply, the mentalis muscle can produce different facial movements (raise the chin and protrude the lip) and topographic appearance changes in the shape and form of facial features (for example, chin wrinkles, dimples, and lip contour), that can then be expressed in many individuals during the psychophysiological states of sadness, anger, disdain, and doubt.
Alteration and complete effacement of facial markers (FEAMs) by modern approaches such as ablations (laser, nerve, etc.), chemical peels, botulinum toxin and other medical approaches have the potential to interfere with both inter- and intra-personal communication of human facial expression signals. These can at times produce unintended and undesirable long-term consequences (e.g., an artificial permanent “look of surprise” or the impression of a “paralyzed” face). Thus, the right personal balance of functional goals (mimetic differentiation, signal communication, aesthetic, facial feedback, etc.) should be clearly and (if possible) more quantitatively measured, specified, designed or modulated if one attempts to properly optimize and modulate facial expressions.
Currently practiced methods to assess, measure, produce, reproduce and modulate individual facial expressions have significant inherent and practical limitations in dealing with personal facial features that occur due to localized neuromotor and functional anatomic variations. The current methods also fail to encompass and predictably manage the functional complexity of the whole face characterizing facial expressions of basic psychophysiological states by not taking into account the differential intensity activation patterns of facial features and muscles that can also have significant interacting and overlapping components with other characteristic facial displays of different basic emotions and other expressive psycho-physiological states. Furthermore, there is a lack of functional testing methods able to determine the quantitative and qualitative perceptual effects that the newly modulated whole faces have on an observer's decoding judgments.
In certain embodiments, the present technology is directed to a method for achieving a desired facial expression on the face of a subject, the method comprising the steps of: (a) measuring the activation intensity or intensities of one or more facial expression activation markers (FEAMs) of the subject affecting a desired facial expression; (b) determining the activation intensity or intensities of one or more optimal FEAMs affecting the desired facial expression; (c) grouping the one or more optimal FEAMs into one or more functional sets, each set comprising a desired optimal pattern of FEAMs and each set corresponding to a muscle activity optimally approximating the desired expression; and (d) applying an agent or procedure to the face, head or neck of the subject to implement the desired optimal pattern of FEAMs and corresponding muscle activity for the desired facial expression.
In certain embodiments, the desired facial expression is indicative of a basic expressive psycho-physiological state chosen from worry, anxiety, fear, rage, anger, irritation, contempt, disgust, pain, despair, distress, sadness, shame, boredom, fatigue/somnolence, calmness, serenity, friendliness, amusement, relief, delight, pleasure, happiness, love, pride, elated joy, awe, interest, surprise, eyebrow flash greeting, novelty, expectation, goal attainment, power/control, external/internal standards disturbance, intrinsic pleasantness, or a combination of any of the foregoing.
In certain embodiments, the determination of the activation intensity or intensities of one or more optimal FEAMs affecting the desired facial expression in step (b) comprises analyzing and testing the interactions and effects of FEAM changes on functionally relevant full face expressions and associated perceptual spaces of two or more basic expressive psycho-physiological states.
In other embodiments, the present technology is directed to a method for modulating the expression of a psycho-physiological state in a subject, the method comprising the steps of: (a) measuring one or more facial expression activation markers (FEAMs) and corresponding muscle activity of the subject for the psycho-physiological state; (b) determining one or more optimal FEAMs and corresponding desired muscle activity of the subject for the psycho-physiological state; (c) relating the optimal FEAM to a desired muscle activity that triggers the FEAM; (d) identifying an agent and optimal dosage of the agent that modulates the desired FEAM and muscle activity; (e) identifying an optimal site on the body of the subject that modulates the desired FEAM and muscle activity when contacted with the agent; and (f) applying the optimal dosage of the agent to the body of the subject to implement the desired FEAM and muscle activity, wherein the implementation of the desired FEAM and muscle activity achieves the optimal modulation in the expression of the psycho-physiological state of the subject.
In other embodiments, the present technology is directed to a method for optimally modulating the expression of a psycho-physiological state in a subject, the method comprising the steps of: (a) eliciting and determining the individual patterns of facial expression markers (FEAMs) and corresponding muscle activity during the expression of a psycho-physiological state in a subject (the Maximal Individual Prototypical Facial Expression of the psycho-physiological state); (b) determining an optimal pattern of FEAMs and corresponding desired muscle activity of the subject for the psycho-physiological state; (c) relating the pattern of FEAMs to a desired muscle activity that triggers the optimal FEAM; (d) identifying an agent and optimal dosage of the agent that modulates the desired FEAM and muscle activity; (e) identifying an optimal site on the body of the subject that modulates the desired FEAM and muscle activity when contacted with the agent; and (f) applying the optimal dosage of the agent to the body of the subject to implement the desired FEAM and muscle activity, wherein the implementation of the desired FEAM and muscle activity achieves the optimal modulation in the expression of the psycho-physiological state of the subject.
In certain embodiments, the present technology is directed to a method for achieving a desired optimal facial expression on the face of a subject, the method comprising the steps of: (a) measuring one or more facial expression activation markers (FEAMs) of the subject affecting a desired facial expression; (b) determining one or more optimal FEAMs affecting a desired facial expression by analyzing the interactions and effects of optimal FEAMs on two or more functionally relevant facial expressions, e.g., on full facial expressions characteristic of basic emotions and other psycho-physiological states; (c) grouping the one or more optimal FEAMs into one or more functional sets, each set comprising a desired optimal pattern of FEAMs and each set corresponding to an optimal muscle activity characterizing the desired expression; and (d) applying an agent or procedure to the face, head or neck of the subject to implement the desired optimal pattern of FEAMs, corresponding muscle activities and perceptual signal values for the desired facial expression and associated expressions of functionally relevant psycho-physiological states.
In other embodiments, the present technology is directed to a method for modulating the expression of a psycho physiological state in a subject, the method comprising the steps of: (a) measuring one or more facial expression activation markers (FEAMs) and corresponding muscle activity of the subject for the psycho physiological state; (b) determining optimal pattern of FEAMs and corresponding desired muscle activity of the subject for the psycho physiological state; (c) relating the optimal FEAM to a desired muscle activity that triggers the FEAM; (d) identifying an agent and optimal dosage of the agent that modulates the desired FEAM and muscle activity; (e) identifying an optimal site on the body of the subject that modulates the desired FEAM and muscle activity when contacted with the agent; and (f) applying the optimal dosage of the agent to the body of the subject to implement the desired FEAM and muscle activity, wherein the implementation of the desired FEAM and muscle activity achieves the modulation in the expression of the psycho physiological state of the subject.
In other embodiments, the present technology is directed to a method for optimally modulating the expression of a psycho-physiological state in a subject, the method comprising the steps of: (a) eliciting and determining the individual patterns of facial expression markers (FEAMs) and corresponding muscle activity during the expression of a psycho-physiological state in a subject (the Maximal Individual Protypical Facial Expression of the psycho-physiological state); (b) relating the pattern of FEAMs to sets of corresponding muscle activity that can be targeted to affect and benefit the expression of the psycho-physiological state; (c) determining an optimal pattern of facial expression activation markers (FEAMs) and corresponding desired muscle activity to optimally modulate the expression of the psycho-physiological state; (d) identifying an optimal site on the body of the subject that modulates the desired FEAMs and muscle activity when contacted with the agent; (e) identifying an agent and optimal dosage of the agent that modulates the desired FEAMs, perceptual signal value and muscle activity to correspond to the optimal expression of the psycho-physiological state; and (f) applying the optimal dosage of the agent to the body of the subject to implement the desired FEAM and muscle activity, wherein the implementation of the desired FEAM and muscle activity achieves the optimal modulation in the expression of the psycho-physiological state of the subject.
In other embodiments, the present technology is directed to a method for optimally modulating the expression of a psycho physiological state in a subject, the method comprising the steps of: (a) eliciting and determining the individual pattern of facial expression markers (FEAMs) and corresponding muscle activity during the expression of a psycho physiological state in a subject (the Maximal Individual Prototypical Facial Expression of the psycho physiological state); (b) determining an optimal pattern of FEAMs and corresponding desired muscle activity of the subject for the psycho physiological state; (c) relating the optimal FEAM to a desired muscle activity that triggers the optimal FEAM; (d) identifying an agent and optimal dosage of the agent that modulates the FEAM and desired muscle activity; (e) identifying an optimal site on the body of the subject that modulates the desired FEAM and muscle activity when contacted with the agent; and (f) applying the optimal dosage of the agent to the body of the subject to implement the desired FEAMs and muscle activity, wherein the implementation of the FEAMs and desired muscle activity achieves the optimal modulation in the expression of the psycho physiological state of the subject.
Some aspects of embodiments may be implemented as a computer system. For example, various implementations may include digital and/or analog circuitry, computer hardware, firmware, software, or combinations thereof. Apparatus elements can be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by a programmable processor; and methods can be performed by a programmable processor executing a program of instructions to perform functions of various embodiments by operating on input data and generating an output. Some embodiments can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and/or at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
Suitable processors for the execution of a program of instructions include, by way of example and not limitation, both general and special purpose microprocessors, which may include a single processor or one of multiple processors of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and, CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits). In some embodiments, the processor and the member can be supplemented by, or incorporated in hardware programmable devices, such as FPGAs, for example.
In some implementations, each system may be programmed with the same or similar information and/or initialized with substantially identical information stored in volatile and/or non-volatile memory. For example, one data interface may be configured to perform auto configuration, auto download, and/or auto update functions when coupled to an appropriate host device, such as a desktop computer or a server.
In some implementations, one or more user-interface features may be custom configured to perform specific functions. An exemplary embodiment may be implemented in a computer system that includes a graphical user interface and/or an Internet browser. To provide for interaction with a user, some implementations may be implemented on a computer having a display device, such as an LCD (liquid crystal display) monitor for displaying information to the user, a keyboard, and a pointing device, such as a mouse or a trackball by which the user can provide input to the computer.
In various implementations, the system may communicate using suitable communication methods, equipment, and techniques. For example, the system may communicate with compatible devices (e.g., devices capable of transferring data to and/or from the system) using point-to-point communication in which a message is transported directly from the source to the receiver over a dedicated physical link (e.g., fiber optic link, point-to-point wiring, daisy-chain). The components of the system may exchange information by any form or medium of analog or digital data communication, including packet-based messages on a communication network. Examples of communication networks include, e.g., a LAN (local area network), a WAN (wide area network), MAN (metropolitan area network), wireless and/or optical networks, and the computers and networks forming the Internet. Other implementations may transport messages by broadcasting to all or substantially all devices that are coupled together by a communication network, for example, by using omni-directional radio frequency (RF) signals. Still other implementations may transport messages characterized by high directivity, such as RF signals transmitted using directional (i.e., narrow beam) antennas or infrared signals that may optionally be used with focusing optics. Still other implementations are possible using appropriate interfaces and protocols such as, by way of example and not intended to be limiting, USB 2.0, Firewire, ATA/IDE, RS-232, RS-422, RS-485, 802.11 a/b/g, Wi-Fi, Ethernet, IrDA, FDDI (fiber distributed data interface), token-ring networks, or multiplexing techniques based on frequency, time, or code division. Some implementations may optionally incorporate features such as error checking and correction (ECC) for data integrity, or security measures, such as encryption (e.g., WEP) and password protection.
A number of implementations have been described. Nevertheless, it will be understood that various modification may be made. For example, advantageous results may be achieved if the steps of the disclosed techniques were performed in a different sequence, or if components of the disclosed systems were combined in a different manner, or if the components were replaced or supplemented with other components. The functions and processes (including algorithms) may be performed in hardware, software, or a combination thereof, and some implementations may be performed on modules or hardware not identical to those described. Accordingly, other implementations are within the scope of the following claims.
The technology is directed to multidimensional methods to selectively analyze, target, modulate and/or optimize personal prototypical facial expression patterns and perceptual spaces that are characteristic of basic emotions and other psycho-physiological states.
Claims
1. A treatment system for modulating, approximating, or optimizing desired facial expressions by applying an agent or procedure to the face, head, or neck of a patient, the system comprising:
- a processor;
- a plurality of non-volatile program-memory locations coupled to the processor;
- a plurality of non-volatile data-memory locations coupled to the processor;
- a display device configured to be controlled by the processor; and,
- a camera configured to be controlled by the processor,
- wherein the plurality of non-volatile program-memory locations contains instructions that, when executed by the processor, cause the processor to perform operations comprising: receiving an image file of a full human face of a patient, the image file captured by the camera, the image file corresponding to a facial expression desired to be changed by the patient; sending for display on the display device the received image file; receiving, from the patient, information representative of a desired change in one or more selected Facial-Expression Activation Markers (FEAMs) of the facial expression desired to be changed, each of the one or more selected FEAMs corresponding to muscles or muscle groups; retrieving from the non-volatile data memory locations a facial expression activation pattern that is associated with the selected FEAMs, the facial expression activation pattern comprising one or more associated FEAMS and corresponding muscles or muscle groups, each of the one or more associated FEAMs and corresponding muscles or muscle groups having an activation intensity level; retrieving from the non-volatile data memory locations psycho-physiological states associated with one of the one or more associated FEAMs and corresponding muscles or muscle groups of the associated muscular activation pattern; receiving image files of facial expressions corresponding with the retrieved psycho-physiological states; receiving, from the user, information representative of a desired amount of change in the activation intensity level of the selected FEAMs; and generating a treatment program comprising one or more treatments approximating the desired facial expression while producing optimized corresponding changes in the expression of each of two or more psychophysiological states, each treatment location targeting one of the one or more selected FEAMs and corresponding muscles or muscle groups, each treatment location having an associated treatment and dosage information.
2. The treatment system of claim 1, further comprising a dosing station configured to be controlled by the processor.
3. The treatment system of claim 1, further comprising instructions that, when executed, send for display on a display device an image file of a human face with treatment locations identified and annotated with treatment and dosage information.
4. The treatment system of claim 1, further comprising instructions that, when executed, displaying on a display device, the information representative of an analysis of the patterns of whole (or selected parts) of the facial expressions, by measuring and reporting relevant personal activation intensity patterns of FEAMs and corresponding muscles that occur and co-occur during the Maximal Individual Prototypic Facial Expressions for each relevant psycho-physiological state.
5. The treatment system of claim 1, further comprising instructions that, when executed, send control commands to the camera and receive image files from the camera.
6. The treatment system of claim 1, further comprising instructions that, when executed, map the image file of the human face onto a model of a human face.
7. The treatment system of claim 6, further comprising instructions that, when executed, identify facial markers from the image file of the human face and correlate the identified markers to corresponding markers of a model of a human face.
8. The treatment system of claim 7, further comprising instructions that, when executed, correlate a model FEAM to the selected FEAMs by calculating a relative location and activation intensity of the selected FEAM for the correlated markers.
9. A computer program product (CPP) tangibly embodied in a computer readable medium and containing instructions that, when executed, cause a processor to perform operations to determine an optimal treatment plan for using agents or procedures capable of modifying FEAMs, the operations comprising:
- receiving an image file of a human face, the image file corresponding to a facial expression desired to be changed by a user;
- sending for display on a display device the received image file;
- receiving, from a user, information representative of a desired change in one or more selected Facial-Expression Activation Markers (FEAMs) to obtain a more desirable facial expression, each of the one or more selected FEAMs corresponding to muscles or muscle groups;
- retrieving a facial expression activation pattern that is associated with the selected FEAM, the facial expression activation pattern comprising one or more associated FEAMs and corresponding muscles or muscle groups, each of the one or more associated FEAMs and corresponding muscles or muscle groups having an activation intensity level;
- retrieving psycho-physiological states associated with one of the one or more associated FEAMs and corresponding muscles or muscle groups of the associated muscular activation pattern;
- receiving image files corresponding with the retrieved psycho-physiological states;
- receiving, from the user, information representative of a desired amount change in the activation intensity of the selected FEAM; and,
- generating a treatment program comprising one or more locations, each location targeting one of the one or more treatments optimally approximating the desired facial expression while producing optimized corresponding changes in the expression of each of two or more psycho-physiological states, each treatment location targeting one of the one or more selected FEAMs and corresponding muscles or muscle groups of a facial activation pattern associated with the one of the one or more selected FEAMs, each treatment location having an associated treatment and dosage information.
10. The computer program product of claim 9, further comprising instructions that, when executed, associates a weighting value with each of the retrieved psycho-physiological states associated with one of the one or more selected FEAMs and associated muscles or muscle groups.
11. The computer program product of claim 9, wherein the information representative of a desired amount of FEAM treatment includes a weighting value representative of the importance of treating the FEAM.
12. The computer program product of claim 9, further comprising instructions that, when executed, send control commands to an image capture device and receive image files from an image capture device.
13. The computer program product of claim 9, further comprising instructions that, when executed, map the image file of the human face onto a model of a human face.
14. The computer program product of claim 13, further comprising instructions that, when executed, identify facial markers from the image file of the human face and correlate the identified markers to corresponding markers of a model of a human face.
15. The computer program product of claim 14, further comprising instructions that, when executed, correlate a model FEAM to the selected FEAM by calculating a relative location and activation intensity of the selected FEAM for the correlated markers.
16. A computer program product (CPP) tangibly embodied in a computer readable medium and containing instructions that, when executed, cause a processor to perform operations to determine an optimal treatment plan for using agents or procedures capable of modifying FEAMs, the operations comprising:
- receiving an image file of a human face, the image file corresponding to a facial expression desired to be changed by a user;
- sending for display on a display device the received image file;
- receiving, from a user, information representative of a desired change in one or more selected Facial-Expression Activation Markers (FEAMs) to obtain a more desirable facial expression;
- retrieving a facial expression activation pattern that is associated with the selected FEAM, the facial expression activation pattern comprising one or more associated FEAMs and corresponding muscles or muscle groups, each of the one or more associated FEAMs and corresponding muscles or muscle groups having an activation intensity level;
- retrieving psycho-physiological states associated with one of the one or more associated FEAMs and corresponding muscles or muscle groups of the associated muscular activation pattern;
- receiving image files corresponding with the retrieved psycho-physiological states;
- receiving, from the user, information representative of a desired amount change in the activation intensity of the selected FEAM; and,
- means for auto-generating a treatment plan based upon the received information representative of a desired amount of FEAM treatment, the treatment plan comprising one or more treatment locations, each treatment location targeting one of the one or more selected FEAMs and corresponding muscles or muscle groups of a facial expression activation pattern associated with the selected FEAM, each treatment location having an associated treatment and dosage information.
17. The computer program product of claim 16, wherein the means for auto-generating a treatment plan comprises means for weighting the retrieved psycho-physiological states associated with one of the one or more selected FEAMs and corresponding muscles or muscle groups of the associated facial expression activation pattern.
18. The computer program product of claim 16, further comprising instructions that, when executed, resolves a conflict between the information representative of a desired amount of FEAM treatment and other associated conflicting information representative of a desired amount of FEAM treatment.
19. The computer program product of claim 18, wherein resolving the conflict includes sending, for display on a display device, the information representative of a desired amount of FEAM treatment with other conflicting information representative of a desired amount of FEAM treatment, for review by a physician.
20. The computer program product of claim 18, wherein resolving the conflict includes using weighting values associated with each of a plurality of conflicting FEAM treatment associated with the conflict derived from multiple expressions of different psycho-physiological states.
Type: Application
Filed: Dec 9, 2013
Publication Date: Jun 11, 2015
Patent Grant number: 9579057
Inventor: Stefan Bradu (Rego Park, NY)
Application Number: 14/100,885
