ROBOTIC CREATURE AND METHOD OF OPERATION
A robotic creature system includes an actuatable head defining eyes, an eyelid mechanism, a body, a drivetrain, and a set of sensors. A method for robotic creature operation includes: detecting an event associated with a technological imperfection and automatically performing a set of expressive actions associated with the event.
This application is a continuation of U.S. patent application Ser. No. 15/817,035, filed 17 Nov. 2017, which claims the benefit of U.S. Provisional Application No. 62/424,315, filed 18 Nov. 2016, and U.S. Provisional Application No. 62/424,308, filed 18 Nov. 2016, all of which are incorporated in their entireties by this reference.
This application is related to U.S. application Ser. No. 29/587,060 filed 9 Dec. 2016, and U.S. application Ser. No. 29/587,059 filed 9 Dec. 2016, both of which are incorporated in their entireties by this reference.
TECHNICAL FIELDThis invention relates generally to the robotics field, and more specifically to a new and useful robotic creature and method of operation in the robotics field.
The following description of the preferred embodiments of the invention is not intended to limit the invention to these preferred embodiments, but rather to enable any person skilled in the art to make and use this invention.
1. Overview.As shown in
As shown in
In some applications, there can be a mismatch between robot capabilities and human expectations. In particular, humans expect robots to be reliable and stable—in other words, humans expect robots to be perfect. However, robots are oftentimes imperfect, due to the drawbacks of the underlying technology. For example, the rangefinding system of a home robot may have a limited field of view that cannot monitor the physical region proximal the robot base. Because of this blind spot, the robot may inadvertently run into objects, which is a mistake that humans typically expect robots to avoid.
The inventors have discovered that user expectations can be better aligned with the robot's capabilities by automatically performing robotic expressive actions (i.e. “romojis”) when the robot encounters situations that implicate or involve a technological imperfection (e.g., underlying technology disadvantage). These expressive actions can be further used to cause the users to “forgive” robot imperfections or miscues (e.g., by acting and looking cute and loveable), and to facilitate a more enjoyable user interaction with the robot and connected technologies. For example, when the robot runs into the object, the robot can detect object contact, play a sound associated with confusion or mistake, actuate the head and/or main body to “look” at the object (e.g., direct the eyes toward the object), and then turn and drive away from the object. When the robot falls over, the robot can perform a “sadness” sequence, including playing a sound associated with sadness, actuating the head to direct the eyes downward, and lowering the eyelids. When the robot fails to understand a command (e.g., due to NLP drawbacks, connection lag, etc.) or complete a mission, the robot can respond with a quizzical sound and actuate the head upward. However, other expressive actions can be performed.
In some variants, user expectations can be further aligned with robot capabilities by designing the robot to be incapable of autonomously speaking in natural language. The inventors have discovered that, in some variants, the lack of an ability to speak in natural language causes users to lower their expectations of the robot's abilities (e.g., the users do not expect the robot to be too intelligent or capable).
In some variants, the limited abilities of the robot (e.g. inability to speak) and/or the design features of the robot (e.g. lack of mouth, large eyes, general ‘creature-like’ appearance) can serve the purpose of preventing or minimizing the “uncanny valley” phenomenon, which is characterized by users' general revulsion to robots that appear to be nearly human.
However, lowering user expectations too much comes at the risk of robot nonuse. To mitigate this risk, the robot can include some basic response capabilities, such as being able to give simple yes/no responses (e.g., by nodding or shaking the robot's head), which can increase continuous robot usage while maintaining the lowered user expectations. The robot can additionally or alternatively promote robot usage by “comprehending” natural language commands from the user (e.g., using natural language processing), in addition to supporting other methods of command input, such as remote control commands. The robot can additionally or alternatively promote robot usage by autonomously performing some tasks, such as roaming around a house (e.g., to autonomously map the house and rooms), docking itself when the battery is low (and/or to receive updates or other data from the dock; example shown in
In some variants, the robot can promote usage by performing data-acquiring actions that remedy technological imperfections through the collection of data. These can be performed during or after an expressive action. In some variants, the expressive action itself facilitates data acquisition and imperfection resolution. For example, while the robot is performing an expressive action, it can concurrently be acquiring information to overcome an event associated with a technological imperfection. This allows the robot to overcome its technological imperfections with little to no user input, which potentially increases the functionality and dependability of the robot. In a specific example, the robotic creature can turn its head to “look around” when in an unrecognized physical space, which can give the robotic creature's mapping and/or navigation system sufficient time to sample the unrecognized space and localize the robotic creature within a predetermined map based on the sampled signals. In this example, the robotic creature can optionally turn its body to “look around” before, after, or during the head animation. This can increase the mapping and/or navigation system's field of view, which, in turn, can increase the number of visual features that can be used to locate the creature within the predetermined map. In a second specific example, the robotic creature can inch forward, toward a user, in response to receipt of an unrecognized (e.g., oral) command from the user. This can reduce the sampled signal's noise by sampling a clearer command (due to increased proximity to the user). In a third specific example, the robotic creature can back up in response to collision with an obstacle, which gives the robotic creature space to clear the obstacle. In this example, the robotic creature can concurrently actuate the head to “look down” at the obstacle as it is backing up, which can function as an expressive action and/or enable the robotic creature to direct a camera system (e.g., monocular, stereocamera, etc.), located in the robotic creature's head, toward the obstacle. This can enable the robotic creature's vision system to determine obstacle parameters (e.g., dimensions, geometry), which can be fed into the navigation system to route the robotic creature around the obstacle. However, any suitable data-acquiring actions can be performed.
The robot can also promote usage by performing expressive actions that cause users to create an emotional bond with the robot. For example, users are more likely to interact with the robot on a continuous basis when the robot recognizes household members (e.g., using facial recognition based on the video sampled by the camera) and automatically interacts with the recognized household member; when the robot occasionally follows users around in a pet-like manner; when the robot performs a “happy” expressive action in response to user interaction with the robot (e.g., looks up and actuates the eyelids such that half-moon lower eyelids partially obscure the eye); and when the robot reacts to stimuli (e.g., with happiness, sadness, confusion, or tiredness, as animated using the robot head, eyelid mechanisms, lights, and/or sounds). The robot can additionally dynamically adjust the parameters of the expressive actions (e.g., based on context, historic data, etc.), such that an expressive action does not become repetitive. However, robot usage can be otherwise promoted.
3. System.As shown in
The head 110 of the robotic creature functions to perform all or part of an expressive action, and additionally functions to define the eyes and support the eyelid mechanism. The head no is preferably substantially spherical, but can alternatively be humanoid, dog-like, or have any other suitable shape. A dimension of the head no (e.g., the head diameter) is preferably approximately ⅔ of the height of the body, but can alternatively be larger or smaller. The head 110 is preferably hollow and functions as a shell that hides the head mechanisms and sensors, but can alternatively be solid or otherwise configured. The head 110 is preferably made of plastic, but can alternatively be made of metal, ceramic, or any other suitable material. The head no is preferably white or cream, but can alternatively be black, red, blue, green, yellow, or any other suitable color. The head no can include a body interface where the head mounts to the body, but can alternatively include any other suitable mounting point. The body interface preferably includes an aperture, but can include a set of linkages or be otherwise configured. The head no is preferably recessed into the body, but can alternatively overlap the body, be connected by a column (e.g., a “neck”), or be otherwise connected to the body. The head 110 can include auxiliary mounting points (e.g., holes, clips, adhesive, etc.) that can function to mount accessories (e.g., ears, hats, etc.), auxiliary sensors (e.g., LIDAR, etc.), or any other suitable auxiliary system. Alternatively, the head no can be substantially smooth, textured, or have any other suitable feature.
The head no preferably defines a set of eyes 120, which function to give the robot an anthropomorphic or creature-like look. The eye 120 can additionally function as apertures connecting sensors housed in the head with the ambient environment. The head preferably defines two eyes 120 separated by a portion of the head (e.g., the “nose”), but can alternatively define any suitable set of eyes 120. The eye 120 is preferably spherical, but can alternatively be obloid, almond-shaped, semicircular, a circle segment, lunular (i.e., crescent-shaped), polygonal (e.g., triangular, octagonal), or any other suitable shape. The eye 120 can be defined by a hole in the head, by an outline drawn on the head exterior, by different material mounted to or embedded within the head, by a thinned-out region of the head, or otherwise defined. In variants where the eye 120 is a hole, the hold can be a countersunk or tapered hole (e.g., a hole with an angled bezel connecting the exterior hole perimeter to the interior hole perimeter), a simple hole, or any other suitable hole. A dimension of the eye 120 (e.g., diameter) is preferably ⅕ the diameter of the head, but can alternatively be larger or smaller. Different eyes 120 on the head preferably have the same size, shape, and profile, but can alternatively have different parameters. The head preferably does not include a mouth, but can alternatively include a static or articulable mouth. The head can support one or more of the sensors. In one example, the head can include one or more cameras (e.g., arranged in an eye socket defined by the head) and an array of capacitive touch sensors arranged along the top of the head (e.g., opposing the body mounting point).
In one variation, the head includes a camera 159 (e.g., used for object recognition, facial recognition, etc.) recessed within the housing and aligned with the eye, wherein the camera 159 is optically connected to the external environment through the eye socket. However, the camera 159 can be otherwise arranged. The camera 159 preferably does not fill the entire eye, but can optionally have a lens the same dimension as and/or fills the eye. The camera 159 is preferably biased toward the upper left, such that less of the frame is occluded by the nose, but can alternatively be coaxially aligned with the eye or otherwise arranged. In one embodiment, the camera assembly can optionally include a camera concealing assembly that functions to hide the camera within the eye. In one example, the camera concealing assembly can include, in order from the camera toward the housing exterior: a camera shroud connecting the bezel of the camera lens and the housing interior, a glass layer (e.g., tinted or transparent) and a tinted film over the eye bracket. The camera-concealing assembly can additionally include blackout paint (e.g., matte), textured black pigmented resin, felt, or any other suitable low-reflectance material that functions to decrease ingressed light reflection out of the eye socket. However, the camera assembly can include any other suitable camera-concealing assembly. In the event that the camera-concealing assembly includes a suitable low-reflectance material, a processing system of the robotic creature can include a color correction process to calibrate the camera footage based on a predetermined color shift. In one example, the camera-concealing assembly includes a black tinted film designed to appear substantially opaque to a user while simultaneously being substantially transparent to allow the camera to record the environment of the robotic creature.
In a second variation, example shown in
The head can additionally or alternatively include a head mechanism, which functions to actuate the head. The head mechanism preferably pans and tilts the head relative to the body housing (e.g., yaws and pitches, respectively), but can alternatively or additionally roll the head (e.g., to create a quizzical look) or otherwise actuate the head. Each head degrees of freedom (head DOF) is preferably directly driven by an independent drive mechanism (e.g., including a force generation mechanism, such as a motor, and a force translation system, such as a series of linkages). Alternatively, all or a subset of the head DOFs can be driven by a common drive mechanism. However, the head DOFs can be indirectly driven or otherwise driven.
The head mechanism is preferably mounted to the body at a first end, extends into the head cavity through the body interface on the head, and mounts to the head interior at a second end. However, the head mechanism can be mounted to the body at a first end and mounted to the bottom of the head at a second end, or otherwise mounted.
In one variation, example shown in
The body mount functions to mount to the body (e.g., body housing interior), but can alternatively mount to any other suitable portion of the robotic creature. The body mount is preferably arranged parallel the robotic creature's transverse plane (e.g., parallel the head sagittal axis), but can be otherwise arranged.
The head chassis functions to support the head components (e.g., processing systems, eyelid mechanism, etc.) and to mount the head mechanism to the head housing interior, but can additionally or alternatively perform any other suitable function. The head chassis is preferably rotatably mounted to the body mount, but can be translationally mounted or otherwise mounted. The head chassis rotational axis is preferably perpendicular to the plane of the body mount, but can be otherwise arranged. The rotational mechanism enabling head chassis rotation relative to the body mount can include: an annular track, bearings within the track, dampening fluid within the track (e.g., to smooth out chassis rotation), or include any other suitable rotational mechanism.
The force generation mechanism functions to generate the force that rotates the head chassis. The force generation mechanism can be mounted to the head chassis, the body mount, or to any other suitable component. The force generation mechanism can be a linear actuator, circular or rotary actuator, or be any other suitable actuator. The force generation mechanism is preferably a motor, but can alternatively be a solenoid or any other suitable force generation mechanism. The motor is preferably an electric motor (e.g., brushed or brushless, DC or AC), but can be any other suitable motor. In a first variation, the force generation mechanism can be arranged with the force generation mechanism rotational axis (e.g., rotor axis) aligned with the rotational mechanism's rotational axis. In a second variation, the force generation mechanism can be arranged with the force generation mechanism rotational axis offset but substantially parallel the rotational mechanism's rotational axis. In a third variation, the force generation mechanism can be arranged with the force generation mechanism output substantially perpendicular the rotational mechanism's rotational axis. However, the force generation mechanism can be otherwise arranged relative to the rotational mechanism's rotational axis.
The force translation mechanism functions to connect the force generation mechanism to the head chassis. The force translation mechanism can be a set of linkages (e.g., 2-bar linkage, etc.) connected to the force generation mechanism's output and a mounting point on the head chassis, a belt or chain drive extending between the output and a gear statically connected to the head chassis, or be any other suitable force translation mechanism.
In one example, the head mechanism includes: a body mount mounted to the body housing interior; a main head chassis rotatably mounted to the body mount and rotatable about a rotational axis perpendicular the body mount plane; a head tilt chassis rotatably mounted to the main head chassis, the head tilt chassis rotatable relative to the main head chassis about a tilt rotation axis parallel the body mount plane and otherwise statically fixed to the main head chassis, the head tilt chassis statically mounted to the head housing interior; a pan motor (e.g., electric motor, such as a brushed DC motor) mounted to the body mount with a rotor axis perpendicular the body mount plane; a pan force translation mechanism connecting the pan motor's rotor to the main head chassis (e.g., two-bar linkage, configured to actuate within a plane substantially parallel the body mount plane); a tilt motor (e.g., second electric motor, such as a brushed DC motor) mounted to the head chassis with a rotor axis parallel the body mount plane (example shown in
The eyelid mechanism 122 of the robotic creature functions to generate expressive actions. In particular, the eyelid mechanism 122 can convey: happiness, by raising the lower semicircular lids 128 to obscure the lower portion of the eye; sadness, by lowering the upper lids 128 to obscure the upper portion of the eye; blink, by lowering, then raising, the upper lids 128; surprise, by lowering the upper lids and/or the lower lids repeatedly (e.g. in a fluttering or blinking motion), or any other suitable expressive action. In a first variation, the eyelid mechanism 122 is operable between an open-eye position, where little to none of the eyelid mechanism 122 obscures the eye; and a smiling position, where the lower lid 128 obscures a portion of the eye. In a second variation, the eyelid mechanism 122 is further operable in a lowered position, where the upper lid 128 obscures a portion or all of the eye. However, the eyelid mechanism 122 can be operable in any other suitable position.
The eyelid mechanism 122 preferably includes a lower lid 128 and a lid drive mechanism (example shown in
The lid drive mechanism functions to actuate the lid between one or more positions. The lid drive mechanism can actuate one or more lid pieces (e.g., first and second lower lids, upper and lower lids, etc.), together or separately. The lid drive mechanism can pull the lower lid up, pull the lower lid down, or otherwise actuate the lower lid. The lid drive mechanism can include a lid actuation mechanism (e.g., a motor) and a lid force translation mechanism (e.g., a 2-bar linkage connected to the lid piece and configured to actuate in a plane perpendicular the body mount plane), but can alternatively or additionally include any other suitable component. In one example, the lid drive mechanism includes a lid motor (e.g., a brushed DC motor) mounted to the head tilt chassis with a rotor axis parallel the body mount plane and a lid force translation mechanism connecting the lid motor's rotor to the lid piece. In a first specific example, the lid motor and the linkage are arranged above the lid piece. In a second specific example, the lid motor can be mounted below the lower lid piece and the lid force translation mechanism is preferably mounted to an upper portion of the lid piece, such that the lid drive mechanism pushes the lower lid up to achieve the smiling position (e.g., raised position), be mounted above the lower lid piece, or be otherwise mounted.
The eyelid mechanism 122 can optionally include an upper lid mechanism. As shown in
The body 130 of the robotic creature functions to mount the head and house the drivetrain, navigation sensors, and other sensors. The body 130 can additionally function to define a handle, which can be used to manually lift the robotic creature. The body 130 can additionally function to mount the outputs (e.g., speakers, LEDs, etc.) and/or any other suitable component The body 130 is preferably rotund (e.g., ogived), but can alternatively be spherical or have any other suitable shape. The body 130 is preferably hollow and functions as a shell, but can alternatively be solid or otherwise configured. The body 130 is preferably made of the same material and color as the housing, but can be made of a different material or have a different color.
The drivetrain 140 of the robotic creature functions to move the robot within a physical volume, example shown in
The sensors 150 of the robotic creature function to monitor the ambient environment surrounding the robot and/or as inputs into the robotic creature. The robotic creature can include: optical sensors, such as ambient light sensors and cameras; orientation sensors, such as accelerometers, gyroscopes, and altimeters; acoustic sensors, such as microphones (example shown in
The robotic creature can additionally include: inputs 170 (e.g. user inputs), outputs, a processing system 180 (e.g., CPU, GPU, microprocessor, etc.), a communications module (e.g., WiFi, BLE, Zigbee, cellular, etc.), a power storage system (e.g., rechargeable battery), or any other suitable component, which can be mounted to the body, head, or any other suitable mounting point. Outputs can include: speakers (example shown in
The robotic creature can additionally include an internal map 160, which functions to serve as a reference for guiding the robotic creature around its environment. Additionally or alternatively, the internal map 160 functions to localize the robotic creature within the environment. The internal map 160 can be determined using the rangefinding system (e.g., mapping sensors) on-board the robotic creature (e.g., incrementally generated, frame by frame, as the robotic creature moves through the space), an indoor positioning system, manually determined (e.g., generated by a user, annotated by a user), or otherwise determined. Preferably, the internal map 160 is a virtual representation of the traversable surroundings of the robotic creature (e.g. the layout of a house), which includes approximate dimensions of the space, the approximate locations and sizes of landmarks (e.g. obstacles, entrances/exits, etc.) within the space, and/or any other feature of the traversable space of the robotic creature. The virtual representation can be a point cloud, semantic object map, feature map (e.g., with a set of feature descriptors), or have any other suitable structure. Preferably, the internal map 160 is stored within a processing system (e.g. microprocessor) of the robotic creature, but can alternatively or additionally be stored within a remote server or elsewhere. Preferably, the internal map 160 is determined during an initialization phase of the robotic creature (e.g. after unboxing of the robotic creature, upon first placement of the robotic creature into an environment, etc.), but can alternatively or additionally be determined at another time, at multiple times, be continuously updated, or be otherwise determined and/or altered. In one variation, the internal map 160 receives input from an authorized user. In one example, the internal map 160 includes “off-limit” room designations (e.g., blacklisted geofences, physical regions, physical volumes), wherein the “off-limit” room designations are assigned by an authorized user and prevent the robotic creature from entering the designated rooms. The room designations can be assigned through a voice command (e.g. telling the robotic creature “stay out of this room” when the robotic creature enters the room during an initialization phase), selected through an application on a user device, or otherwise designated.
4. Method.As shown in
The method can include receiving a command or instruction to perform an action S210, which functions to initiate the performance of a robotic creature action (e.g. movement to a specified location). The command can be received from a user (e.g., authorized user, any user), client instance associated with the user (e.g., user logged into their account), an auxiliary device (e.g., Google Home™, a Lifx™ bulb, etc.), from a remote computing system, automatically generated, or otherwise determined. The command can be received at a command time, which can be within a predetermined time window (e.g., before, after, during) of technological imperfection (e.g., failure event) determination. Preferably, the command is received at a sensor and transmitted to a processing system of the robotic creature, but can alternatively be directly received at the processing system of the robotic creature, predetermined, determined using an algorithm (e.g. a machine learning algorithm), accessed from a lookup table or remote server, or received or determined in any suitable way. Preferably, S210 is performed first in the method, but can additionally or alternatively be performed at any point in the method. Additionally, the method can include performing any part or all of the action. Examples of the command include: voice commands to perform an action, such as “go to the kitchen” (e.g., wherein the command content, signal patterns, or other input features can be mapped to a predetermined library of known commands); component operation commands, such as instructing an imaging system (e.g., camera) to record images (e.g., still frames, videos, etc.); navigation instructions to move to a waypoint; or any other suitable command.
The method can additionally include receiving a user input S220, which functions to initiate a command to a robotic creature. Additionally or alternatively, S220 can function to determine a robotic creature mood. Preferably, the user input is received from a user in physical proximity to a user, but can alternatively be received from a user via a user device or access point, predetermined, or otherwise received. Preferably, the user input is received concurrently with receiving a command to perform an action, but can additionally or alternatively be performed prior to receiving a command to perform an action, after receiving a command to perform an action, at multiple times throughout the method, or at any other time. Preferably, the user input is received at a sensor of the robotic creature, but can alternatively be received directly at a processing system of the robotic creature. In a first variation, the user input is a voice command received at an audio sensor (e.g. microphone). In a second variation, the user input is a touch (e.g. petting) received at a contact sensor (e.g. capacitive touch sensor in the robotic creature head). In a third variation, the user input is the location of a user determined by the robotic creature's rangefinding system. In a fourth variation, the user input is a specific user identification determined through a facial detection method.
The method can additionally include scoring the user input, such as assigning a positivity score to a user input S230, which functions to assess a user's response to the robotic creature. Additionally or alternatively, S230 can function to determine a robotic creature's actions (e.g. toward a user), assess user satisfaction, and/or assess user mood. Preferably, a positivity score is assigned to a user input immediately after the user input is received or determined by the robotic creature, but can additionally or alternatively be determined at any point during the method, predicted using an algorithm (e.g. machine learning algorithm) or mathematical model, predetermined, or otherwise determined in any suitable way at any suitable time. Preferably, the positivity score is assigned to a user input in a processing system onboard the robotic creature, but can alternatively be assigned in a remote processing system, in a processing system on a user device, or elsewhere. Preferably, the positivity score is determined from a predetermined library (e.g. lookup table) but can alternatively be determined from a dynamically-determined library (e.g. a machine learning library), determined by an authorized user, or determined in any other way. Preferably, the positivity score is determined based on qualities of the user input, such as, but not limited to: the quality of a voice command (e.g. tone, pitch, content, frequency, volume, etc.), the quality of contact (e.g. on capacitive touch sensors, bump shell, etc.) with the robotic creature (e.g. frequency, duration, type, strength, etc.), the expression on a user's face (e.g. smile, frown, laughter, etc.), or any other quality. The positivity score can be a numeric score determined from a range of scores, a ranking relative to other user inputs, a score associated with a fixed number of categories (e.g. “high positive”, “low positive”, “neutral”, “low negative”, and “high negative”), a binary score assignment (e.g. “negative” or “positive”), or any other type of score. In one variation, the positivity score includes a weight factor, which can be incorporated into a weighted positivity score (e.g.
In the example shown in
The method can additionally include detecting a face S290, which functions to detect the presence of a user. Additionally or alternatively, S290 can function to seek help (e.g. receive a user input) in overcoming an event associated with a technological imperfection, and/or contribute to the performance of any robotic creature action. S290 can additionally include identifying a face (e.g. performing facial recognition), which functions to identify a specific user. The term ‘user’ herein refers to any person or animal (e.g. pet) located within the detectable vicinity of the robotic creature (e.g. the region able to be detected though the robotic creature's sensors). Preferably, S290 is performed with a sensor onboard the robotic creature (e.g. camera arranged in eye socket defined by the head, example shown in
The method can additionally include sampling sensor signals associated with an action from a sensor S240, wherein the action is preferably the action determined in S210, but can alternatively be any other robot action. S240 functions to detect an event associated with a technological imperfection. Additionally or alternatively, S240 can function to enable the appropriate performance of the action, monitor the performance of the action, determine a user response to the action, or any other suitable function. Preferably, S240 is performed using sensors and a processing system, wherein both the sensors and the processing system are onboard the robotic creature, but can alternatively be performed using remote elements (e.g. processor in a user device). Preferably S240 is performed after receiving a command to perform an action, but can alternatively be performed multiple times throughout the method, continuously throughout the method, at the onset of an event, or at any other time. In one variation, S240 is performed only at the onset of certain events (e.g. historically problematic events).
Detecting an event associated with a technological imperfection S250 functions to trigger expressive action performance. Additionally or alternatively, S250 can function to trigger data-acquiring action performance. Preferably, the event is associated with the action commanded in S210 but can alternatively occur independently or in relation to any other robotic creature action or event. The event (e.g., failure event) can be detected based on: signals sampled by the robot sensors (example shown in
In a first variation, the event includes unexpected robot contact with an obstacle (e.g.
In a second variation, the event includes unexpected obstacle movement (e.g.
In a third variation, the event includes determining that the robot is in an environment that the robotic creature is known to have problems with (problematic environment). The problematic environment(s) is preferably associated with a set of predetermined parameter values or range, but can be otherwise defined. Problematic environments can include: an environment with light of a given wavelength exceeding a threshold intensity value, cliffs (e.g., stairs) within the robot traversal path, overhangs lower than or the same level as the robot height within the robot traversal path, rough terrain (e.g., carpet), or any other suitable problematic environment. The problematic environment can be determined from: geographic location, ambient light sensors, wheel drop sensors, onboard rangefinding systems, beacons, terrain differences, historic sensor measurements (e.g., increased wheel resistance for a given geographic location), or from any other suitable set of sensors.
In a fourth variation, the event includes determining unexpected robotic creature motion. Examples of unexpected robotic creature motion include: tilt beyond a threshold angle (e.g., falling over), lift (e.g., determination that the robotic creature is being lifted), unexpected robot motion (e.g., moving faster than expected, moving slower than expected, coming to a complete stop), or any other suitable motion along any other suitable axis. The unexpected robotic creature motion can be determined by comparing the expected motion with the actual motion, or otherwise determined. The expected motion can be determined from path planning processes, drivetrain control instructions, or otherwise determined. The actual motion can be determined from orientation sensors, such as IMUs, accelerometers, and gyroscopes; rangefinding systems such as TOF systems and stereocamera systems; wheel drop sensors or wheel encoders; capacitive sensors (e.g., within the robotic creature handle); and/or any other suitable set of sensor outputs.
In a fifth variation, the event includes determining robotic creature operation parameters satisfying a predetermined condition. In one example, this can include determining that the robotic creature's state of charge is below a threshold SOC level. However, satisfaction of any other suitable set of predetermined conditions can be determined.
In a sixth variation, the event includes receiving an unrecognized command. In one example, this can include determining that a voice command received by the robotic creature is not included in a library of known voice commands.
In a seventh variation, the event includes determining that the robotic creature is in an unknown location (e.g., determining a new location event). In one example, this can include determining that the surroundings of the robot are not included in its internal map (e.g., based on features detected from signals sampled by the rangefinding system, camera system, etc.). In a second example, this can include robotic creature transitioning from a power-off to a power-on state. In a third example, this can include satisfaction of conditions associated with robotic creature delocalization (e.g., pick up and placement; arrangement in an unknown or new location; etc.). Examples of these conditions include: wheel unloading and re-loading (e.g., sampled by wheel drop sensors, motor encoders, drivetrain position sensors, etc.), handle actuation (e.g., wherein a touch-sensitive surface within the handle is actuated), inertial sensor patterns indicative of a pickup event (e.g., pitch or roll beyond a threshold angle; vertical acceleration above a threshold value), or any other suitable condition.
In an eighth variation, the event includes an initial setup. In one example, this can include movement away from a charging station for the first time. In another example, this can include a restart (e.g. reboot) of the operating system of the robotic creature, determination that the robotic creature is unassociated with an internal map, receipt of a setup command from a user, or any other suitable initial setup event. However, any other suitable event associated with a technological imperfection can be determined by the robotic creature.
The method can additionally include performing a data-acquiring action based on the event S260, which functions to determine information associated with overcoming the event (e.g., failure event) associated with a technological imperfection. Additionally or alternatively, S260 can function to determine information associated with the performance of an action (e.g. the action commanded in S210) or any part of the method, determine an alternative action, improve a user impression of the robotic creature (e.g. perform an action predicted to result in a high positivity score), or any other function. The data-acquiring action can be associated with the failure event, with the expressive action, with the underlying data used to perform the failed action, or otherwise associated. Preferably, the data-acquiring action is determined based on the type of event associated with a technological imperfection (e.g. collision, unrecognized voice command, etc.), but can additionally or alternatively be determined based on a sensor signal, a user specification, a predetermined command, or any other suitable signal. The data-acquiring action preferably includes a predetermined sequence of actions performed by the robotic creature outputs (e.g. head mechanism, eyelid mechanism, lights, speakers, and/or drivetrain), but can alternatively be any other suitable set of actions performed by any other suitable system (e.g., robotic creature of otherwise). Preferably, S260 is performed after S250, but can additionally or alternatively be performed in the absence of an event associated with a technological imperfection, throughout a robot action, multiple times throughout the method (e.g. continuously), routinely (e.g. to update the robotic creature's internal map), or at any other time.
Each action in the sequence is preferably associated with a set of action parameter values defining different parameters of the performed action. The action parameters can include: operated subcomponent (e.g., rangefinding system, camera system, head motor, drivetrain, etc.), subcomponent operation parameters (e.g., power provision, frequency, duration, speed, timing, etc.), action duration, action intensity (e.g., output amplitude or magnitude), or any other suitable set of parameters. For example, a “scanning” data-acquiring action can include a sequence of side-to-side and top-to-bottom head actuations to mimic scanning across and up-and-down a room, respectively. In this example, the action parameters for each head actuation include: the distance the head is actuated, the direction the head is actuated, the speed of head actuation, and the time duration separating successive head actuation actions. In one variation, the data-acquiring action and its action parameter values are chosen to prompt a user to provide additional information. For example, a “quizzical” data-acquiring action can include a vocal output to mimic the high-rise-terminal (e.g. uptalk) structure of a question, which prompts a user to repeat or rephrase their original command. In this example, the action parameters for each vocal output include the duration of the vocal output and its pitch progression (e.g. rising pitch intonation). However, the data-acquiring actions can be otherwise characterized.
The performed data-acquiring action (including the sequence of actions and the action parameters for each action within the subset) is preferably pre-associated with the detected event (e.g., technological imperfection), but all or part of the data-acquiring action can alternatively or additionally be determined: using a state machine (e.g., based on context and robotic creature mood), using an artificial intelligence module (e.g., classification, regression, etc.), selected (e.g., based on operation context, sampled information), randomly, user-determined, or otherwise determined. The performed data-acquiring action is preferably automatically determined by the robotic creature (e.g., retrieved from on-board memory, using an on-board adjustment module, etc.), but can additionally or alternatively be entirely or partially determined by a remote computing system (e.g., server), user device (e.g., connected smartphone or tablet), or by any other suitable computing system. In the latter instances, event-to-data-acquiring action maps can be periodically received by the robotic creature from the external system, the data-acquiring action can be retrieved by the robotic creature from the external system based on the detected event, the event or parameters thereof (e.g., underlying sensor measurements) can be sent to the external system and the data-acquiring action received, or the data-acquiring action otherwise determined. The data-acquiring action can be associated with a set of technical requirements, which can include a minimum, maximum, preferred range, or other measure of performance time, motion or actuation pattern (e.g., moving the data-acquiring component along a predetermined trajectory, etc.), power draw, or any other suitable technical requirement.
Automatically performing a set of expressive actions S270 associated with the event functions to reduce negative user responses to the robotic creature's technological imperfections. Additionally or alternatively, S270 can function to distract a user while the robotic creature is performing a data-acquiring action, prevent a user from becoming angry that the robotic creature has experienced a technological imperfection (e.g. reduce the likelihood of receiving a user input with a low positivity score), or any other function. The expressive action preferably includes a predetermined sequence of actions performed by the robotic creature outputs (e.g., head mechanism, eyelid mechanism, lights, speakers, and/or drivetrain), but can alternatively be any other suitable set of actions performed by any other suitable system (e.g., robotic creature or otherwise). Each action in the sequence is preferably associated with a set of action parameter values defining different parameters of the performed action. For example, for a “tired” expressive action can include a sequence of successive eyelid lowering and raising by the eyelid mechanism (e.g., simulating a blink), wherein each successive blink is lower and slower than the previous. In this example, the action parameters for each eyelid actuation include: the distance the eyelid is actuated, the speed of eyelid actuation, and the time duration separating successive eyelid actuation actions. However, the expressive actions can be otherwise characterized. Preferably, the expressive action's action parameters (and/or instructions) are determined based on the technical requirements of the data-acquiring action (e.g. the values of the expressive action are determined based on the action parameter values of a data-acquiring action), but can be manually determined, empirically determined (e.g., increased if the data-acquiring action did not have sufficient time to adequately sample data for technological imperfection resolution), or otherwise determined. For example, the expressive action parameters can be selected such that the data-acquiring action has sufficient time and/or data points to resolve the technological imperfection. In a specific example, the expressive action animation duration can be substantially equal to (e.g., within 1 s, 5 s, 10%, etc.), longer than, or shorter than the data-acquiring action. In a second specific example, the expressive action can move the data-acquiring component (e.g., camera, rangefinding system) along a data-acquisition trajectory. However, the expressive action parameters can be otherwise determined.
The expressive action can be performed concurrently, asynchronously (e.g., before, after), or at any suitable time relative to the data-acquisition action. Alternatively or additionally, the expressive action can function as the data-acquisition action. In a first example, one or more of the duration, speed, frequency, timing, and any other parameter of the expressive action can be chosen to ensure that the expressive action and the data-acquiring action occur partly or completely concurrently. In this example, the expressive action can function to distract the user from the data-acquiring action. In another example, the expressive action is chosen to be performed with the same mechanism (e.g. head mechanism) as the data-acquiring action. Alternatively, the action parameter values of the expressive action can be predetermined, specified by an authorized user, or determined in any other way.
The performed expressive action (including the sequence of actions and the action parameters for each action within the subset) is preferably pre-associated with the detected event, but all or part of the expressive action can alternatively or additionally be determined: using a state machine (e.g., based on context and robotic creature mood), using an artificial intelligence module (e.g., classification, regression, etc.), randomly, user-determined, or otherwise determined. The performed expressive action is preferably automatically determined by the robotic creature (e.g., retrieved from on-board memory, using an on-board adjustment module, etc.), but can additionally or alternatively be entirely or partially determined by a remote computing system (e.g., server), user device (e.g., connected smartphone or tablet), or by any other suitable computing system. In the latter instances, event-to-expressive action maps can be periodically received by the robotic creature from the external system, the expressive action can be retrieved by the robotic creature from the external system based on the detected event, the event or parameters thereof (e.g., underlying sensor measurements) can be sent to the external system and the expressive action received, or the expressive action otherwise determined.
In one variation, the determined event is associated with a predetermined sequence of actions, each with a set of baseline action parameter values, wherein the action parameter values are dynamically adjusted based on the robotic creature mood and/or instantaneous operation context. Dynamically adjusting the action parameter values can include: scaling the values as a function of the mood and/or context; selecting a new equation to determine the action parameter values based on the mood and/or context; or otherwise adjusting the action parameter values. The action parameter value adjustment can be learned, changed over time according to a predetermined pattern, or otherwise determined.
In one variation, the data-acquiring action is an expressive action. For example, a sequence of head actuations to mimic scanning a room may function to gather information to compare with an internal map (data-acquiring action), allow sufficient time for a rangefinding system to sample the ambient environment and/or localize the robot within an internal map (data-acquiring action), and/or look pleasing to a user and convey that the robotic creature looks lost (expressive action). This animation can optionally include a second stage, which includes a sequence of body actuations to mimic scanning a room (e.g., turning the body side to side), which can allow the rangefinding system to sample a larger field of view and gather more information for robotic creature localization. The second stage can automatically follow the first stage, be concurrently performed with the first stage, be triggered when data gathered during the first stage is insufficient to localize the robot (e.g., within a predetermined certainty threshold). In a second variation, the expressive action is a data-acquiring action. For example, a vocal output with rising pitch intonation of the robotic creature may be comical to a user (expressive action), while prompting a user to repeat a command (data-acquiring action).
The robotic creature mood can be determined based on historic user interaction with the robotic creature (e.g., happiness increased as a function of positive interactions, such as petting and detection of users smiling, degraded as a function of negative interactions, such as pushing the robotic creature over or detection of users yelling at the robotic creature, degraded as a function of time away from a given user, determined based on positivity scores), successful completion of assigned tasks by the robotic creature (e.g., happiness augmented upon successful completion and degraded upon failure), a baseline mood (e.g., happy), or otherwise determined. The baseline robotic creature mood can optionally be recovered at a predetermined rate (e.g., a happy robotic creature will react less happily over time, unless positive interactions are received in the interim).
The robotic creature context can be determined based on: obstacles within the proximal environment (e.g., presence of objects, object classes, etc.), sounds recorded by the acoustic sensors (e.g., classification of the sounds as happy or angry, loud or soft, etc.), facial expressions of users proximal the robotic creature (e.g., classification of expressions as happy, curious, sad, angry, etc.), presence of users within the robotic creature environment (e.g., determined based on whether the associated user device is connected to a shared communications network, determined from security camera measurements, etc.), events in the associated user's calendars, recent social networking posts by an associated user, or any other suitable parameter indicative of the operating context for the robotic creature.
In one example, dynamically adjusting the parameter values based on the robotic creature mood includes: when the robotic creature mood is happy (e.g., as determined from a high robotic creature happiness score), increasing the acoustic frequency of the emitted sounds, increasing the brightness of the chest light, and increasing the duration of expressive action playback. When the robotic creature is sad (e.g., as determined from a low happiness score), decreasing the acoustic frequency of emitted sounds, decreasing the chest light brightness, and slowing the expressive action playback. However, the parameter values can be otherwise adjusted.
The method can additionally include performing an action based on information determined during the data-acquiring action S280. Preferably, the action is the action that the robotic creature is initially commanded to perform (the action interrupted by the event, such as a failure event, associated with a technological imperfection). Alternatively, the action can be a modified version of the original action (e.g. when the robotic creature cannot overcome the event associated with a technological imperfection), or any other suitable action. The different action can be selected form a set of predetermined related actions, be randomly selected, or be any other suitable action. The action is preferably performed after performance of a data-acquiring action and an expressive action, but can alternatively be performed after one of a data-acquiring action and an expressive action, in the absence of an expressive action, multiple times throughout the method, or at any other time. The action is preferably performed with information determined during the data-acquiring action, but can additionally or alternatively be performed with information determined during an expressive action. In one variation, the action is movement toward a specified room in a house and the information determined during the data-acquiring action is the location of the robotic creature with reference to its internal map. In another variation, the action is playing a specific song through the robotic creature's speaker and the information determined during the data-acquiring action is the song title.
As shown in
As shown in
A third example of the method includes: detecting a region of the ambient environment with intense sunlight (e.g., which can wash out the rangefinder signal), and performing an expressive action including: playing a scared sound (e.g., “whoa!”) and actuating the drive train to turn and drive the robotic creature away from the sunlight. The expressive action can optionally include: actuating the eyelid mechanism to lower the upper eyelids and actuating the head mechanism to lower the head.
As shown in
As shown in
As shown in
Embodiments of the system and/or method can include every combination and permutation of the various system components and the various method processes, wherein the method processes can be performed in any suitable order, sequentially or concurrently.
As a person skilled in the art will recognize from the previous detailed description and from the figures and claims, modifications and changes can be made to the preferred embodiments of the invention without departing from the scope of this invention defined in the following claims.
Claims
1. An eyelid mechanism of a robotic creature, the eyelid mechanism comprising: wherein the eyelid mechanism is operable between:
- an eyelid piece, the eyelid piece comprising: a first lid section arranged posterior to a first eye aperture of the robotic creature, the first lid section comprising: a first upper lid; a first lower lid statically connected to the first upper lid; a first tinted film segment arranged between the first upper lid and the first lower lid; a second lid section arranged posterior to a second eye aperture of the robotic creature, the second lid section comprising: a second upper lid; a second lower lid statically connected to the second upper lid; a second tinted film segment arranged between the second upper lid and the second lower lid;
- wherein the first lid section is statically connected to the second lid section;
- a lid drive mechanism connected to the eyelid piece, the lid drive mechanism comprising: a motor arranged above the eyelid piece; a two-bar linkage comprising a first end connected to the motor and a second end connected to the eyelid piece;
- a raised position, wherein the first upper lid and the second upper lid are fully obscured from view through the first and second eye apertures, respectively;
- a lowered position, wherein the first lower lid and the second lower lid are fully obscured from view through the first and second eye apertures, respectively;
- an open-eye position, wherein the first upper lid and the first lower lid are fully obscured from view through the first eye aperture, and the second upper lid and the second lower lid are fully obscured from view through the second eye aperture; and
- a closed-eye position, wherein the first tinted film segment and the first lower lid are fully obscured from view through the first eye aperture, and the second tinted film segment and the second lower lid are fully obscured from view through the second eye aperture.
2. The eyelid mechanism of claim 1, wherein the first lower lid is statically connected to the first upper lid by a frame, further wherein the second end of the two-bar linkage is connected to the frame.
3. The eyelid mechanism of claim 1, wherein:
- each of the first upper lid and the second upper lid define a concave edge; and
- each of the first lower lid and the second lower lid define a convex edge.
4. The eyelid mechanism of claim 1, wherein the eyelid mechanism is mounted to a head tilt chassis of the robotic creature.
5. The eyelid mechanism of claim 1, wherein a length of the first tinted film segment is at least equal to a diameter of the first eye aperture.
6. The eyelid mechanism of claim 1, wherein the first tinted film segment is statically mounted to the first upper lid and the first lower lid.
7. A robotic system, the robotic system comprising:
- a head defining an eye aperture;
- an eyelid mechanism comprising an eyelid piece arranged posterior to the eye aperture, the eyelid piece comprising: an upper lid; a lower lid statically connected to the upper lid; a tinted film segment arranged between the upper lid and the lower lid;
- a motor arranged above the eyelid piece; and
- a two-bar linkage comprising a first end connected to the motor and a second end connected to the eyelid piece.
8. The robotic system of claim 7, further comprising a control system electrically connected to the motor, the control system configured to operate the eyelid mechanism between:
- a raised position, wherein the upper lid is fully obscured from view through the eye aperture;
- a lowered position, wherein the lower lid is fully obscured from view through the eye aperture;
- an open-eye position, wherein the upper lid and the lower lid are fully obscured from view through the eye aperture; and
- a closed-eye position, wherein the tinted film segment and the lower lid are fully obscured from view through the eye aperture.
9. The robotic system of claim 7, wherein:
- the eye aperture has an aperture diameter;
- the tinted film segment has a tinted film length extending between the upper lid and the lower lid, wherein the tinted film length is not less than the aperture diameter.
10. The robotic system of claim 9, wherein the tinted film segment is statically mounted to the upper lid and the lower lid.
11. The robotic system of claim 7, wherein the upper lid defines a concave edge.
12. The robotic system of claim 7, wherein the lower lid defines a convex edge.
13. The robotic system of claim 12, wherein the upper lid defines a concave edge proximal the lower lid.
14. The system of claim 7, further comprising:
- an eye socket posterior to the eye aperture; and
- a camera arranged in the eye socket.
15. The robotic system of claim 14, wherein the camera is arranged with a bias toward an upper distal quadrant of the eye aperture.
16. The robotic system of claim 14, wherein the eye socket comprises a blackout painted surface.
17. The robotic system of claim 7, further comprising a head tilt chassis arranged in the head, wherein the eyelid mechanism is mounted to the head tilt chassis.
18. The robotic system of claim 7, wherein the motor comprises a brushed DC motor.
19. The robotic system of claim 7, further comprising a second eyelid mechanism, the second eyelid mechanism comprising a second eyelid piece arranged posterior to a second eye aperture of the robotic system, the second eyelid piece comprising:
- a second upper lid;
- a second lower lid statically connected to the second upper lid; and
- a second tinted film segment arranged between the second upper lid and the second lower lid.
20. The robotic system of claim 19, wherein the second eyelid mechanism is statically connected to the first eyelid mechanism by a frame, wherein the first and second eyelid mechanisms are both actuatably connected to the motor.
Type: Application
Filed: Feb 5, 2018
Publication Date: Aug 2, 2018
Inventors: Stephanie Lee (Redwood City, CA), Douglas Dooley (Redwood City, CA), Sarah Osentoski (Redwood City, CA), Kaijen Hsiao (Redwood City, CA)
Application Number: 15/889,137