Abstract: System and method for generating acupuncture points on reconstructed 3D human body mesh for physical treatment are disclosed. The computing device obtains a first two-dimensional image of a human subject that captures at least a predefined portion of the human subject; processes the first two-dimensional image of the first human subject using a trained human body recovery model to obtain a plurality of parameters representing a three-dimensional human body mesh with corresponding acupuncture points. The trained human body recovery model includes an iterative three-dimensional regression module that is supervised by a discriminator and that minimizes a combined loss below a preset threshold. The combined loss includes a discriminator error that provides a measure of whether the obtained three-dimensional human body mesh with corresponding acupuncture points correspond to real human shape, pose, and acupuncture points.

Abstract: System and method for providing remotely guided physical treatment are disclosed. A computing device is communicably coupled to a therapeutic robot and a remote guidance device. The computing device receives full-body images of a treatment subject. The computing device generates a plurality of key physical points for the treatment subject by identifying a three-dimensional human body template that corresponds to the shape and posture parameters of the treatment subject and fitting the identified first three-dimensional human body template to the full-body images of the treatment subject. The fitting causes adjustments to locations of the corresponding set of key physical points of the first three-dimensional human body template. The adjusted locations of the corresponding set of key physical points of the first three-dimensional human body template are provided to the guidance device as basis for a treatment procedure that is specified by the treatment guidance provider during the physical treatment session.

Abstract: A method for providing virtual aid templates includes capturing images of a physical environment including a physical object at a first location, rendering a 3-D virtual environment including a representation of the physical object, capturing a first hand gesture in the physical environment, translating the first hand gesture into a first operation of displaying a virtual aid template associated with the physical object, rendering the virtual aid template associated with the physical object adjacent to the location of the representation of the physical object, capturing a second hand gesture in the physical environment, translating the second hand gesture into a first interaction with the representation of the physical object, determining a second operation on the virtual aid template associated with the physical object in accordance with the first interaction, and rendering the second operation on the virtual aid template associated with the physical object.

Abstract: A method for providing real-time virtual experience of user interactions includes rendering a 3-D virtual environment based on one or more images of a physical environment including one or more representations of physical objects, receiving a user input to place a first preset virtual object at a space in the 3-D virtual environment, rendering the first preset virtual object, detecting, by the one or more cameras, a user's hand gesture interacting with the first preset virtual object, translating the user's hand gesture into an interaction with the first preset virtual object, identifying a first part of the first preset virtual object that is subject to the interaction with the first preset virtual object, concurrently displaying in real time on the display a representation of the user's hand gesture and a movement of the first part of the first preset virtual object caused by the interaction in the 3-D virtual environment.

Abstract: A method and system of controlling an appliance includes: receiving, from a first home appliance, a request to start video image processing for detecting a motion gesture of a user; processing a sequence of image frames captured by a camera corresponding to the first home appliance to identify a first motion gesture; selecting a second home appliance as a target home appliance for the first motion gesture in accordance with one or more target selection criteria, including first target selection criteria based on a location of the user relative to the first home appliance and second target selection criteria based on a level of match between the first motion gesture and a first control gesture corresponding to the second home appliance; and generating a control command to control the second home appliance in accordance with the first control gesture corresponding to the second home appliance.

Abstract: A method and system of facilitating product research and development, comprising: obtaining respective product-related data from a plurality of data sources, including product-specific data for a plurality of products, and non-product-specific data including at least one of talent profile data and technology description data; performing topic extraction on the respective product-specific data and the non-product-specific data to obtain respective topics associated with the plurality of products and corresponding numerical statistics for the respective topics; performing sentiment analysis on the respective product-specific data for a plurality of products for the respective topics to obtain respective values of a measure of consumer sentiment corresponding to the respective topics for a respective product; and presenting an integrated sentiment review of a selected product based on the respective values of the measure of consumer sentiment corresponding to one or more of the respective topics for the selected

Abstract: A method and system of facilitating product research and development, comprising: providing, in a first user interface region, a plurality of filters for selecting product research data; receiving a request to display an integrated sentiment review for a respective collection of products corresponding to respective user selected values; obtaining results of topic extraction on selected product research data; obtaining results of sentiment analysis on the selected product research data; and presenting, in a second user interface region, the integrated sentiment review of the respective collection of products, including, for each of a plurality of top-ranked topics in the results of topic extraction on the selected product research data corresponding to the respective user selected values for a first filter and a second filter, a first visual representation of a quantitative measure of positive consumer sentiment adjacent a second visual representation of a quantitative measure of negative consumer sentiment.

Abstract: A method and system of providing a control user interface at a first home appliance are disclosed, the method including detecting presence of a user within a threshold range of the first home appliance, and performing image processing on one or more real-time images of the user to determine one or more characteristics of a facial image of the user; determining at least a first parameter that is configured to trigger a first change in a current control user interface configuration for the first home appliance; and activating a first control user interface configuration corresponding to the first parameter for the first home appliance while the presence of the first user continues to be detected within the threshold range of the first home appliance.

Abstract: A virtualized environment corresponding to a physical environment currently surrounding a robot is displayed. The virtualized environment is updated in accordance with streaming environment data received from sensors collocated with the robot. A first user input inserting a first virtual object at a first location in the virtualized environment is detected. The virtualized environment is modified in accordance with the insertion of the first virtual object at the first location. The first virtual object at the first location causes the robot to execute a first navigation path in the physical environment. A second user input is detected that moves the first virtual object along a movement path to a second location in the virtualized environment. The movement path is constrained by simulated surfaces in the virtualized environment, and the first virtual object at the second location causes the robot to execute a modified navigation path in the physical environment.

Abstract: A virtual pointer object is displayed within a virtualized environment corresponding to a physical environment currently surrounding a robot. The virtualized environment is generated and updated in accordance with streaming environment data received from sensors collocated with the robot. First user input is detected via a haptic-enabled input device that causes the virtual pointer object to move along a movement path in the virtualized environment, where the movement path is constrained by simulated surfaces in the virtualized environment. Haptic feedback is generated via the haptic-enabled input device in accordance with simulated material and/or structural characteristics of the movement path. The virtualized environment is modified at the locations of marking inputs along the movement path and affects path planning for the robot within the first physical environment.

Abstract: A method and system of performing interactive object segmentation from streaming surfaces is disclosed. An environment data stream, including correlated image and depth data, is received from a set of sensors collocated with a robot. A virtualized representation of a physical environment is displayed and updated in accordance with the environment data stream in real-time. A marking input is received from a haptic-enabled input device. A position in the virtualized representation of the physical environment is determined in accordance with the marking input and is constrained by a first virtualized surface in the virtualized representation of the physical environment. Object segmentation is performed from the position of the marking input on the correlated image and depth data.

Abstract: A method includes: deploying a frontend system (FIES); registering a current deployment location of the FIES; detecting a close-proximity interaction between a user and the FIES that specifies a first category; generating a product recommendation that includes a first product of the first category; detecting a respective prior purchase record of the user for a second product in a second category associated with the first category; if the prior purchase record indicates a purchase or delivery location outside of a geographic region of the registered current deployment location, automatically augmenting the product recommendation to include a third product from the second category selected based on characteristics of the second product; and otherwise, automatically refining the first product recommendation to further define characteristics of the first product based on the characteristics of the second product; and providing the product recommendation to the first user after the automatic augmenting or refining

Abstract: A virtualized environment corresponding to a physical environment currently surrounding a robot is displayed. The virtualized environment is updated in accordance with streaming environment data received from sensors collocated with the robot. A first user input inserting a first virtual object at a first location in the virtualized environment is detected. The virtualized environment is modified in accordance with the insertion of the first virtual object at the first location. The first virtual object at the first location causes the robot to execute a first navigation path in the physical environment. A second user input is detected that moves the first virtual object along a movement path to a second location in the virtualized environment. The movement path is constrained by simulated surfaces in the virtualized environment, and the first virtual object at the second location causes the robot to execute a modified navigation path in the physical environment.

Abstract: A method includes: deploying a frontend system (FIES); activating input streams from on-site cameras located at a current deployment location of the FIES; in accordance with the respective input stream of a first camera, registering an inspection event of a user in association with a first sample product on display; detecting a close-proximity interaction between the user and the FIES; automatically generating a first product recommendation, including: if the inspection event meets enhanced inspection criteria, which are met when a second inspection event of the first user exists in previously stored inspection events associated with the respective first sample product, automatically adding a product-specific description of the first sample product in the first product recommendation; and otherwise, forgoing including the product-specific description of the first sample product in the first product recommendation; and providing, through the frontend information exchange system, the first product recommendatio

Abstract: A virtualized environment corresponding to a physical environment currently surrounding a robot is displayed. The virtualized environment is updated in accordance with streaming environment data received from sensors collocated with the robot. A first user input inserting a first virtual object at a first location in the virtualized environment is detected. The virtualized environment is modified in accordance with the insertion of the first virtual object at the first location. The first virtual object at the first location causes the robot to execute a first navigation path in the physical environment. A second user input is detected that moves the first virtual object along a movement path to a second location in the virtualized environment. The movement path is constrained by simulated surfaces in the virtualized environment, and the first virtual object at the second location causes the robot to execute a modified navigation path in the physical environment.

Abstract: System and method for generating acupuncture points on reconstructed 3D human body mesh for physical treatment are disclosed. The computing device obtains a first two-dimensional image of a human subject that captures at least a predefined portion of the human subject; processes the first two-dimensional image of the first human subject using a trained human body recovery model to obtain a plurality of parameters representing a three-dimensional human body mesh with corresponding acupuncture points. The trained human body recovery model includes an iterative three-dimensional regression module that is supervised by a discriminator and that minimizes a combined loss below a preset threshold. The combined loss includes a discriminator error that provides a measure of whether the obtained three-dimensional human body mesh with corresponding acupuncture points correspond to real human shape, pose, and acupuncture points.

Abstract: System and method for providing remotely guided physical treatment are disclosed. A computing device is communicably coupled to a therapeutic robot and a remote guidance device. The computing device receives full-body images of a treatment subject. The computing device generates a plurality of key physical points for the treatment subject by identifying a three-dimensional human body template that corresponds to the shape and posture parameters of the treatment subject and fitting the identified first three-dimensional human body template to the full-body images of the treatment subject. The fitting causes adjustments to locations of the corresponding set of key physical points of the first three-dimensional human body template. The adjusted locations of the corresponding set of key physical points of the first three-dimensional human body template are provided to the guidance device as basis for a treatment procedure that is specified by the treatment guidance provider during the physical treatment session.

Abstract: A virtual pointer object is displayed within a virtualized environment corresponding to a physical environment currently surrounding a robot. The virtualized environment is generated and updated in accordance with streaming environment data received from sensors collocated with the robot. First user input is detected via a haptic-enabled input device that causes the virtual pointer object to move along a movement path in the virtualized environment, where the movement path is constrained by simulated surfaces in the virtualized environment. Haptic feedback is generated via the haptic-enabled input device in accordance with simulated material and/or structural characteristics of the movement path. The virtualized environment is modified at the locations of marking inputs along the movement path and affects path planning for the robot within the first physical environment.

Abstract: A method and system of performing interactive object segmentation from streaming surfaces is disclosed. An environment data stream, including correlated image and depth data, is received from a set of sensors collocated with a robot. A virtualized representation of a physical environment is displayed and updated in accordance with the environment data stream in real-time. A marking input is received from a haptic-enabled input device. A position in the virtualized representation of the physical environment is determined in accordance with the marking input and is constrained by a first virtualized surface in the virtualized representation of the physical environment. Object segmentation is performed from the position of the marking input on the correlated image and depth data.

Abstract: A method for providing real-time virtual experience of user interactions includes rendering a 3-D virtual environment based on one or more images of a physical environment including one or more representations of physical objects, receiving a user input to place a first preset virtual object at a space in the 3-D virtual environment, rendering the first preset virtual object, detecting, by the one or more cameras, a user's hand gesture interacting with the first preset virtual object, translating the user's hand gesture into an interaction with the first preset virtual object, identifying a first part of the first preset virtual object that is subject to the interaction with the first preset virtual object, concurrently displaying in real time on the display a representation of the user's hand gesture and a movement of the first part of the first preset virtual object caused by the interaction in the 3-D virtual environment.