Scalable Smart Environment with Gateway Thermostat

Abstract

Building blocks for a smart device such as a thermostat include a user interface (UI) unit and a terminal (TML) unit. A UI unit may support one or more input data from a user and/or sensors and/or one or more control terminals. The UI unit may process each input datum or a combination of the input data, generate a control signal to one or more control terminals based on the processing, and send the control signal to one or more control terminals over a communication channel. A terminal unit, which may consist of one or more control terminals, transforms the received control signal into one or more controls to one or more associated environmental generators. One or more UI units may control one or more controlled apparatuses in conjunction with a mobile app to allow a unified user experience.

Description

This patent application is a divisional application of and claims priority to U.S. patent application Ser. No. 16/037,140, entitled “A Scalable Smart Environment Using A Gateway Thermostat” filed on Jul. 17, 2018 which claims priority to provisional patent application Ser. No. 62/581,910 entitled “A Scalable Smart Environment Using A Gateway Thermostat” filed on Nov. 6, 2017, which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

Aspects of the disclosure relate to a smart control system. Some embodiments may include a user interface and a terminal unit to control one or more controlled apparatuses.

BACKGROUND OF THE INVENTION

There is frequently a need to support a scalable smart environment that may span environmental controllers, smart devices, and/or other appliances.

SUMMARY OF THE INVENTION

An aspect supports building blocks for a smart device, such as a thermostat. Building blocks include one user interface (UI) unit and one terminal (TML) unit. A UI unit may support one or more input data from a user and/or sensors and/or one or more control terminals. The UI unit may process each input datum or a combination of the input data, generate a control signal to one or more control terminals based on the processing, and send the control signal to one or more control terminals over a communication channel. A terminal unit, which may consist of one or more control terminals, transforms the received control signal into one or more controls to one or more associated environmental generators. Environmental generators (controlled apparatuses) may include furnaces, boilers, air conditioners, heat pumps, fans, de-humidifiers, thermostatic radiator valves (TRVs), and so forth.

With another aspect, a user interface unit communicates to a terminal unit via a communication channel. The communication channel may utilize a wired or wireless transmission medium.

With another aspect, the number of UI and terminal units is flexible. The wired/wireless connections/communications between UI and terminal units are also flexible, to form a scalable smart control system.

With another aspect, a smart control system comprises two individual smart devices. Examples include a thermostat (having one UI unit and one terminal unit) and a home console (having one UI unit and multiple terminal units including those for lightings, switches, TV's, air conditioners, and so forth).

With another aspect, one or more UI units may communicate with one or more terminal units via a connection hub. Control commands and status reports may be collected in the connection hub and may be routed to target devices within the web. The connection hub may be implemented in a public cloud provided by a service provider or within one of the many UI units within the smart control system.

With another aspect, a UI unit may be a gateway which connects all other smart devices to a public cloud.

With another aspect, a UI unit may support a private cloud, where controls and status reports are manipulated within the user's home network before going to a public domain. The UI unit further supports a rule engine for smart devices within a home environment and only user-defined control and information are sent to public cloud for further manipulation. The UI unit may be equipped with the capabilities of a connection hub, a gateway and private cloud.

With another aspect, different UI units may have different user interface capabilities such as an LCD with touchscreen, 7-segment LEDs with mechanical buttons, and so forth.

With another aspect, there is no UI unit at all and the control of terminal units is done via mobile devices, such as a smart phone or a tablet.

With another aspect, a terminal unit may support one or more control terminals for heating units, cooling units, heat pumps, fans, de-humidification, and the like.

With another aspect, the terminal units may be extended to include those for controlling different electrical appliances. Flexibility may be supported for interactions between different electrical appliances and different UI units. The electrical appliances can be, but not limit to, smart home devices (such as thermostats, light bulbs, smart sensors, smart plugs, and so forth), home appliances (such as air conditioners, washing machines, electric grill, oven, and so forth), a game console, media devices, healthcare devices, and the like.

With another aspect, a mobile application is scalable such that it can select all the smart devices which connect to a UI unit (which has the capabilities of user interface, gateway, connection hub and private cloud). Conversely, it may de-select a smart device from the list.

With another aspect, the user experience design (UX) and UI graphic of each UI unit in the mobile app is identical to the actual UX and UI graphic of the pair to improve user experience. No extra learning for using the mobile app is thus needed.

With another aspect, a rule engine may execute on a terminal unit, and/or a UI unit, and/or public cloud.

With another aspect, a UI unit may comprise a mobile app that controls all the associated terminal units via either WiFi or BLUETOOTH® Low Energy (BLE). In this case, a rule engine may be downloaded to individual terminal units to reduce the latency caused by network performance or when the network connection is unavailable or when the mobile phone is absent. More complex rule engines and learning algorithms may be implemented in the mobile phone or public cloud.

With another aspect, a number of UI and terminal units may also be connected and one of the UI units which may have enough computational and memory power to implement the functions of a connection hub, a gateway and a private cloud which has a rule engine with a learning algorithm to execute.

With another aspect, an individual terminal unit may have its own UI unit. A rule engine and learning algorithm may be implemented separately into each UI/terminal pair and the associated input/output devices to form a local loop. Status reports for each UI/terminal pair may be sent to a public cloud via one UI/terminal pair. Rule engine and learning algorithms for the interactions between of all the UI/terminal pairs may be implemented in the public cloud.

With another aspect, a number of local loops can be grouped together to form a network, such as a home network, a building network, a community network, a regional network, etc.

With another aspect, rule engines and learning algorithms may be implemented with four different levels. First, a simple rule engine may be implemented in the terminal unit based on its individual internal inputs (for example, sensor inputs) and associated external inputs (for example, external sensor inputs). The rule engine may be downloaded to the terminal unit using WiFi or BLE in the mobile phone. Second, a rule engine may be implemented within individual UI unit which controls one or more terminal units. Third, a rule engine between different UI/terminal pairs may be implemented in the private cloud (which may be one of the UI/terminal pairs). Learning algorithms may be implemented in the private cloud to reduce the latency introduce by the internet. Fourth, a rule engine and learning algorithm may involve external Things of Interest (for example, weather information, location data, sales information, and the like) that may be implemented in a public cloud.

With another aspect, rules may be automatically generated via machine learning algorithm. Machine learning may be applied locally in the UI unit of a local loop. Machine learning can also applied to the information received from multiple UI units of a larger network. Machine learning may be conducted within a single UI unit which has better system hardware. Learning may also be distributed to different UI units to reduce the computation loading for individual UI unit and summarized within the primarily UI units.

With another aspect, the configuration of each UI/terminal pair may be set in the mobile app and committed to an individual UI/terminal pair one after another by using near-field communications (NFC). Alternatively, a user can commit to the private cloud (which may be one of the UI/terminal pairs) via NFC and the private cloud can then route the configuration to an individual UI/terminal pair.

With another aspect, system security may be realized via audio signature when audio service is available in the private cloud or individual UI/terminal pair. A user may use voice commands to unlock the system. Alternatively, the NFC of the user's mobile device may also be used, assuming that only the user can unlock his/her mobile device. Voice signature or NFC access may also be used for system personalization to configure the system based on the individual user's historical data to meet his/her preference system behavior(s).

With another aspect, a scalable smart control system controls at least one controlled apparatus, where the control flow and the user interface graphics of a UI unit in a mobile app are identical to that in a physical UI unit. The status of each UI unit in the dash board of the mobile app is identical to the current status as shown in the physical UI unit. A user may choose a UI unit from the dash board of the mobile app for further manipulation, as it is done on the spot. This eliminates the learning needs in using the mobile app and allows a unified user experience in using the UI unit as well as the mobile app.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary of the invention, as well as the following detailed description of exemplary embodiments of the invention, is better understood when read in conjunction with the accompanying drawings, which are included by way of example, and not by way of limitation with regard to the claimed invention.

FIG. 1 shows system building blocks of a gateway thermostat in accordance with an embodiment.

FIG. 2 shows multiple user interfaces (UI) units controlling a single terminal unit in accordance with an embodiment.

FIG. 3 shows a single UI unit controlling multiple terminal units in accordance with an embodiment.

FIG. 4 shows a connection hub between multiple UI units and terminal units in accordance with an embodiment.

FIG. 5 shows a user interface unit in accordance with an embodiment.

FIG. 6 shows a terminal unit in accordance with an embodiment.

FIG. 7 shows a system with multiple UI units, multiple terminal units, a mobile app, and a web browser app in accordance with an embodiment.

FIGS. 8-12 show exemplary embodiments with one or more UI units and one or more terminal units.

It will be apparent to one skilled in the art after review of the entirety disclosed that the steps illustrated in the figures listed above may be performed in other than the recited order, and that one or more steps illustrated in these figures may be optional.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown, by way of illustration, various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized, and structural and functional modifications may be made, without departing from the scope of the present disclosure.

It is noted that various connections between elements are discussed in the following description. It is noted that these connections are general and, unless specified otherwise, may be direct or indirect, wired or wireless, and that the specification is not intended to be limiting in this respect.

Illustrative embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments of the invention are shown. Indeed, the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

As will be appreciated by one of skill in the art in view of this disclosure, the present invention may be embodied as an apparatus (for example, a system, computer executable product, and/or other device), a method, or a combination of the foregoing. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code), or an embodiment combining software and hardware aspects that may generally be referred to herein as a “system.” In addition, various signals representing data or events as described herein may be transferred between a source and a destination in the form of light or electromagnetic waves traveling through signal-conducting media such as metal wires, optical fibers, or wireless transmission media (e.g., air or space). In general, the one or more computer-readable media may comprise one or more non-transitory computer-readable media. Embodiments of the present invention are described below with reference to flowchart illustrations and/or block diagrams of processes or apparatuses (the term “apparatus” including systems and computer executable products). It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer executable instructions. These computer executable instructions may be provided to a processor of a special purpose computer or other executable data processing apparatus to produce a particular machine, such that the instructions, which execute by the processor of the computer or other executable data processing apparatus, create mechanisms for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer executable instructions and algorithms described herein may also be stored in a computer-readable memory that can direct a computer or other executable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions, which implement the function/act specified in the flowchart and/or block diagram block or blocks.

FIG. 1 shows system building blocks 101 and 102 of gateway thermostat 100 in accordance with an embodiment.

System 100 includes terminal unit 102 and user interface (UI) unit 101. Each unit may be housed separately and connected via either wireless means or wired means 151.

A wireless connection may support Sub-1G, ZWave, WiSun, Zigbee, Thread, BLE, WiFi, NFC, 2G/3G/4G modem or IrDa or any combination of these wireless technologies.

A wired connection may support a direct connection of the system control signals. It may also support any wired interface, such as SPI, I2C, UART, USB, CAN, Ethernet, Opentherm, RS232/RS485 running FSK, eBus, EMS, MP Bus, Modbus, BACnet, and the like.

A proprietary protocol may be defined for exchanging system information as well as terminal controls.

System flexibility for multiple connection scenarios may be enabled by using the design topology shown in FIG. 1.

FIG. 2 shows multiple user interfaces (UI's) 201-204 controlling single terminal unit 205 in accordance with an embodiment.

Multiple UI units 201-204 may be able to control single terminal unit 205. The feature requirements for different UIs may be different. With one embodiment, UI unit 201 (for example, located in the sitting room) may be a comprehensive UI unit that is configured to control other appliances, such as a TV (not explicitly shown), game console 206, and/or multimedia console 207. UI unit 202 may be the control panel of a home appliance 208, such as an oven.

FIG. 3 shows single UI 301 controlling multiple terminal units 302-305 in accordance with an embodiment.

Single UI unit 301 is able to control a range of terminal units 302-305. The feature requirements for different terminal units 302-305 may be different. For example, UI unit 301 may control a first terminal unit at home (which may have three-stages of heat and cool, a heat pump, a fan and a de-humidifier) and a second terminal unit in an individual store room away from home (which may only have one-stage heat and cool, a heat pump, and a fan).

The following is an example illustrating the operation between UI unit 301 and a terminal unit 302. A user sets a target temperature to UI unit 301. UI unit 301 then sends the target temperature to terminal unit 302. Terminal unit 302 may act as a thermostat or may control an external thermostat (not explicitly shown). Using its rule engine to estimate how long it will take to reach the target temperature, terminal unit 302 decides when to use different stages of a heating and cooling system (not explicitly shown). If a first stage of heating/cooling cannot reach the desired temperature in a reasonable amount of time, terminal unit 302 may activate additional stages of heating or cooling to ensure achieving the target temperature.

The following is another example illustrating the operation between UI unit 301 and a terminal unit 302. A user is on the way back home and performs the following actions via a smart home app executing on his/her mobile device.

• • Sets the target temperatures of his/her sitting room and executes the setting.
  • Sets the target time and temperatures of his/her bed room as well as his/her children's rooms.
  • Programs the washing machine
  • Sets the time and temperature of the oven
  • When the user arrives home, the user puts all the dirty clothes into the washing machine and a turkey into the oven. Referring to FIG. 3, the user then puts the near field communications (NFC) device of his/her mobile device near central UI unit 301 in the sitting room (which also functions as the connection hub of his home). Central UI unit 301 then routes the settings to the corresponding terminal units 302-305 within the local loop, including the HVAC terminal units for the bed rooms, the terminal unit of the washing machine, and the terminal unit of the oven.

FIG. 4 shows a connection hub between multiple UI units 401-404 and terminal units 405-408 in accordance with an embodiment.

With some embodiments, multiple UI units 401-404 and terminal units 405-408 are connected together via connection hub 412. Control commands and status reports are collected in connection hub 412 and may be routed to target devices within the web. Connection hub 412 may be implemented in a public/private cloud 413 provided by a service provider or within one of the UI units 401-404.

In the embodiments shown in FIGS. 2-4, the UI unit can be a gateway which connects all other devices to a public cloud. Alternatively, the UI unit may be a connection hub which routes the commands and status reports to target devices within the web. Alternatively, the UI unit may be a private cloud, in which all the controls and status report may be manipulated within the user's home network before going to a public domain. Embodiments may include a rule engine (for example mapping events to actions) for all the smart devices within a home network and only the user-defined control and information are sent to public cloud for further manipulation. The UI unit may also be equipped with capabilities of a gateway, a connection hub and private cloud.

FIG. 5 shows a user interface (UI) unit 500 in accordance with an embodiment. Components of UI unit 500 comprise a basic subsystem, a sensor subsystem, a UI subsystem, a connectivity subsystem, and a security subsystem.

The basic subsystem includes main processor 501, memory device 502, power management unit 503, and other system components as will be discussed.

Main processor 501 may be a MCU, a SoC of a single processor core, or a multi-core SoC with numerous co-processor or accelerators. Multiple processors may also be used depending on the system architecture design.

Different combination of memory device(s) 502 may be used (such as SSD, eMMC, NAND, NOR, DDR3, SDRAM, EEPROM) depending on the feature requirements. If a private cloud is implemented into a UI unit, a large size eMMC or an SSD may be used for data warehousing and data mining as well as machine learning.

The core building block of the power management unit (PMU) 503 may be discrete components or a complex power management IC (PMIC) which may include multiple DC-DC converts, LDOs as well as battery charging circuitry 521 and/or external battery 522. Some embodiments may require separate real-time clock (RTC) 523 with backup power source (a re-charge coil cell battery or a super-cap 524), in order to provide an accurate clock source when system 500 in deep sleep mode.

Other basic system components may include debug port(s) 504 for system debugging, keys 525 for resetting system 500 and configuring different system boot modes, and information apparatus 526 that is indicative of the hardware version and the like.

A sensor subsystem may include multiple sensors to enable a “smart environment”. The sensors include, but not limit to the following:

Temperature and relative humidity sensor (T/RH) 505: A combo temperature (T) and relative humidity (RH) sensor may be used to monitor the ambient temperature and relative humidity.

Temperature sensor(s): Multiple temperature sensors (Temp) 506 may be installed for measuring the ambient temperature as well as the internal temperature of the UI unit for temperature compensation.

Accelerometer: Accelerometer 507 may be installed to detect the orientation of the UI unit and to use the corresponding UI/UX. Accelerometer 507 may also be used as user inputs, through different tapping patterns and directions.

PIR sensor: Passive infrared (PIR) sensor 508 may be installed to detect the presence of human object(s) within the area of interests. One of the applications is directed to energy saving, in which some appliances may only be operated when human object(s) are detected. For example the backlight of a UI unit may be turned on when a human object is detected.

Proximity sensor: Proximity sensor 509 may be equipped to detect the proximity of a human object. In such case, the UI unit may prepare all the essence information for device operation, for example prompting up the setting page for temperature and operation mode when an object is detected within a pre-defined distance.

Ambient light sensor: An ambient light sensor (not explicitly shown) may be equipped for backlight adjustment according to ambient light intensity for user experience enhancement.

The UI unit can also receive inputs from external sensor through wired or wireless means.

Inputs from combination of sensors can be used together to provide means to enhance user experience in the smart environment.

As will be discussed, the user interface subsystem may support inputs from touch panel 511, microphone array 512, and feature keys 513 as well as support outputs to display 514, speaker array 515, LED(s) 516, and vibrator 517. Other user inputs and outputs means may also be added to enhance the user experience of the smart environment.

Display and capacitive touch panel 511 may include traditional UI means.

Depending on the application requirement, a display may be segment LEDs, or a segment TN LCD, or a PMOLED LCD, or a dot-matrix TN LCD, or a color TFT LCD, or an IPS TFT LCD, or an AMOLED LCD.

For touch panels, resistive touch or capacitive touch may be supported. Gesture inputs as well as multiple touch inputs may also be supported.

Regarding speaker array 515 and microphone array 512, audio UI means may also be used to provide a more natural user experience. Microphone array 512 together with audio processing algorithms (supported by device 527) may be used to provide a clear voice command understand noisy environment. The speech recognition algorithm may be deployed to understand the commands. Speaker array 515 may be used to provide a voice feedback.

Microphone array 512 may be used to detect the position of a user and hence the voice feedback can be focused towards the user's position when a more complex speaker array is used.

Other system inputs include the count and direction of taps from accelerometers 507, feature keys 513, and the like.

Other system outputs include system LEDs 516 and vibrator 517. System outputs may incorporate a combination of different LED colors, mark space ratios of the LED's, vibration patterns, and so forth.

The connectivity subsystem includes wireless connectivity and wired connectivity.

Wireless connectivity subsystem, 518 depending on the application requirements, may be a combination of Sub-1G, Wisun, Zwave, Zigbee, Thread, BLE, WiFi, NFC, 2G/3G/4G modem or IrDa, and so forth.

Wired connectivity subsystem 519 may be a combination of I2C, SPI, UART, RS232, RS485, USB, CAN, Ethernet, Opentherm, and so forth.

Security subsystem 520 may be included to avoid un-authorized system access and to protect user identity and data from the smart devices. This may be important for IoT application, where data of smart devices are uploaded to the public cloud for data mining and machine learning of user behavior.

General system access can be done through password input.

Voice signature may also be used to prevent un-authorized system access, verification of user identity and hence to enable personalized system profile.

Encryption may be applied when data is exchanged between a UI unit and public cloud.

Data encryption may be done by SW implementation. Alternatively, an external data encryption IC (not explicitly shown) can be used to further strengthen data security.

FIG. 6 shows terminal unit 600 in accordance with an embodiment.

The building blocks of terminal unit 600 include a basic subsystem, a connectivity subsystem, and a terminal control subsystem.

Similar to the UI unit, the basic subsystem of terminal unit 600 may include main processor 601, its corresponding memory devices (not explicitly shown), power management unit 602, and other system components.

Main processor 601 may be a MCU with limited memory that is powerful enough to detect and control its associated terminals as well as communicating to its associated devices, including smart sensor(s) and/or UI unit(s).

Power management unit (PMU) 602 may be implemented with discrete components, including AC/DC converter, DC/DC converters and LDOs.

RTC 604 may be used to provide the clock when terminal unit 600 is in deep sleep mode and no RTC is available from the UI unit(s).

Other basic system components include debug port(s) 605 for system debugging, keys 606 for resetting system 600 and configuring different system boot modes, information apparatus 607 for storing information indicative of the hardware version and so forth.

The terminal control subsystem includes terminal detection and control.

Detection circuit 608 is implemented to detect if an appliance is connected to a terminal and/or if the appliance is disconnected (malfunction) during operation.

Terminal control (ON/OFF) 609 is enabled when an appliance is detected.

The connectivity subsystem includes wireless connectivity and/or wired connectivity, depending on the application requirements.

Wireless connectivity subsystem 610 may support a unique protocol which may be sub-1G, Wisun, Zwave, Zigbee, Thread, BLE, WiFi, NFC, 2G/3G/4G modem or IrDa, and the like.

Wired connectivity subsystem 611 may include an I2C, SPI, UART, RS232, RS485, USB, CAN, Ethernet, Opentherm interfaces. Wired protocol(s) may be implemented using the connection, such as Modbus, BACnet, EBus, EMS Bus, MP Bus, FSK or a proprietary protocol. Direct connection of the control signal may be allowed if UI unit 500 and terminal unit 600 are at the same location.

With reference to FIGS. 5 and 6, the computing system environment may include a computing device wherein the processes discussed herein may be implemented. The computing device may have a processor for controlling overall operation of the computing device and its associated components, including RAM, ROM, communications module, and memory device. The computing device typically includes a variety of computer readable media. Computer readable media may be any available media that may be accessed by computing device and include both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise a combination of computer storage media and communication media.

Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. A program module may include computer-executable instructions that are executed by one or more processors. Computer storage media include, but is not limited to, random access memory (RAM), read only memory (ROM), electronically erasable programmable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device.

Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

The following capabilities may be supported by the embodiments.

• • A thermostat system which consists of two separate sub-blocks for system flexibility.
  • The first sub-block is for terminal control, the terminal unit. The number of control terminals and their functions are configurable according to the product requirements.
  • The second sub-block is for user interface (UI), the UI unit. The complexity of the UI unit may be changed according to the product requirements, the complexity of user experience and the graphics. Examples include portrait or landscape detection, voice control and feedback, user identification, human presence detection, user proximity detection, ambient light tracking, fun lighting, haptic feedback, 2D/3D animations, and so forth.
  • The communication means between the two sub-blocks may depend on the user cases, including:
    • a. Directly connecting the control signals through connectors when the two sub-blocks are located at the same spot.
    • b. Wireless communication when the two units are separated apart from each other.
    • c. Wired communication in scenario where wireless communication is not feasible.
  • Embodiments may utilize the thermostat design topology shown in Figure
    • a. A UI unit is able to control a single terminal unit or multiple terminal units.
    • b. A terminal unit may be controlled by a single Li unit or multiple UI units.
    • c. Multiple UI units and terminal units may be connected together through a connection hub.
  • A thermostat system may be extended to include different smart devices. Moreover, flexibility is allowed for different terminal units to be associated with different UI units, for interactions. The smart devices may be, but not limit to, smart home devices (e.g. thermostats, light bulbs, smart sensors, smart plugs, etc.), home appliances, a game console, media devices, healthcare devices, and so forth.
  • The terminal unit implements only one or numerous control terminal. Examples include:
    • a. A comprehensive set of control terminals including heats, cools, heat pumps, fans, fan speeds, de-humidifier, auxiliary, and so forth.
    • b. A single control terminal for boiler on/off.
    • c. A single control terminal to produce a pulse width modulation output, such as those used for motor control or heating elements.
  • The UI unit is equipped with gateway capability to connect the terminal unit and other smart devices to public cloud.
  • The UI unit is also equipped with private cloud capability, including but not limited to:
    • a. Data warehouse to record the history of the terminal unit and other smart devices.
    • b. rule engine which implements all the rules for the interaction between the terminal unit and other smart devices,
      • As an example, a user sets a target temperature to warm up a room. The rule engine may set a first stage algorithm to control a terminal unit or more terminal units to reach the target temperature in a reasonable time. The first stage algorithm may be obtained from a user's input or predefined heuristic algorithm or may be generated by artificial intelligence (AI) of the rule engine based on collecting data from the environment. The terminal unit(s) may feedback temperature(s) to the rule engine. If the target temperature is not achieved in a reasonable time, the rule engine may control terminal(s) with a second stage algorithm or even a third stage algorithm in order to achieve the target temperature and thus ensure comfort to the user.
      • As another example, a user always turns on the thermostats of the bed rooms and turns off the thermostat of the sitting room, the oven, the microwave and cooker in the kitchen. All of these actions may be captured by the UI units in the corresponding smart device and sent to the central UI unit. The central UI unit learns the user profile. One day, the user goes to bed and forgets to turn off the oven. The central UI unit automatically sends a turn off command to the UI unit of the oven and then sends a warning to the user.
    • c. A learning model to learn and adapt to the user behavior in using the terminal unit and other smart devices.
  • The wireless connectivity subsystem of the UI unit supports, but is not limited to, WiFi, BLE, Zigbee, ZWave, and NFC.
    • a. The WiFi subsystem enables internet access to the UI unit and other smart devices connected to it. The WiFi subsystem also allows mobile app control via WiFi direct.
    • b. The Zigbee subsystem, ZWave subsystem and BLE subsystem connect to the smart devices under the same protocols, for example, Zigbee thermostat, smart plugs and light bulbs, ZWave thermostats and sensors, BLE heartbeat monitor and weight scale, and so forth.
    • c. The BLE subsystem also enables directly control of smart devices through mobile app.
    • d. The NFC subsystem provides a direct and easy way for UI units to pair with terminal units and/or transfer the profiles to the UI units, thereafter routing to the target UI unis and/or terminal units.
  • The wired connectivity subsystem of the UI unit consists of, but not limited to, an Ethernet and an RS485 connections.
    • a. The Ethernet connects UI unit directly to internet.
    • b. The RS485 allows wired protocol to be implemented for controlling the smart devices which only have wired connectivity available, for example Modbus or BACnet
  • The UI unit equips with an accelerometer for portrait or landscapes UI mode. Moreover, it allows user inputs by tapping the UI unit from different directions with different count of tabs.
  • The audio subsystem of the UI unit consists of a mic array with two digital microphones and a speaker array with two speakers.
    • a. The audio subsystem is used for user identification, speech recognition, audio feedback as well as audio playback.
    • b. User identification is used for access control to the UI unit.
    • c. User identification is also used for retrieving personal profile for automatic control of all the smart devices as well as learning of user behavior.
    • d. Speech recognition is used for controlling the smart devices. During the speech recognition process, the direction of the user is also estimated.
    • e. The audio feedback can be focused towards the direction of the user estimated.
  • The UI unit has a large memory to store the data from a range of connected smart devices. A rule engine may be implemented to enable the interaction between smart devices. The interactive rules may be set by users through the UI unit or via a mobile device. Alternatively, it may be learnt directly via a learning algorithm implemented within the UI unit(s), private cloud or public cloud.
  • A learning model is implemented to enable learning of user behavior.
    • a. Supervised learning is started by using the rule engine to understand the “Things of Interests” (TOI).
    • b. The model learns the user behavior through the data from the smart devices and associated user inputs.
    • c. The model repeats the user behavior when a pre-defined confidence level is reached.
    • d. Reinforce learning is implemented when user correction is made to the machine action.
    • e. Machine learning may be conducted in the public cloud server or private cloud server (which reside within a UI unit) or distributed amongst the number of UI units within a network.
  • A simple rule engine is also implemented in the terminal unit, when a UI unit or network connection is not available, which take the inputs from associated smart devices to execute the controls. An example would be the terminal control would be enabled when user presence is confirmed from an external PIR sensor.
  • Rules related to the terminal unit a downloaded to the terminal unit when it is established and executed accordingly within the terminal unit.
  • A mobile application clones the user interface of the UI unit. The usage model is such that a user is using the UI unit on the spot. For example, when a user uses a mobile app to control a smart device, the UI unit and the control flow of the mobile app cooperate together. An advantage of this approach is a unique user experience in using the smart device and its associated mobile app so that a user does not have to learn two user interface schemes for a single smart device (one in the UI unit and the one in the mobile app).
  • The mobile application is scalable such that it can select all the smart devices which connected to the UI unit. Conversely, it can de-select a smart device from the list.
  • Referring to the embodiment shown in FIG. 7, system 700 includes 2 UI units 701 and 702a, 4 TML units 702b, 703, 704, and 705, mobile app 706, and PC web browser app 707. While not explicitly shown, some of the elements may interact with each other via wired or wireless communication channels. UI1 701 comprises a home console that supports a home gateway, a rule engine, and a connection hub. Device control from UI1 701 may be performed via a capacitve touch panel and/or voice commands. UI2/TML2 pair 702a,702b supports a thermostat in which UI unit 702a and terminal unit 702b are wiredly connected via a proprietary control protocol. TML3 703 may be a simple terminal unit which controls a relay of a heating element. TML4 704 and TML5 705 may be terminal units controling the LED lighting at different zones. There was no dedicated UI for TML3 703, TML4 704, and TML5 705; rather, control was performed via mobile app 706, PC web browser app 707, and/or UI1 701. All the devices may be added to mobile app 706, PC web browser app 707, and/or UI1 701 for smart home control. As shown in FIG. 7, the UI/UX for all UI devices 701 and 702a, mobile app 706, and PC web browser app 707 are the same to enable a good user experience. (Display components 701-755 correspond to UI1 701, UI2/TML2 702a,702b, TML3 703, TML4 704, and TML5 704, respectively.) The status of all the devices are synchronized and displayed in mobile app 706, and PC web browser app 707.

Exemplary embodiments include:

As shown in FIG. 8, a scalable smart control system 800 controls a plurality of controlled apparatuses, where the system comprises:

first and second user-interface (UI) units 801 and 802;
a terminal unit 803 controlling a first controlled apparatus 804;

• • when the first UI unit 801 receives first information, the first UI unit 801 sends the first information to the terminal unit 803 and controls the first controlled apparatus 804 via the terminal unit 803;
  • when the second UI unit 802 receives second information, the second UI unit 802 sends the second information to the terminal unit 803 and controls the first controlled apparatus 804 via the terminal unit 803; and
  • when the first UI unit 801 receives third information, the first UI unit 801 directly controls a second controlled apparatus 805.

As shown in FIG. 9, a scalable smart control system 900 controls a plurality of controlled apparatuses, where the system comprises:

a user-interface (UI) unit 901;
a first terminal unit 902 controlling a first controlled apparatus 904;
a second terminal unit 903 controlling a second controlled apparatus 905;
the UI unit 901 receiving first target data and second target data;
the UI unit sending the first target data to the first terminal unit 902 and the second target data to the second terminal unit 903;
the first terminal unit 902 controlling the first controlled apparatus 904 in accordance with the first target data; and
the second terminal unit 903 controlling the second controlled apparatus 905 in accordance with the second target data.

As shown in FIG. 10, a scalable smart control system 1000 controls a plurality of controlled apparatuses, where the system comprises:

first and second user-interface (UI) units 1001 and 1002;
first and second terminal units 1003 and 1004 controlling first and second controlled apparatuses 1005 and 1006, respectively;
a connection hub 1007;
the first UI unit 1001 receiving first control data and sending a subset of the first control data to a cloud computing resource 1008;
the first UI unit 1001 receiving, from the cloud computing resource 1008, a processed subset of the first control data;
the first UI unit 1001 obtaining processed first control data from the processed subset;
the second UI 1002 unit receiving second control data;
the first UI unit 1001 sending the processed first control data and the second UI unit 902 sending the second control data to the connection hub 1007;
the connection hub 1007 collecting and routing the processed first control data and the second control data to the first and second terminal units 1003 and 1004, respectively;
the first terminal unit 1003 controlling the first controlled apparatus 1005 based on the processed first control data; and
the second terminal unit 1004 controlling the second controlled apparatus 1006 based on the second control data.

As shown in FIG. 11, a scalable smart control system 1100 controls at least one controlled apparatus, where the control flow and the user interface graphics of a UI unit in a wireless device app 1107 (executed by a wireless device) is identical to that in the physical UI unit 1101, 1102, and 1103. The status of each UI unit in the dash board of the mobile app 1107 is identical to the current status shown in the physical UI unit 1101, 1102, and 1103. A user can choose a UI unit from the dash board of the wireless device app 1107 for further manipulation, as it is done on the spot. This eliminates the learning needs in using the wireless device app 1107 and allows a unified user experience in using the UI unit as well as the wireless device app 1107.

Physical UI unit 1101, 1102, 1103 controls smart devices 1104, 1105, and 1106, respectively, and may do so through terminal units not explicitly shown in FIG. 11.

The wireless device may include a computing device for executing wireless device app 1107. may include a computing device wherein the processes discussed herein may be implemented. The computing device may have a processor for controlling overall operation of the computing device and its associated components, including RAM, ROM, communications module, and memory device. The computing device typically includes a variety of computer readable media. Computer readable media may be any available media that may be accessed by computing device and include both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise a combination of computer storage media and communication media.

As shown in FIG. 12, a scalable smart control system 1200 controls at least one controlled apparatus, where the system comprises:

first and second user-interface (UI) units 1201 and 1202;
a terminal unit 1203 controlling a controlled apparatus through the second UI unit 1201;
the first UI unit 1201 comprising a near-field communications (NFC) interface 1206;
the first UI unit 1201 receiving configuration data about the controlled apparatus 1204 from a NFC device 1205 through the NFC interface 1206;
the first UI unit 1201 routing the configuration data to the second UI unit 1202;
the second UI unit 1202 sending the configuration data to the terminal unit 1203; and
the terminal unit 1203 controlling the controlled apparatus 1204 in accordance with the configuration data.

With another aspect of the embodiments, rules may be automatically generated via machine learning program module. Machine learning may be applied locally in the UI unit of a local loop. Machine learning may also be applied to information received from multiple UI units of a larger network. With some embodiments, the machine learning program module may be executed within a single UI unit that has system hardware with better performance. A machine learning program module may also be distributed over different UI units to reduce the computation loading for individual UI units. The resulting rules determined by the distributed machine learning module may be loaded to the corresponding terminal units to reduce response latency. With some embodiments, the machine learning program module and resulting rules may be executed when the internet is disconnected.

One or more aspects of the disclosure may be embodied in computer-usable data or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices to perform the operations described herein. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types when executed by one or more processors in a computer or other data processing device. The computer-executable instructions may be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, RAM, and the like. The functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGA), and the like. Particular data structures may be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated to be within the scope of computer executable instructions and computer-usable data described herein.

As can be appreciated by one skilled in the art, a computer system with an associated computer-readable medium containing computer-executable instructions for controlling the computer system can be utilized to implement the exemplary embodiments that are disclosed herein. The computer system may include at least one computer such as a microprocessor, digital signal processor, and associated peripheral electronic circuitry.

Aspects of the invention have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications and variations within the scope and spirit of the disclosed invention will occur to persons of ordinary skill in the art from a review of this entire disclosure. For example, one of ordinary skill in the art will appreciate that the steps illustrated in the illustrative figures may be performed in other than the recited order, and that one or more steps illustrated may be optional in accordance with aspects of the disclosure.

Claims

1. One or more non-transitory computer-readable media storing instructions that, when executed by a computing device comprising at least one processor, memory, and a communication interface, cause the computing device to:

generate a first control flow and a first user graphics of a first generated user interface identical to that in a first physical user interface;

generate a second control flow and a second user graphics of a second generated user interface identical to that in a second physical user interface;

display a dash board with a first status of the first generated user interface and a second status of the second generated user interface;

receive an selection input to select one of the generated user interfaces of the dash board, wherein the selected generated user interface corresponds to a corresponding physical user interface; and

modify the selected generated user interface based on inputted manipulations.

2. The one or more non-transitory computer-readable media of claim 1 storing instructions that, when executed by the computing device, further cause the computing device to:

control at least one smart device associated with the corresponding physical user interface.

3. The one or more non-transitory computer-readable media of claim 2 storing instructions that, when executed by the computing device, further cause the computing device to:

select one or more of the at least one smart device associated with the corresponding physical user interface.

4. The one or more non-transitory computer-readable media of claim 1 storing instructions that, when executed by the computing device, further cause the computing device to:

synchronize a current status of the first physical user interface with the first status of the first generated user interface.

5. The one or more non-transitory computer-readable media of claim 1 storing instructions that, when executed by the computing device, further cause the computing device to:

control a controlled device in accordance with the inputted manipulations through the selected generated user interface.

6. The one or more non-transitory computer-readable media of claim 5 storing instructions that, when executed by the computing device, further cause the computing device to:

update a third status of the corresponding physical user interface from a fourth status of the selected generated user interface.

7. The one or more non-transitory computer-readable media of claim 5 storing instructions that, when executed by the computing device, further cause the computing device to:

subsequently control the controlled device in accordance with a user input through the corresponding physical user interface.

8. The one or more non-transitory computer-readable media of claim 7 storing instructions that, when executed by the computing device, further cause the computing device to:

update the fourth status of selected generated user interface from the third status of the corresponding physical user interface.

9. A method for controlling a controlled apparatus in conjunction with a plurality of user interface (UI) units and at least one terminal unit, wherein the plurality of UI units includes first and second UI units, wherein the at least one terminal unit includes a first terminal unit, and wherein the controlled apparatus is controlled through the terminal unit, the method comprising:

distributing execution of a machine learning program module over the first and second UI units;

receiving, by the first UI unit, operational information from the second UI unit;

determining, by the first UI unit, a first rule about operation of the controlled apparatus;

determining, by the second UI unit, a second rule about the operation of the controlled apparatus;

downloading the first and second rules into the terminal unit; and

instructing, by the terminal unit, the controlled apparatus based on the first and second rules.

10. The method of claim 9, further comprising:

executing the machine learning program module, the first rule, and the second rule when the plurality of UI units and the at least one terminal unit are disconnected from an internet.

11. The method of claim 9, further comprising:

locally applying machine learning in one of the plurality of UI unit of a local loop, wherein said one UI unit is paired with an individual terminal unit.

12. A method for controlling a controlled apparatus in conjunction with a plurality of user interface (UI) units, wherein the plurality of UI units includes first and second UI units, the method comprising:

obtaining first and second performance metrics from the first and second UI units, respectively;

selecting one of the first and second UI units based on the first and second performance metrics;

executing a machine learning programing module on the selected UI unit;

obtaining, by the selected UI unit, operational information from at least one of the first and second UI units;

determining, by the selected UI unit, a rule about operation of the controlled apparatus from the operational information; and

instructing the controlled apparatus based on the rule.

13. The method of claim 12, further comprising:

downloading the rule into a terminal unit; and

instructing, by the terminal unit, the controlled apparatus based on the rule.

14. The method of claim 12, further comprising:

executing the machine learning program module and the rule at the selected UI unit when the selected UI is disconnected from an internet.

15. A mobile computing device for controlling a plurality of smart devices, the mobile computing device comprising:

at least one processor;

a wireless communication interface; and

memory storing computer-readable instructions that, when executed by the at least one processor, cause the mobile computing device to: generate a first control flow and a first user graphics of a first generated user interface identical to that in a first physical user interface; generate a second control flow and a second user graphics of a second generated user interface identical to that in a second physical user interface; display a dash board with a first status of the first generated user interface and a second status of the second generated user interface; receive an selection input to select one of the generated user interfaces of the dash board, wherein the selected generated user interface corresponds to a corresponding physical user interface; modify the selected generated user interface based on inputted manipulations; and control, through the wireless communication interface, at least one smart device associated with the corresponding physical user interface.

16. The mobile computing device of claim 15, wherein the memory storing computer-readable instructions that, when executed by the at least one processor, further cause the mobile computing device to:

select one of the at least one smart device associated with the corresponding physical user interface.

17. The mobile computing device of claim 15, wherein the memory storing computer-readable instructions that, when executed by the at least one processor, further cause the mobile computing device to:

control a first smart device in accordance with the inputted manipulations via the selected generated user interface.

18. The mobile computing device of claim 17, wherein the memory storing computer-readable instructions that, when executed by the at least one processor, further cause the mobile computing device to:

update a third status of the corresponding physical user interface from a fourth status of the selected generated user interface.

19. The mobile computing device of claim 17, wherein the memory storing computer-readable instructions that, when executed by the at least one processor, further cause the mobile computing device to:

subsequently control the first smart device in accordance with a user input through the corresponding physical user interface.

20. The mobile computing device of claim 19, wherein the memory storing computer-readable instructions that, when executed by the at least one processor, further cause the mobile computing device to:

update the fourth status of selected generated user interface from the third status of the corresponding physical user interface.

21. A control system for controlling a controlled device, the system comprising:

a first user-interface (UI) unit;

a second UI unit;

a terminal unit;

a terminal unit is configured to control the controlled device;

the first UI unit and the second UI unit configured to execute a distributed machine learning program module;

the first UI unit configured to receive operational information from the second UI unit and determine a first rule about operation of the controlled device;

the second UI unit configured to determine a second rule about the operation of the controlled device;

the first and second UI units configured to download the first and second rules, respectively, into the terminal unit; and

the terminal unit configured to instruct the controlled device based on the first and second rules.