Abstract: In some aspects, a computing device receives a scan of a code displayed on an order post located near a restaurant, determines that the code is associated with the restaurant, and automatically opens a software application and navigates the software application to an ordering page associated with the restaurant. The computing device initiates receiving, via the software application, input associated with an order, sends the input to a machine learning based software agent executing on a server, receives a predicted response to the input, provides the predicted response as audio output and/or displays the predicted response on the touchscreen display device. After the order is complete, the computing device sends order data associated with the order to the restaurant. After receiving an indication from the restaurant that the order is ready, the computing device indicates that the order is ready to be picked up.

Abstract: In some aspects, an edge appliance is placed in an active mode and causes a software agent that is based on a machine learning algorithm to engage in a conversation to take an order from a customer that is located at an order post. The edge appliance provides, using a communication interface, audio data that includes the conversation, to a communications system of a restaurant. The edge appliance provides, using the communication interface, a content of a cart associated with the order to a point-of-sale terminal of the restaurant. If the edge appliance determines, using the communication interface, that a microphone of the communication system is receiving audio input from an employee, the edge appliance automatically transitions the edge appliance from the active mode to an override mode, enabling the employee to receive a remainder of the order from the customer.

Abstract: In some examples, a server may receive an utterance from a customer. The utterance may be included in a conversation between the artificial intelligence engine and the customer. The server may convert the utterance to text and determine a customer intent based on the text and a user history. The server may determine a user model of the customer based on the text and the customer intent. The server may update a conversation state associated with the conversation based on the customer intent and the user model. The server may determine a user state based on the user model and the conversation state. The server may select, using a reinforcement learning based module, a particular action from a set of actions, the particular action including a response and provide the response to the customer.

Abstract: In some examples, a software agent executing on a server receives a communication comprising a first utterance from a customer and predicts, using an intent classifier, a first intent of the first utterance. Based on determining that the first intent is order-related, the software agent predicts, using a dish classifier, a cart delta vector based at least in part on the first utterance and modifies a cart associated with the customer based on the cart delta vector. The software agent predicts, using a dialog model, a first dialog response based at least in part on the first utterance and provides the first dialog response to the customer using a text-to-speech converter.

Abstract: In some aspects, an order post detects, using one or more sensors, a presence of a customer, determines an identity of the customer, retrieves previous orders of the customer, indicates at least one item in the previous orders, receives an order comprising input that includes an utterance of the customer, modifies the utterance to create a modified utterance, sends the modified utterance to a software agent comprising a natural language processor and one or more classifiers, receives a predicted response to the modified utterance from the software agent, plays back the predicted response via the speaker, determines that the order is complete, receives payment information for the order from the customer, sends order data associated with the order to a restaurant, receives an indication from the restaurant that the order is ready for pickup, and instructs the customer to pick up the order.

Abstract: In some examples, a software agent executing on a server an utterance from a customer. The software agent converts the utterance to text. The software agent creates an audio representation of the text and performs a comparison of the audio representation and the utterance. The software agent creates edited text based on the comparison. For example, the software agent may determine, based on the comparison, audio differences between the audio representation and the utterance, create a sequence of edit actions based on the audio differences, and apply the sequence of edit actions to the text to create the edited text. The software agent outputs the edited text as a dialog response to the utterance.

Abstract: A computer system may connect to various customer-facing devices and manage or automate the order process between a retail store and the customer. The computer system may perform the dialogue and receive an order for items from the retail store and may perform quality control monitoring of the dialogue between customers and employees taking orders. The ordering system may utilize the ordered items in combination with various contextual cues to determine a customer identity which may then be linked to past orders and/or various order preferences. Based on the determined customer identity, the system may provide recommendations of additional order items or order alterations to the customer before personally identifying information has been collected from the customer. The determination of the customer identity and the determination of recommendations may be performed by machine learning algorithms that were trained on customer data and the retail store products.