METHOD, COMPUTER PROGRAM PRODUCT, AND SYSTEM FOR PROVIDING A SENSOR-BASED ENVIRONMENT

Abstract

Method, computer program product, and system to influence a person within an environment having a plurality of items for selection. The method includes capturing, using a first visual sensor disposed within the environment, field of view information for the person, performing analysis on the field of view information using a computing device, and identifying, based on the analysis, one or more first items of the plurality of items that are included within the field of view of the person. The method further includes selecting, based on the identified one or more items, at least one second item of the plurality of items for presentation to the person; and presenting information related to the at least one second item to the person.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit of U.S. provisional patent application Ser. No. 62/064,323, filed Oct. 15, 2014, entitled “Integrated Shopping Environment,” which is herein incorporated by reference.

BACKGROUND

The present disclosure relates to a sensor-based environment, and more specifically, to providing an adaptive personal experience within such an environment using a determined field of view of the person.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates an integrated shopping environment, according to one embodiment.

FIG. 2 illustrates a shopping environment system, according to one embodiment.

FIG. 3 illustrates an integrated shopping environment, according to one embodiment.

FIG. 4 illustrates a system of influencing shopping experience based on a customer field of view, according to one embodiment.

FIGS. 5A and 5B illustrate an example wearable computing device for use in a shopping environment, according to one embodiment.

FIGS. 6A-6C illustrate determining a customer field of view and identifying items included within the field of view, according to one embodiment.

FIG. 7 illustrates several example views of determining a customer focus on an item within the customer's field of view, according to one embodiment.

FIGS. 8A-8C illustrate several example presentations of information related to an item, according to one embodiment.

FIG. 9A illustrates a method of influencing a shopping experience of a customer within a shopping environment, according to one embodiment.

FIG. 9B illustrates a method of determining content to present to a customer based on a customer interest score, according to one embodiment.

To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized on other embodiments without specific recitation. The illustrations referred to here should not be understood as being drawn to scale unless specifically noted. Also, the drawings are often simplified and details or components omitted for clarity of presentation and explanation. The drawings and discussion serve to explain principles discussed below, where like designations denote like elements.

DETAILED DESCRIPTION

Aspects of the current disclosure relate to an integrated environment capable of providing a personalized, automated, and adaptive experience for a person within the environment. A number of different sensor devices may be employed within the environment, and networked with various computing devices such as point-of-sale (POS) terminals, digital signage, servers, and mobile or handheld computing devices to provide a seamless integration of mobile technologies and e-commerce into a traditional experience.

Using one or more visual sensors within the environment, a retailer or other provider may determine a person's field of view, and may compile personal behaviors and determine personal preferences. This data may then be used to provide timely, tailored recommendations in real-time to the person in order to more effectively influence their experience. While generally discussed within the context of a shopping environment, such as a retail store or other commercial environment, it is contemplated that the techniques disclosed herein may be applied to other environments (some non-limiting examples include libraries, museums, classrooms, hospitals, etc.) to provide an adaptive experience for persons included therein.

FIG. 1 illustrates an integrated shopping environment, according to one embodiment. As shown, environment 100 includes a plurality of terminals 105, a plurality of servers 110₁, 110₂ coupled to a network 115, one or more sensors 120 of different types, one or more user devices 140, and one or more other devices 150. In some embodiments, the environment 100 may be integrated into the layout of a retail store, market, or other commercial environment that is known or hereinafter developed.

Terminals 105 generally include any structure that is capable of receiving input from customers and/or producing output to customers within the environment 100. The terminals 105 may include computing systems, portions of computing systems, or devices controllable by computing systems. In one example, a terminal may include a computing device that is communicatively coupled with a visual display and audio speaker(s), as well as being communicatively coupled with one or more input devices. In another example, a terminal may include a visual display and associated driver hardware, but a computing device coupled to the terminal and providing data for display is disposed separately from the terminal. In some embodiments, terminals 105 may be implemented as standalone devices, such as a kiosk disposed on the store floor or monolithic device disposed on a shelf or platform. In some embodiments, terminals 105 may be integrated partially or wholly with other components of the environment 100, such as input or output devices included with shelving or other structural components in the environment (e.g., components used for product display or storage). In some embodiments, terminals 105 may be modular and may be easily attachable and detachable to elements of the environment 100, such as the structural components.

Generally, terminals 105 may be distributed throughout the environment 100 and may enhance various phases of the shopping experience for customers. For example, terminals 105 may include digital signage 108 disposed throughout the environment, such as included in or near aisles, endcaps, displays, and/or shelving in the environment. A customer may view and/or interact with the digital signage 108 as he/she moves through the store environment. The digital signage may be included in a static display or may be movable, such as including digital signage within a customer's shopping cart or basket. Terminals 105 may also include POS terminals 106 that provide a checkout functionality, allowing the customer to complete his/her shopping transaction (e.g., make payment for selected items). In some embodiments, terminals 105 may provide an integrated functionality. For example, the terminals may function in one mode as digital signage, and when engaged by a customer, the terminals function as a POS terminal.

Servers 110₁, 110₂ generally include processors, memory, and communications capabilities, and may perform various computing tasks to support the commercial operation of the environment 100. Servers 110₁, 110₂ communicate using various wired and/or wireless communications methods with terminals 105, sensors 120, and with other networked devices such as user devices 140 and other devices 150. Servers 110₁, 110₂ generally execute computer program code in which input data is received from networked devices, the input data is processed and/or stored by the servers, and output data is provided to networked devices for operation of the environment 100.

Sensors 120 may include various sensor devices, such as video sensors 125, audio sensors 130, and other sensors 135. The other sensors 135 generally include any sensors that are capable of providing meaningful information about customer interactions with the environment, e.g., location sensors, weight sensors, and so forth. Sensors 120 may be deployed throughout the environment 100 in fixed and/or in movable locations. For example, sensors 120 may be statically included in walls, floors, ceilings, displays, or other devices, or may be included in shopping carts or baskets capable of being transported around the environment. In one embodiment, sensors 120 may include adjustable position sensor devices, such as motorized cameras attached to a rail, wire, or frame. In one embodiment, sensors 120 may be included on one or more unmanned vehicles, such as unmanned ground vehicles (UGVs) or unmanned aerial vehicles (UAVs or “drones”). Sensors 120 may also include sensor devices that are included in user devices 140 or other devices 150 (which in some cases may include body-worn or carried devices). User devices 140 and other devices 150 may include passive or actively-powered devices capable of communicating with at least one of the networked devices of environment 100. One example of a passive device (which may be worn or carried) is a near-field communication (NFC) tag. Active devices may include mobile computing devices, such as smartphones or tablets, or wearable devices such as a Google Glass™ interactive eyepiece (Glass is a trademark of Google Inc.). The user devices 140 generally denotes ownership or possession of the devices by customers, while the other devices 150 denotes ownership or possession by the retailer or other administrator of the environment 100. In some cases, other devices 150 may be carried by employees and used in the course of their employment. User devices 140 and other devices 150 may execute applications or other program code that generally enables various features provided by the servers and/or other networked computing devices.

FIG. 2 illustrates a shopping environment system, according to one embodiment. Generally, the system 200 corresponds to the environment 100 described above. System 200 includes one or more processors 205, memory 210, and input/output 250, which are interconnected using one or more connections 240. In one embodiment, system 200 may be included in a singular computing device, and the connection 240 may be a common bus. In other embodiments, system 200 is distributed and includes a plurality of discrete computing devices that are connected through wired or wireless networking. Processors 205 may include any processing element suitable for performing functions described herein, and may include single or multiple core processors, as well as combinations thereof. Processors 205 may be included in a single computing device, or may represent an aggregation of processing elements included across a number of networked devices such as user devices 140, POS terminals 105, etc.

Memory 210 may include a variety of computer-readable media selected for their size, relative performance, or other capabilities: volatile and/or non-volatile media, removable and/or non-removable media, etc. Memory 210 may include cache, random access memory (RAM), storage, etc. Storage included as part of memory 210 may typically provide a non-volatile memory for the networked computing devices (e.g., servers 110₁, 110₂), and may include one or more different storage elements such as Flash memory, a hard disk drive, a solid state drive, an optical storage device, and/or a magnetic storage device. Memory 210 may be included in a single computing device or may represent an aggregation of memory included in networked devices. Memory 210 may include a plurality of modules 211 for performing various functions described herein. The modules 211 include program code that is executable by one or more of the processors 205. As shown, modules 211 include user identification 212, item identification 214, advertising 216, recommendations 218, virtual cart 220, assistance 222, security 224, power management 226, gaming 228, audit 230, loyalty program 232, and inventory 234. The modules 211 may also interact to perform certain functions. For example, a loyalty program module 232 during operation may make calls to user identification module 212, item identification module 214, advertising module 216, and so forth. The person of ordinary skill will recognize that the modules provided here are merely non-exclusive examples; different functions and/or groupings of functions may be included as desired to suitably operate the shopping environment. Memory 210 may also include customer profiles 236 and customer images 238, which may be accessed and/or modified by various of the modules 211. In one embodiment, the customer profiles 236 and customer images 238 may be stored on the servers 110₁, 110₂ or on a separate database.

Input/output (I/O) 250 may include a number of different devices interfacing with various computing devices and with the shopping environment. I/O 250 includes sensors 120, which have been described above. I/O 250 may further include input devices 252 and output devices 254 that are included to enhance the shopping experience for customers. In one embodiment, terminals 105, user devices 140, and other devices 150 may include visual displays and/or audio speakers (examples of the output devices 254), and various input devices 252 (such as cameras, keyboards or keypads, touchscreens, buttons, inertial sensors, etc.). I/O 250 may further include wired or wireless connections to an external network 256 using I/O adapter circuitry. Network 256 may include one or more networks of various types, including a local area or local access network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet). In one embodiment, various networked computing devices of the system 200 are interconnected using a LAN, and one or more computing devices (e.g., servers 110₁, 110₂, user devices 140) include connections to the Internet.

FIG. 3 illustrates an integrated shopping environment, according to one embodiment. The environment 300 includes a plurality of sensor modules 302 disposed in the ceiling 301 of the store. The sensor modules 302 may each include one or more types of sensors, such as video sensors (e.g., cameras), audio sensors (e.g., microphones), and so forth. Sensor modules 302 may also include actuating devices for providing a desired sensor orientation. Sensor modules or individual sensors may generally be disposed at any suitable location within the environment 300. Some non-limiting examples of alternative locations include below, within, or above the floor 330, within other structural components of the environment 300 such as a shelving unit 303 or walls, and so forth. In some embodiments, sensors may be disposed on, within, or near product display areas such as shelving unit 303. The sensors may also be oriented toward an expected location of a customer interaction with items, to provide better data about a customer's interaction, such as determining a customer's field of view.

Environment 300 also includes a number of kiosks (or terminals) 305. Generally, kiosks 305 may be configured for performing customer checkout and/or other shopping functions. Each kiosk 305 may each include computing devices or portions of computing systems, and may include various I/O devices, such as visual displays, audio speakers, cameras, microphones, etc. for interacting with the customer. In some embodiments, a customer 340 may have a mobile computing device, such as smartphone 345, that communicatively couples with the kiosk 305 for completing the checkout transaction. In one embodiment, the mobile computing device may execute a store application that is connected to the networked computing systems (e.g., through servers 110₁, 110₂), or may be directly connected to kiosk 305 through wireless networks within the environment (e.g., over Wi-Fi or Bluetooth). In one embodiment, the mobile computing device may couple to the kiosk 305 when brought within range, e.g., using Bluetooth or NFC.

Environment 300 also includes one or more shelving units 303 having shelves 310 that support various store items 315. Though not shown, multiple shelving units 303 may be disposed in a particular arrangement in the environment 300, and the space between adjacent shelving units may form aisles through which customers may travel. In some embodiments, the shelving unit 303 may include visual sensors or other sensor devices or I/O devices. The sensors or devices may couple to a customer's smartphone 345 and/or other networked computing devices (including servers) within the environment 300. For example, the front portions 320 of shelves 310 may include video sensors oriented outward from the shelving unit 303 to capture customer interactions with items 315 on the shelving unit 305, and the data from the video sensors may be provided to back-end servers for storage and/or analysis. In some embodiments, portions of the shelving unit 303 (such as the front portions 320 of shelves 310) may include indicator lights or other visual display devices or audio output devices that are used to communicate with a customer.

FIG. 4 illustrates a system of influencing shopping experience based on a customer field of view, according to one embodiment. System 400 may be used in coordination with the various shopping environments described herein. Generally, system 400 may share at least portions of several components with the shopping environment system 200, such as processors 205, memory 210, and I/O 250. System 400 may also utilize one or more of the modules 211 to provide various aspects of the system's functionality, such as item identification module 214, advertising module 216, recommendation module 218, and so on.

I/O 250 includes output devices 254 and sensors 120. Output devices 254 include one or more devices for presenting information to customers and include audio output devices 440 and visual output devices 445. The audio output devices 440 may include conventional audio speakers having any suitable form factor (e.g., in a stereo, headphones, etc.), as well as devices using alternative methods of producing sound to a customer, such as bone conduction transducers in a worn device. Visual output devices 445 may include visual displays and various visual indicators such as light emitting diodes (LEDs). Other output devices 450 may provide information to customers through tactile feedback (e.g., haptic devices) or other sensory stimuli. Sensors 120 may include visual sensors 455 which may be carried or worn sensors 460, and distributed sensors 465 that are disposed throughout the shopping environment. Other sensors 470 may be included that are suitable for collecting information about a customer and his/her interactions within the shopping environment. Examples of other sensors 470 may include infrared (IR) sensors, thermal sensors, weight sensors, capacitive sensors, magnetic sensors, sonar sensors, radar sensors, lidar sensors, and so forth.

The visual sensors 455 may be used to capture one or more images 415 of the customer and/or the shopping environment, which may include views from various perspectives (e.g., a customer-worn visual sensor, static or movable visual sensors at various locations in the environment). The images 415 may be stored in memory 210, and may be individually or collectively processed to determine information about customers in the environment and their respective interactions with items in the environment.

Memory 210 includes one or more programs 435 that collectively receive data about customers and the shopping environment, process the received data, and transmit information to customers in order to influence the customers' shopping experiences. Programs 435 may include program code to determine a customer's field of view at a given time, including which items are included in the field of view. In one embodiment, the customer's field of view may be determined directly. For example, a body-worn device may include a visual sensor (i.e., a worn visual sensor 460) that, when the device is worn, gives the visual sensor an orientation that is similar to the orientation of customer's head or eyes (e.g., a forward-looking camera). Images captured from the worn visual sensor may generally reflect the customer's field of view.

In some embodiments, the customer's field of view may be estimated (or determined indirectly) using other sensor measurements. In one embodiment, the customer's field of view may be estimated by determining the orientation of one or both of the customer's eyes. Eye orientation may be determined using worn visual sensors 460 (e.g., an inward-facing camera on a head-worn device) and/or distributed visual sensors 465 (e.g., capturing images of the customer's face and image processing to determine an eye orientation). In other embodiments, the customer's field of view may be estimated by determining the position and/or orientation of the customer's head and/or body using various visual sensor measurements. The customer's field of view may be represented in any suitable data format, such as an image or as coordinate data (e.g., Cartesian, polar, spherical coordinates).

While it is possible that a single visual sensor 455 may be used to determine a customer's field of view, several embodiments employ a combination of a plurality of visual sensors 455 to determine a field of view. These embodiments may be preferred as providing additional data to support a more accurate estimate of the field of view. Additionally, the plurality of visual sensors used to determine a field of view may include visual sensors selected from different categories (worn sensors 460, distributed sensors 465) to provide additional robustness to the collected data.

Programs 435 may also include program code to identify one or more items included within the customer's field of view. The identification process may be performed directly or estimated. One example of direct identification is performing image processing on images collected from a worn, forward-looking camera to visually identify one or more items. Estimating items within a customer's field of view may require combining sensor data with known information about the shopping environment, such as the items in the environment (item data 420) and their relative arrangement or layout within the environment (location data 425).

Programs 435 may also include program code to present information to customers based on the identified one or more items. In some embodiments, the information may be used to influence the customer's shopping experience. The information presented to the customer may include information about the identified items (e.g., nutritional data, pricing, a customer's purchase history of the items, etc.), information encouraging the purchase of identified items (e.g., bringing particular items to the customer's attention, touting the items' features, offering discounts or other promotions on the items, etc.), and information encouraging the purchase of alternatives to the identified items (e.g., highlighting differences of the items, offering discounts, etc.).

To present relevant and persuasive information in real-time to a particular customer, programs 435 may access and analyze various additional data in memory 210 that is related to the customer and perhaps other customers of the shopping environment. In one embodiment, programs 435 may analyze shopping data 430 collected from previous shopping experiences for the particular customer and/or for other customers or groups of customers. For example, shopping data 430 may include customer views and the items included therein, progressions of customer views (showing a customer's interest over time), selection or purchase history for items, and so forth. While shopping data 430 may be compiled and used to generate information to present to customers and influence their shopping experiences in real-time, the shopping data 430 may also be used by the retailer or administrator of the shopping environment to modify the layout of the environment. The shopping data 430 may help the retailer identify trends in customer shopping, and to optimize placement of items within the environment to improve customer sales.

System 400 may also present information to customers based on their personal preferences 405. The preferences 405 may generally be stored in a corresponding customer profile 236, and may reflect preferences that are explicitly specified by a customer, or may be determined based on the customer's historical shopping behavior (e.g., included in shopping data 430). For example, a customer may have an allergy to a particular ingredient, and the customer may enter this allergy information in preferences 405, e.g., using a mobile phone app for the retailer. Accordingly, the system 400 when determining which information to present to the customer may present information that highlights items within the customer's field of view that include the ingredient, and may further suggest alternative items that do not include the ingredient.

A customer's shopping history may also be used to determine customer preferences 405. For example, the customer's determined fields of view and purchase history from shopping data 430 may be processed to deduce which items, combinations of items, and/or aspects of items are preferred by the customer. For example, preferred aspects might include preferred brands, costs, quantities, sizes, ingredients, nutritional properties (e.g., calorie content, fat, sugar, vitamins, minerals, etc.), and so forth. For example, the customer may specify a preference for low-fat foods, and the system may determine recommended items based on the items included in the customer's field of view and the customer's preferences. This may include suggesting a particular item within the field of view for purchase (or alternatively, an item located outside the field of view) and/or advising the customer about the item's properties vis-à-vis the customer's preferences (e.g., reporting fat content).

Of course, a customer's preferences may be included as a logical combination of a plurality of these aspects (e.g., a customer prefers Brand X items to Brand Y, so long as the cost of the Brand X item is no more than 150% of the Brand Y item). In some embodiments, other customers' shopping data may also be used to deduce a particular customer's preferences. Of course, the preferences may be dynamically updated to identify whether deduced preferences are accurate or not. The dynamic updating may be caused by the customer's explicit indication and/or by the customer's shopping patterns following the deduced preference. For example, the system 400 may deduce that a customer has a categorical preference for Brand X items over similar Brand Y items. However, the customer's historical shopping data indicated that the customer looked at a Brand X item (e.g., field of view data) before selecting and purchasing a similar Brand Y item (e.g., field of view data and/or purchase history data). The system in response may adapt or may entirely remove the deduced (and determined inaccurate) preference.

In some embodiments, system 400 may present information to customers that is also based on other programs 410. Examples of programs 410 may include fitness, nutrition, or health goals, money management goals, etc. In some embodiments, the programs 410 themselves may be integrated into a store application and accessible by the customer's mobile computing device or wearable device. In other embodiments, the store application may interface with applications from other providers to determine the customer's goals and present appropriate information during the shopping experience. For example, the system 400 could include a nutrition-oriented program, and may make suggestions for more nutritious items to a customer who is looking at junk food items (e.g., candy).

FIGS. 5A and 5B illustrate an example wearable computing device for use in a shopping environment, according to one embodiment. Portions of wearable computing device 500 may be head-worn or worn on other portions of the body. The wearable computing device 500 includes a housing 505 that includes several structural components. The band 505 may be used as a structural frame, supporting other portions while itself being supported by a customer's head when worn. Other structural components may include nose pieces 520, ear piece 515, and enclosure 512. Enclosure 512 may include a computing device 525, which includes video I/O components 530 and audio I/O components 540. The enclosure formed by the earpiece 515 may also house components, such as a battery 545 for powering the computing device 525. Although not shown, computing device 525 also includes wireless networking components for communicating with other computing devices. The video I/O components 530 may include a forward-looking camera 513 that provides an estimate of the wearer's field of view based on their head orientation, and a transparent prism 514 that is used to project light onto the wearer's retina, displaying information to the wearer over the wearer's natural field of view. Other video I/O components may include an inward-looking camera that is configured to capture the eye orientation of the wearer, a conventional display device (e.g., a LCD), and so forth. Audio I/O components 540 may include one or more microphones and/or audio output devices, such as speakers or bone-conducting transducers.

FIG. 5B illustrates the wear of wearable computing device 500. As shown, scene 550 includes the wearer's natural view 560. A portion of the area of the natural view 560 may be used for displaying an overlay 565 (e.g., using the prism 514) that provides additional information to the wearer. The display of the overlay 565 may be adequately transparent to permit the wearer to continue to observe their natural view through the overlay area. As shown, the overlay 565 includes a map view of the wearer's current location (e.g., the wearer is at W 34^th street). In some embodiments, information may be selected and visually presented in text and/or graphics to a customer wearing such a device to influence his/her shopping experience.

FIGS. 6A-6C illustrate determining a customer field of view and identifying items included within the field of view, according to one embodiment. In scene 600, a shelving unit 603 is depicted having a plurality of shelves 610 that each support and display a number of items 612 that are available for selection and purchase by a customer.

Within the scene 600 are defined a customer's field of view 615 and an area 605 outside the customer field of view. In one embodiment, the customer's field of view 615 may be represented by an image captured from a forward-looking camera. While shown as generally rectangular, the customer field of view 615 may have any suitable alternative shape and size. For example, the customer's actual vision may encompass a significantly larger area, but determining the field of view for purposes of the shopping environment may include applying a threshold or weighting scheme that emphasizes areas that are closer to the center of a customer's vision. Of course, data provided by various visual sensors and/or other sensors may be used to make these determinations.

The field of view 615 may include a plurality of fully included items 620, as well as a plurality of partially included items 625. When determining which items to identify as “included” in the field of view, certain embodiments may categorically include or exclude the partially included items 625. An alternative embodiment may rely on image processing to determine whether a partially included item 625 should be identified as included. For example, if the processing cannot recognize the particular item with a certain degree of confidence, the item may be excluded. In another alternative embodiment, partially included items 625 may be included, and the amount of item inclusion (e.g., the percentage of surface area of the item included) may be used to determine a customer focus or calculate a customer interest score, which are discussed further below.

Items that are fully or partially included in the customer field of view 615 may be recognized by performing image processing techniques on images captured by various visual sensors. For example, images that include the items may be compared against stock item images stored in a database or server. To aid image processing, items may also include markers or distinctive symbols, some of which may include item identification data such as barcodes or quick response (QR) codes. Of course, other processing techniques may be employed to recognize a particular item, such as textual recognition, determining the item's similarity to adjacent items, and so forth.

Scene 650 of FIG. 6B depicts a customer 660 in a shopping environment. The customer 660 is standing in an aisle 655 adjacent to a shelving unit 603, which has a plurality of shelves 610. Visual sensors may capture one or more images of scene 650 from various spatial perspectives, and the images may be used to determine the customer's field of view. Specifically, various aspects of the scene that are captured in the images may be used to estimate the customer's field of view.

In one embodiment, the relative position and/or orientation of portions of the customer's body may be determined. In one embodiment, the position and orientation of the customer's eyes 680 may be determined. For example, eye position within the environment may be determined in Cartesian coordinates (i.e., determining x, y, and z-direction values) and eye orientation may be represented by an angle a defined relative to a reference direction or plane (such as horizontal or an x-y plane corresponding to a particular value of z). In other embodiments, other portions of the customer's body may (also) be used to determine the field of view, such as the position and orientation of the customer's head 665, or of one or both shoulders 670. In other embodiments, the customer's interaction with the shelving unit 603 by extending her arm 675 may also be captured in one or more images, and the direction of the extended arm may be used to determine her field of view.

Of course, embodiments may use combinations of various aspects of the scene to determine the customer's field of view. In some embodiments, the combinations may be weighted; for example, data showing a customer 660 reaching out her arm 675 towards a specific item may be weighted more heavily to determine her field of view than the orientation of her shoulders. In some embodiments, the weights may be dynamically updated based on the customer's shopping behaviors following an estimate of the customer's field of view. For example, if a customer reached for (or selected) an item that was not included in the determined field of view, the system may adjust the relative weighting in order to accurately capture the customer's field of view. This adjustment may include determining correlation values between particular captured aspects of the scene to the selected item; for example, the customer's head may be partly turned towards the selected item, but their eye orientation may generally be more closely tied to the selected item. In some embodiments, the correlation values may be more useful where one or more aspects of the scene cannot be determined (e.g., the system may be unable to determine eye orientation for a customer wearing sunglasses, non-optimal visual sensor positioning, etc.).

Scene 685 of FIG. 6C illustrates an overhead view of several customers 660 in a shopping environment. In one embodiment, the view of scene 685 may be represented by an image captured from a ceiling mounted camera, or from a drone.

Certain additional aspects depicted in scene 685 and captured in images may be used to estimate a customer's field of view. In one example, the orientation of customer 660A may be estimated using the relative position of his/her shoulders 670. As shown, a line connecting the two shoulders may be compared to a reference direction or plane (e.g., parallel to the length of shelving unit 603A) and represented by an angle β. In another example, the orientation of customer 660B may be estimated using the orientation of his/her head 665, comparing a direction of the customer's head to a reference direction or plane, which may be represented by an angle γ. Images may also capture a customer 660C interacting with the shelving unit 603B, and the position and/or orientation of the customer's arm 675 may be used to determine the customer's field of view.

FIG. 7 illustrates several example views of determining a customer focus on an item within the customer's field of view, according to one embodiment. While the computing systems may determine information to present to a customer based on all of the items identified within a determined field of view, in some embodiments it may be advantageous to make a further identification of one or more items that the customer is specifically focused on. Presenting information to the customer that is based on the customer-focused items may generally provide a more relevant and more persuasive information to influence the customer's shopping experience.

Items 715A-D are included on a shelf 710 of a shelving unit 703. The determined field of view 705 of a customer may include different groups of items at different times, and the progression of the customer's field of view over time may help determine which item(s) within the field of view are specifically being focused on by the customer. Generally, the customer's focus on a particular item may indicate that the item is merely attracting the customer's attention, or that the customer is deciding whether or not to purchase the item. Either way, understanding the object of a customer's focus may help retailers or suppliers to improve packaging and placement of items or to influence the customer's shopping experience in real-time.

View 1 illustrates a field of view 705 that includes items 715A, 715B, and a portion of 715C. In one embodiment, items 715A and 715B may be included in a customer focus determination due to the items' full inclusion within the field of view 705. Conversely, item 715C may be excluded from a customer focus for being only partially included in the field of view 705. In an alternative embodiment, all three items may be included by virtue of being at least partially included in the field of view 705. In some embodiments, a customer focus on particular items may be a time-based determination. For example, if the customer's field of view 705 remained relatively steady during a preset amount of time (e.g., 5 or 10 seconds), such that both items 715A, 715B remained within the field of view during this time, the computing system may determine that the customer is focused on items 715A-C. In some embodiments, the particular item(s) must continuously remain in the field of view 705 during the preset amount of time (e.g., remain across several samples of the field of view during this time).

View 2 illustrates a field of view 705 that includes item 715B, and portions of items 715A and 715C. View 2 could represent the same field of view as View 1 at a later time. In some embodiments, the customer's focus may be determined to include item 715B but not 715A. For example, the system could determine that 715B is the lone customer-focused item based on its relatively central position within the field of view 705, or perhaps based on the changes to the field of view from View 1 (i.e., a shift away from previously fully included item 715A). In one embodiment, item 715B must also remain within the field of view 705 for the predetermined amount of time to be considered as a customer-focused item. Of course, other methods are possible to identify one or more relatively prominent items within a field of view to determine a customer's focus, such as determining of the percentage of the item that is included within the field of view, determining the percentage of the field of view occupied by the item, and so forth. In alternative embodiments, the customer-focused items may still include item 715A and/or item 715C.

View 3 illustrates a field of view 705 that includes item 715B, and portions of items 715A and 715C. View 3 differs from View 2 in that the field of view 705 is “closer” to the shelving unit 703 and items 715 in View 3 than in View 2. View 3 could represent the same field of view as View 2 at a later time, e.g., as the customer moves towards the shelving unit 703. Here, the system could determine that 715B is a customer-focused item based on its relatively central position within the field of view 705 and/or based on the changes to the field of view from View 2 (i.e., item 715B occupies an increased percentage of the field of view 705).

View 4 also illustrates a field of view 705 that includes item 715B, and portions of items 715A and 715C. View 4 could represent the same field of view as View 3 at a later time. In View 4, the customer has selected item 715B and is holding the item in his/her hand 725. Here, the system could determine that 715B is a customer-focused item based on its relatively central position within the field of view 705, based on the changes to the field of view from View 3 (e.g., item 715B occupies an increased percentage of the field of view 705), and/or based on the presence of a portion 730 of the customer's hand 725 included in the field of view 705.

In some embodiments, a customer interest score may be determined for items included within the customer's field of view. The customer interest score is generally time-based, and score may be used to timely interject desired information in order to influence a customer's shopping decisions (e.g., whether to buy a particular item) in real-time. In one embodiment, a customer interest score is calculated and updated concurrent with determining a customer focus, and the customer interest score for an item exceeding a threshold value may be used to determine that the customer has focused on the item. The customer interest score may also be adjusted based on various other aspects shown in the Views 1-4, such as position within the field of view (e.g., a central position may mean more interest), percentage occupied within the field of view (e.g., a larger percentage may mean more interest), customer selection and/or manipulation of the item, and so forth. Of course, the calculation of a customer's interest score may reflect their particular shopping behaviors (e.g., a customer is determined to be 95% likely to pick up and manipulate a customer-focused item within their field of view, so the mere fact of picking up an item might not indicate an increased level of customer interest in the item).

After determining that a customer has focused on a particular item, the customer interest score may generally decrease over time as it becomes more likely that the customer will not purchase the particular item. To maintain or improve customer interest, and to thereby increase the likelihood that the customer will purchase the item, the system may provide information about the item at desired times (corresponding to customer interest scores). This may include providing information about the items or alternatives, and/or offering promotional pricing on the item (or an alternative item). In one embodiment, to provide more effective promotions (e.g., more customer-tailored and more timely), and to prevent customer abuse of a time-based promotional pricing scheme, the customer shopping history may include data regarding the number of promotions already offered to the customer, the length of time (or customer interest score) before offering the promotion, the customer utilization rate of the previous promotions, etc.

FIGS. 8A-8C illustrate several example presentations of information related to an item, according to one embodiment. As discussed above, the presentation of information may be based upon some or all of the items that are identified as being within the customer's field of view. For ease of description, FIG. 8A illustrates a single item 805 within the customer's field of view 705; this could also represent an example in which one item is identified from several items as being a customer focused item. However, similar techniques may be applied to a field of view or a customer focus that includes more than one item. In some embodiments, the computing system may present information to the customer about a particular item, based on the item 805. The presented information may relate to the same item 805 and/or to one or more different items. The different items may include other items located within the customer's field of view 705, and may include items (such as item 810) that are not included within the field of view 705. In some embodiments, the information may be determined and timely presented so as to influence the customer to purchase the particular item(s).

FIG. 8B illustrates several example presentations of information to a customer. The presentations may be delivered in real-time to the customer using any suitable manner, such as a visual output (e.g., graphical and/or textual), audio-based output, and/or other sensory output (e.g., haptic). While the presentations 815, 820, 825 are shown as textual presentations for simplicity, it is possible for the presentations to be spoken or presented in any suitable alternative manner. In one example, a presentation may be displayed by a wearable computing device of the customer, such as a visual overlay on a display of the wearable computing device as discussed above with respect to FIGS. 5A and 5B. In another example, a presentation may be transmitted as a message to the customer's mobile computing device, causing text or graphics to be displayed on the mobile computing device and causing the device to vibrate.

Presentation 815 indicates a customer preference for a second item 810 to the currently viewed or focused item 805. Presentation 815 generally suggests that the customer may also prefer the second item 810. The customer preference may be determined based on compiled previous shopping data for other customers. The shopping data may include separate statistics for the two items, for example, based on historical purchase data for each item. The shopping data may also include compiled information that directly interrelates the two items. For example, in previous shopping experiences, other customers also focused on item 805 (e.g., based on determined fields of view) but ultimately declined to purchase item 805 in favor of item 810.

Presentation 820 indicates that a promotion is available for the second item 810. Presentation 820 may reflect a sales promotion or prices already in effect (e.g., a reporting function), or the promotion may be dynamically generated to encourage the customer to purchase item 810. For example, manufacturers or stores may wish to promote item 810 over item 805, and the promotion may be presented to the customer in real-time. As discussed above, in some embodiments, the promotion may be determined, as well as the timing of its presentation to the customer, based on a determined level of customer interest in an item.

Presentation 825 indicates that item 805 would have a negative impact on the customer's goals. The presentation 825 may be based on customer profile data such as customer preferences and/or programs. Examples of programs may include fitness, nutrition, or health goals, money management goals, etc. Although not shown here, one embodiment may present the information to the customer upon which the negative impact determination was made. If the system determines that item 805 is incompatible with specified goals or preferences, or that better alternative items exist consistent with the customer's goals/preferences, the system may recommend one or more alternative items for presentation to the customer.

FIG. 8C illustrates several example presentations of information to a customer. Like the presentations discussed above, the presentations may be delivered in real-time to the customer using any suitable manner, such as a visual output (e.g., graphical and/or textual), audio-based output, and/or other sensory output (e.g., haptic). The presentations may be output using a customer's worn or carried computing device, or using output hardware (e.g., displays, speakers, etc.) that is disposed within the shopping environment. For example, LEDs or other visual display elements may be included in the shelving units, such as display elements 835a-b and 836c-d.

As shown, a single item 805 is included within a customer's field of view 705, and item 810 is located outside the field of view. Of course, similar techniques may be applied to a field of view or a customer focus that includes more than one item. In cases where the item 805 (already included in the field of view 705) is recommended for presentation to the customer, the system may display various indicators that highlight the item 805 to the user. For example, if the field of view 705 represents the customer view through a head-worn computing device such as wearable computing device 500, the system may display colors, shapes, etc. that visually enhance the item 805, or may reduce the visibility of other items in the field of view (e.g., gray out). The system may also display location information of the item 805, for example, by illuminating one or more of the display elements 835a-b nearest to the item 805.

In cases where an item is recommended for presentation that is not disposed within the field of view 705 (e.g., item 810), the system may output directional information from the current field of view 705 to the item. For example, the system may display an arrow 830 overlay indicating the relative direction from the field of view 705 or item 805 to the target item 810. The system may also provide directional information using one or more of the display elements 835, 836. For example, the display elements 835, 836 may be illuminated in sequence from left to right to guide the customers' eye toward item 810. Of course, the various types of information discussed here (e.g., highlighting, location, and directional) may additionally or alternately be provided by non-visual means to the customer, such as audio or haptic outputs.

FIG. 9A illustrates a method of influencing a shopping experience of a customer within a shopping environment, according to one embodiment. Method 900 may generally be used in conjunction with the various environments and systems discussed above. Method 900 begins at block 905, where a field of view is determined for the customer. Determining a field of view may include a “direct” determination, such as capturing an image from a body-worn, forward-looking camera (i.e., the image strongly corresponds to the customer's eye gaze). Alternatively (or in addition to the direct determination), the field of view may be determined “indirectly” by capturing images of the shopping environment and performing image processing to determine various aspects. The aspects may include the determined position and orientation of one or more of the customer's body, head, shoulders, eyes, arms, hands, etc. The images may be captured by visual sensors that are worn or disposed throughout the shopping environment.

At block 915, one or more first items that are included within the customer's field of view are identified. In one embodiment, identifying items may include performing image processing on a captured image that reflects the customer's field of view (e.g., from a forward-looking camera). In another embodiment, identifying items may include referencing coordinate data determined for the customer's field of view against a known store layout that includes item location information. In such an embodiment, image processing may not be necessary to identify the one or more first items.

At block 925, at least one second item is selected for presentation to the customer. The selection is based on the identified one or more first items. In one embodiment, at least one of the second item(s) may be included in the identified first items. In other embodiments, each of the second item(s) may be different than the identified first items. The selection may be based on decisional logic for recommending a particular item for sale, based on historical shopping data for the customer (and/or other customers) and/or customer preferences or programs.

At block 935, information related to the at least one second item is presented to the customer. Presenting information may be done in any suitable manner, such as displaying textual or graphical information in a visual overlay, displaying directional information to the customer, or outputting sound or other feedback. The information may be related to the properties of the second item (e.g., price, nutrition information, etc.) or to historical shopping data, promotions, or other customer-specific preferences or programs. The timing of the presentation may be determined based on customer interest scores or other timing considerations that are collectively intended to influence a customer's shopping purchases.

FIG. 9B illustrates a method of determining content to present to a customer based on a customer interest score, according to one embodiment. Method 940 may generally be used in conjunction with the various shopping environments and systems discussed above. In one embodiment, method 940 is performed as part of performing method 900. Method 940 begins at block 945, where a customer focus is determined on at least one item selected from one or more items that are included within the customer's field of view. The customer focus may be time-based, such as requiring the item to remain within the field of view for a minimum amount of time.

At block 955, a customer interest score is determined for the at least one customer-focused item. The customer interest score may also be time-based, and may be influenced by various characteristics of the interaction of the customer with the item, which may be observed through the determined customer field of view. In one embodiment, the customer interest score may be determined concurrently with determining the customer focus on an item. In one embodiment, the customer-focused item is determined based on the customer interest score for that item reaching a predetermined threshold value. In another embodiment, the customer-focused item is determined by the item's score that exceeds scores for other items within the field of view.

At block 965, the content of the information to present to the customer is determined based on the determined customer interest score. Determining the content may include determining whether to present information for the customer-focused item or for an alternate item, whether or not to present a promotion for the item or the alternate item, determining an amount of the promotion, and so forth.

The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

In the preceding, reference is made to embodiments presented in this disclosure. However, the scope of the present disclosure is not limited to specific described embodiments. Instead, any combination of the following features and elements, whether related to different embodiments or not, is contemplated to implement and practice contemplated embodiments. Furthermore, although embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the scope of the present disclosure. Thus, the following aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s). Likewise, reference to “the invention” shall not be construed as a generalization of any inventive subject matter disclosed herein and shall not be considered to be an element or limitation of the appended claims except where explicitly recited in a claim(s).

Aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module,” or “system.”

The present disclosure may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

Embodiments of the disclosure may be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Thus, cloud computing allows a user to access virtual computing resources (e.g., storage, data, applications, and even complete virtualized computing systems) in “the cloud,” without regard for the underlying physical systems (or locations of those systems) used to provide the computing resources.

Typically, cloud computing resources are provided to a user on a pay-per-use basis, where users are charged only for the computing resources actually used (e.g., an amount of storage space consumed by a user or a number of virtualized systems instantiated by the user). A user can access any of the resources that reside in the cloud at any time, and from anywhere across the Internet. In context of the present disclosure, a user may access applications (e.g., a retail store app for a mobile computing device) or related data (e.g., compiled shopping data) available in the cloud. Doing so allows a user to access this information from any computing system attached to a network connected to the cloud (e.g., the Internet).

While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.

Claims

1. A computer-implemented method to influence a person within an environment having a plurality of items for selection, the method comprising:

capturing, using a first visual sensor disposed within the environment, field of view information for the person;

performing analysis on the field of view information using a computing device;

identifying, based on the analysis, one or more first items of the plurality of items that are included within the field of view of the person;

selecting, based on the identified one or more first items, at least one second item of the plurality of items for presentation to the person; and

presenting information related to the at least one second item to the person.

2. The computer-implemented method of claim 1, wherein capturing field of view information for the person includes using the first visual sensor to determine at least one of an eye orientation, a head orientation, and a body orientation of the person.

3. The computer-implemented method of claim 1, wherein performing analysis on the field of view information includes comparing the field of view information with location data for the plurality of items of the environment.

4. The computer-implemented method of claim 1, wherein performing analysis on the field of view information includes performing image processing upon one or more images of portions of the environment captured using at least a second visual sensor disposed within the environment.

5. The computer-implemented method of claim 1, further comprising:

determining a focus of the person on at least one item selected from the one or more first items included within the person's field of view, wherein determining a focus is based on inclusion of the at least one selected item within the person's field of view for at least a predetermined first length of time.

6. The computer-implemented method of claim 5, further comprising:

calculating an interest score of the person for the at least one selected item based on a length of time of inclusion within the person's field of view.

7. The computer-implemented method of claim 6, wherein presenting information related to the at least one second item includes determining content of the information based on the interest score.

8. The computer-implemented method of claim 1, wherein selecting the at least one second item is based on data compiled from one or more previous experiences of one or more persons.

9. The computer-implemented method of claim 1, wherein selecting the at least one second item is based on personal preferences included in a personal profile associated with the person.

10. The computer-implemented method of claim 1, wherein the at least one second item is not included within the person's field of view, and wherein presenting information related to the at least one second item includes indicating a direction to a location of the at least one second item within the environment.

11. A computer program product to influence a person within an environment having a plurality of items for selection, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to: capture, using a first visual sensor disposed within the environment, field of view information for the person; perform analysis on the field of view information using a computing device; identify one or more first items of the plurality of items that are included within the field of view of the person; select, based on the identified one or more first items, at least one second item of the plurality of items for presentation to the person; and present information related to the at least one second item to the person.

12. The computer program product of claim 11, wherein performing analysis on the field of view information includes performing image processing upon one or more images of portions of the environment captured using at least a second visual sensor disposed within the environment.

13. The computer program product of claim 11, wherein the computer-readable program code is further executable to:

determine a focus of the person on at least one item selected from the one or more first items included within the person's field of view, wherein determining a focus is based on inclusion of the at least one selected item within the person's field of view for at least a predetermined first length of time.

14. The computer program product of claim 13, wherein the computer-readable program code is further executable to:

calculate an interest score of the person for the at least one selected item based on a length of time of inclusion within the person's field of view.

15. The computer program product of claim 14, wherein presenting information related to the at least one second item includes determining content of the information based on the interest score.

16. A system to influence a person within an environment having a plurality of items for selection, the system comprising:

one or more computer processors;

a first visual sensor disposed within the environment and communicatively coupled with the one or more computer processors; and

a memory containing a program which, when executed by the one or more computer processors, performs an operation comprising: capturing, using a first visual sensor disposed within the environment, field of view information for the person; performing analysis on the field of view information using a computing device; identifying, based on the analysis, one or more first items of the plurality of items that are included within the field of view of the person; selecting, based on the identified one or more first items, at least one second item of the plurality of items for presentation to the person; and presenting information related to the at least one second item to the person.

17. The system of claim 16, wherein the first visual sensor is included in a body-worn computing device of the person.

18. The system of claim 16, further comprising a plurality of visual sensors distributed throughout the environment, and wherein performing analysis on the field of view information includes performing image processing upon one or more images of portions of the environment captured using the plurality of visual sensors.

19. The system of claim 16, further comprising an output device communicatively coupled with the one or more computer processors, wherein the output device is configured to present the information related to the at least one second item to the person using at least one of audio and visual output.

20. The system of claim 19, wherein the output device is one of a handheld computing device and a body-worn computing device of the person.