INFORMATION HANDLING SYSTEM AUGMENTED REALITY THROUGH A VIRTUAL OBJECT ANCHOR
A virtual object anchor stores an offset in memory that defines a virtual object position, orientation and scale relative to the location of the virtual object anchor. Information handling systems retrieve and apply the offset to generate a virtual object in a head mounted display that is presented at a location relative to the virtual object defined by the offset. In one embodiment, gestures detected by information handling system sensors that change the position, orientation or scale of the virtual object are applied to update the offset and communicated for storage at the virtual object anchor. In another embodiment, gestures detected by sensors of the virtual object anchor are applied to update the offset so that the updated offset is communicated to the information handling systems for presentation of the virtual object at an updated position, orientation and/or scale determined from the gestures.
Latest Dell Products L.P. Patents:
- Cable free networking device stacking system
- Automatically processing user request data using artificial intelligence techniques
- Handle release for locking and unlocking a riser cage to a stabilizing feature of a computing device
- Outcome-based compliance and security configurations for information technology environments
- Recycled aluminum with glass fiber reinforced polylactic acid (PLA) bioplastic for an information handling system
The present invention relates in general to the field of information handling system visual information presentation, and more particularly to an information handling system augmented reality through a virtual object anchor.
Description of the Related ArtAs the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
Information handling systems often process information to present the information at a display as a visual image. Typically information handling systems include graphics systems that process visual information into pixel values that define an image at a display. Conventional information handling systems typically interface with flat panel displays that present the visual information as two dimensional visual objects. Enterprises that do design work often use information handling systems with powerful processing capabilities and graphics chips to rapidly process and present complex visual information. For example, computer aided design (CAD) applications define a product, building or other designed object in software and then render the object as a visual image that an end user can manipulate, such as by changing the object's orientation or peeling back layers of the object to view structures under the object. CAD applications greatly simplify design work by aiding end user visualization of the designed object before prototypes are built.
Recently, head mounted displays have entered enterprise work places as a tool to help designers visualize designed objects. Head mounted displays generally operate by projecting an image within head gear worn by an end user and in front of the end user's eyes. Head mounted displays can project the designed object in three dimensions to appear as if the object is in space at a location in front of the end user. An engineer wearing a head mounted display can view a three dimensional image of a designed object within arm's length and then reach out and manipulate the designed object with gestures, such as grasping at the projected location of the three dimensional image. Head mounted displays generally work in a virtual reality or an augmented reality. Virtual reality images are presented against a darkened background so that the end user views just the displayed virtual reality. Augmented reality images are presented against an opening that lets the end user see the “real” world in front of him while projecting the three dimensional image against the real world background. Thus, for example, an end user manipulating a virtual object in virtual reality cannot directly view his hands during the manipulation; in contrast, an end user manipulating a virtual object in augmented reality can view his hands as they touch and gesture relative to the virtual object. Augmented reality tends to provide a more intuitive interaction with end user gestures.
Often enterprise designers collaborate on design projects with different individuals making different contributions to the design. A difficulty with augmented reality in a collaborative environment is that each end user has his own three dimensional virtual object presented through his own head mounted display. Thus, a “real-life” collaboration when end users each wear their own head mounted display does not generally involve coordinated presentation of the virtual object so that end users interact at the same virtual object location. Efforts in industry to implement collaborative solutions for augmented reality that let each end user work on the same virtual object generally attempt to merge and synchronize the “virtual spaces” of separate head mounted displays using inside-out tracking data. Head mounted display tracking for collaboration purposes has limitations in distance accuracy, field of view, processing power and user tracking.
SUMMARY OF THE INVENTIONTherefore, a need has arisen for a system and method which provides information handling system augmented reality through a virtual object anchor.
In accordance with the present invention, a system and method are provided which substantially reduce the disadvantages and problems associated with previous methods and systems for collaborating between multiple users interacting with an augmented reality object. A virtual object anchor stores an offset that defines a virtual object position, orientation and scale relative to a position of the virtual object anchor. Information handling systems retrieve the offset an apply the offset to generate a virtual object in a head mounted display at a location relative to the virtual object anchor defined by the offset.
More specifically, a virtual object anchor integrates a processor, memory, network interface device and sensors in a portable housing configured to rest on a desktop surface. Instructions stored in non-transitory memory of the virtual object anchor execute on the processor to store an offset that defines a position, orientation and scale of a virtual object relative to the housing and that communicates the offset to information handling systems proximate the object. Information handling systems apply the offset to generate a three dimensional visual image of a virtual object in a head mounted display at a location defined by the offset. As an information handling system detects gestures of an end user that change the virtual object's presentation, the information handling system updates the offset and communicates the updated offset to the virtual object anchor. The virtual object anchor stores the updated offset and communicates the updated offset to other information handling systems so that the other information handling systems render the virtual object as updated by the gestures. Alternatively, sensors integrated in the virtual object anchor detect gestures at the position defined by the offset and apply the detected gestures to update the offset so that information handling systems can retrieve the updated offset and render the virtual object against a common coordinate system. In one embodiment, position information determined to a head mounted display by sensors of the virtual object anchor is compared with position information determined to the virtual object anchor by sensors of the head mounted display to calibrate the presentation position of the virtual object.
The present invention provides a number of important technical advantages. One example of an important technical advantage is that a virtual object presented at a head mounted display correlates in its relative physical position to virtual objects of other head mounted displays so that end users wearing the head mounted displays can collaborate with interactions at the virtual objects. The location of the virtual object is defined by offset values stored in a physical device, referred to herein as the virtual object anchor, that each head mounted display retrieves. Each head mounted display then applies the offset and the head mounted display's position relative to the virtual object anchor to determine a location of the virtual object, and presents the virtual object at the determined location. As an end user interacts with the virtual object, changes made to the presentation of the virtual object are stored locally on the virtual object anchor with updated offset information that allows other head mounted displays to match the virtual object's relative position. In one embodiment, multiple virtual object anchors cooperate through networked communications so that the virtual object anchors at different physical locations coordinate the relative positioning of the virtual object.
The present invention may be better understood, and its numerous objects, features and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference number throughout the several figures designates a like or similar element.
A virtual object anchor coordinates presentation of a virtual object by plural information handling systems through plural head mounted displays by storing an offset of the virtual object's position relative to the virtual object anchor and providing the offset to the plural information handling systems. For purposes of this disclosure, an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an information handling system may be a personal computer, a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, and a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.
Referring now to
In the example embodiment, information handling system 10 interfaces with a head mounted display 32 that rests on an end user head to present visual images as three dimensional objects as the end user views real objects through a clear visor 52. In the example embodiment, head mounted display 32 has a CPU 12 that executes instructions stored in RAM 14 to coordinate presentation of visual images, such as operating code retrieved by CPU 12 to RAM 14 from non-transient flash memory. Head mounted display 32 includes a wireless network interface card 20 that communicates through a wireless or wired (e.g. HDMI, USB, DP) interface 38 to information handling system 10, such as to retrieve visual information to present as visual images at display 32. Display 32 is, for instance an LCD projector integrated in head mounted display 32 presents a three dimensional object to appear as a virtual object 70 at a location beyond the clear visor 52, such as at an arm's reach of the end user. Head mounted display 32 includes one or more accelerometers to detect motion, such as configured as an inertial motion detector that senses orientation. Head mounted display 32 includes a magnetic compass 36 that provides a reference axis, such as true north. Various sensors, such as depth camera 30 or ultrasonic sensor, integrated in head mounted display 32 detect objects, such as end user hands that perform gestures. For example, an end user viewing a virtual object 70 can reach with hands to make a rotational movement at object 70 that is detected by depth camera 30. The detected gestures are then communicated to information handling system 10 where augmented reality application 28 and GPU 18 cooperate to adapt the presentation of virtual object 70 to reflect the gesture. In an alternative embodiment, information handling system 10 may be integrated into head mounted display 32 to process and present a virtual object as a contiguous unit.
In order to coordinate presentation of virtual object 70 for multiple end users, a virtual object anchor 54 placed on desktop 46 locally stores a virtual object offset 50 that defines a presentation location, orientation and scale for virtual object 70. Virtual object anchor 54 is a physical object that provides a physical reference point for presentation of a virtual object. For example, the offset is nine data points per frame, including three for position, three for rotation and three for scale. The nine data points describe the virtual object's location in space relative to virtual object anchor 54 so that an information handling system 10 that knows virtual object anchor 54's position and rotation along with the offset can render virtual object 70 relative to virtual object anchor 54 in common with other information handling system's 10. In the example embodiment, two separate information handling systems 10 render virtual object 70 for two separate end user head mounted displays 32. Each information handling system 10 retrieves the same offset from virtual object anchor 54, such as with a Bluetooth interface between each information handling system 10 and virtual object anchor 54. Each head mounted display 32 includes sensors that define a user position vector 48 from the head mounted display 32 to virtual object anchor 54. Once a virtual object 70 offset 50 and user position vector 48 are known to augmented reality application 28, augmented reality application 28 cooperates with GPU 18 to render virtual object 70 in head mounted display 32 to have a common location presentation across multiple head mounted displays 32. For example, each end user sees virtual object 70 in a position, orientation and scale fixed relative to virtual object anchor 54. For instance, end user's located at opposite sides of virtual object 70 will see opposite sides of virtual object 70. Although
In the example embodiment, virtual object 70 has presentation bifurcated across two or more databases. Information handling systems 10 interface through a network 40 with a server information handling system 42 that has a CAD database 44 that stores a CAD model of virtual object 70. A CAD application 26 on each information handling system 10 retrieves the model from CAD database 44 to render the model with GPU 18 as a three dimensional visual image for presentation by a head mounted display 32. Augmented reality application 28 adjusts the presentation position, orientation and scale based upon virtual object offset 50 to render the image in a fixed location relative to virtual object anchor 54. If the end user alters virtual object 70's position, orientation and/or scale with a gesture, an update to offset 50 is determined based upon the end user interaction and stored to virtual object anchor 54 so that all end user's viewing virtual object 70 can retrieve the updated offset and render virtual object 70 with updated position, orientation and scale based upon the detected gestures. Bifurcation of model and offset data provides a powerful collaboration tool. For example, an end user may elect to manipulate virtual object 70 without storing an updated offset so that the end user can consider alternative views alone, and then update offset 50 when ready to share with others. At any time, all end users viewing virtual object 70 may share a common view rendered at each head mounted display 32 by retrieving offset 50 from memory of virtual object anchor 54 and applying offset 50 with user position vector 48 to render virtual object 70. In alternative embodiments, CAD database 44 may be collocated with offset information, such as at the same server information handling system or stored in virtual object anchor 54. For example, virtual object anchor 54 may itself include resources to act as a server information handling system and/or a data storage server that has memory and network bandwidth sufficient to support communication of a CAD model to client information handling systems for presentation at head mounted displays.
Referring now to
Advantageously, storage and sensors integrated in virtual object anchor 54 provide flexible interactions with multiple information handling systems for presenting a common virtual object and manipulating that object. Storing a virtual object offset in memory of virtual object anchor 54 lets multiple information handling systems retrieve the offset information an present the virtual object at a common coordinate system referenced to virtual object anchor 54, such as true north or a visually distinct marking on virtual object anchor 54. In one embodiment, information handling systems retrieve the offset as needed, such as at regular intervals or upon receiving an alert that the offset has changed. For example, if one end user makes a gestured detected by the end user's sensors, the end user's information handling system communicates an updated offset to virtual object anchor 54 for storage in memory. The information handling system may alert collaborating information handling systems of the change or, alternatively, virtual object anchor 54 may broadcast an alert that an updated offset is available for download. In one embodiment, virtual object anchor 54 may broadcast the offset at regular intervals, such as with a Bluetooth beacon, so that any information handling system in range get the offset by listening to the broadcast. In an alternative embodiment, sensors 56 of virtual object anchor 54 detect end user gestures, compute an updated offset based upon the detected gestures, and provide the updated offset to the information handling to apply in rendering the virtual object. As is set forth below in greater detail, sensors 56 also provide a way to calibrate end user presentation and gesture input by measuring the relative position of head mounted displays in proximity to virtual object anchor 54.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Although the present invention has been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without departing from the spirit and scope of the invention as defined by the appended claims.
Claims
1. An information handling system comprising:
- a processor operable to execute instructions to process information;
- a memory interfaced with the processor and operable to store the instructions and information;
- a graphics processor interfaced with the processor and memory, the graphics processor operable to generate visual information that defines a visual image for presentation at a display;
- one or more sensors operable to detect a position of a predetermined virtual object anchor;
- a wireless network interface card operable to establish a wireless interface with the predetermined virtual object anchor and to retrieve offset information stored on the virtual object anchor; and
- a head mounted display interfaced with the graphics processor and operable to generate a visual image presented as a three dimensional virtual object at a location; and
- an augmented reality application executing on the processor to define the location of the three dimensional virtual object by applying the position of the virtual object anchor determined from the sensors and the offset retrieved from the virtual object anchor, the graphics processor interfaced with the augmented reality application and generating the visual information to present the visual image in the head mounted display at the location.
2. The information handling system of claim 1 wherein:
- the one or more sensors detect end user manipulation of the three dimensional virtual object;
- the graphics controller adjusts the appearance of the three dimensional virtual object in response to the detected end user manipulation; and
- the augmented reality application determines an updated offset from the detected end user manipulation and communicates the updated offset to the virtual object anchor for storage on the virtual object anchor.
3. The information handling system of claim 2 wherein the offset comprises position, rotation and scale information.
4. The information handling system of claim 3 wherein:
- the augmented reality application receives updated offset information stored at the virtual object anchor by an external information handling system, the updated offset information reflecting end user manipulation of the three dimensional virtual object by an end user of the external information handling system; and
- the augmented reality application defines an updated location of the three dimensional virtual object that includes the end user manipulation by applying the position of the virtual object anchor determined from the sensors and the updated offset retrieved from the virtual object anchor, the graphics processor interfaced with the augmented reality application and generating the visual information to present the visual image in the head mounted display at the location.
5. The information handling system of claim 4 wherein the virtual object anchor beacons a time reference of the most recent change to the offset information and the augmented reality application retrieves the offset information if a time reference of offset information in use by the augmented reality application is older than the beaconed time reference.
6. The information handling system of claim 4 further comprising a virtual object anchor database interfaced through a network with the virtual object anchor and storing the offset information, the augmented reality application retrieving the offset information through the network from the virtual object anchor database in response to an alert communicated from the virtual object anchor.
7. The information handling system of claim 6 wherein the virtual object anchor is at a first location and a second virtual object anchor is at a second location, each virtual object anchor storing the offset information synchronized through the network and virtual object anchor database.
8. The information handling system of claim 1 wherein the virtual object anchor includes one or more sensors operable to detect a position of one or more head mounted displays and the virtual object anchor stores the position of the one or more head mounted displays.
9. The information handling system of claim 8 wherein the augmented reality application retrieves the head mounted display position stored on the virtual object anchor and applies the head mounted display position to define a location of the virtual object.
10. A method for presenting a virtual object at an information handling system head mounted display, the method comprising:
- retrieving an offset stored on a virtual object anchor to the information handling system;
- determining a position of the virtual object anchor relative to the head mounted display;
- applying the offset and the position of the virtual object anchor to determine a location of a virtual object in the head mounted display; and
- presenting the virtual object at the location in the head mounted display.
11. The method of claim 10 further comprising:
- detecting with sensors integrated in the head mounted display and interfaced with the information handling system a manipulation of the virtual object by a gesture of an end user of the information handling system;
- applying the gestures to manipulate the presentation of the virtual object;
- determining an updated offset from the gestures at the information handling system;
- communicating the updated offset from the information handling system to the virtual object anchor; and
- storing the updated offset on the virtual object anchor.
12. The method of claim 10 further comprising:
- detecting with sensors of the virtual object anchor manipulation of the virtual object by a gesture at a location associated with the offset;
- applying the gesture to update the offset;
- storing the updated offset at the virtual object anchor;
- communicating the updated offset from the virtual object anchor to the information handling system; and
- applying the updated offset to adjust the presentation of the virtual object.
13. The method of claim 12 further comprising:
- communicating the updated offset from the virtual object anchor through a network to a second virtual object anchor disposed at a distal location;
- storing the offset at the second virtual object anchor;
- communicating the updated offset from the second virtual object anchor to a second information handling system located proximate the second virtual object anchor; and
- applying the updated offset to adjust the presentation of the virtual object at a second head mounted display associated with the second information handling system.
14. The method of claim 12 further comprising:
- communicating the updated offset from the virtual object anchor to a second information handling system proximate to the virtual object anchor; and
- applying the updated offset at the second information handling system to adjust presentation of the virtual object in a second head mounted display associated with the second information handling system.
15. The method of claim 10 wherein the offset comprises position, rotation and scale information of the virtual object.
16. The method of claim 10 where in the determining a position of the virtual object anchor relative to the head mounted display further comprises:
- detect with sensors of the virtual object anchor a position of the head mounted display;
- detect with sensors of the head mounted display a position of the virtual object anchor; and
- calibrate the presentation of the virtual object by comparing the virtual object anchor position and the head mounted display position.
17. A virtual object anchor comprising:
- a housing;
- a processor integrated in the housing and operable to execute instructions to process information;
- a memory interfaced with the processor and operable to store the instructions and memory;
- a network interface device interfaced with the processor and operable to communicate with external devices; and
- non-transitory memory interfaced with the processor and storing the instructions;
- wherein the instructions when executed on the processor;
- store in the memory a virtual object offset defining a virtual object position, orientation and scale relative to the housing; and
- communicate the offset through the network interface to plural external devices, the external devices applying the offset to present the virtual object at a location determined from the offset.
18. The virtual object anchor of claim 17 further comprising an orientation sensor that detects the housing orientation relative to a predetermined coordinate system, the instructions defining the offset based at least in part upon the housing orientation detected by the orientation sensor.
19. The virtual object anchor of claim 17 further comprising:
- one or more sensors integrated in the housing and operable to sense gestures by an end user at a location relative to the housing defined by the offset;
- wherein the instructions when executed on the processor apply the sensed gestures to update the offset stored in memory to adjust the virtual object position, orientation and scale according to the sensed gestures.
20. The virtual object anchor of claim 19 wherein the network interface device beacons the offset as wireless signals.
Type: Application
Filed: Mar 1, 2018
Publication Date: Sep 5, 2019
Applicant: Dell Products L.P. (Round Rock, TX)
Inventors: Ryan N. Comer (Austin, TX), Yagiz C. Yildiz (Austin, TX), Anantha K. Boyapalle (Cedar Park, TX), Jace W. Files (Round Rock, TX)
Application Number: 15/909,108