Control of display content by movement on a fixed spherical space
The present invention addresses the above described problems by simplifying the assumptions on possible motions made by a user. This simplification process can take place by making an assumption that the motion of the device will travel along a certain preferential motion arc, which will be referred to herein as the “Javal arc.” Calculations of motions of the device can be made based on these Javal arcs. Additionally, the double integration of each accelerometer is now eliminated and the expensive accelerometers are replaced by two magnetic sensors.
Latest Facebook Patents:
- Methods and devices for haptic communication
- Identifying content to present to a group of online system users based on user actions and specified by a third-party system
- SURFACE RELIEF GRATING AND METHOD OF MAKING THE SAME
- VOLUME BRAGG GRATING, FABRICATION METHOD AND SYSTEM
- INTRALINK BASED SESSION NEGOTIATION AND MEDIA BIT RATE ADAPTATION
This application is a reissue of U.S. Pat. No. 7,365,734, filed Feb. 9, 2004 and issued Apr. 29, 2008, and is a Continuation of PCT/US02/24981, filed Aug. 6, 2002, which claims the benefit of U.S. Provisional Application No. 60/310,870, filed Aug. 7, 2001.
FIELD OF THE INVENTIONThis invention relates to electronic devices, more specifically, to the control of display content for electronic devices.
BACKGROUND OF THE INVENTIONPortable electronic devices are generally controlled by input devices located on the portable device, such as a button, or pen touch computing. A typical portable electronic device 20 is depicted in Prior Art
Other methods have been developed to control the display of a portable electronic device with the use of user initiated motion. One such method invented by the applicant is embodied in U.S. Pat. No. 6,184,847, which is represented by prior art
Additionally, the calculation required from the input of several accelerometers can add greatly to the computational problems of controlling a display with such types of motions sensors. Although, simple threshold types of motions such as a 45 degree tilt can easily control a display for simple commands such a scroll left or scroll right, calculation that requires more fine tuning will use a great deal more computing power to integrate the data from the three accelerometer or gyroscope readings. For example the motions of the portable device will result in logic having to integrate the voltage readings from two accelerometers.
The accelerometer can be used to measure motion in any direction, including a “tilting” of the device. The distance an object has been moved is the integral of velocity over time. Essentially, the integral is a fairly complicated calculation on a device that may be running less than 8 M or RAM and while single calculations do not present a serious problem, continuous calculation from constant movement will require a great deal of system resources.
Normally, to measure the distance and object has traveled with accelerometers, it is necessarily to integrate twice. This amount of integration can lead to a great deal of error, even with the most accurate of accelerometers, what is needed is a way to predict some of the parameters of motion based on a normal users movement, instead of integrating several accelerometers at the same time.
What is needed is a method of calculating the motion or position of a portable device that does not require the expense and computational complexity of multiple accelerometers or other motion sensor, in order to control the display of the device.
SUMMARY OF THE INVENTIONThe present invention addresses the above described problems by simplifying the assumptions on possible motions made by a user. This simplification process can take place by making an assumption that the motion of the device will travel along a certain preferential motion arc, which will be referred to herein as the “Javal arc.” Calculations of motions of the device can be made based on these Javal arcs. Additionally, the double integration of each accelerometer is now eliminated and the expensive accelerometers are replaced by two magnetic sensors.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
The Javal arc is mainly used in connection with optical measurement devices and it is generally used as a medical term. However, since the display of a computer or electronic device has to do with vision, the Javal arc can be applied to the system of screen display viewing as well. Javal was a French physician who developed measurements for lines of sight, which has been applied mainly to opthamalogic devices for almost a century. However, the Javal arc can be used to measure the natural motion of lines of sight and motion related thereto. In the present invention the Javal arc is used as a computational assumption or a constant in order to eliminate expensive accelerometers and valuable computing power in a portable electronic device. There are 3 components to the Javal arc that extends out from the comfortably stretched arm of the user.
In order to avoid the difficulty of double integration in the calculation method for determining the motion of the device, a simpler and more accurate method of the present invention teaches a way to more accurately calculate movement from less expensive components, such as magnetometers. In one embodiment this control method requires only one accelerometer, and two inexpensive magnetometers. In yet another embodiment, two inexpensive magnetometers and a gyroscope may be used in the place of an accelerometer.
Distance traveled is the integral of velocity over time. Needless to say, for motions that last a fraction of a second, this can be very taxing on the calculation resources on a small electronic device where computing resources are already scarce. It can also be wildly inaccurate as can be appreciated by those skilled in the art.
Generally speaking, a small portable display screen will not have the capacity to display an entire desktop computer screen worth of graphics and text at one time. Although, some solutions, like for those on PDAs have proposed a new set of display requirements, such as web clipping, which are wholly inadequate to display the graphics of a normal computer display. The invention teaches a way to control the buffered display of an entire virtual display which will not fit on the display screen of the device, using the Javal arc motion and related computation aspects described herein.
Control of the display 26 of the portable electronic device 20 will be effectuated by motion of the device by a user. This is described above in the prior art description of control of the display of a device through a motion of the user. However, the user of a portable electronic device will not move the device equivalent to a normal Cartesian set of coordinates, or any other perfectly ordered geometrical system (cylindrical, etc.). The user will move the device in a manner that is natural to the user's eye and hand coordination, and is described with respect to a Javal arc movement coordinate system.
The present invention uses the expression “virtual display space” to define the entire of amount of graphics that may be presented.
Other expressions used in this capacity are “performance space” and “virtual desktop.” The expression “virtual screen” is a related concept to virtual display space, but is more limited in that the virtual screen has the equivalent virtual display space of only one desktop screen. The present invention distinguishes between virtual display space and a virtual screen, because of the manner in which the display memory will store and access such data. For example, a PDA graphic memory may store an entire virtual screen in a easily accessed buffer, but other screens are stored in cache or loaded as they become needed in the buffer. A more powerful device with a larger display buffer may be able to load an entire document. For the purposed of this disclosure, the expression virtual display space is primarily used. When discussing content from the Internet or an equivalent computer display, the expression virtual screen may be used but is distinguished from virtual display space.
Referring now to
Referring now to
Thus the “panning” motion of the portable electronic device 20 will generally travel on a portion of a spheroid 101 like coordinate system 100 in which the two Javal arcs which meet at the central viewing point 199 and define the surface of motion in which the user moves the device. The user does not need to move the device along either of the arcs 104 110 to effectuate control of the display of the device but will tend to move the device anywhere on the surface of the spheroid 101. The angle of the movement along either arc 104 110 will be proportional the angle that the user 10 is holding the device 20 away from its normal axis. These angles are described in detail below.
Referring now to
Referring now to
In
The invention includes logic means 2000, which is shown in
Because accelerometers can effectively measure motion in one direction without integration, that one direction can be easily calculated, especially in the accelerometers most sensitive mode which is vertical because of the measure of gravity. Normally, to detect the motion in two directions, two accelerometers can be placed on a device, but this requires a great deal of integration which leads to enormous inaccuracies. In the present invention, the single accelerometer 1002 will measure the vertical movement or position of the portable electronic device 20.
Magnetometers can also easily measure the angle of the device 20 relative to its normal orientation as depicted in
The resulting voltage inputs to 2200 and 2300 from the magnetometers calculations of the changes in the magnetic field H, generally means that each axis of sensitivity of both H(x) and H(y) will have to be determined from H(⊥) and H(//) based on the angles α(2) 40 and α(4) 60 from the axes. The formula for the determination of the resulting voltage signal output into the inputs 2200 and 2300 is included herein.
The voltage signals from the changes in the respective magnetic fields H(x) and H(y) and which are detected by the magnetometers 1004 and 1006, are compiled with the output from the accelerometer 1002 and then compiled in the motion calculation unit 2500 based on the formulas described herein. It should be noted that the entire motion detection logic and control system 2000 is virtual, and any one of the components of the system may be located at any place through out the hardware of the device 20. For example, the accelerometer input unit 2100 may actually be located on the accelerometer 1002 instead of in the ASIC 2001 or in a discrete location in the distributed system alternative. The logic of the motion calculation unit 2500 for determining the motion and position of the device of is based on formulas disclosed herein.
One method for calculating position and distance traveled along the Javal spheroid 101 is the instantaneous velocity of an origin point 2 on the electronic device 20 is defined as: V(x)=R*dα(1)/dt; V(y)=R*sin α(1)*dα(2)/dt in the “portrait” orientation, and V(x)=R*dα(4)/dt; V(y)=R*sin α(1)*dα(3)/dt in the “landscape” orientation. Although this requires the conversion of coordinates based on the changing coordinate systems, the conversion of the Cartesian coordinates to Spherical coordinates in well known to those skilled in the art and is described in appendix A in the additional disclosure of the invention.
Referring now to
Depending on the user preferences, the movement and position of the device can be programmed to control the display differently. For example a user may wish to have the upward movement along the vertical Javal arc 104 to have the PDA content screen 600 to pan downward. Other users may wish to have such movement pan upward in an alternate embodiment.
In other preferred embodiments, means for detecting the distance a user 10 is away from the device may include echo location functions, light intensity functions. Zooming functions may be activated in different ways depending on the efficiency of manufacture. Other useful features of the invention may include such embodiments which include (1) threshold control for the elimination of the movement “noise;” (2) threshold control for the activation of the display control mechanism, (3) alternate display controls, such as horizontal movement is a display vertical scroll and vice versa; (4) lock and unlock controls of the arc movement navigation (where the horizontal is locked, but the vertical continues to scroll) and other useful navigation features.
In another embodiment represented by
When a user holding a PDA or smart phone device in his hand, moves it around while looking at the device display, the following assumptions apply:
- 1) The user will maintain the display at a distance that is constant. The distance can be adjusted to a comforatable reading distance. Assume that the comfortable reading distance is denoted by the value Rcomf.
- 2) The user will maintain the display approximately normal to his line of sight.
These constraints define a situation in which the center of the display O1 will move over the surface of its sphere, centered at the middle point of the segment joining the user's two eyes O, and of radius Rconf; with the display remaining tangent to the sphere at all times.
It can be assumed, without any loss of generality, a global frame of reference (O, X, Y, Z) so that OZ is vertical with the gravity vector oriented downward, and so that the Earth magnetic field {right arrow over (H)} is in the (OX, OZ) plane. One defines a device frame that (O1, X111, Y111, Z111) attached to the device display.
The device frame is related to the global frame by three successive rotations. A first rotation of the global frame around its OZ axis by angle Φ (the azimuth angle), defines the first intermediate frame (O1, X1, Y1, Z1), with Z1=Z. Then a rotation of the first intermediate frame around the Y1 axis by an angle θ (the tilt angle), defines the second intermediate frame (O1, X11, Y11, Z11), with Y11=Y1. Finally a rotation of the second intermediate frame around the Z11 axis by an angle γ (the orientation angle), defines the device frame (O1, X111, Y111, Z111), with Z111=Z11.
γ=zero corresponds to a portrait orientation of the device, while γ=π/2 corresponds to a landscape orientation.
Assuming that the motion of the device in space is constrained as previously explained, the instantaneous velocity of O1 in the global frame, projected along the base vectors of the device frame, is given by:
1) in portrait orientation:
2) in landscape orientation: γ=±π/2
From the previous equations it can be seen that the determination of the instantaneous velocity of O1 has been reduced to the determination of the time derivatives of θ and Φ.
Model of the Sensor Assembly:
In order to simplify the formulation of the problem, and without any significant loss of generality, one assumes that all the sensors are in the plane of the display (the (O′X′″, O′Y′″) plane).
One infers that:
With the angle ξα assumed to be known (calibration).
Tilt and Orientation Determination:
Assuming two accelerometers in the (O′X′″, O′Y′″) plane, with sensitivity axis
one has:
And the output of these two sensors are given by:
Where the offset voltage Vα0 (in V), and the sensitivities Sα (in V m−1, δ), are assumed to be known (calibration).
From (10), one has:
From (9), one has:
i.e.,
One introduces:
From (7) and (8), one has:
i.e.,
i.e.,
i.e.,
As by definition
and
If
for portrait orientation
for landscape orientation with ϵ=Sig (Sin γ)
Azimuth Determination:
One has the in-plane component of the local magnetic field given by:
Where the amplitude Hpoc and the inclination angle δ may be slowly varying functions off position and time.
And the perpendicular component:
Assuming two magnometers in the (O′X′″, O′Y′″) plane, with sensitivity axis
and
one has:
And the output of these two sensors are given by:
Where the offset voltage V0α (in V), and the sensitivities Sα (in V·Oe−1) are assumed to be known (calibration).
From (26), one has:
i.e., from (15), one has:
i.e.,
For (6), (7) and (8), one has:
Sin θ is known from (21):
And Sin γ and Cos γ are known from (19):
Thus,
ϵ Cos γ|Cos θ|(H// Cos Φ)−Sin γ(H// Sin Φ)=[hx′″−H⊥ Cos γ Sin θ]
ϵ Sin γ|Cos θ|(H// Cos Φ)−Cos γ(H// Sin Φ)=[hy′″−H⊥ Sin γ Sin θ]
i.e.,
hX′″=ϵ Cos γ|Cos θ|(H// Cos Φ)−Sin γ(H// Sin Φ)+H⊥ Cos γ Sin θ (35.1)
hY′″=ϵ Sin γ|Cos θ|(H// Cos Φ)−Cos γ(H// Sin Φ)+H⊥ Sin γ Sin θ (35.2)
Sin γhX′″−Cos γhY′″=2ϵ Sin γ Cos γ|Cos θ|(H// Cos Φ)−Cos γ Sin2 γ(H// Sin Φ)
Sin γhX′″+Cos γhY′″=−(H// Sin Φ)+2H⊥ Sin γ Cos γ Sin Φ
i.e.,
2ϵ Sin γ Cos γ|Cos θ|(H// Cos Φ)+Cos2 γ Sin2 γ(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (36.1)
(H// Sin Φ)+(2 Sin γ Cos γ Sin θ)H⊥=(Sin γhX′″+Cos γhY′″) (36.2)
H⊥=ϵM(1−H//2)1/2 (37)
Where ϵM is known (north or south of magnetic equator).
Thus:
2ϵ Sin γ Cos γ|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (38.1)
(H// Sin Φ)+ϵM(Sin γ Cos γ Sin θ)(Hpoc2−H//2)1/2=(Sin γhX′″+Cos γhY′″) (38.2)
If the magnetic inclination δ and the field amplitude Hpoc are assumed to be known:
H//=Hpoc Cos δ (9.1)
H⊥=Hpoc Sin δ (39.2)
[2H// Sin γ Cos γ(ϵ|Cos θ|)]Cos Φ+(Cos2 γ−Sin2 γ)(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (40.1)
H// Sin Φ+2(Sin γ Cos γ Sin θ)H⊥=(Sin γhX′″+Cos γhY′″) (40.2)
i.e.,
H// Sin Φ=2(Sin γ Cos γ Sin θ)H⊥−(Sin γhX′″+Cos γhY′″) (41.1)
γ≃0 then H// Sin Φ≃Cos γhY′″
hX′″≃ϵ|Cos θ|(H// Cos Φ)+H⊥ Sin θ
i.e.,
ϵ|Cos θ|(H// Cos Φ)≃(hX′″−H⊥ Sin θ)
(Cos γhX′″−Sin γhy′″)=ϵ|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)H⊥ Sin θ
(Sin γhX′″+Cos γhy′″=−(H// Sin Φ)+2(Sin γ Cos γ)H⊥ Sin θ
i.e.,
ϵ|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)(H⊥ Sin Φ)=(Cos γhX′″−Sin θhy′″) (42.1)
−(H// Sin Φ)+2(Sin γ Cos γ)(H⊥ Sin θ)=(Sin γhX′″+Cos γhy′″) (42.2)
If γ=0, one has in first approximation:
ϵ|Cos θ|(H// Cos Φ)+(H⊥ Sin θ)=hx′″
−H// Sin Φ=hy′″
i.e.,
Claims
1. A method for controlling a display on a portable electronic device, the method comprising the computer-implemented acts of:
- detecting sensor data indicative of a motion of said portable electronic device relative to a fixed point, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device;
- calculating a position on a spheroid of said portable electronic device based on said sensor data indicative of said motion that is relative to said fixed point;
- calculating said motion with respect to said spheroid given said sensor data indicative of said motion and analyzing said sensor data indicative of said motion with respect to said spheroid;
- determining that said motion corresponds to a display command; and
- adjusting said display based on said motion and on an angle of said position, if on said spheroid in response to determining that said motion corresponds to a user said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
2. The method of claim 1, wherein said act of detecting said sensor data indicative of said motion further comprising comprises:
- receiving input from motion sensors which that can detect said sensor data indicative of said motion of said portable electronic device.
3. The method of claim 1, wherein said act of calculating a said position on said spheroid is based on a Javal coordinate system.
4. The method of claim 1, wherein said act of calculating said motion with respect to said spheroid and analyzing said sensor data indicative of said motion is are based on a Javal arc coordinate system.
5. The method of claim 1, wherein said act of calculating said motion with respect to said spheroid and analyzing said sensor data indicative of said motion further comprises measuring changes in a magnetic field.
6. The method of claim 1, wherein said act of calculating said motion with respect to said spheroid and analyzing said sensor data indicative of said motion further comprises measuring one or more angles corresponding to a position of said portable electronic device relative to said fixed point.
7. The method of claim 1, wherein said motion is modeled mathematically by a virtual surface of a virtual thin-walled shell that is approximately spherical in shape.
8. The method of claim 4, wherein an origin of said Javal arc is said fixed point.
9. A controller for a display on a portable electronic device, the controller comprising:
- at least one accelerometer for measuring a sensor data indicative of a motion of said portable electronic device relative to a fixed point, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device;
- at least one magnetic sensor for measuring said motion; and
- logic for calculating said motion with respect to a spheroid given said senor data indicative of said motion, and for calculating a position on said spheroid of said portable electronic device based on one or more measurements from said at least one accelerometer and said at least one magnetic sensor;
- logic for determining that said motion corresponds to a display command; and
- logic for adjusting said display based on said motion and on an angle of said position on said spheroid in response to determining that said motion corresponds to said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
10. The controller of claim 9, wherein said motion is modeled by a pre-defined path.
11. The controller of claim 9, wherein said motion is modeled by a Javal arc.
12. The controller of claim 9, wherein said motion is based on a Javal coordinate system.
13. A method for controlling a portable electronic device, the method comprising the computer-implemented acts of:
- calculating and analyzing a motion of said portable electronic device with respect to a spheroid given sensor data indicative of said motion with respect to said spheroid;
- analyzing said sensor data indicative of said motion with respect to said spheroid;
- calculating a position on said spheroid of said portable electronic device based on said sensor data indicative of said motion that is relative to a fixed point from said portable electronic device, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device; and
- determining if that said motion corresponds to a user display command; and
- adjusting at least one aspect of a display on said portable electronic device based on said motion when said motion corresponds to a user command and on an angle of said position on said spheroid in response to determining that said motion corresponds to said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
14. The method of claim 13, wherein said acts of calculating said motion of said portable electronic device and analyzing said sensor data indicative of said motion further comprising comprises:
- receiving input from motion sensors which that can detect said sensor data indicative of said motion of said portable electronic device.
15. The method of claim 14, wherein said motion sensors are attached to said portable electronic device.
16. The method of claim 13, further comprising:
- calculating a position of said device based on said motion that is relative to said fixed point.
17. A controller for a display on a portable electronic device, the controller comprising:
- only two accelerometers for measuring sensor data indicative of a motion of said portable electronic device relative to a fixed point, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device;
- at least one magnetic sensor for measuring said motion; and
- logic for calculating said motion with respect to a spheroid given said sensor data indicative of said motion, and for calculating a position on said spheroid of said portable electronic device based on one or more measurements from said only two accelerometers and said at least one magnetic sensor;
- logic for determining that said motion corresponds to a display command; and
- logic for adjusting said display based on said motion and on an angle of said position on said spheroid in response to determining that said motion corresponds to said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
18. A controller for a display on a portable electronic device, the controller comprising:
- one gyroscope for measuring sensor data indicative of a motion of said portable electronic device relative to a fixed point, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device;
- at least one magnetic sensor for measuring said motion; and
- logic for calculating said motion with respect to a spheroid given said sensor data indicative of said motion, and for calculating a position on said spheroid of said portable electronic device based on one or more measurements from said one gyroscope and said at least one magnetic sensor;
- logic for determining that said motion corresponds to a display command; and
- logic for adjusting said display based on said motion and on an angle of said position on said spheroid in response to determining that said motion corresponds to said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
19. A controller for a display on a portable electronic device, the controller comprising:
- only one accelerometer for measuring sensor data indicative of a motion of said portable electronic device relative to a fixed point, wherein said motion follows a preferential motion arc that is natural to eye and/or hand coordination of a holder of said portable electronic device;
- only two magnetic sensors for measuring said motion; and
- logic for calculating said motion with respect to a spheroid given said sensor data indicative of said motion, and for calculating a position on said spheroid of said portable electronic device based on one or more measurements from said only one accelerometer and said only two magnetic sensors;
- logic for determining that said motion corresponds to a display command; and
- logic for adjusting said display based on said motion and on an angle of said position on said spheroid in response to determining that said motion corresponds to said display command, said angle indicating an orientation of said portable electronic device relative to a line of sight of said holder.
5506605 | April 9, 1996 | Paley |
5602566 | February 11, 1997 | Motosyuku et al. |
5910797 | June 8, 1999 | Beuk |
6072467 | June 6, 2000 | Walker |
6184847 | February 6, 2001 | Fateh et al. |
6201554 | March 13, 2001 | Lands |
6228704 | May 8, 2001 | Flack et al. |
6288704 | September 11, 2001 | Flack et al. |
6466198 | October 15, 2002 | Feinstein |
6690358 | February 10, 2004 | Kaplan |
6847351 | January 25, 2005 | Noguera |
- PCT International Search Report, PCT Application No. PCT/US2002/024981, Dec. 10, 2002, one page.
- United States Office Action, U.S. Appl. No. 10/775,762, dated May 31, 2007, seven pages.
Type: Grant
Filed: Aug 20, 2015
Date of Patent: Jun 25, 2019
Assignee: Facebook, Inc. (Menlo Park, CA)
Inventors: Sina Fateh (Sunnyvale, CA), Ray Valdes (Davis, CA), John Masiewicz (San Jose, CA)
Primary Examiner: Dennis G Bonshock
Application Number: 14/831,870
International Classification: G09G 5/08 (20060101); G06F 1/16 (20060101);