Mobile communication terminal and method
It is shown a method to move focus from a first user interface element to a second user interface element shown on a display of a mobile communication terminal, said mobile communication terminal further comprising an input device. The method comprises the steps of: detecting a directional input via said input device, said input indicating a desired direction to move; determining a set of candidate user interface elements being eligible to receive focus; for each candidate user interface element in said set of candidate user interface elements, determining a weighted distance between said candidate user interface element and said first user interface element by applying a function including, as an input parameter, an overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction; and determining said second user interface element to be a particular candidate user interface element in said set of candidate user interface elements that has a minimum weighted distance to said first user interface element.
This application is a continuation-in-part of U.S. patent application Ser. No. 11/135,624 filed on May 23, 2005.
FIELD OF THE INVENTIONThe present invention generally relates to user interfaces of mobile communication terminals, and more particularly to changing focus of user interface elements shown on displays of mobile communication terminals.
BACKGROUND OF THE INVENTIONMobile communication terminals have changed dramatically in the last decade. With the first 2G terminals, the only real purpose was to make normal phone calls. Now with 2.5G (GPRS), CDMA2000 and UMTS technology, mobile communication terminals not only facilitate voice communication, but also digital communication such as text and multimedia messaging, as well as browsing content provided by Internet servers.
The browser applications, also known as user agents, that are responsible for rendering documents, such as HTML, SVG, SMIL, containing focusable user interface elements, for example links or form controls, need to provide a method that takes input from the user, e.g. an event generated by the hardware input controller, and translates that input into an action that changes the state of the document by removing the focus from one user interface element and setting it on another.
In case the user agent is running on a device that allows a pen or a mouse as an input device, then the user can directly indicate the user interface element that will receive focus by simply tapping the screen or moving the mouse and clicking, in a position where a desired user interface element is rendered. However, if the device only has a four-way or five-way navigation key or a joystick, the user agent has the much more difficult task of determining the user interface element that will receive focus solely based on:
the currently focused user interface element,
the desired direction (given by the joystick, etc.), and
the position of the other focusable user interface elements relative to the currently focused user interface element.
Two important requirements that should be fulfilled by a user agent that provides such a method are:
the newly focused user interface element should usually be the same as the one intended by the user, and
the method should provide a certain degree of reversibility (e.g. a “left” press on the joystick, etc. followed by a “right” press should transfer the focus between the same two user interface elements).
The European patent EP 0 671 682 B1 presents a method, apparatus and computer readable storage device for positioning a cursor on one of a plurality of controls being displayed on a screen. However, the presented method is unsuitable for use in modern mobile communication terminals capable of displaying complex documents with focusable user interface elements, as the method provides a low degree of reversibility.
Consequently, there is a problem on how to manage navigation in documents displayed in a mobile communication terminal having only a limited input device.
SUMMARY OF THE INVENTIONIn view of the above, an objective of the invention is to solve or at least reduce the above-identified and other problems and shortcomings with the prior art, and to provide improvements to a mobile communication terminal.
Generally, the above objectives and purposes are achieved by methods, mobile communication terminals and computer program products according to the attached independent patent claims.
A first aspect of the present invention is a method to move focus from a first user interface element to a second user interface element shown on a display of a mobile communication terminal, said mobile communication terminal further comprising an input device. The method comprises the steps of:
detecting a directional input via said input device, said input indicating a desired direction to move;
determining a set of candidate user interface elements being eligible to receive focus;
for each candidate user interface element in said set of candidate user interface elements, determining a weighted distance between said candidate user interface element and said first user interface element by applying a function including, as an input parameter, an overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction; and
determining said second user interface element to be a particular candidate user interface element in said set of candidate user interface elements that has a minimum weighted distance to said first user interface element.
This provides a method with improved predictability of the shift of focus from a first user interface element to a second user interface element in a complex user interface environment.
In one embodiment, said mobile communication terminal further comprises a current focus position related to said first user interface element, said method comprising the further step of determining a new focus position related to said second user interface element, such that a component distance, along an axis orthogonal to said desired direction, of an absolute distance between said current focus position and said new focus position, is minimized. The focus point furthermore increases the reversibility and provides a way for the user to more predictably move focus from the first user interface element to the second user interface element.
In one embodiment, said in step of determining a new focus position, said new focus position is determined such that it is placed inside said second user interface element, at least a margin distance from any border thereof. This avoids the focus point from being placed right on the border of a user interface element, as the visible part of the user interface element is actually often placed a distance from the selectable border of the user interface element.
In one embodiment, said step of detecting an input involves detecting an input from a directional input device. It is especially important to improve predictability for users when a directional input device is employed.
In one embodiment, said step of detecting an input involves detecting an input from a device selected from the group consisting of a four way input device, a five way input device, an eight way input device, a nine way input device, a joystick, joypad and a navigation key.
In one embodiment, said overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction is restricted to an overlap being visible on said display. This prevents longer user interface elements always winning focus over shorter user interface elements when all user interface elements completely overlap with the currently focused user interface element.
In one embodiment, the method further comprises a step of providing a visual representation of said focus position on said display. This gives the user an indication of the position of the focus point which allows the user to improve the prediction of the movement of the focus point. The visual representation may be a graphical symbol. This allows the user to determine the position of the focus points by means of familiar user interface symbols or icons.
In one embodiment, the method further comprises a step, after said step of detecting an input, of determining a search area in a currently displayed document in which said first and second user interface elements are included, wherein said step of determining a set of candidate user interface elements is confined to user interface elements included in said search area. The introduction of a search area decreases the required processing.
In one embodiment, said search area is a combination of a part of said document currently visible on said display and a part of said document that would be visible if said document is scrolled in said desired direction. This search area should contain all user interface items the user would expect to be able to navigate to, given the desired direction.
In one embodiment, said applied function is:
distbasic+distparallel+2*distorthogonal−√{square root over (overlap)},
wherein distbasic is a Euclidian distance between a current focus position related to said first user interface element and a candidate focus position related to said candidate user interface element;
distparallel is a component along an axis parallel to said desired direction of a distance between a first point of said first user interface element utmost in said desired direction and a second point of said candidate user interface element utmost in a direction opposite said desired direction;
distorthogonal is a component along an axis orthogonal to said desired direction of a distance,between a third point, which may be the same as said first point, of said first user interface element closest to said candidate user interface element and a fourth point, which may be the same as said second point, of said candidate user interface element closest to said first user interface element; and
overlap is a distance of overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction.
The formula provided gives a good level of reversibility while being reasonably simple to calculate.
In one embodiment, distparallel is determined to be 0 if said first user interface element and said candidate user interface element overlap in a direction parallel to said desired direction.
In one embodiment, distorthogonal is determined to be 0 if said first user interface element and said candidate user interface element overlap in a direction orthogonal to said desired direction.
In one embodiment, if a component of said current focus position on an axis orthogonal to said desired direction equals a component of said candidate focus position on an axis orthogonal to said desired direction, distbasic is determined to be 0.
A second aspect of the present invention is a mobile communication terminal comprising a display and an input device, said terminal being configured to allow movement of focus from a first user interface element to a second user interface element shown on said display, said terminal furthermore comprising:
means for detecting a directional input via said input device, said input indicating a desired direction to move;
means for determining a set of candidate user interface elements being eligible to receive focus;
means for determining, for each candidate user interface element in said set of candidate user interface elements, a weighted distance between said candidate user interface element and said first user interface element by applying a function including, as an input parameter, an overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction; and
means for determining said second user interface element to be a particular candidate user interface element in said set of candidate user interface elements that has a minimum weighted distance to said first user interface element.
This provides a mobile communication terminal with improved predictability of the shift of focus from a first user interface element to a second user interface element in a complex user interface environment.
A third aspect of the present invention is a computer program product, directly loadable into a memory of a digital computer, comprising software code portions for performing a method according to the first aspect. This provides a computer program product, when executed, providing improved predictability of the shift of focus from a first user interface element to a second user interface element in a complex user interface environment.
Other objectives, features and advantages of the present invention will appear from the following detailed disclosure, from the attached dependent claims as well as from the drawings.
BRIEF DESCRIPTION OF THE DRAWINGSEmbodiments of the invention will now be described in more detail, with reference to the enclosed drawings.
The pocket computer 1 of the illustrated embodiment comprises an apparatus housing 2 and a display 3 provided at the surface of a front side 2f of the apparatus housing 2. Next to the display 3 a plurality of hardware keys 5a-d are provided, as well as a speaker 6.
More particularly, key 5a is a five-way navigation key, i.e. a key which is depressible at four different peripheral positions to command navigation in respective orthogonal directions (“up”, “down”, “left”, “right”) among information shown on the display 3, as well as depressible at a center position to command selection among information shown on the display 3. Key 5b is a cancel key, key 5c is a menu or options key, and key 5d is a home key.
At the surface of a short side 21 of the apparatus housing 2, there is provided an earphone audio terminal 7a, a mains power terminal 7b and a wire-based data interface 7c in the form of a serial USB port.
A second digital network 26 is shown in
The memory 54 is used for various purposes by the controller 50, one of them being for storing data and program instructions for various pieces of software in the pocket computer 1. The software may include a real-time operating system, drivers e.g. for a user interface 51, as well as various applications 57.
Many if not all of these applications will interact with the user 9 both by receiving data input from him, such as text or navigational input through the input device(s) 53, and by providing data output to him, such as visual output in the form of e.g. text and graphical information presented on the display 52. Non-limiting examples of applications are a www/wap browser application, a contacts application, a messaging application (email, SMS, MMS), a calendar application, an organizer application, a video game application, a calculator application, a voice memo application, an alarm clock application, a word processing application, a spreadsheet application, a code memory application, a music player application, a media streaming application, and a control panel application. GUI (graphical user interface) functionality 56 in the user interface 51 controls the interaction between the applications 57, the user 9 and the user interface elements 52, 53 of the user interface.
The search area 323 comprises the part 321 of the document currently visible on the display 52 plus a search range 322, which is the document area that would become visible as a result of scrolling the document in the desired direction 320.
A set of candidate user interface elements is populated by recursively traversing a rendering tree of the browser and testing boxes in nodes of the tree for overlap with the search area 323. Those boxes that are found to be overlapping are used to get pointers to corresponding user interface elements in the rendering tree. The rendering tree nodes, that correspond to user interface elements that are focusable are added to the set of candidate user interface elements.
The focus position movement follows the navigational input given by the user: given that a user interface element 320 has focus, and the focus position is in the position 325, navigating downward 328 will determine the focus position to move in the same direction. Initially, the focus position encounters a user interface element 322, resulting in the focus position moving to a new position 326. An additional user input event indicating a desire to move further downwards, analogously moves the focused position to a new position 327 in a user interface element 323. When moving upward from user interface element 323, the focus position will be determined such that the same user interface elements will receive focus (322 followed by 320). Here is a difference to solutions in the prior art for solving the same problem: when traveling upwards from user interface element 323, the focus arrives at 322. At this point, an algorithm that relies solely on a mathematical function to compute the distance between user interface elements would most likely choose user interface element 321 as the next focus target. One reason for this can be that a geometrical center 328 of the user interface element 322 is closer to a geometrical center 329 of user interface element 321 than the geometrical center 330 of user interface element 320. However, this would lead to a rather bad user experience, since the expectation is that 320, and not 321, would receive focus.
For example, a user triggers a scroll in the browser application along a direction 336 to the left from an original display view 335a such that the display view changes to 335b, and in response to an additional user input event, to a display view 335c. Content, such as user interface elements 331, 332 and 333, are moved correspondingly as an effect of the scrolling. Additionally, synchronized with the scrolling, the focus position moves from an original position 334a to an intermediate position 334b and finally to a position 334c. The focus position keeps, if possible, the same relative position inside the display 52.
The focus position is originally in a position 339 in a user interface element 340, displayed on the display 52. The user indicates a desired direction 337 of movement, in this case being downwards, resulting in the focus position moving to a new position 338. This new position can then be the starting point for applying a distance function to the user interface elements 342-345 inside the image 341 as is described in detail with reference to
In
In
The distance function is used to determine a weighted distance between a currently focused user interface element position and a candidate user interface element. This distance function is applied to calculate a weighted distance between each candidate user interface element and the currently focused user interface element. The new user interface element to receive focus is then determined to be the candidate user interface element with the smallest weighted distance to the currently focused user interface element. The weighted distance is calculated by means of the following formula:
distweighted=distbasic+distparallel+2*distorthogonal−√{square root over (overlap)}
The parameter distbasic is a Euclidian distance between the current focus position of the currently focused user interface element and the candidate focus position of the candidate user interface element. If the two positions have the same coordinate on the axis orthogonal to the desired direction, distbasic is forced to be 0. For example, in
The parameter distparallel is a component along an axis parallel to the desired direction of a distance between a first point 360 of the first user interface element utmost in the desired direction, and a second point 361 of the candidate user interface element utmost in a direction opposite the desired direction. The parameter distparallel is in other words a component of the distance between the points 360 and 361, projected along an axis parallel to the desired direction. Preferably, if the first point is further along the desired direction than the second point, distparallel is set to be 0. Note that in
The parameter distorthogonal is set to 0 if there is an overlap between the first user interface element and the candidate user interface element in a direction orthogonal to the desired direction; otherwise distorthogonal is a component along an axis orthogonal to the desired direction of a distance between a third point, which may be the same as the first point, of the first user interface element closest to the candidate user interface element and a fourth point, which may be the same as the second point, of the candidate user interface element closest to the first user interface element. For example, in
The parameter overlap is a distance of overlap between the currently focused user interface element and the candidate user interface element in a direction orthogonal to the desired direction. For example, in
In a detect directional input step 410, a directional input signal is detected via an input device such as the five-way navigation key 5a. This input gives information about the desired direction in which the user wishes to move focus from the currently focused user interface element to a target user interface element.
In a determine search area step 411, a search area is determined, as described in detail with reference to
In a determine a set of candidate user interface elements step 412, user interface elements of the search area are all considered candidate user interface elements and references to these are collected in a set.
In a determine a test candidate user interface element step 413, a test candidate user interface element, for which a weighted distance has not been calculated yet, is selected from the set of candidate user interface elements.
In a calculate weighted distance step 414, a weighted distance is calculated between the test candidate user interface, element and the currently focused user interface element, as described in detail with reference to
In a conditional uncalculated user interface elements step 415, it is tested whether there are any more uncalculated user interface elements. If there are more uncalculated user interface elements, the method proceeds to the determine a test candidate user interface element step 413, otherwise the method proceeds to a determine target user interface element step 416.
In the determine target user interface element step 416, the target user interface element is determined as an element in the candidate set of user interface elements having a minimum weighted distance to the currently focused user interface element.
The invention has mainly been described above with reference to a number of embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims. It is to be noted that the invention may be exercised in other kinds of mobile communication terminals than the pocket computer of
Claims
1. A method to move focus from a first user interface element to a second user interface element shown on a display of a mobile communication terminal, said mobile communication terminal further comprising an input device, said method comprising the steps of:
- detecting a directional input via said input device, said input indicating a desired direction to move;
- determining a set of candidate user interface elements being eligible to receive focus;
- for each candidate user interface element in said set of candidate user interface elements, determining a weighted distance between said candidate user interface element and said first user interface element by applying a function including, as an input parameter, an overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction; and
- determining said second user interface element to be a particular candidate user interface element in said set of candidate user interface elements that has a minimum weighted distance to said first user interface element.
2. The method of claim 1, wherein said mobile communication terminal further comprises a current focus position related to said first user interface element, said method comprising the further step of:
- determining a new focus position related to said second user interface element, such that a component distance, along an axis orthogonal to said desired direction, of an absolute distance between said current focus position and said new focus position, is minimized.
3. The method according to claim 2, wherein in said step of determining a new focus position, said new focus position is determined such that it is placed inside said second user interface element, at least a margin distance from any border thereof.
4. The method of claim 1, wherein said step of detecting an input involves detecting an input from a directional input device.
5. The method of claim 4, wherein said step of detecting an input involves detecting an input from a device selected from the group consisting of a four way input device, a five way input device, an eight way input device, a nine way input device, a joystick, a joypad and a navigation key.
6. The method of claim 1, wherein said overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction is restricted to an overlap being visible on said display.
7. The method of claim 1, further comprising the step of:
- providing a visual representation of said focus position on said display.
8. The method of claim 7, wherein said visual representation is a graphical symbol.
9. The method of claim 1, further comprising the step, after said step of detecting an input, of:
- determining a search area in a currently displayed document in which said first and second user interface elements are included, wherein said step of determining a set of candidate user interface elements is confined to user interface elements included in said search area.
10. The method of claim 9, wherein said search area is a combination of a part of said document currently visible on said display and a part of said document that would be visible if said document is scrolled in said desired direction.
11. The method of claim 1, wherein said applied function is: distbasic+distparallel+2*distorthogonal−√{square root over (overlap)},
- wherein distbasic is a Euclidian distance between a current focus position related to said first user interface element and a candidate focus position related to said candidate user interface element;
- distparallel is a component along an axis parallel to said desired direction of a distance between a first point of said first user interface element utmost in said desired direction and a second point of said candidate user interface element utmost in a direction opposite said desired direction;
- distorthogonal is a component along an axis orthogonal to said desired direction of a distance between a third point, which may be the same as said first point, of said first user interface element closest to said candidate user interface element and a fourth point, which may be the same as said second point, of said candidate user interface element closest to said first user interface element; and
- overlap is a distance of overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction.
12. The method of claim 11, wherein distparallel is set to 0 if said first user interface element and said candidate user interface element overlap in a direction parallel to said desired direction.
13. The method of claim 11, wherein distorthogonal is set to 0 if said first user interface element and said candidate user interface element overlap in a direction orthogonal to said desired direction.
14. The method of claim 11, wherein if a component of said current focus position on an axis orthogonal to said desired direction equals a component of said candidate focus position on an axis orthogonal to said desired direction, distbasic is determined to be 0.
15. A mobile communication terminal comprising a display and an input device, said terminal being configured to allow movement of focus from a first user interface element to a second user interface element shown on said display, said terminal furthermore comprising:
- means for detecting a directional input via said input device, said input indicating a desired direction to move;
- means for determining a set of candidate user interface elements being eligible to receive focus;
- means for determining, for each candidate user interface element in said set of candidate user interface elements, a weighted distance between said candidate user interface element and said first user interface element by applying a function including, as an input parameter, an overlap between said first user interface element and said candidate user interface element in a direction orthogonal to said desired direction; and
- means for determining said second user interface element to be a particular candidate user interface element in said set of candidate user interface elements that has a minimum weighted distance to said first user interface element.
16. A computer program product, directly loadable into a memory of a digital computer, comprising software code portions for performing a method according to claim 1.
Type: Application
Filed: Jun 22, 2005
Publication Date: Nov 23, 2006
Inventors: Antti Koivisto (Cambridge, MA), Andrei Popescu (Helsinki), Wei Liu (Acton, MA), Guido Grassel (Espoo), Virpi Roto (Espoo), Lasse Pajunen (Espoo), Matti Vaisanen (Helsinki)
Application Number: 11/158,921
International Classification: G09G 5/00 (20060101);