Assisted application operation service for mobile devices using screen sharing
A system and corresponding methods to provide assistance to a mobile user (42) to operate applications in the mobile device (31) is described. A call center is enhanced to enable routing mobile device application screens (43) to call center agents in addition to routing voice calls. Such a call center is referred to as remote help center. When a mobile user (42) needs assistance in operating applications (44) at the mobile device (31), he makes a call to remote help center and sends display contents from the mobile device (31) into the remote help center which routes the display contents to corresponding remote help center agents who can provide remote assistance to operate applications (44) at the mobile device (31).
Not applicable
FEDERALLY SPONSORED RESEARCHNot applicable
SEQUENCE LISTING OR PROGRAMNot applicable
BACKGROUND OF THE INVENTION1. Field of Invention
The present invention generally relates to assisted application operation services and specifically to assistance provided from a remote service center using screen sharing.
2. Prior Art
Application operation assistance is necessary for mobile device users who want to operate their applications while they are mobile. With advent of smartphones, the number of applications in a mobile device has increased exponentially. This has lead to a complex user interaction with several menus and submenus to get to a desired set of features in applications. Hence it is desirable to have assistance to operate the complex set of menus to access features quickly and efficiently.
Technology in smartphones is improving rapidly providing faster processors and increased memory, enabling complex-desktop like applications to be executed in a smartphone. But complex desktop like applications provide a lot of features and accessing all of these features without compromising user experience is a big challenge. Hence mobile device users and application designers face an increasing challenge to balance ease of use with complex features of an application. Most mobile devices such as smartphones only provide a small keypad or a touch sensitive screen which is not sufficient to easily access complex features of desktop like software applications.
There have been several attempts at providing a better user experience in mobile devices for accessing features of complex applications. These include providing a small keypad to enable entering data into applications, providing touch sensitive screen with finger gestures to enable quick access to some features and providing a larger display to enable better content viewing experience.
But all these attempts cannot replicate a user experience provided by a desktop or laptop hardware that has a large keyboard, mouse and a large display to enable efficient use of complex desktop application software.
A small keypad in a smartphone can only allow thumb based typing and that can be very limiting while editing large documents.
A gesture based touch screen may enable faster access to some often used application features, but getting access to several hundred features is impossible using gestures as a user cannot remember all the gestures necessary. Accessing large number of features is only possible by providing menus and toolbar items in predetermined locations of an application.
Providing a slightly larger display helps in viewing content such as a web page, parts of which may be scaled and panned to provide a better view of localized content in a screen. But such a system is useful only for applications that are tailored for viewing and not editing. Applications that require a large amount of user interaction cannot be operated with frequent scaling and panning of a screen.
Another solution to this problem in prior art is to provide a mini laptop called the netbook. Such netbooks provide a reasonable sized keypad and screen to enable a user to operate almost all of the features available in a desktop application. But such devices are too big to be carried by people on person, without having an extra case or a bag for it. These devices cater to a different market segment and are not a replacement for a smartphone device that can be carried in a pocket and operated by a single hand.
Even with such devices, it is impossible to get access to all the features while being mobile. Being mobile, a person can only use one hand and hence it becomes quite difficult to operate all applications with one hand using a netbook.
Hence there is a need for assistance to operate applications for a mobile device user from a remote service center. Such assistance can be provided by a live person further referred to as a helper or an agent at the service center who is equipped with a desktop computer with a large keyboard, screen and a mouse and can control and operate the applications at the mobile device of the user remotely.
Currently there are no methods in prior art that offer a solution to this problem. It is the intent of present invention to provide a solution to this problem by using screen sharing methods. The present invention proposes using screen sharing methods combined with a call center like service center that is capable of routing screens of mobile device users to corresponding helpers who can remotely control and operate the screens of a mobile device in addition to routing voice calls.
Call centers usually route voice calls from a user to appropriate call center agent who can handle a call.
The present invention proposes extending call center features by routing screens of mobile device users to call center agents in addition to routing calls. This particular combination of routing mobile device screens to call center agents in addition to routing calls does not exist in any known prior art. Such a service center shall be further referred to as remote help center.
Prior art methods use screen sharing methods to enable multiple users to communicate within a group by allowing them to view common data. But all these users are from a trusted group and such systems cannot be used by individual users who might need help in entering large amounts of data into forms or editing a spreadsheet with complex formulae and graphs.
There are several products in prior art that use screen sharing protocols such as webex, vnc and citrix. But all of these products setup a communication channel among trusted participants and do not provide an individual mobile device user assistance to operate applications in a mobile device.
Most such systems enable a mobile device user to view screens from a desktop user and join a communication session, but none offer a solution where screens from a mobile device are shared and routed to an agent at a service center who can operate all applications at the mobile device using a larger desktop.
Hence it can be seen that there is a need for a service that enables a mobile device user to get help in operating applications in a mobile device.
Following paragraphs in current section describe relevant prior art in this field.
Prior art U.S. Pat. No. 6,980,953 proposes a real time transcription service where a user gets documents transcribed from a remote service center. The transcription could be viewed by the user seeking this service on a screen in proximity to the user. But this does not address the need of a mobile device user to get help to operate applications on the mobile device. This only caters to a particular need of transcribing documents.
Prior art U.S. Pat. No. 7,398,470 proposes a remote printing service where a user is able to view the display of service personnel at a remote service center so that the user can influence design of items being printed. But this does not address the need of a mobile device user who wants help in operating complex applications at the mobile device. This is the opposite of what a mobile device user who wants to operate applications at the mobile device would want.
Prior art U.S. Pat. No. 6,295,551 proposes a system where a call center agent and a user share a browsing session. This is achieved by providing an intermediate java applet that communicates changes in browser state from a user to call center agent and vice versa. But this is a specific implementation to provide a joint browsing session where a java applet is provided by the call center agent to the browser of the user. This makes use of two different application instances of browser application, one on user device and another on call center agent device. This does not transport all screen contents from a mobile device but just show cases how joint browsing can be achieved. This cannot be used for providing a user with operation assistance for all applications in the mobile device. Hence this does not address the need of a mobile device user who wants help in operating complex applications at the mobile device.
As can be seen from above, all known prior arts suffer from some limitations in offering a solution that enables a mobile device user to get help in operating a complex set of applications in the mobile device from a remote service center.
3. Objects and Advantages
Accordingly, several objects and advantages of the present invention are:
-
- a) to provide a mobile device user access to a remote help center from where the user can get help in operating applications in the mobile device;
- b) to provide a remote help center that is capable of routing screens of a mobile device user in addition to voice calls to appropriate agents who can operate applications in the mobile device; and
- c) to provide a protocol that enables efficient routing of screens of a mobile device user in addition to routing voice calls to remote help center agents.
In accordance with present invention a system is provided that enables a mobile device user to get help in operating applications in the mobile device by a person at a remote service center.
This is achieved by enhancing a call center with screen sharing protocol that enables routing display screens of a mobile device user to call center agents who are live persons who can operate upon applications at the mobile device remotely, in addition to routing calls.
Combining screen sharing protocol with a call center is unique to this invention and provides an unobvious value to mobile device users who can get help in operating complex desktop like applications while on the move.
The enhanced call center is able to route mobile device screens to appropriate agents who can provide the needed assistance. This enhanced call center is further referred to as remote help center.
Mobile device application that may be operated with remote assistance could be any application in a mobile device including phone call application, calendar application, email application, productivity application such as spreadsheet and browser application.
Such applications may be completely executed at the mobile device that includes both service logic and graphical user interface, or may be executed as a split mode application where graphical user interface may be executed on the mobile device and the service logic may be executed at a server.
Hence all applications whether running completely at the mobile device or in split mode will benefit from this invention in providing assistance to mobile device users in operating complex applications from a remote service center.
- 31 mobile device
- 32 cellular packet data
- 33 internet
- 34 mobile call interface
- 35 voice switching network
- 36 remote help center server
- 37 call routing function
- 38 screen routing function
- 39 helper stations
- 40 desktop personal computers
- 41 telephones
- 42 mobile user
- 43 mobile application screen
- 44 mobile application
- 45 voice path
- 46 data connection
A remote help center server 36 provides support for assisted application operation, which includes support for call routing function 37 and screen routing function 38. Application operation assistance is provided by pool of helper stations 39 that are connected to remote help center server 36 through intranet or internet and also through voice switching network 35. Helper stations 39 are equipped with desktop personal computers 40 and telephones 41. Voice switching network 35 can either be Public Switched Telephone Network (PSTN) also known as circuit switched network, or Voice over Internet Protocol (VoIP). In preferred embodiment of this invention, mobile device 31 is equipped to support complex applications that are not always convenient to operate with one hand and may involve complex interactions such as menu navigations, and significant data entry.
A mobile user 42 using mobile device 31 initiates assisted application operation service while using mobile application screen 43 of mobile application 44 by sending assistance request to remote help center server 36 through internet 33. Remote help center server 36 allocates one of the available helper stations 39 to assist mobile user 42. Assisted application session involves establishing voice connection as well as application screen sharing session simultaneously between mobile device 31 and helper station 39.
In one of the embodiment of this invention, voice connection between mobile device 31 and helper station 39 is facilitated using Voice Over Internet Protocol (VoIP). In this embodiment, remote help center server 36 performs call routing functions 37 to establish voice path 45 between mobile device 31 and helper station 39.
In another embodiment of this invention, voice connection between mobile device 31 and helper station 39 is established using Public Switched Telephone Network (PSTN).
Remote help center server 36 also establishes application screen sharing session for assisted mobile application 43 between mobile device 31 and helper station 39. Screen sharing session is established using a data connection 46 through internet 33. Assisted mobile application 43 screen appears in display of helper station 39. Helper using helper station 39 can engage with mobile user 42 using mobile device 31 in voice conversation and also navigate application menu and enter data in assisted mobile application 43 screen in helper station 43. Data modifications and menu selections of assisted mobile application 43 in helper station 39 are automatically reflected in assisted mobile application 43 screen in mobile device 39.
A mobile application 44 executing at the mobile device 31 can be one of call processing application, electronic mail application, calendar application, memo pad application, browser application, and remotely rendered application running at a server.
Mobile application 44 executing at the mobile device 31 can be fully executing at the mobile device 31 or can be executed in a split mode of operation.
When mobile application 44 is executed fully at the mobile device 31, both graphical user interface logic and service logic are collocated at the mobile device 31.
When mobile application 44 is executed in split mode of operation, the graphical user interface logic is executed at the mobile device 31, but service logic may be executed at another device other than the mobile device 31. Such applications are called remotely rendered applications.
The remotely rendered application could be an application that displays graphical output at the mobile device 31, but its service logic maybe executing at a different device such as a remote server. Examples of such a system could be applications using X window protocol, or other protocols than enable screen sharing such as virtual network computing (VNC).
Routing of voice call and screen data to a specific agent can be controlled by using agent selection criteria that can be one of availability of agents, domain expertise of agents, performance of agents, geographical location of agents, network latency between said mobile device 31 and said agent, and language preferences of a user of said mobile device 31.
From the description above a number of advantages of the system that extends a call center service with screen sharing protocol become evident:
-
- a) a method is provided where a mobile device user gets access to a remote help center from where the user can get help in operating applications in the mobile device;
- b) a remote help center is provided that is capable of routing mobile device user screens in addition to voice calls to appropriate remote help center agents who can operate applications in the mobile device from a remote screen; and
- c) a protocol is provided to enable efficient routing of screens of a mobile device user in addition to routing voice calls to remote help center agents.
Accordingly, the reader will see that enhancing a call center application to route mobile device screens in addition to call routing provides a solution to a mobile device user who can get help in operating a growing set of complex applications at the mobile device.
Although the description above contains many specificities, these should not be construed as limiting the scope of invention but merely as providing illustrations of some of the presently preferred embodiments of this invention. Thus the scope of this invention should be determined by appended claims and their legal equivalents, rather than by example given.
Claims
1. A method of providing assistance to operate applications associated with a mobile device from a remote service center further referred to as a remote help center comprising:
- a) displaying graphical user interface output of said applications at a display associated with said mobile device;
- b) capturing said graphical user interface output to produce a data stream in a network data representation format;
- c) sending said data stream to said remote help center;
- d) receiving said data stream at said remote help center; and
- e) routing said data stream to a agent at said remote help center further referred to as agent providing assistance, wherein said agent is a person capable of operating said applications remotely.
2. The method of providing assistance of claim 1, further comprising:
- a) converting said data stream into graphical user interface elements to form a copy of display contents of said graphical user interface output to be displayed at a display associated with said agent providing assistance; and
- b) displaying said copy of display contents into said display of said agent providing assistance.
3. The method of providing assistance of claim 2, further comprising:
- a) capturing user interface input events from said agent providing assistance who is using a mouse and a keyboard to generate user interface input events;
- b) associating said user interface input events with said copy of display contents; and
- c) transferring said user interface input events of said agent to said application associated with said mobile device.
4. The method of providing assistance of claim 3, further comprising:
- a) initiating a voice call to said remote help center and associating said voice call with said mobile device;
- b) receiving said voice call at said remote help center; and
- c) routing said voice call to said agent providing assistance.
5. The method of providing assistance of claim 4, wherein said voice call is selected from group consisting of voice over internet protocol call, and circuit switched call.
6. The method of providing assistance of claim 4, wherein said voice call initiation is from a device selected from group consisting of said mobile device, device capable of call processing networked with said mobile device, and a separate device capable of call processing.
7. The method of providing assistance of claim 1, wherein said mobile device is selected from group consisting of personal digital assistant, mobile phone, smartphone, and portable media player.
8. The method of providing assistance of claim 1, wherein said application is selected from group consisting of call processing application, electronic mail application, calendar application, memo pad application, browser application, and remotely rendered application running at a server.
9. The method of providing assistance of claim 1, wherein said applications are executed using application execution methods selected from group consisting of complete application execution at said mobile device, and partial application execution method at said mobile device, wherein said complete application execution method provides for executing both graphical user interface logic and service logic of said application at said mobile device and wherein said partial application execution method provides for executing graphical user interface logic of said application at said mobile device and service logic of said application at a remote server.
10. The method of providing assistance of claim 1, wherein said capturing of said graphical user interface output associated with said application is done at a device where said application service logic is executed.
11. The method of providing assistance of claim 1, wherein said routing of said data stream to said agents is dependent on agent selection criteria selected from group consisting of availability of agents, domain expertise of agents, performance of agents, geographical location of agents, network latency between said mobile device and said agent, and language preferences of a user of said mobile device.
Type: Application
Filed: May 11, 2009
Publication Date: Oct 1, 2009
Inventors: Sudharshan Srinivasan (Fremont, CA), Kothandraman Ramchandran (Fremont, CA), Jai Kumar (Cupertino, CA)
Application Number: 12/454,049
International Classification: H04M 3/42 (20060101); G06F 3/00 (20060101);
