User-aware video display

Abstract

The present invention provides methods and systems for displaying a video signal depending upon a user interaction. An example system includes a display for receiving the video signal, at least one transmitter for transmitting location signals, and at least three transceivers for receiving the location signals from the transmitter and for transmitting modified location signals. The geometric locations of the at least three transceivers are known to a computing device. The computing device is then able to derive a transmit location of the location signals based on the received modified location signals, wherein the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction.

Description

FIELD OF INVENTION

This invention relates to a method and system for displaying a video signal on a display depending upon a user interaction. The provided system is particularly well suited for, but not limited to, wireless communication and positioning technologies exploiting the ultra-wideband (UWB) radio channel recently made available by the US Federal Communications Commission (FCC) in the frequency range between 3.1 GHz and 10.6 GHz.

BACKGROUND

Short-range wireless technologies in the wireless local area network (WLAN) space as well as wireless personal and body area networks (WPAN and WBAN) continue to proliferate rapidly. Similarly, wireless technology is increasingly applied to feed high-resolution video signals over a digital video interface (DVI) to hand-held or detachable flat panel displays (FPDs) as well as to stationary displays, e.g., display panels mounted on a wall. A missing feature in these applications using DVIs is the capability to support user-aware, real-time or quasi real-time interactions in response to the user's position and orientation relative to the video display. Typically, conventional wireless video display systems operate by transmitting compressed video signals to the display, such that the latter requires decompression circuits, over license-free but narrow designated radio spectrum bands; however, these narrowband wireless systems can generally not support the desired high data rates for the transmission of raw high-resolution video signals. Furthermore, narrowband wireless systems are generally not capable to support applications requiring a precise indoor location tracking capability.

To mitigate the threat of a future spectrum shortage, additional radio spectrum in the form of the ultra-wideband (UWB) radio channel was recently made available for use in the USA in the range 3.1 GHz-10.6 GHz. European and Asian authorities are also preparing rulings to enable commercial marketing and use of devices based on UWB radio technology (UWB-RT).

High-definition television (HDTV) and high-resolution video display rendering will use DVIs with the added disadvantage that high transmission data rates are to be used between the video signal source and the display. For example, the digital DVI (DVI-D) standard link allows a digital connection from a set-top box (STB) to a display eliminating unnecessary digital to analog conversions and keeping the signal digital. Almost all STBs use DVI-D and do not carry any analog signal. An HDTV capable TV set has maximum resolution of 1920×1080 pixels (60 Hz AC power supply) and should only use a single link (cable or wireless), as do almost all DVI equipped displays. However, plasma screens and flat panel displays (FPDs) can have much higher resolutions and refresh rates and could use a dual or multiple DVI-D links. The requirement for very high data rates between the video signal sources and high-resolution displays is particularly demanding if the video signal is to be transmitted wirelessly to the display. Thus, there is a need for new solutions to increase both the effective data rate and the link distance between the video source and the display. This problem prevails equally when connecting a personal computer (PC) to a high-resolution video display through DVIs or other digital interfaces.

One possibility to overcome this problem is to split a high-rate data stream into multiple lower-rate data streams to achieve a compound data rate equal to the desired high-data rate. In such cases, designers are faced with the problem of choosing the best possible multi-stream technology for a given set of system parameters and criteria. There exist two different approaches: a) support of multiple data streams by means of co-located and non-interfering networks, also called piconets, and b) support of multiple data streams within a single communication cell based on a suitable multiple access (MA) technique. All of these approaches are based on an efficient channel access method and the most popular prior art MA methods are briefly described hereafter, listing their known features and suitability when applied to networks with terminals based on UWB-RT.

MA techniques, e.g., Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Code Division Multiple Access (CDMA), and in particular Rate Division Multiple Access (RDMA) as described by Martin Weisenhorn and Walter Hirt, “Novel Rate-Division Multiple-Access Scheme for UWB-Radio-Based Sensor Networks”, IEEE Int. Zurich Seminar on Communications, pages 76-81, February 2004, have been known for some time. Some of these techniques could be applied to mitigate the high-rate video distribution problem. For example, when using RDMA, each data stream is transmitted by a binary antipodal signal, preferably with a low duty cycle, that can be distinguished at the destination receiver by an appropriate, unique choice of a user-specific pulse rate or a combination of at least two user-specific pulse rates. However, it may be advantageous in certain applications to combine the features and benefits of the basic RDMA method with one or more of the prior art multiple access methods.

Known wireless (video) displays include:

• • PDA-like screens of low pixel resolution (320×240), near/full-motion video (15/30 fps (frames per second)), in high or true color format (12, 16 or 24 bit/pixel);
  • tablet—SVGA up to XGA resolution, 16/24 bit/pixel, with full or reduced refresh rate; if full-motion video is supported, then the video stream is commonly MPEG-encoded when sent over a wireless channel, with data rates of 1-10 Mbps; and
  • video projectors and TV displays with internal MPEG decoding engine and Wi-Fi® capability with nominal data rates of 11-54 Mbps (Wi-Fi is a registered trademark of the Wi-Fi Alliance).

While some of the above will support full-motion video in DVD or even HDTV quality, none of these display devices can sustain the high-resolution full-rate refresh requirements of a standard PC over a wireless channel. A distinction is made between a cinematographic and “full-motion” video display of 25, resp. 30 fps, and a more demanding notion of “full-rate” computer monitor, with refresh rates in excess of 60 fps. Here the requirements of full-rate display refresh are addressed.

A question worth asking is: How to wirelessly connect a full-rate high resolution PC monitor? A brief step-wise calculation of the bandwidth requirements yields the following results:

a) A typical PC monitor resolution ranges from XGA (1024×768) up to QUXGA-W (3840×2560); most common resolutions are SXGA/+(1280×1024) and UXGA (1600×1200). The large scale introduction of HDTV will drive the standard PC resolution up to QXGA (2048×1536).

b) The minimum refresh rate for PC displays in TFT LCD technology is 60 fps.

c) Unlike video codecs such as MPEG-2/4, there seem to be no general codecs for GUIs (Graphical User Interfaces). A universal GUI display that works with a common operating system uses the least common denominator, i.e., un-encoded video transmission between the computer video source and its display. Otherwise, the lack of a large enough market would not allow the widespread use of full rate/motion wireless displays.

Thus the net bandwidth required by an XGA display at 60 fps is: (1024×768 pix)×(24 bit/pix)×(60 fps)=1132 Mbps or 1.132 Gbps. Assuming a 25% overhead for wireless encoding and protocol, the raw data rate for XGA is ˜1.4 Gbps; for QXGA: ˜5.7 Gbps; for QUXGA-W: ˜17.7 Gbps.

Considering SXGA/UXGA, the most likely resolution used for PC, notebook and HDTV flat screens, it is found that the raw data rates are also very high for wireless transmission (SXGA: ˜2.4 Gbps; UXGA: ˜3.5 Gbps).

From the above it follows that there is still a need in the art for a scheme that allows displaying of a video signal on a display without cables and in particular depending upon a user interaction. It would be further advantageous if the scheme enables a user-aware wireless video display offering full rate refresh at high resolution.

SUMMARY OF THE INVENTION

Therefore, in accordance with the invention, there is provided a system for displaying a video signal depending upon a user interaction. An example of a system comprises: at least one transmitter to transmit a video signal, a display for receiving the video signal, at least one transmitter for transmitting location signals, and at least three transceivers for receiving the location signals from the transmitter and for transmitting modified location signals. The geometric locations of the at least three transceivers are known to a computing device. The computing device is then able to derive a transmit location of the location signals based on the received modified location signals, where afterwards the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction. The user interaction changes or moves the transmit location of the location signals when, for example, a user moves the display or himself.

In accordance with another aspect of the invention, there is provided a method for displaying a video signal on a display depending upon a user interaction. An example of a method comprises the steps of: receiving by the display the video signal from a computing device; transmitting location signals via at least three transceivers to the computing device; receiving by the computing device modified location signals from the at least three transceivers, the geometric locations of the at least three transceivers being known to the computing device; deriving by the computing device a transmit location of the location signals based on the received modified location signals; and modifying the content of the video signal in response to the derived transmit location depending upon the user interaction.

DESCRIPTION OF FIGURES

Advantageous embodiments of the invention are described in detail below, by way of example only, with reference to the following schematic figures, in which:

FIG. 1a illustrates a first wireless multi-channel video distribution system with a set-top box (STB) and a flat panel display (FPD). In the STB, the video source's data stream is divided into four parallel data sub-streams. Each receiver built into the display is at least capable of receiving one of the four sub-streams;

FIG. 1b illustrates a further wireless multi-channel, high-resolution video distribution system with a graphic and video sub-system (GVS) and a flat panel display (FPD). In the GVS, the video source's data stream is divided into four parallel data sub-streams. Generally, each receiver built into the display and connected to the respective antenna (A1, A2, A3 or A4) is at least capable of receiving one of the four data sub-streams;

FIG. 2a shows a basic system setup usable to support high-resolution video signal distribution to an un-tethered FPD when the data sub-streams are sent from a GVS to wireless transceivers (TR1, TR2, TR3 and TR4) by means of wires;

FIG. 2b shows the basic system setup of FIG. 2a supporting localization and tracking of one antenna (A1) of the FPD within the reference frame defined by the indicated (x,y,z) coordinate system with origin O=(0,0,0). The position of the antenna A1 is determined by the position vector of antenna A1 (PVA1) defined by the geometric position coordinates p₁=(x₁,y₁,z₁).

FIG. 2c illustrates an extension of the setup shown in FIG. 2b to all four antennas of the FPD, where the position of antenna An is determined by the position vector of antenna n (PVAn) as defined by the coordinates p_n=(x_n,y_n,z_n), n=1,2,3,4 (for reasons of clarity, only PVA4 is shown);

FIG. 3a shows a further basic system setup usable to support high-resolution video signal distribution to an un-tethered FPD when the data sub-streams are sent to the wireless transceivers (TR1, TR2, TR3 and TR4) by means of further wireless links, preferably based on UWB-RT;

FIG. 3b shows the basic system setup of FIG. 3a supporting localization and tracking of the spatial position of antenna A1 mounted on the FPD within a reference frame, defined by the indicated (x,y,z) coordinate system with origin O=(0,0,0) Assuming that the positions of the other three antennas (A2, A3 and A4) have been determined in a similar way (e.g., see FIG. 2c), this figure also illustrates how the position and orientation of the FPD can be described by means of i) a panel position vector (PPV) pointing to the center of the display panel and ii) a perpendicular panel orientation vector (POV) that defines the geometric orientation of the panel plane;

FIG. 4a illustrates a further basic wireless multi-channel video distribution system with a graphics and video subsystem (GVS) and a wall mounted, stationary flat panel display (FPD). In the GVS, the video source's data stream is divided into four parallel data sub-streams that are carried by cables to corresponding transceivers (TR1, TR2, TR3 and TR4). The latter transmit the data sub-streams either by means of i) four independent piconets or ii) by means of a multi-user communication network operating within a single communication cell. Generally, each receiver built into the display and connected to the respective antenna (A1, A2, A3 or A4) is at least capable of receiving one of the four data sub-streams;

FIG. 4b illustrates another basic wireless multi-channel video distribution system capable to support applications based on user interaction. The system comprises a graphics and video subsystem (GVS) and a wall mounted flat panel display (FPD). In the GVS, the video source's data stream is divided into four parallel data sub-streams that are carried by cables to corresponding transceivers (TR1, TR2, TR3 and TR4). Generally, each receiver built into the display and connected to the respective antenna (A1, A2, A3 or A4) is at least capable of receiving one of the four data sub-streams. The basic system is enhanced with a transmitting beacon (B1) carried by the user that allows measurement of the user's position and orientation relative to a stationary display; and

FIG. 4c illustrates the system of FIG. 4b but with the difference that the wired links between the GVS and the transceivers (TR1, TR2, TR3, TR4) have been replaced by wireless links (for reasons of clarity, only the links between the GVS and TR1 and TR2 are shown).

The drawings are provided for illustrative purposes only.

DESCRIPTION OF THE INVENTION

This invention provides methods and systems for displaying a video signal on a display depending upon a user interaction. Interesting applications become feasible which are based on a user-aware wireless video display offering full rate refresh at high resolution. The system provided is particularly well suited for wireless communication and positioning technologies exploiting the ultra-wideband (UWB) radio channel recently made available by the US Federal Communications Commission (FCC) in the frequency range between 3.1 GHz and 10.6 GHz.

Thus, the present invention provides a system for displaying a video signal depending upon a user interaction. An embodiment of a system comprises at least one transmitter to transmit a video signal, a display for receiving the video signal, at least one transmitter for transmitting location signals, and at least three transceivers for receiving the location signals from the transmitter and for transmitting modified location signals. The geometric locations of the at least three transceivers are known to a computing device. The computing device is then able to derive a transmit location of the location signals based on the received modified location signals, where afterwards the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction. The user interaction changes or moves the transmit location of the location signals when, for example, a user moves the display or himself. In advantageous embodiments multiple transmitters for transmitting location signals are used in order to increase the accuracy of the location determination.

The invention also provides a method for displaying a video signal on a display depending upon a user interaction. The method comprises the steps of: the display receiving the video signal from a computing device; transmitting location signals via at least three transceivers to the computing device; the computing device receiving modified location signals from the at least three transceivers, the geometric locations of the at least three transceivers being known to the computing device; the computing device deriving a transmit location of the location signals based on the received modified location signals; and modifying the content of the video signal in response to the derived transmit location depending upon the user interaction.

The method can further comprise the steps of: each of the at least three transceivers receiving the location signals, determining arrival times from the received location signals, and each of the at least three transceivers transmitting the determined arrival times within modified location signals to the computing device. This allows to provide the computing device with the relevant information for the subsequent determination of the transmit location.

When the display is a mobile or portable, tablet-like user-aware wireless display or flat panel display (FPD) that is freely movable by a user, then the user can decide by user interaction how something is shown or from which angle, e.g., of a picture, movie, or construction, it is displayed.

The position (location) of the display can be fixed or the display can mounted on a wall, and then the location signals can be transmitted by a beacon transmitter that can be attached to a user. Thus, the calculated transmit location of the location signals is directly related to the movement of the user. The fixed or mounted display can then be a stationary large size user-aware display usable for immersive virtual reality or other applications.

The connection between the computing device and the display can be established by cables (partly), various wireless links, or a combination thereof. Best suited for the wireless links are UWB radio or other very wideband RF (radio frequency) technologies. The transmission of the video signal from the computing device to the display and the transmission of the location and modified signals to the computing device via the at least three transceivers can be performed by wireless links. Then, no cables are required, which allows more freedom of movement for the user.

As indicated, user awareness is enabled through two basic approaches: i) by determining the geometric position and orientation of the display with the transceivers, e.g., of a handheld wireless display panel, carried by a user within a reference frame; ii) by determining a user's geometric position and orientation in front of a stationary wireless display, while the user has a beacon transmitter attached.

When the display is stationary, e.g. mounted on a wall, then the geometric locations of the display are known to the computing device. The computing device is then able to derive the transmit location of the beacon transmitter relative to the display's geometric location based on the received modified location signals and to modify the content of the video signal in response to the derived transmit location of the beacon transmitter depending upon the user interaction, i.e. movement of the beacon transmitter.

The video signal or channel can be split into at least three sub-channels, which can be realized, for example, with UWB radio technology, based on coordinated piconets. The at least three sub-channels can be de-multiplexed by a Graphical Processing Unit (GPU), or by a standard chipset at the read port of the video RAM. That means a high-rate video data stream can be split into multiple sub-streams, such that each sub-stream can be transmitted wirelessly to a full-rate high-resolution display, where the sub-streams are combinable into a composite video data stream (video signal). A suitable arrangement of the transceivers and the antennas associated with the display enable user-aware applications. The wireless display can feature full-rate/full-motion refresh at high resolutions.

At the receiving end, the display will process the video sub-channels by allocating and mapping each of them into one of the adjacent vertical stripes. Whereas the mapping to the actual screen circuitry can be done differently, a relevant feature is the geometrical location of the transmitter, as this defines the transmit location of the location signals. This location should be eligible or appropriate for wireless triangulation.

Based on today's technological developments and the assumption that further advanced state-of-the-art UWB wireless links will become available in the future, it is estimated that three or four parallel channels for SXGA and SXGA+ displays will be suitable to achieve a compound data rate of well above 1 Gbps over an acceptable distance.

The desire to split a high-rate video data stream into multiple sub-streams, such that each sub-stream can be transmitted wirelessly to the same destination, e.g., to a high-resolution display, can be exploited further and thus has further advantages. An implementation of a suitable arrangement of receiving antennas on the display can be used to enable user-aware applications or interactions, such that the user can interact with graphic or video shown on said display. Below are listed some advantages relating to displays, particularly high-resolution video displays, which are capable to support applications based on user awareness and/or interaction:

• • a triangulation-based localization—which enables user-awareness, virtual reality (VR) etc.—adds only a minor cost to the display, this comes as a by-product of having a full-rate high-resolution wireless display;
  • in the future, the implementation is possible with commercial-off-the-shelf (COTS) components;
  • wide range of applications, from scientific and engineering, design, business, e-learning, to consumer and entertainment;
  • potential products can be marketed both as commodity h/w and as dedicated, professional systems;
  • the affordability of implementations lowers the entry barriers of established virtual reality (VR) applications and may enable new, un-anticipated applications;
  • transmission of multiple sub-streams of data enables high aggregate data rates suitable for high-resolution video signal display;
  • suitable placement of multiple antennas on the display, that can be a flat panel display (FPD), can be exploited to enable applications desiring user-aware interaction;
  • the combination of multiplexed, full-rate high-resolution wireless video streaming to a movable display and simultaneous wireless 3D position location and tracking of the display enables novel user-aware applications, one such application is the portable tablet usable for virtual reality (VR) and 3D interaction, the innate hand-eye coordination property of the human sensorial system (retina>cortex>motor nerve>muscle) can be exploited;
  • expensive electro-mechanics are replaced by wireless triangulation (i.e., position location and tracking);
  • the combination of multiplexed, full-rate high-resolution wireless video streaming to a fixed display and simultaneous wireless 3D position location and tracking of a user in front of the fixed display enables novel user-aware applications, e.g., for engineers or designers,
  • such an application is a large, wall-mounted user-aware display, where the same type of triangulation scheme enables an “immersive VR”-class of applications, without the need of wiring the users with intrusive sensors;
  • the precision of the 3D positioning can be optimized by proper geometric placement of the antennas on the display, e.g., by maximizing the spatial separation between antennas and mounting them in the same plane;
  • the precision of the 3D positioning can be further optimized by proper geometric placement of the at least three transceivers within the room where the display is used, e.g., in order to obtain minimal GDP (geometric dilution of precision), thus avoiding that all transceivers are placed within a plane;
  • 3D positioning is preferably based on a TDOA (time difference of arrival) scheme.

In addition, the proposed space-multiplexed wireless video transmission scheme provides and supports:

• • multi-purpose video sources: from PCs to game consoles, DVD players, HDTV tuners, video-cams, video servers;
  • multi-purpose displays: the same wireless panel(s) could be used as tablet, monitor or HDTV display;
  • mobility within a range of up to 4 m, or farther at reduced frame rate;
  • a low cost wireless system that consists of four COTS chipsets, e.g., adapted from the USB2.0 or other connecting interfaces; and
  • reduced costs of new applications and by-products, such as affordable virtual reality (VR).

Of particular interest are the user-aware displays and “immersive VR” systems, when built on commodity hardware. Such systems can be used for scientific and engineering visualization, remote learning, telemedicine, conferencing systems, e-retail, arcade and multi-user/Net-based games.

FIG. 1a illustrates a wireless multi-channel video distribution system 1 comprising a set-top box (STB) 10 and a flat panel display (FPD) 30 which are connected via wireless links 5. The set-top box 10 comprises a video source interface 11, an STB Controller & De-Multiplexer unit 12, and STB transmit/receive units 13 that are connected to STB antennas 14 and the STB Controller & De-Multiplexer unit 12, respectively. The STB Controller & De-Multiplexer unit 12 receives the video signal from the video source interface 11 that is connected to a computing device 50 that provides the video signal, also referred to as video data stream, via a video link 7. The computing device 50 receives modified location signals via an information link 8 from the STB Controller & De-Multiplexer unit 12.

The flat panel display (FPD) 30 comprises a video sink unit 31, an FPD Controller & Multiplexer unit 32, and display transmit/receive units 33 that are connected to FDP antennas 34 and the FPD Controller & Multiplexer unit 32, respectively. The FPD Controller & Multiplexer unit 32 provides the received video signal to the sink unit 31 for further processing and displaying.

In the set-top box 10, the video data stream coming from the computing device 50 is here divided into four parallel data sub-streams to feed i) four independent piconets or ii) a multi-user communication network operating within a single communication cell, where each display transmit/receive unit 33 built into the display 30 is at least capable of receiving one of the four sub-streams. When receiving one of the four sub-streams each display transmit/receive unit 33 together with its corresponding FDP antenna 34 is also referred to as display receiver. In this setup, the set-top box 10 transmits preferably un-compressed video signals in each sub-channel to remove the burden of signal decompression in the display 30, thereby reducing power consumption and cost.

Similarly, FIG. 1b illustrates a wireless multi-channel, high-resolution video distribution system 2 comprising a graphic and video sub-system (GVS) 20 and the flat panel display 30. The same reference numbers are used to denote the same parts or the like. The graphic and video sub-system 20, short GVS 20, comprises a high-resolution video source interface 21, a GVS Controller & Video De-Multiplexer unit 22, and GVS transmit/receive units 23 that are connected to GVS antennas 24 and the GVS Controller & Video De-Multiplexer unit 22, respectively. The GVS Controller & Video De-Multiplexer unit 22 receives a high-resolution video signal from the computing device 50 via the video link 7 and the high-resolution video source interface 21. On the other hand, the computing device 50 receives modified location signals via the information link 8 from the GVS Controller & Video De-Multiplexer unit 22.

In the graphic and video sub-system 20, the high-resolution video data stream coming from the computing device 50 is divided by the GVS Controller & Video De-Multiplexer unit 22 into four parallel data sub-streams to feed i) four independent piconets or ii) a multi-user communication network operating within a single communication cell. Generally, each display transmit/receive unit 33 built into the display 30 and connected to the respective FDP antenna 34 (A1, A2, A3 or A4) is at least capable of receiving one of the four data sub-streams transmitted via the GVS antennas 24. A1, A2, A3 and A4 indicate the transmit location of the respective FDP antenna 34 of the display 30. The positions of the FPD antennas 34 at the display 30 and particularly the distances between the antennas are a priori known by design. For both embodiments indicated in FIGS. 1a and 1b, the four FPD antennas 34 and respective transmit/receive units 33 offer three different modes of operation that can be used separately or in combination:

High Data Rate Video Streaming

Aggregation of the four parallel sub-system data streams to obtain a four-fold compound data rate enabling high rate video streaming to the display 30. This mode solves the problem that a single wireless link 5 may not be able to sustain the desired high compound data rate over the preferred distance.

Improved Link Range and/or Link Robustness

The set-top box 10 and the flat panel display 30 form a multiple-input/multiple-output (MIMO) transmission channel, where the set-top box 10 emits the same video source data over all sub-channels, thereby introducing a spatial diversity effect that can be exploited. Correspondingly, all four FPD antennas 34 and display transmit/receive units 33 receive up to four versions of the same video source data stream. This allows the display system to exploit the advantages of MIMO communication by i) achieving a higher data rate over a given link distance or ii) achieving an increased link distance for a given lower data rate or iii) achieving a higher link robustness or iv) using any combination of i) to iii).

Position Location and Tracking

With each of the four independent FPD antennas 34 placed in one of the corners of the display 30 (maximum spatial separation) the display 30 can be enabled with user-awareness by sensing the geometric position of the four FPD antennas 34. From knowledge of the positions of at least three of these antennas 34 the orientation of the panel's viewing plane and its absolute position within a reference frame can be determined; such information is useful for graphic rendering applications requiring user awareness for interactive display operation and feedback. Further details of this operational mode will be discussed below.

FIGS. 2a, 2b and 2c illustrate a basic system setup in an office or residential environment with wireless transceivers 41, 42, 43, 44, also referred to as transceivers 41-44. FIG. 2a shows the setup usable to support high-resolution video signal distribution, over four sub-channels (C1-C4) to an un-tethered display 30 when the video data is sent to the wireless transceivers 41, 42, 43, 44, also labeled with TR1, TR2, TR3 and TR4, by means of cables or cable links 4. Depending on the application, the wireless links 5 between the transceivers 41, 42, 43, 44 and the display 30 may be unidirectional or bidirectional, as indicated in the figure. In this first arrangement where the primary application is to feed a video signal to the display 30, the actual positions of the transceivers 41-44 is not critical as long as the desired quality of service (QoS) can be maintained over the wireless links 5.

FIG. 2b illustrates how the video streaming system setup can be used to support localization and tracking of one FDP antenna 34, here also referred to as antenna A1, of the display 30 within a reference frame defined by the indicated (x,y,z) coordinate system with origin O=(0,0,0). The respective location or position of the antennas are indicated by and labeled with An, with n=1,2,3,4. As indicated in the figure, the position of antenna A1 (e.g., the antenna's feed point may be used to define its position) is determined by the position vector of antenna A1 (PVA1) defined by the geometric coordinates p₁=(x₁,y₁,z₁), which can be determined by the following procedure. For reasons of simplicity, detailed descriptions are only given for antenna A1; similar procedures can be applied to the other FDP antennas, labeled with A2, A3 and A4 as indicated in FIG. 2c.

1) Antenna A1 emits a so-called positioning signal (PS), e.g., in the form of an acknowledgment (ACK) signal after reception of a video data packet, that is receivable by each of the four transceivers 41-44 (TR1, TR2, TR3 and TR4) and that comprises unambiguous information to identify it as a signal emitted from antenna A1. The time instant of this emission is not known and thus not available anywhere in the system.

2) The transceivers 41-44 (TR1, TR2, TR3 and TR4) register reception of the PS from antenna A1 and relay the source identification information and a time count to the GVS 20 in the manner as described in the United States Patent Application 20030174086, presently assigned to the assignee of the instant application and the disclosure of which is incorporated herein by reference. Alternatively, the transceivers 41-44 have the capability to measure the absolute arrival time of the PS sent by antenna A1, in which case the transceivers 41-44 forward the source identification information and the absolute arrival time to the GVS 20 within the modified location signals.

3) Given the known propagation times between each transceiver 41, 42, 43, 44, and the GVS 20, a processing unit (not shown) in the computing device 50 first computes the time difference of arrival (TDOA) between three pairs of received transceiver signals. Alternatively, the TDOA values can be computed from measured absolute arrival times, which are sent by transceivers 41-44 via the GVS 20 to the processing unit (computing device) 50. This procedure yields sufficient information to find a solution based on known methods for the position vector of antenna A1 (PVA1): p₁=(x₁,y₁,z₁).

4) The above procedure can be repeated in parallel or sequentially for each FPD antenna 34 of the display 30 to yield position vectors for each FPD antenna 34: p_n=(x_n,y_n,y_z) n=1,2,3,4. Any three of these vectors can be used to define the orientation of the FPD 30 plane while the fourth redundant position vector may be used to improve the plane's parameter estimates (see also FIG. 3b).

5) The information of the FPD plane orientation within the reference frame can be used by the application that renders the video signal for display.

A further embodiment of the proposed system is indicated in FIG. 3a, showing the basic system setup usable to support high-resolution video signal distribution to the un-tethered display 30 when the data sub-streams are sent to the wireless transceivers 41, 42, 43, 44 (TR1, TR2, TR3 and TR4) by means of further wireless links 6, thereby replacing the cable links 4 illustrated in FIG. 2a.

Consequently, FIG. 3b shows an alternative to the embodiment illustrated in FIG. 2b, where FIG. 3b indicates how the orientation of the FPD plane can be defined after at least three of the four antenna positions (A1, A2, A3, A4) within the FPD plane have been determined. In particular, the position and orientation of the FPD plane is defined by i) the panel position vector (PPV) pointing to the geometric center of the FPD plane and ii) the plane orientation vector (POV) positioned perpendicular to the FPD plane; without loss of generality, the POV may be a normalized unit vector. The coordinates of the PPV and the POV, derivable from the TDOA measurements described above, provide sufficient information for an application program rendering video signals in response to the FPD's geometric position and orientation within the geometric reference frame defined by the (x,y,z) coordinate system with origin O=(0,0,0).

In operation, the content of a video signal on the display 30 is modified depending upon a user interaction, i.e., when a user moves the display 30. Generally, the display 30 receives the video signal from the computing device 50.

Location signals are transmitted from the FDP antennas 34 via the at least three transceivers 41-44 and the GVS 20 to the computing device 50. In the transceivers 41-44, whose geometric locations are also known to the computing device 50, the received location signals are transformed to modified location signals which then include arrival time information. The computing device 50 that then receives the modified location signals from the transceivers 41-44 is able to derive each transmit location (A1, A2, A3, A4) of the location signals based on the received modified location signals. The content of the video signal is then modified by the computing device 50 in response to the derived transmit location (A1, A2, A3, A4) depending upon the user interaction, that is the movement of the display 30 in all directions.

A further application of the display 30 is illustrated in FIG. 4a, where the display 30 remains in a stationary position, for example by mounting it on a wall. Depicted is a wireless multi-channel video distribution system with the GVS 20 and the display 30 that here is a wall mounted flat panel display 30. In the GVS 20, the video source's data stream is divided into four parallel data sub-streams (C1-C4) that are carried by cable links 4 to the respective transceivers 41-44. The latter transmit the data sub-streams via the wireless links 5 either by means of i) four independent piconets or ii) by making use of a multi-user communication network operating within a single communication cell. Generally, each display transmit/receive unit 33 built into the display 30 and connected to the respective antenna 34 is at least capable of receiving one of the four data sub-streams via the wireless links 5.

FIG. 4b illustrates further the basic wireless multi-channel video distribution system introduced in FIG. 4a to support applications based on user interaction. The system comprises again the GVS 20 and the wall mounted flat panel display, i.e., the display 30. In the GVS 20, the video source's data stream is divided into four parallel data sub-streams (C1-C4) that are carried by the cables links 4 to the corresponding transceivers 41-44. The latter transmit the data sub-streams either by means of i) four independent piconets or ii) by making use of a multi-user communication network operating within a single communication cell. Generally, each display transmit/receive unit 33 built into the display 30 and connected to the respective antenna 34 (with position A1, A2, A3 or A4) is at least capable of receiving one of the four data sub-streams. The system is enhanced with a transmitting beacon 60, also referred to as beacon transmitter 60, carried by a user 62. The place of the beacon transmitter 60 defines here the transmit location. This arrangement allows measurement of the user's position p_u=(x_u,y_u,z_u) and his orientation with respect to the stationary display 30. In this setup, the display's position is known and defined as its geometric center with the coordinates p_d=(x_d,y_d,z_d), thus the user direction vector (UDV), indicating a user's position relative to the FPD's known position and orientation, is obtained by the vector difference p_UVD=p_u−p_d=(x_u−x_d,y_u−y_d,z_u−z_d). The coordinates of the UVD, p_u=(x_u,y_u,z_u), are derivable from the TDOA measurements as described above; together with the display's known position and geometric orientation, p_d=(x_d,y_d,z_d), sufficient information can be provided to an application program rendering video signals in response to the user's geometric position that is the transmit location of the beacon transmitter 60, indicated with B1. Thus, the user's location within the geometric reference frame defined by the (x,y,z) coordinate system with origin O=(0,0,0) can be determined. The application program that modifies the video signal executes on the computing device 50 which for simplicity is not shown in the figure. The computing device 50 can be any computer or sever that is able to influence the video signal based on the calculations.

In operation, the display 30 receives the video signal from the computing device 50 via the GVS 20 and the transceivers 41-44. As indicated, the geometric location and orientation of the display 30 is known. Location signals are transmitted by the beacon transmitter 60 via the transceivers 41-44 and the GVS 20 to the computing device 50. In the transceivers 41-44, whose geometric locations are also known to the computing device 50, the received location signals are transformed to modified location signals which then include arrival time information. The computing device 50 that then receives the modified location signals from the transceivers 41-44 is able to derive a transmit location (B1) of the location signals relative to the display's geometric location based on the received modified location signals. The content of the video signal is then modified by the computing device 50 in response to the derived transmit location (B1) depending upon the user interaction by the user 62.

Alternatively, in the setup of FIG. 4b, it is not necessary to know the display's position a priori. Similarly to the embodiment illustrated in FIG. 3b, it is possible to determine the display position coordinates p_d=(x_d,y_d,z_d) by measuring TDOA parameters and to compute the display position coordinates from location signals transmitted via the FPD's antennas (A1-A4).

A further alternative to the system shown in FIG. 4b is indicated in FIG. 4c, where the cable links 4 between the GVS 20 and the transceivers 41-44 have been replaced by further wireless links 6, i.e., the four data sub-streams (C1-C4) are sent wirelessly to the reference transceivers 41-44. Variations described for the present invention can be realized in any combination desirable for each particular application. Thus particular limitations, and/or embodiment enhancements described herein, which may have particular advantages to a particular application need not be used for all applications. Also, not all limitations need be implemented in methods, systems and/or apparatus including one or more concepts of the present invention. Methods may be implemented as signal methods employing signals to implement one or more steps. Signals include those emanating from the Internet, etc.

The present invention can be realized in hardware, software, or a combination of hardware and software. A visualization tool according to the present invention can be realized in a centralized fashion in one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system—or other apparatus adapted for carrying out the methods and/or functions described herein—is suitable. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein. The present invention can also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which—when loaded in a computer system—is able to carry out these methods.

Computer program means or computer program in the present context include any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after conversion to another language, code or notation, and/or reproduction in a different material form.

Thus the invention includes an article of manufacture which comprises a computer usable medium having computer readable program code means embodied therein for causing a function described above. The computer readable program code means in the article of manufacture comprises computer readable program code means for causing a computer to effect the steps of a method of this invention. Similarly, the present invention may be implemented as a computer program product comprising a computer usable medium having computer readable program code means embodied therein for causing a function described above. The computer readable program code means in the computer program product comprising computer readable program code means for causing a computer to affect one or more functions of this invention. Furthermore, the present invention may be implemented as a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for causing one or more functions of this invention.

It is noted that the foregoing has outlined some of the more pertinent objects and embodiments of the present invention. This invention may be used for many applications. Thus, although the description is made for particular arrangements and methods, the intent and concept of the invention is suitable and applicable to other arrangements and applications. It will be clear to those skilled in the art that modifications to the disclosed embodiments can be effected without departing from the spirit and scope of the invention. The described embodiments ought to be construed to be merely illustrative of some of the more prominent features and applications of the invention. Other beneficial results can be realized by applying the disclosed invention in a different manner or modifying the invention in ways known to those familiar with the art.

Claims

1. A display method comprising displaying a video signal on a display depending upon user interaction, the step of displaying comprising the steps of:

the display receiving the video signal from a computing device;

transmitting location signals from at least one transmitter via at least three transceivers to the computing device;

the computing device receiving modified location signals from said at least three transceivers having geometric locations known to the computing device;

the computing device deriving a transmit location of the location signals based on the received modified location signals; and

modifying content of the video signal in response to the derived transmit location depending upon the user interaction.

2. The method according to claim 1, further comprising the steps of:

each of said at least three transceivers receiving the location signals,

determining arrival times from the received location signals, and

each of said at least three transceivers transmitting the determined arrival times within modified location signals to the computing device.

3. The method according to claim 1, further comprising employing wireless links in connecting the computing device to the display.

4. The method according to claim 1, wherein the display is a fixed display, and further comprising a step of transmitting the location signals by a beacon transmitter.

5. The method according to claim 4, wherein the beacon transmitter is attached to a user.

6. The method according to claim 1, wherein the video signal is comprised of a video data stream, and further comprising splitting the video signal into multiple sub-streams, such that each sub-stream is transmittable wirelessly to the display, and the sub-streams are combinable into a composite video data stream.

7. The method according to claim 1, wherein the display is one of: a portable and a mobile user-aware display freely movable by a user, and further comprising the user deciding by user interaction how an object is shown.

8. The method according to claim 1, wherein a connection between the computing device and the display is established by at least one of: cables; various wireless links; UWB radio, and very wideband radio frequency technologies.

9. The method according to claim 1, wherein the video signal is split into at least three sub-channels employing UWB radio technology based on coordinated piconets, and further comprising de-multiplexing said at least three sub-channels employing one of: a Graphical Processing Unit and a standard chipset at the read port of the video RAM.

10. The method according to claim 9, further comprising the display processing the video sub-channels by allocating and mapping each of them into one of the adjacent vertical stripes.

11. An article of manufacture comprising a computer usable medium having computer readable program code means embodied therein for causing display of a video signal, the computer readable program code means in said article of manufacture comprising computer readable program code means for causing a computer to effect the step of displaying the video signal on a display depending upon user interaction, the step of displaying comprising the steps of:

the display receiving the video signal from a computing device;

transmitting location signals from at least one transmitter via at least three transceivers to the computing device;

the computing device receiving modified location signals from said at least three transceivers having geometric locations known to the computing device;

the computing device deriving a transmit location of the location signals based on the received modified location signals; and

modifying content of the video signal in response to the derived transmit location depending upon the user interaction.

12. An article of manufacture as recited in claim 11, the computer readable program code means in said article of manufacture further comprising computer readable program code means for causing a computer to effect the steps of:

each of said at least three transceivers receiving the location signals,

determining arrival times from the received location signals, and

each of said at least three transceivers transmitting the determined arrival times within modified location signals to the computing device.

13. An article of manufacture as recited in claim 11, the computer readable program code means in said article of manufacture further comprising computer readable program code means for causing a computer to effect the step of employing wireless links in connecting the computing device to the display.

14. An article of manufacture as recited in claim 11, wherein the display is a fixed display, and the computer readable program code means in said article of manufacture further comprising computer readable program code means for causing a computer to effect the step of transmitting the location signals by a beacon transmitter.

15. A display system comprising:

a display for receiving the video signal, said system for displaying a video signal depending upon a user interaction

at least one transmitter for transmitting location signals;

at least three transceivers for receiving the location signals from said at least one transmitter and for transmitting modified location signals, the geometric locations of said at least three transceivers being known; and

a computing device for deriving a transmit location of the location signals based on the received modified location signals, wherein the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction.

16. The display system according to claim 15, wherein said at least three transceivers are adapted to determine arrival times from the received location signals, the determined arrival times are transmittable by said at least three transceivers within the modified location signals to the computing device.

17. The display system according to claim 15, wherein the display is one of: a mobile display and a portable display.

18. The display system according to claim 15, further comprising wireless links for the transmission of the video signal from the computing device to the display and the transmission of the location and modified signals to the computing device via said at least three transceivers.

19. A computer program product comprising a computer usable medium having computer readable program code means embodied therein for causing display of a video signal, the computer readable program code means in said computer program product comprising computer readable program code means for causing a computer to effect the functions of:

receiving for a display the video signal, said display for displaying the video signal depending upon a user interaction;

at least one transmitter for transmitting location signals;

at least three transceivers for receiving the location signals from said at least one transmitter and for transmitting modified location signals, the geometric locations of said at least three transceivers being known; and

a computing device for deriving a transmit location of the location signals based on the received modified location signals, wherein the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction.

20. A computer program product as recited in claim 19, the computer readable program code means in said computer program product further comprising computer readable program code means for causing a computer to effect the function of wireless links for the transmission of the video signal from the computing device to the display and the transmission of the location and modified signals to the computing device via said at least three transceivers.