A MOBILE VIRTUAL REALITY SERVICE
Inconventional distributed and networked virtual re- ality systems users’ capabilities to move are very
limited, if they can move at all. These systems provide applications like
teleconferencing or concurrent engi- neering, but users are tied to their
desktop workstation or very near them. For example in all of the most well-
known distributed virtual reality systems like AVIARY [lo, 111, DistributiveVirtualEnvironment[3],dVS/dVISE[6] or The MR Toolkit [9] participants use
their personal computers to interact with other users and applications
connected by a wired network like internet.
One
example of such a network is a seamless wireless ATM network (SWAN) [l], which
is based on room-sized picocells and mobile multimedia endpoints like portable
multimedia terminals. However, few re- search results have been reported in the
literature so far concerning the integration of virtual reality and mo- bile wireless
communications systems. One project is going on in Nara Institute of Science
and Technology in Japan [5], where the research concentrates on aug- mented
virtual reality merging both real and virtual environments so that a mobile
observer is able to treat virtual objects as if they actually existed. A
real-time distributed mobile virtual reality is a new research project, started
at VTT (Technical research Centre of Finland) Eletronics and University of
Oulu. The research concentrates on combining a virtual re- ality environment and
a wireless mobile network so that users could move freely inside the network
area (for example inside a building) and interact with real and virtual objects
and users. The system is based on in-building picocellular mobile personal
surround- ing networks. The diameter of a personal surrounding network is very
short, want
to construct very low-powered hand-held termi- nals, still capable of
transmitting high bandwidth mul- timedia information. Because of the small cell
size the number of base station needed in the system would be enormous, if each
cell had a fized base station like in conventional mobile networks. We on the
other hand try to minimize the number of fixed base stations by al- lowing
personal surrounding networks of different users to communicate directly with
each other. In the proto- type implementation we are going to use the mobileIP
protocol so that mobile users could easily interact with fixed users through
Internet. In the future user’s per- sonal surrounding network will connect all
user’s ter- minal equipment wirelessly so that the user does not have to wear
any uncomfortable and limiting cables tied around his body. In the beginning of
the project we shall study applications based on augmented virtual reality like
new virtual three dimensional interfaces of common household devices. In
Section 2 we describe a system architecture of our mobile VR system. In
addition, we present the idea of personal surrounding networks and personal
mobile terminals capable of routing traffic. Section 3 contains the description
of the protocol architecture of the sys- tem based on mobileIP, IP multicast
and streaming protocols. Real-time requirements and some solutions satisfying
the requirements are discussed in Section 4. Finally in Section 5 we describe some
possible applica- tions and the outlines of the future study and in Section 6
we give some concluding remarks.
The mobile VR system can
be divided into virtual re- ality and communication subsystems. Virtual reality
subsystem is responsible for generating feasible aug- mented VR services for
users. The picocellular per- sonal surrounding network (PSN) implements the
com- munication subsystem, which is responsible of mobile connections
management between users and their per- sonal mobile terminals (PMT). A global mobile network consists of several
national mobile networks, which in turn contain several macro networks. For
example a cell created by a GSM base station can be considered as a macro
network. A macro network can contain many smaller micro networks like a
wireless LAN inside a building. Each micro network may contain several
picocells, the diameter of which is typically some ten meters. A hierarchy of
different mobile
Figure 1. Hierarchy of different mobile
net- works.
The reason for making cells smaller and
smaller is obvi- ous. The smaller the cell size, the higher the through- put,
because there are fewer users in each cell and higher transmission frequencies
can be used. Usually high frequencies are not used, because of a quick signal
attenuation, but if the transmission range is just a few meters, the effect of
attenuation is almost negligible. In addition, smaller cell size enables
greater frequency reuse. For all of these reasons we have decided to con- struct
very small cells, which we call picocellular per- sonal surrounding networks.
The diameter of a PSN is some two meters, which enables the construction of
small very low-powered hand held terminals still capa- ble of transmitting high
bandwidth data. Systems con- sisting of very small cells have one serious
drawback; the number of base stations needed in the system is enormous. In our
system adjacent PSNs can change information directly without a base station.
Trans- mitters of user’s terminals are thus very low-powered making it possible
to build smaller user terminals. In addition, our system enables the
communication be- tween users, although there is no base station nearby.
Mobility
management in picocellular networks is a very demanding task because of
frequent handovers and dy- namic changes of mobile links between PSNs. In [4] a
three layer network structure is given to avoid the problem of connection
breaks during handovers. A cor- responding structure adapted to our mobile VR
system is depicted in Fig. 2. At the bottom level the structure consists of
mobile hosts (MH), i.e, mobile users carry- ing VR equipment, terminals and
transmitters. Each fixed room-sized cell has typically one mobile support
station (MSS), which manages wireless connections to MHs. MSSs are wired to
server hosts (SH), which han- dle connections between MHs in different cells
and be- tween MHs and nodes in a wired network. Our system contains also
wireless MSSs, i.e., PSNs, which are or- dinary MHs capable of relaying packets
between each other. Each SH maintains a table indicating all MHs inside the
cells, the SH manages. So when a MH moves from one cell to another, the SH can
inform the involv- ing MSSs beforehand to make the handover procedure as
transparent to the user as possible. For example if in Fig. 2 MH1 is moving
from cell MSS2 to cell MSS1, the SH can inform MSSl in advance that a new user
is going to enter the cell. In conventional mobile net- works this may be
difficult, because the exact position of a user in a cell is not known.
However, in a VR envi- ronment user’s position must be determined precisely all
the time, so it is easy to determine the exact time when the handover should
happen.
In
[4] a transparent handover is provided by multicas- ting packets addressed to
some MH to those MSSs, in which the MH is located or into which the MH is going
to move. Multicast groups is a well-known so- lution to a problem of scaling
very large distributed systems containing interacting objects like networked
virtual reality environments. The objects must know how other objects move or
how their appearance or size changes, to be able to smoothly co-operate with
them. For example a DIS [13] simulation may contain 10,000 simultaneous users.
To send the state of each user to every other user using point-to-point
connections be- tween users would require an enormous network band- width. To
reduce this bandwidth requiremen
Figure 2. Distributed mobile virtual
reality system.
and other objects are divided into classes
using for ex- ample their spatial, temporal or functional relations. Each class
forms a multicast group and an object in a group can send a message to all
other objects in the same group by using only one multicast address. This method
is successfully used in many distributed envi- ronments for example in DIS [8].
MSSs as a multicast group In our system a
SH maintains a table, which contains all MHs which are in some cell the SH
manages. Note that there may be several MHs in the system such that none of
them is located in a cell managed by a SH. In that case the MHs can communicate
with each other like MH3 and MH4 in Fig. 2, but of course connections to
services provided by a fixed network, are not available. The SH is connected to
a server performing location manage- ment. The location manager determines the
location of each MH inside cells managed by the SH from the tracking sensor
data sent by the MHs. The manager informs the SH how MHs are moving. According
this information a SH generates a multicast group for each MH. The multicast
group changes all the time accord- ing to user’s movements.
Multicast versus packet forwards All the
data packets addressed to a MH are not only transmitted to a MSS, where the MH
is located but also to a MSS, towards which the MH is going. The importance of
ac- curate position tracking is very significant, if we want to reduce the
number of unnecessary packet forwards and stores. If the SH does not know
exactly, where the MH is moving, it may unnecessarily transmit informa- tion to
some MSS, so the storage space in MSSs must be quite large. However, if a
multicast group is not used, an old MSS must store the packets addressed to a
MH that left the cell, until the SH informs the new cell, where a MH is located.
After the MSS knows the new location, it forwards the packets to a new MSS.
Thus, the same packets must be transmitted several times through a network.
Assume further that a MH is a person walking along a corridor and thus cross-
ing cell boundaries frequently. When forwarded pack- ets arrive to a new MSS,
the MH has possibly already moved into another cell and the packets must be
for- warded to a third MSS. If packets are multicasted to several MSSs, an old
MSS can delete all the packets designated to a MH that has left the cell. In
[4] it is calculated that in a building consisting of 200 rooms, if
approximately two users crosses the cell boundary per second and the available
bandwidth to each user is 2 Mbit/s and each user’s load is 0.9, the network must
forward packets at the speed of 48-96 Mbit/s.
In most papers concerning mobile
communicating only mobility of users is discussed. In addition to users,
services should be mobile too. Consider a situation, in which a user is
executing a VR application or service implemented only in one cell. Because it
is very likely that the user is going to change the cell quite frequently, the
service disconnects as soon as the user gets used to it. We are going to use
mobile agents (71 to make services mobile too. So in addition to multicast
groups, the SH should take care of the sending of mobile agents, providing the
same services in the new cell as the user has applied in the previous cell.
The virtual reality subsystem consists of VR
input and output devices and an advanced position tracking sys- tem. As an
output device we use a see-through high resolution head mounted display (HMD).
The field of view is not so important as in immersive VR appli- cations,
because the user can always turn his head to see augmented objects outside his
current field of view. AS an input device we are going to use a 3D mouse or a
pen. The drawback of commercial 3D mouses is the position tracking. They are
usually tracked us- ing a magnetic field, the range which is very limited. In
the future instead of a mouse we shall use a data glove, which uses an accurate
position tracking system. Precise position tracking systems have been developed
at VTT Automaton based on acceleration sensors and global positioning system
and acoustic methods. Us- ing acoustic or electromagnetic methods objects can
be tracked very precisely, but the tracking system needs a lot of
infrastructure in all the rooms, where a user can move. On the contrary we
would like to develop a tracking system that the user can easily carry with
him. We are now developing a head tracking system, which uses a gyroscope and
acceleration sensors and which can calibrate itself very precisely using
machine vision.
In the first prototype implementation of
our system different user’s peripherals are connected to each other through
wires. In the future the PSN will take care of wireless connections between
user’s equipment, dif- ferent users and a fixed network. The components of
user’s mobile equipment consisting of personal mobile terminals and a MSS are
depicted in Fig. 3
The protocol architecture
of our virtual reality system is given in Fig. 4. Note that MHs may have the
same protocol layers as MSSs, because user’s PSN contains both the
functionality of a HM and a MSS. In the be- ginning we will use Internet based
protocols for several ing possibly inexpensive off-the-thelf components.
Figure 3. Picocellular personal surrounding
network connecting the components of VR subsystem.
reasons. Firstly, we can directly connect
our system to existing distributed VR environments like DIVE just implementing
a wireless DIVE node. Secondly, a lot of useful protocol layers like mobileIP,
multicast IP and streaming protocols have already been implemented in Internet.
In addition, a lot of virtual reality applica- tions implemented e.g. with virtual
reality modelling language (VRML) can be executed through Internet. These
applications provide a broad test bench for our mobile VR system. Thirdly,
Internet suffers from sim- ilar undeterministic and load dependable
transmission delays, which will occur in our system, in case packets are be
routed through several users’ PSNs. The meth- ods guaranteeing certain quality
of service or enabling the transmission of continuous data through Internet are
thus worth researching in our project too.
It is not decided yet, how the physical
layer of the ra- dio interface will be implemented and which medium access
method will be used (time division multiple ac- cess, code division multiple
access, combination). In the beginning we shall use ordinary mobile phones or communicators,
broadband wireless local networks and cordless telephone systems to provide
wireless ser- vices. Later we are going to implement our own wire- less
spread-spectrum based communicatio
Figure 4. Protocol architecture for the
mobile virtual reality.
In order to keep personal mobile terminals as
simple as possible, they should not have much storage space or computation
capacity. Data routed by MHs may be buffered, if the destination MH is
temporarily un- reachable, but has not left its position from the routing
chain. A MH leaving the chain notifies immediately its downlink MH, if such
exists and uplink MH or MSS, so that they can create an other routing chain, if
there is enough MHs in the area. Data addressed to such a MH, which has left
the routing chain is deleted with- out buffering, because it will be delivered
to the right MH by multicasting it to a new cell or a new routing chain. To
decrease the amount of computation power needed by a MH all processes producing
information for it are distributed across the network and executed in different
network nodes. For example one server node can execute haptic rendering,
another may be a visual renderer and the third can generate 3D audio data, as
depicted in Fig. 5. One of the most important pro- cesses is a position
tracking manager, which determines the locations of the MHs from the data it
periodically receives from them. One MH can contain several posi- tion sensors,
because in addition to user’s head it may be desirable to locate his hands and
legs or the whole body too. In addition to a SH, position information is used
by a collision manager, which is again executed in an own network node.
Renderer processes use both collision and position information to give a user a
real- world-like visual and haptic feedback of virtual objects.
Finally there is an object manager
responsible of dis- tribution of virtual objects on demand. mobility, HMD,
manager, gloves position manager manager access Collision object manager are
still delayed due to irrecoverable transmission er- rors, the user’s rendering
processes use dead reckoning algorithms to extrapolate the state changes of the
user. The situation is more problematic, when a MH and a SH do not have a
direct communication link, but there may be several relaying MHs between them.
Then it would be feasible to change position tracking or any other information
directly between interacting users and allow the user to process the
computation instead of distributing all the computation in a fixed network. The
more undeterministic transmission delays the wire- less link has the more
computatiop power a user should have to process the information locally. In the
areas, where there are a lot of users, a small group of adjacent users can
configurate into a form of a ring. With a suc- cessful frequency reuse pattern
each user can send data at the same time to its neighbour user. Transmission
delay around the whole ring depends on the number of users in the ring, but is
deterministic enabling real-time co-operation between the users.
Figure 5. Distribution of system processes.
We are
going to implement a distribution platform, which is a common interface for all
applications. Dis- tribution platform enables users and network nodes to
execute their processes transparently around the net- work so that they do not
know where their processes are executed physically. In addition, the platform
en- ables the split of heavily loaded network processes. If for example the
load of a SH or a MSS increases over a pre-determined threshold, a SH can
allocate some idle network node and create a new SH, which will take care of
half of the connections, previously managed by one SH.
The most time critical data of our system is
position tracking information. If this information is delayed or lost,
collisions may remain undetected or user’s aug- mented view may contain
incomplete information. The data link between a user and a SH is created and
all the transmission characteristics of the link are requested on demand so
that the successfully created data link guar- antees some pre determined
quality of service as long as the link exists. So the transmission delays
between the fixed network and a mobile user are known and deter- ministic. If
some position tracking information packets
Using a lot of infrastructure, i.e., a
separate position tracking system in each fixed cell an accurate real-time
position tracking system can be easily constructed. For example in [2] a
position tracking method is presented, which allows the usage of multiple
transmitter-receiver pairs and position accuracies in the millimeter range. The
method is based on the code division multiple ac- cess (CDMA) and precision
ranging characteristics of spread-spectrum communication technology. The re-
ceiver must know the exact times the spread-spectrum signal was transmitted and
received in order to deter- mine the amount of time the radio signal took to
travel i.e., the position of the receiver with respect to the transmitter. The
timing information is embedded in the received signal. The disadvantage of this
is that both the transmitter and the receiver must have ac- curate synchronized
clocks. For example, if the timing error is off by 10 nanoseconds, the range
error would be three meters. More accurate precision is accomplished by
detecting the phase difference in the code sequences of the transmitter and the
receiver. Four transmitters are placed in the virtual environment corresponding
the x-, y-, z- and time-axis. All four transmit using the same spread-spectrum
frequency band, but each trans- mitter has its own unique code sequences so
mutual in- terference is avoided. As many as receivers as needed may be
positioned on the body of a person. Each re- ceiver receives the transmitted
signal from all four sta- tionary transmitters and calculates position with respect to the room
origin. The position information of all the receivers is transmitted to a
location manager, which informs a SH, a collision manager, and render- ing
processes about the changes in position. In this way the SH can prepare for the
handover procedure early enough so that when a user enters a new cell, a con-
nection and other necessary resources are immediately allocated for him.
Consistency is a
vital aspect in the design of distributed and networked environments and even
more important, when virtual reality is involved. Consistency means that all
objects in a virtual world know the states and state changes of other objects
with which they may interact. Our system uses two different synchroniza- tion
schemes. We use the timing information of po- sition tracking receivers for
intra-cell synchronization. It gives accurate enough system clock for objects
in- side a cell. Another scheme is used for synchronizing distributed processes
in a fixed network and for inter- cell synchronization. Position tracking
messages sent periodically by a MH contain a time stamp. A MSS polls every MH
in a cell in turn and synchronizes to the tracking information flow it
receives. The same information flow is used to synchronize a SH and dis-
tributed processes. Normally inter-cell synchronization is not used, because
users in different cells cannot in- teract with each other. If inter-cell
synchronization is needed, a SH performs the synchronization by using the time
stamps of position tracking messages received from the MSSs of the cells.
The
implementation of an experimental system of a distributed mobile virtual
reality continues in the com- puter engineering laboratory of university of
Oulu and VTT Electronics. In addition to the implementation work, we start
examining two important research ar- eas, which have remained untouched so far,
namely the fault tolerance and security. The picocellular personal surrounding network
could be used more generally than in just virtual reality applications. For
example in a department store the PSN could communicate with a cashier so that
the network would take care of all the functions, in which we earlier needed a
credit card. The exchange of credit information wirelessly requires reli- able
data security and fault tolerant operations.
At present we are designing new
applications, where a mobile virtual reality could be used. In Fig. 6 one ex-
ample is given, where user’s senses are augmented by increasing the amount of
visual information received by the user from his environment. The user can e.g.
see the outside temperature by just focusing his eyes to the window or the
temperature of a coffee mug by just looking at the mug. The user can have very
simple interfaces to household equipment. For example above a television there
could be just one button, where the user can switch the television on after
which he can control the volume and change channels by command- ing the
television with his voice. Furthermore, the user could have very complicated
interfaces like a virtual interface to a portable computer with 3D windowing
environment.
Figure 6. Mobile augmented virtual reality
system.
We have presented a system and protocol architec- ture for a
picocell-based real-time mobile virtual re- ality system. The system is based
on picocellular per- sonal surrounding networks having very low possibly human
powered terminals yet capable of transmitting high bandwidth multimedia and
virtual reality data. Picocellular networks allow users a real ability to move
freely and in a natural way in room-sized cells inside a building. In addition,
personal surrounding networks can communicate directly without any fixed base sta-
tions, so two users locating near each other can apply the system virtually
anywhere. The protocol architec- ture of the system is based on IF protocols
allowing easily co-operation between a mobile user and an im- movable user of a
fixed network. First applications will be based on augmented reality and
include virtual in- terfaces to common household equipment and to new
telecommunication devices.
[l] P. Agrawal, E. Hyden, P. Krzyzanowski, P. Mishra, M. Srivastava,
and J. Trotter. SWAN: A Mobile Mul- timedia Wireless Network. IEEE Personal
Communi- cations, 3(2):18-33, April 1996. 121 S. Bible, M. Zyda, and D.
Brutzman. Using Spread- Spectrum Ranging Techniques for Position Tracking in a
Virtual Environment. In Proceedings of the 1995 Workshop on Networked
Realities, Boston, MA, Octo- ber 26-28, 1995, 1995. 131 C. Carlsson and 0.
Hagsand. DIVE-a Multi-User Virtual Reality System. In Proceedings of the IEEE
Virtual Reality Annual Symposium (VRAIS'93), September 18-22, Seattle,
Washington, pages 394-400, 1993. 141 R. Ghai and S. Singh. An Architecture and
Com- munication Protocol for Picocellular Networks. IEEE Personal
Communications, 1(3):36-46, 1994. [5] http://isw3.aist-nara.ac.jp/IS/
Chihara-lab /. Mobile Virtual Reality. WWW-site. [6] http://www.division.com.
dVS/dVISE. WWW-site. [7] G. Liu and G. M. Jr. A Virtual Distributed System
Architecture for Supporting Global-distributed Mobile Computing. Technical
Report TRITA-IT R 95-01, Royal Institute of Technology, 1994. [8] M. Macedonia,
M. Zyda, D. Pratt, D. Brutzman, and P. Barham. Exploiting Reality with
Multicast Groups: A Network Architecture for Large-scale Virtual Envi-
ronments. In Proceedings of the Virtual Reality Annual International Symposium
'95, Research Triangle Park, North California, March 11-15, 1995, Los
Alamistos, California, 1995. IEEE Computer Society Press. [9] C. Shaw, M.
Green, J. Liand, and Y. Sun. De- coupled Simulation in Virtual Reality with The
MR Toolkit. ACM Transactions on Information Systems,
[lo] D. Snowdon and A. West. The AVIARY
VR-system. A Prototype Implementation. In Proceedings of the 6th ECRIM
Workshop, Stockholm, Sweden June, 1994, 1994. [11] D. Snowdon, A. West, and T.
Howard. Towards the Next Generation of Human-Computer Interface. In Proceedings
of the Informatique '93: Interface to Real & Virtual Worlds, Montpellier,
France, March 24-26, 1993, pages 399-408, 1993. 1121 T. Starner. Human Powered
Wearable Computing. Submitted for publication in the IBM Systems Journal, page
16, 1996. 11(3):287-317, 1993.
No comments:
Post a Comment