Experimental Study of Voice Over IP Services Over
Experimental Study of Voice Over IP Services Over
Experimental Study of Voice Over IP Services Over
2. RELATED WORK
2.1 IEEE802.11 Performance Studies
Li, et al. [6] examined the interactions of the IEEE802.11
Medium Access Control (MAC) protocol and ad hoc forwarding
and the effect on capacity for several simple configurations and
traffic patterns. By measuring the throughputs of different packet
sizes, they showed that traffic pattern determines whether an ad
hoc network’s per node capacity will scale to large networks.
They discovered that IEEE802.11 is more efficient for orderly
local traffic patterns, such as a lattice network with only
horizontal flows. They also argued that the locality of traffic
determines whether large ad hoc networks are feasible. The less
local the traffic pattern is, the faster per node capacity degrades
with network size.
Figure 1. Overall Physical Hybrid Network Deployment.
2.2 IEEE802.16 Performance Studies
Hoymann [7] performed an analytical performance evaluation of
the IEEE802.16 standard by taking overall system performance 3. NETWORK DEPLOYMENT
measurements. Simulation results obtained from a software-based The overall physical hybrid network deployment of the
simulator were also compared with the numerical results obtained experimental setup, shown in Figure 1 above, consists of one
from an analytical model implemented in MATLAB. It has been 5.8GHz WiMAX network and two 2.4GHz WiFi networks with
shown that the IEEE 802.16 MAC overhead can be assumed to be all links operating at 54Mbps. L1 and L2 are laptops used to
10% and this can be reduced by optional fragmentation to fill up initiate the tests and collect experimental data for analysis. R1 and
the MAC frame optimally. Next, the simulation results showed R2 are IEEE802.11g routers used for setting up the two separate
maximum MAC throughput and packet delay values close to that WiFi networks. These WiFi networks represent the WLANs and
obtained by theoretical analysis. LANs within buildings. Next, SS1 and SS2 are the IEEE802.16
SSes which are linked to the IEEE802.16 BS in a Point-to-Multi-
2.3 IEEE802.11-IEEE802.16 Interoperability Point (PMP) mode to form a WiMAX network. This WiMAX
network represents the wireless backbone infrastructure. Next, R1
Berlemann, et al. [8] realized the coexistence and interworking of
and R2 are linked to SS1 and SS2 respectively using Ethernet
IEEE 802.16 and IEEE 802.11e. A central coordinating device
(UTP) cables, establishing the connections between the WiMAX
called Base Station Hybrid Coordinator (BSHC) supporting both
network and WiFi networks to create a hybrid network. The
the IEEE802.16 and IEEE802.11e protocols that operate in the
WiMAX network and WiFi networks are also tested separately to
same frequency band was implemented. The interworking of the
get additional results for comparison with that of the hybrid
two protocols is done by inserting the IEEE802.11 transmission
network. These results give a better understanding of how each of
sequences into the MAC frame structure of IEEE802.16. For a
the two networks performs individually and helps to diagnose any
transmission of a 512-byte data packet, the IEEE802.11e MAC
problems encountered by the hybrid network.
protocol has a lower efficiency than the IEEE802.16 protocol
which resulted in longer transmission time. It was also shown that
the overall system capacity based on the relay link throughput 4. VoIP CALL QUALITY ASSESSMENT
decreases as the number of QSTAs (QoS supporting 802.11e While a voice call involves bidirectional data flow, we first study
Stations) increases. a connection as a unidirectional flow that lasts for five minutes,
which is the length of a typical voice call [10]. The ITU-T
2.4 Multimedia Performance Studies specified E-Model [11] is adopted as the means for assessing the
Sun, et al. [9] evaluated the performance of video and voice VoIP call quality (cf: Section 4.1.) The E-model is applied to two
traffic through multi-hop wireless paths and studied the capacity voice codecs, viz., G.711 and G.729, to study their performance
of the mesh network using IEEE802.11b. They set up a UCSB in the hybrid network scenario. In the following, we addressed
MeshNet testbed and selected five nodes to conduct the how the various performance metrics should be maintained so that
performance study. The multimedia performance metrics used are voice quality is ensured.
packet latency, packet loss rate, inter-flow fairness and packet
4.1 The E-Model
jitter. They concluded that the number of hops in the transmission
The E-Model [11] is a computational model that uses the
path restrict the capacity of the network. Data rate and packet size
combined effects of several transmission parameters to predict the
do not influence the number of flows supported by the network. It
835
subjective quality of a voice call. A transmission rating factor R is 5. PERFORMANCE TESTS PROCEDURES
calculated by combining all the transmission parameters relevant This section discusses the detailed procedures for both network
for a connection, which can then be used to predict subjective performance and VoIP performance tests. The parameters
user reactions. The factor R is made up of the approximate sum of involved and details on how each performance metric is measured
the impairments caused by these transmission parameters. The and obtained are also given.
mathematical representation of R is shown in the following
formula:
5.1 Network Performance Tests
R = Ro – Is – Id – Ie + A (1) The two network performance tests conducted are: ‘ping’ test
The basic signal-to-noise (SNR) ratio Ro takes into account the using the Fast Pinger tool [16] and UDP throughput test using the
effects of background noise and circuit noise. The simultaneous Iperf tool [17].
impairment factor Is is the sum of all impairments which may
occur more or less simultaneously with the voice transmission. 5.1.1 Ping Test
The delay impairment factor Id consists of all impairments due to The goal of using the Fast Pinger tool is to emulate the behaviour
delay of voice signals such as mouth-to-ear delay [12]. The of a specific codec to be tested. First, the payload size of the ping
equipment impairment factor Ie encompasses the distortion packet has to be determined. From Figure 2, it can be seen that in
impairment caused by low bit-rate codecs and packet loss [13]. addition to the ping payload size matching the voice payload size,
The advantage factor A measures the amount of decrease in the another 12 bytes is needed to represent the RTP header.
factor R that a user is willing to tolerate when using a given
technology over traditional wired telephony. For the purpose of
comparing with PSTN calls, the factor A is set to 0 [11]. The
value of R lies in the range 0 to 100, where R=0 represents an
extremely bad quality and R=100 represents a very high quality.
A traditional PSTN call has a minimum R-value of 70 and this
value serves as the lower limit for a VoIP call to be satisfactory Figure 2. Determination of Packet Size for Ping Test
and acceptable.
Each ping session is conducted for one Round Trip Time (RTT)
4.2 G.711 Codec measurement between the source and the destination. The ping
session runs for five minutes at an interval rate stated in Table 1
G.711 is the conventional codec used by a PSTN call. It uses
for a specific payload size of the codec. (The default parameter
Pulse Code Modulation (PCM) and operates at a high bit-rate of
values for the codecs are highlighted.) Three ping sessions are
64Kbps with a typical voice payload size of 160 bytes sent out at
conducted for each codec at three different payload sizes and the
intervals of 20 ms. The quantization distortion unit (qdu) that
corresponding interval timings. The average RTT measured for
arises from PCM in G.711 causes impairments and this brings the
each session is recorded and halved to obtain the end-to-end delay
R-value of G.711 down to its intrinsic R-value of 93.2, the highest
for that session.
among all other codecs [13]. This is different from the G.711
intrinsic R-value of 94.3 previously reported [14][15]. This is due Table 1. Parameters of G.711 Codec and G.729 Codec
to the revision of the ITU-T Recommendation G.107 in 2000 G.711 G.729
which resulted in the change of this intrinsic R-value for G.711. Bit-rate Payload Interval Bit-rate Payload Interval
(Kbps) (bytes) (ms) (Kbps) (bytes) (ms)
Thus, G.711 has an R-value budget of 23.2 for impairments
before the call quality drops below 70 and becomes unacceptable. 160 + 12 20 20 + 12 20
From Table I.3/G.113 in [13], G.711 has a tolerance of up to 64 240 + 12 30 8 40 +12 40
0.928% packet loss, which equates to an equipment impairment
320 + 12 40 60 + 12 60
factor Ie of 23.2.
836
5.2 VoIP Performance Test End-to-end Delay (ms) against G.729 Payload (bytes)
The rtpplay tool in the set of RTP Tools [18] is used to run VoIP
calls from RTP data files generated from recorded voice 12
conversations for both G.711 and G.729 codecs. This rtpplay tool
10
is run from the laptop to initiate the VoIP call. The laptop at the
conducted. The first run starts off with two simultaneous VoIP 4
calls and the simultaneous VoIP calls increase by two for each 2
subsequent run. As a result, the last run ends with 20
simultaneous VoIP calls. Both the RTP end-to-end delay and RTP 0
160 240 320
packet loss are recorded for each run. The R-value for each run is
G.729 Payload (bytes)
then calculated and determined if each VoIP call is valid and
acceptable. Figure 4. End-to-end Delay of G.729 using Ping
10 10
Hybrid
8 9
WiMAX
8
UDP Throughput (Mbps)
6 Wi-Fi
7
4 6 Hybrid
5 WiMAX
2
4 Wi-Fi
0 3
160 240 320 2
G.711 Payload (bytes) 1
0
Figure 3. End-to-end Delay of G.711 using Ping 5 10 15 20 25 30
UDP Input Load (Mbps)
837
Packet Loss (%) against UDP Input Load (Mbps) 7.2 R-value Results
The R-value of the calls using the G.711 codec and G.729 codec
80
are shown in Figure 8 and Figure 9 respectively. From Figure 8,
70 we observe a 41% overall decrease in the R-value for calls using
60 the G.711 codec from 88.7 at two simultaneous calls down to 52.2
Packet Loss (%)
50
Hybrid
at 20 simultaneous calls. It can also be observed that the R-value
40 WiMAX for these calls drops below the acceptable lower limit of 70 as the
30
Wi-Fi number of simultaneous calls increase beyond 12. This indicates
20
that the hybrid network can only support up to 12 simultaneous
calls using G.711 codec.
10
The packet losses of simultaneous VoIP calls for both G.711 and 80
G.729 codecs are shown in Figure 7. The packet loss is calculated Acceptable
70
R-Value
by averaging all the packet losses of the individual calls within Unacceptable
each set of simultaneous-calls test run. It can be observed that 60
G.729 codec is more robust to packet loss than the G.711 codec. 50
The increase in packet loss for G.729 calls is gradual at the start 40
and remains almost constant from 6 to 12 simultaneous calls.
30
Beyond that, the packet loss increases at a significantly faster rate. 2 4 6 8 10 12 14 16 18 20
This packet loss robustness is due to the in-built Packet Loss No. of G.711 Calls
1.40
85
Packet Loss (%)
1.20
1.00
80
0.80
0.60 75
0.40 Acceptable
R-Value
0.20 70
0.00
Unacceptable
65
2 4 6 8 10 12 14 16 18 20
No. of Calls 60
G.711 G.729 55
2 4 6 8 10 12 14 16 18 20
Figure 7. Packet Loss of Simultaneous VoIP Calls No. of G.729 Calls
Scheme (PLC) scheme in the G.729 codec [21]. Figure 9. R-value of G.729 Calls
As for the calls using the G.711 codec, the packet loss is about From the VoIP performance test results, it is clear that the hybrid
15% to 110% more than those using the G.729 codec. The 110% network can support more calls using the G.729 codec than with
gap occurs at the point when there are 14 simultaneous calls; the the G.711 codec. Specifically, the hybrid network can support up
packet loss for the calls using the G.711 codec is doubled from to 12 simultaneous VoIP calls using the G.711 codec and 20
case of 12 simultaneous calls. This is the threshold point which (possibly more) simultaneous VoIP calls when the G.729 is used.
indicates that the hybrid network is unable to support more than
12 simultaneous calls when the G.711 codec is used.
838
8. CONCLUSION AND FUTURE WORK [7] C. Hoymann, “Analysis and Performance Evaluation of the
In this paper, we present an experimental study in which a hybrid OFDM-based Metropolitan Area Network IEEE 802.16,”
network consisting of one WiMAX and two WiFi links is set up Computer Networks, vol. 49, no. 3, pp. 341-363, Oct 2005.
and used as a testbed for VoIP performance tests. The WiMAX [8] L. Berlemann, et al., “Coexistence and Interworking of IEEE
and Wi-Fi networks are also studied independently of the hybrid 802.16 and IEEE 802.11(e),” Proceedings of the IEEE 63rd
configuration, and results from tests conducted on these two Vehicular Technology Conference, Melbourne, Australia,
networks are then compared with that of the hybrid network. It is May 7–10, 2006.
found that the two networks making up the hybrid network are
[9] Y. Sun, et al., “An Experimental Study of Multimedia
able to connect seamlessly with minimal or no additional delays
Traffic Performance in Mesh Networks,” Proceedings of the
to the transmission of packets through the hybrid network.
International Conference on Mobile Systems, Applications
Next, VoIP performance tests are conducted over this hybrid and Services, Seattle, Washington, USA, Jun 6 – 8, 2005.
network with the aim of determining how many simultaneous [10] A. Hafslund, T. T. Hoang, and O. Kure, “Push-to-talk
VoIP calls the hybrid network can support using the G.711 and Applications in Mobile Ad Hoc Networks,” Proceedings of
G.729 codecs. The ITU-T specified E-Model is used to assess the IEEE 61st Vehicular Technology Conference,
whether a VoIP call is valid and acceptable using two Stockholm, Sweden, May 30 – Jun 1, 2005.
performance metrics, using end-to-end delay and packet loss of a
voice call. It is then found that the hybrid network can support up [11] ITU-T Recommendation G.107, “The E-model, a
to 12 simultaneous VoIP calls using the G.711 codec and more Computational Model for Use in Transmission Planning,”
than 20 simultaneous VoIP calls using the G.729 codec. Since no Mar 2005.
effort is made to add in adaptation layer at the two protocol stacks [12] ITU-T Recommendation G.114, “One-way Transmission
to further improve the interoperability between WiMAX and Time,” May 2003.
WiFi, the network utilization is rather low. Nevertheless, the [13] ITU-T Recommendation G.113, “Transmission Impairments
procedure spelt out to determine the maximum number of VoIP due to Speech Processing,” Feb 2001.
calls which meet the QoS requirements will be useful and
valuable to service providers who want to perform such [14] J. Janssen, D. De Vleeschauwer, and G. H. Petit, “Delay and
measurement. This objective assessment method used adds value Distortion Bounds for Packetized Voice Calls of Traditional
and justification to the VoIP call results, as compared to other PSTN Quality,” Proceedings of the 1st IP-Telephony
subjective methods like the Mean Opinion Score. Workshop, Berlin, Germany, Apr 12–13, 2000.
A possible extension to this study is to find the capacity of dual- [15] A. P. Markopoulou, F. A. Tobagi, and M. J. Karam,
directional VoIP calls over this hybrid network topology. Another “Assessment of VoIP Quality over Internet Backbones,”
alternative is to deploy the testbed over a bigger area to test the Proceedings of the 21st Annual Joint Conference of the IEEE
hybrid network over longer distances, utilizing the long range of Computer and Communications Societies (INFOCOM), New
WiMAX and minimizing the interference between nodes. York, USA, Jun 23–27, 2002.
[16] Kwakkelflap, Fast Pinger by Wouter Dhondt,
http://www.kwakkelflap.com/fping.html
9. REFERENCES
[1] ANSI/IEEE Std 802.11-1999, “IEEE Standard for [17] NLANR/DAST, Iperf 2.02, 2005,
Information Technology – Part 11: Wireless LAN Medium http://dast.nlanr.net/Projects/Iperf/
Access Control (MAC) and Physical Layer (PHY) [18] H. Schulzrinne, RTP Tools Version 1.18
Specifications,” Mar 1999. http://www.cs.columbia.edu/IRT/software/rtptools/
[2] IEEE Std 802.11g-2003, “IEEE Standard for Information [19] Ethereal – A Network Protocol Analyser,
Technology – Part 11: Wireless LAN Medium Access http://www.ethereal.com/
Control (MAC) and Physical Layer (PHY) Specifications –
Amendment 4: Further Higher Data Rate Extension in the [20] A. L. Wijesinha, et al., “Throughput Measurement for UDP
2.4 GHz Band,” Jun 2003. Traffic in an IEEE 802.11g WLAN,” Proceedings of the 6th
International Conference on Software Engineering, Artificial
[3] IEEE Std 802.16-2004, “IEEE Standard for Local and Intelligence, Networking and Parallel/Distributed Computing
Metropolitan Area Networks – Part 16: Air Interface for and 1st ACIS International Workshop on Self-Assembling
Fixed Broadband Wireless Access Systems,” Oct 2004. Wireless Networks, Towson, MD, USA, May 23–25, 2005.
[4] IEEE Std 802.16.2-2004, “IEEE Recommended Practice for [21] J. Janssen, et al., “Delay Bounds for Voice over IP Calls
Local and Metropolitan Area Networks – Coexistence of Transported over Satellite Access Networks,” Mobile
Fixed Broadband Wireless Access Systems,” Mar 2004. Networks and Applications, vol. 7, no. 1, pp. 79-89, Jan
[5] Proxim Wireless Corporation, “The Broadband Wireless 2002.
Access Market,” White Paper – Broadband Wireless, 2005.
[6] J. Li, et al., “Capacity of Ad Hoc Wireless Networks,”
Proceedings of the 7th Annual International Conference on
Mobile Computing and Networking, pp. 61-69, Rome, Italy,
Jul 2001.
839