Spirent TestCenter IPTV V3a
Spirent TestCenter IPTV V3a
Spirent TestCenter IPTV V3a
IPTV Testing
www.spirentcampus.com
Topic Overview
IPTV Overview
IPTV Wizard
Client Settings
Command Sequencer
Results
2
Spirent TestCenter IPTV Testing
IPTV Overview
3
Spirent TestCenter IPTV Testing
Typical IPTV Architecture
4
Spirent TestCenter IPTV Testing
Impact Analysis
With HDTV, every channel could be up to 8 Mbps
A neighborhood of 200 STBs could receive 1.6 Gbps of
traffic
People change channels frequently to see what else is on
TV
Questions to consider:
Did the channel change fast enough for good quality of experience
(QoE)?
Will the users who had poor QoE switch channels again trying to get
a good channel causing additional channel overlap?
How fast did the traffic stop? Was there any overlap of channels?
How much extra traffic was on the subscriber port?
How good is the Video Quality compared to Cable or Satellite?
5
Spirent TestCenter IPTV Testing
Avoiding the Death Spiral
6
Spirent TestCenter IPTV Testing
Impact of Packet Loss (0%)
0% Packet Loss
7
Spirent TestCenter IPTV Testing
Impact of Packet Loss (0.5%)
8
Spirent TestCenter IPTV Testing
Impact of Packet Loss (5%)
5% Packet Loss
9
Spirent TestCenter IPTV Testing
Testing IPTV
10
Spirent TestCenter IPTV Testing
Infrastructure Testing
Bandwidth
Can the infrastructure handle hundreds of channels?
Channel zapping
How fast can a user change channels?
Channel validation
Is the correct information being received?
Benchmarking
How do real servers compare to the Lab environment?
Triple-Play
What is the effect on Voice and Data?
11
Spirent TestCenter IPTV Testing
Test Modes
Channel Zapping
Used to monitor zapping characteristics for a large number of clients and
groups (virtually unlimited)
Only verifies that a channel was changed successfully
Only verifies using the last two bits of the IP multicast address
Channel Validation
Used to monitor zapping characteristics for a limited number of clients and
groups (32,768 total when multiplying the two).
Fully verifies that a channel change was successful using all 32 bits of the
multicast IP address
12
Spirent TestCenter IPTV Testing
Key Statistics
Last Mcast Pkt
Join Latency Arrives
Leave
Time between sending IGMP join and receiving Latency
multicast data for the channel specified in the join
message.
Leave Latency
Leave
Time between sending the IGMP leave for a channel Ch. 2
and when multicast data stopped. Channel
Overlap
Channel Change Latency
The time from the sending of the IGMP leave for one
channel to the receipt of the first “new” multicast Channel 2 Leave Channel
packet after sending an IGMP join for another channel. Ch. 3 Gap
Channel Gap
The time between the last multicast packet received
for the multicast stream for which a leave was sent
v
and the receipt of the first multicast packet of the Channel 3
newly joined multicast stream.
Channel Overlap
The time between the last multicast packet received Channel 4
for the multicast stream for which a leave was sent Join
Join Latency Ch. 3
and the receipt of the first multicast packet of the
newly joined multicast stream.
NOTE: The difference between Channel Gap and Channel
Overlap is whether or not the first multicast stream is Join
Channel
still being received when the newly joined multicast Change Ch. 4
stream arrives. Latency
13
Spirent TestCenter IPTV Testing
Key Parameters
View Duration Last Mcast Pkt
Simulates viewing a channel for an extended Arrives for Ch. 2
period. If view duration is set 0, then the test is a
pure zapping test. Leave Join Leave
2 3 3
Leave-Join Delay
Delay between sending the IGMP leave message
for one channel to sending the join message for the
next. Simulates responsiveness of STB.
Join Latency Threshold
Used as a timer as to when to start the multicast Channel 2
packet-to-leave interval timer if no multicast data
was received. And to count the Join as failed
v
Leave Latency Threshold
A value to measure for failure/error condition in a
test when the IGMP engine has sent a leave for a
particular group and multicast data is still being Leave-Join
received after the threshold period. Delay Channel 3
Multicast packet-to-leave Interval (View)
The delay period overrides the leave-to-leave
interval if specified. The multicast packet-to-leave
interval is the interval to wait between receiving the
first multicast packet and sending the IGMP leave Leave Latency
message for the given channel. Join Latency Threshold
Threshold
Leave-to-Leave Interval (Fixed) Mcast packet-to-
The delay period from an IGMP leave message for View Duration leave interval
one channel to sending the next IGMP leave Leave-to-leave
message for the following channel. First Mcast Pkt
interval
Arrives for Ch. 3
14
Spirent TestCenter IPTV Testing
Margin of Error in Group Join/Leave
Latency
Margin
of Error in
Join Requests
Margin
of Error in
Leave Requests
15
Spirent TestCenter IPTV Testing
Topic Overview
IPTV Overview
IPTV Wizard
Client Settings
Command Sequencer
Results
16
Spirent TestCenter IPTV Testing
Configuration Overview
The IPTV Test Wizard is now turn-key
It can be used to set up the entire test configuration
It can now even be used to create the Hosts (since 2.30)
The Command Sequencer will automatically populated with IPTV
commands upon the IPTV Wizard Completion
Host(s) on the Client and/or Server Port(s) could have
PPPoX/DHCP Emulation or Static IP addresses .
Individual Hosts should have unique VLAN ID’s in case of
Overlapping Channels for Correct Latency Calculation.
Otherwise the Join Latency is only counted for the first Host and the
Leave Latency is only counted for the last Host to Join/Leave a
group respectively.
For Non-Overlapping Channels/Clients, this limitation doesn’t apply.
17
Spirent TestCenter IPTV Testing
What is Different?
18
Spirent TestCenter IPTV Testing
IPTV Wizard Configuration
19
Spirent TestCenter IPTV Testing
IPTV Test Config
Set IPTV Test Specific Parameters in this screen.
20
Spirent TestCenter IPTV Testing
Help is on the way!
21
Spirent TestCenter IPTV Testing
Test Modes
Channel Zapping
Used to monitor zapping characteristics for a large number of clients and
groups (virtually unlimited)
Only verifies that a channel was changed successfully
Only verifies using the last two bits of the IP multicast address
Channel Validation
Used to monitor zapping characteristics for a limited number of clients and
groups (32,768 total when multiplying the two).
Fully verifies that a channel change was successful using all 32 bits of the
multicast IP address
22
Spirent TestCenter IPTV Testing
Test Config Parameters
Test Duration - the length (in days, hours, minutes, and seconds) of the test.
UDP port and UDP port filter - the source or destination UDP ports for the
multicast channel data.
These fields are only enabled if Real World Environment was selected in the
Environment field.
VLAN subfilter type - Only for ports with PPP and VLAN stacking enabled on the
Hosts.
One subfilter is dedicated to the PPP session ID while the other available subfilter can
be used for either the outer VLAN ID or inner VLAN ID.
Total Number of Channels - the total number of channels for the test. Set this
count to a multiple of 4 if Channel Zapping Test was selected in the Type field.
Starting Channel IP and Number- the IP address of the first multicast group in the
test.
Inter-block start delay - the delay between starting consecutive set-top host
blocks on a port.
23
Spirent TestCenter IPTV Testing
Join/fail Percentage Threshold
Join/fail percentage threshold - if the number of failed joins divided by the
total number of channel changes is greater than this percentage, then
the IPTV test aborts.
This enables you to make the necessary changes so you do not have to
sit through the entire test only to realize that the traffic never made it
through.
A failed join occurs when a Join message is sent and the corresponding
channel data are not received within a certain time period (Join Latency
Threshold).
Set this percentage to 100% for the test to continue regardless of the
number of failed joins.
Set this percentage to 0% for stopping on the first failure.
Regardless, failed joins are “Flagged” in the results.
24
Spirent TestCenter IPTV Testing
Selecting the Multicast Source(s)
Select the Port(s) to be used as Multicast Source(s).
This is only available for the “Testing Environment.”
When Spirent TestCenter is simulating the multicast source(s).
Number of
Multicast Sources
can be edited to
have different
Channel Profiles
25
Spirent TestCenter IPTV Testing
Add Multicast Source Hosts
26
Spirent TestCenter IPTV Testing
Host Wizard – Server Ports
Select the Server Port(s) for the IPTV Test.
Usually Multicast Source Ports are on the Internet side of the topology,
so Traffic only would be the likely selection.
More on the Host Wizard in the IPTV Clients Hosts section later.
27
Spirent TestCenter IPTV Testing
Multicast Group Configuration
Multicast Channel Parameters like Channel Range, Rate,
Channel Name can be defined in this screen.
28
Spirent TestCenter IPTV Testing
Add IPTV Client Hosts
29
Spirent TestCenter IPTV Testing
Host Wizard – Client Ports
30
Spirent TestCenter IPTV Testing
Host Wizard – Select Protocols
For static IP Hosts, select the ‘Traffic only’ button.
Or select the ‘Access/Multicast’ button and then select one or more
protocols for the Host(s) to Emulate.
The IPTV Test Wizard will enable multicast for you automatically.
31
Spirent TestCenter IPTV Testing
Host Wizard – VLAN Encapsulation
32
Spirent TestCenter IPTV Testing
VLAN Encapsulation Parameters
33
Spirent TestCenter IPTV Testing
Host Wizard – Configure Hosts
Configure the Host’s MAC and IP information
Pay attention to the addresses and steps
34
Spirent TestCenter IPTV Testing
Host Wizard – Emulation Parameters
Emulation Specific Parameters can be configured from the
Host Wizard as well.
35
Spirent TestCenter IPTV Testing
IPTV Client Settings
Edit the IPTV Client(s) profiles in this screen.
36
Spirent TestCenter IPTV Testing
Topic Overview
IPTV Overview
IPTV Wizard
Client Settings
Command Sequencer
Results
37
Spirent TestCenter IPTV Testing
IPTV Client Settings 1 of 3
Channel Block Name – Derived from the Server Multicast group Blocks.
If there are >1 Channel Block Defined, a drop down Menu is available for
associating that Channel Block to the particular Client Profile.
Channel Block Associations can be repeated across Client Profiles.
Be careful of overlapping channels with non-VLAN enabled clients!
Starting/Ending Channel Numbers inherited from Server Channel Block
associations.
Or editable for a Real World Test
Zap Behavior – Zap Only or Zap and View
Zap and View enables editable Changes Before View and View Duration
Zap Direction - Up, Down or Random.
38
Spirent TestCenter IPTV Testing
IPTV Client Settings 2 of 3
Zap Interval Type – See next Slide
Zap Interval – determines when to send the next leave message
Changes Before View – Number of Channel Changes (Zaps) before
Viewing a Channel.
Only configurable in Zap and View
View Duration – Time a particular Channel will be Viewed before Channel
Change.
Only configurable in Zap and View
Set Top Leave Join Delay – Time between sending a Leave for Current
Channel and sending a Join for the Next Channel.
39
Spirent TestCenter IPTV Testing
Zap Interval Type
Select the method for determining when to send a leave message to
change channels.
While the value of Zap Interval determines how long to wait before
changing channels, the Zap Interval Type determines when the timer
starts.
Leave to Leave – Start timing the Zap Interval when a leave message is
sent.
This is a more stressful test since it does not give the system a chance to
catch its breath.
It can also lead to more failed joins because the data did not come in before
the view duration expired.
Multicast Packet to Leave – Start timing the Zap Interval when the first
multicast packet for the joined channel is received.
This is more realistic since people generally want to see what is on before
changing the channel again.
In many cases it gives the system a chance to breath in between channel
changes.
40
Spirent TestCenter IPTV Testing
IPTV Client Settings 3 of 3
Join Latency Threshold – Time to wait for multicast traffic to be received
after sending a Join.
If no multicast traffic is received within this time, it is marked as a Flagged
(i.e., failed) Join.
Leave Latency Threshold – Time to wait for multicast traffic to stop being
received after sending a Leave.
If multicast traffic is still being received within this time, it is marked as a
Flagged (i.e., failed) Leave.
Additional multicast traffic is ignored after this time.
Save Time Stamps – IPTV Host Block results are aggregated in the
Real-time results.
However, if you select to “Save Timestamps," the Results Reporter can
break them out in Detailed Template View.
41
Spirent TestCenter IPTV Testing
Why Client VLANs are Needed
42
Spirent TestCenter IPTV Testing
Typical Topology and Necessary
Timestamps
C1 DUT
Chan_1
MC Source
C2 Chan_2
Assume that in the above topology, the MC source is transmitting channels 1 and 2 (denoted
chan_1 and chan_2 respectively) and that clients C1 and C2 are changing from chan_1 to
chan_2 independently of each other.
To get the various timestamps (TS), we have to look at both the control plane and data plane.
The control plane TSs can be obtained as the join/leaves are sent from the client side port.
The data plane TSs are obtained from capturing the MC packets as they arrive on the client
side port. Note here that chan_1 (or chan_N) packets that go to C1 and/or C2 have the exact
same L2 and L3 source and destination addresses.
43
Spirent TestCenter IPTV Testing
Clients on Physically Separate
Connections
Spirent TestCenter
Analyzers
SPT-5000A
C1 DUT
Pow
Res
Tem
p
er
et
Chan_1
MC Source
Fan
Link
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
SPT-5000A
C2 Chan_2
er
Pow
et
Res
p
Tem
Fan
Link
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
44
Spirent TestCenter IPTV Testing
Spirent TestCenter Emulates Multiple
Clients on a Single Physical Port
Spirent TestCenter
Analyzer DUT
SPT-5000A
C1
Chan_1
Pow
Res
Tem
Fan
p
Link
er
et
MC Source
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
C2
However, since Spirent TestCenter, from a single port, is emulating multiple clients (or set top
boxes) there is really only one Analyzer to capture the TSs for all the MC traffic for C1 and C2.
And since the L2, L3 addresses and UDP Port for any given chan_N is the same. For example:
• L2 = (MC MAC dest, DUT MAC)
• L3 = (MC group, MC source)
• UPD = (3347, 3347)
As such, the Analyzer will detect substreams for each chan_N but cannot distinguish if the packets
are for C1 or C2. That is, the Analyzer will have 2 substream filters (1 = MC group IP, 2 = UDP
Port). Why is this important?
.
45
Spirent TestCenter IPTV Testing
C1 and C2 Join Chan_1
Spirent TestCenter
Analyzer DUT
SPT-5000A
C1
Chan_1 Chan_1
Pow
Res
Tem
Fan
p
Link
er
et
Chan_1 MC Source
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
C2 Chan_1
However, since Spirent TestCenter, from a single port, is emulating multiple clients (or set top
boxes) there is really only one Analyzer to capture the TSs for all the MC traffic for C1 and C2.
And since the L2, L3 addresses and UDP Port for any given chan_N is the same. For example:
• L2 = (MC MAC dest, DUT MAC)
• L3 = (MC group, MC source)
• UPD = (3347, 3347)
As such, the Analyzer will detect substreams for each chan_N but cannot distinguish if the packets
are for C1 or C2. That is, the Analyzer will have 2 substream filters (1 = MC group IP, 2 = UDP
Port). Why is this important?
.
• Well, say C1 and C2 have joined chan_1 and then after some time …
46
Spirent TestCenter IPTV Testing
C1 Leaves Chan_1
Spirent TestCenter
Analyzer DUT
SPT-5000A
C1
Chan_1
Pow
Res
Tem
Fan
p
Link
er
et
Chan_1 MC Source
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
C2 Chan_1
However, since Spirent TestCenter, from a single port, is emulating multiple clients (or set top
boxes) there is really only one Analyzer to capture the TSs for all the MC traffic for C1 and C2. And
since the L2, L3 addresses and UDP Port for any given chan_N is the same. For example:
• L2 = (MC MAC dest, DUT MAC)
• L3 = (MC group, MC source)
• UPD = (3347, 3347)
As such, the Analyzer will detect substreams for each chan_N but cannot distinguish if the packets
are for C1 or C2. That is, the Analyzer will have 2 substream filters (1 = MC group IP, 2 = UDP
Port). Why is this important?
.
• Well, say C1 and C2 have joined chan_1 and then after some time …
• C1 leaves chan_1. Now the Analyzer will continue to detect the chan_1 substream since C2
hasn’t left it. So at this point the Analyzer cannot obtain the TS of last packet for chan_1 for C1.
47
Spirent TestCenter IPTV Testing
So We Need …
Spirent TestCenter
Analyzer DUT
SPT-5000A
C1
Chan_1
Pow
Res
Tem
Fan
p
Link
er
et
Chan_1 MC Source
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
C2 Chan_1
So, what is needed is a way for the Analyzer to distinguish between C1’s and C2’s MC traffic.
We know it can’t be in the L2 MAC or L3 addresses or UDP ports. But if we …
48
Spirent TestCenter IPTV Testing
A Separate VLAN Per Client
Spirent TestCenter
Chan_1
Pow
Res
Tem
Fan
p
Link
er
et
Chan_1 MC Source
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
Chan_1
C2, VLAN=2
So, what is needed is a way for the Analyzer to distinguish between C1’s and C2’s MC traffic.
We know it can’t be in the L2 MAC or L3 addresses or UDP ports. But if we …
49
Spirent TestCenter IPTV Testing
A 3rd Subfilter for VLAN IDs then Voila,
Latency Measurements!
Spirent TestCenter
Chan_1
Pow
Res
Tem
Fan
p
er
et
Chan_1 MC Source
VLAN=2
Link
Chan_2
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
Chan_1
So, what is needed is a way for the Analyzer to distinguish between C1’s and C2’s MC traffic.
We know it can’t be in the L2 MAC or L3 addresses. But if we …
add a 3rd subfilter to the Analyzer (1 = MC group IP, 2 = UDP Port, 3 = VLAN ID) then the
Analzyer would see that (e.g., chan_1, 3347, VLAN=1) MC traffic has stopped. At this point,
we can now obtain the needed TS of last packet for chan_1 for C1 (VLAN=1).
50
Spirent TestCenter IPTV Testing
Summary: Why VLANs are Needed
Spirent TestCenter
C1, VLAN=1
Analyzer DUT
Chan_1
SPT-5000A
Pow
Res
er
et
Chan_1 MC Source
VLAN=2
p
Chan_2
Tem
Fan
Link
tus
Sta
er
Pow
CAUTION
STATICSENSITIVE
Chan_1
• Each client needs to be on its own VLAN (or VCC for ATM).
51
Spirent TestCenter IPTV Testing
Topic Overview
IPTV Overview
IPTV Wizard
Client Settings
Command Sequencer
Results
52
Spirent TestCenter IPTV Testing
Command Sequencer
53
Spirent TestCenter IPTV Testing
Command Sequencer notes
There is a self check in the IPTV Start Test Command,
which will highlight any configuration/logical errors.
You can still edit the Test Time, Type of Test, Number of
Clients, Join/Fail Threshold percentage etc. by double
clicking on the IPTV:Start Test command.
To View Detailed Results, wait till the end of the test and
Results Reporter will automatically launched.
Results will also be available in the Results Browser during
the Test Execution.
54
Spirent TestCenter IPTV Testing
IPTV Command Editor
55
Spirent TestCenter IPTV Testing
Running the Test
This of course is done from the Command Sequencer.
Start Step Pause Stop Break loop
56
Spirent TestCenter IPTV Testing
Topic Overview
IPTV Overview
IPTV Wizard
Client Settings
Command Sequencer
Results
57
Spirent TestCenter IPTV Testing
Log Messages
58
Spirent TestCenter IPTV Testing
Results Browser
59
Spirent TestCenter IPTV Testing
Port Traffic > Basic Traffic Results
60
Spirent TestCenter IPTV Testing
View the Analyzer Filters
The IPTV Test Wizard will also set up specific filters to sort the received
traffic on.
This will help make the results more intuitive.
And also help to differentiate the received traffic.
Includes the Group IP Address, VLAN ID, PPPoE Session ID, as well as
the upper 16 bits of the Spirent Signature ID.
61
Spirent TestCenter IPTV Testing
Stream Results > Filtered Stream Results
Traffic is sorted based on the settings in the Analyzer Filters.
The varying RX Frame Counts are the results of Channel Zapping.
62
Spirent TestCenter IPTV Testing
Results Reporter
The aggregate results are collected into a .db file at the end
of the test.
This file in turn is read and displayed using the Results
Reporter.
This process was automated when you inserted the EOT
Results command into the Command Sequencer.
63
Spirent TestCenter IPTV Testing
IPTV Aggregate Results Template
Create
PDF
64
Spirent TestCenter IPTV Testing
End of Topic
www.spirentcampus.com
65
Spirent TestCenter IPTV Testing