ECN TLP Prefix 2008-12-15
ECN TLP Prefix 2008-12-15
ECN TLP Prefix 2008-12-15
Emerging usage model trends indicate a requirement for increase in header size fields to provide
additional information than what can be accommodated in currently defined TLP header sizes.
The TLP Prefix mechanism extends the header size by adding DWORDS to the front of headers
that carry additional information. The base size of a TLP Prefix is a single DWORD and the
mechanism allows for multiple TLP Prefixes to be present in a TLP to either provide space for
additional fields or to allow for multiple separate TLP Prefixes to be present in a TLP. The TLP
Prefix mechanism changes the format of a TLP such that a TLP is comprised of an optional TLP
Prefix, TLP Header, optional Data Payload and optional TLP Digest.
There are two types of TLP Prefixes defined, namely Local TLP Prefix and End-End TLP Prefix.
The Local TLP Prefix scope is limited to the Link whereas the End-End TLP Prefix scope is
between the originator and target of a Request or Completion TLP. For End-End TLP Prefixes, the
originator and target may be directly connected or may be connected through a hierarchy of
routing elements.
The TLP Prefix mechanism provides architectural headroom for PCIe headers to grow in the
future. Switches and Switch related software can be built that are transparent to the encoding of
future End-End TLPs. The End-End TLP Prefix mechanism defines rules for routing elements to
route TLPs containing End-End TLP Prefixes without requiring the routing element logic to
explicitly support any specific End-End TLP Prefix encoding(s).
TLP Prefix is an optional normative feature and the impact is incurred by those devices that
choose to support this mechanism. The End-End TLP Prefix mechanism requires an increase in
the buffering requirements for Receivers without impacting the flow control accounting. The
header flow control credit takes into account the amount of buffering required for TLP Prefixes.
TLP processing logic is required to comprehend the presence of the Optional TLP Prefix in a TLP
and, for Functions that support TLP Prefix, will have to handle the control of TLP Prefixes as a
requestor or completer or both as the case may be.
For Local TLP Prefix the impact is dependent on the Local TLP Prefix type.
Page 1
4. Analysis of the Hardware Implications
TLP Prefix is an optional normative feature and the impact is incurred only by those devices that
choose to support this mechanism. Functions supporting TLP Prefix have to consider the buffering
requirements associated with TLP Prefix. For Functions that support End-End TLP Prefixes, the
header flow control credit unit takes into account the maximum size of TLP Prefixes. For End-End
TLP Prefixes the maximum allowed size is 4 DWORDs.
The hardware implications of the Local TLP Prefix are dependent on the TLP Prefix type encoding.
Devices that dont support Local TLP Prefixes (or dont support a specific Local TLP Prefix) are
required to terminate the TLP as a Malformed TLP.
There are new fields in the Device Capabilities 2 and Device Control 2 registers for software to
broadly manage and control the TLP Prefix mechanism. Additionally it is expected that there will
be TLP Prefix specific management and control registers to configure specific TLP Prefix related
attributes.
For End-End TLP Prefixes, software must ensure that the entire communications path from
requestor (completer) to target supports TLP Prefix before enabling the feature.
End-End, SR-IOV: A Virtual Intermediary (VI) can hide the presence of support for End-End
TLP Prefixes from guest operating systems by hiding the appropriate
capabilities. Depending on the nature of the End-End Prefix, the VI may
enable its use even when the associated Guest OS does not comprehend
the Prefix (e.g. crypto capabilities).
Local, SR-IOV: Local Prefixes affect all Functions on either end of a Link and will likely be
managed by the VI.
End-End, MR-IOV: End-End prefixes operate within a specific VH. Software operating in that
VH enables or disables usage. Malfunctioning software that enables End-
End Prefixes over Link(s) that do not support them will result in Malformed
TLP or TLP Prefix Blocked errors signaled within the affected VH.
Local, MR-IOV: Local TLP Prefixes affect all Functions in all VHs on either end of a Link
and will likely be managed by an MR-PCIM. The MR-IOV TLP Prefix is a
Local Prefix that is automatically enabled when a Link trains in MR-IOV
mode.
Page 2
Part II
Detailed Description of the change
End-End TLP Prefix A TLP Prefix that is carried along with a TLP from source to
destination. See Section 2.2.10.2.
Local TLP Prefix A TLP Prefix that is carried along with a TLP on a single Link.
See Section 2.2.10.1.
TLP Prefixes
(optional)
Page 3
+0 +1 +2 +3
7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
Byte 0 > Fmt Type {Fields in bytes 1 through 3 depend on Fmt and Type Fields}
Page 5
EP indicates the TLP is poisoned (see Section 2.7) bit 6 of byte 2
Length[9:0] Length of data payload in DW (see Table 2-4) bits 1:0 of byte 2
concatenated with bits 7:0 of byte 3
TLP data must be 4-byte naturally aligned and in increments of 4-byte Double Words
(DW).
Reserved for TLPs that do not contain or refer to data payloads, including Cpl,
CplLk, and Messages (except as specified)
+0 +1 +2 +3
7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
T E
Byte 0 > Fmt Type R TC R Attr AT Length
D P
Page 6
Table 2-3: Fmt[1:0] and Type[4:0] Field Encodings
TLP Type Fmt Type Description
[2:0] 1 [4:0]
(b) (b)
MRd 000 0 0000 Memory Read Request
001
MRdLk 000 0 0001 Memory Read Request-Locked
001
MWr 010 0 0000 Memory Write Request
011
IORd 000 0 0010 I/O Read Request
IOWr 010 0 0010 I/O Write Request
CfgRd0 000 0 0100 Configuration Read Type 0
CfgWr0 010 0 0100 Configuration Write Type 0
CfgRd1 000 0 0101 Configuration Read Type 1
CfgWr1 010 0 0101 Configuration Write Type 1
TCfgRd 000 1 1011 Deprecated TLP Type2
TCfgWr 010 1 1011 Deprecated TLP Type 2
Msg 001 1 0r2r1r0 Message Request The sub-field r[2:0]
specifies the Message routing mechanism
(see Table 2-12).
MsgD 011 1 0r2r1r0 Message Request with data payload The
sub-field r[2:0] specifies the Message
routing mechanism (see Table 2-12).
Cpl 000 0 1010 Completion without Data Used for I/O and
Configuration Write Completions and Read
Completions (I/O, Configuration, or
Memory) with Completion Status other than
Successful Completion.
CplD 010 0 1010 Completion with Data Used for Memory,
I/O, and Configuration Read Completions.
CplLk 000 0 1011 Completion for Locked Memory Read
without Data Used only in error case.
CplDLk 010 0 1011 Completion for Locked Memory Read
otherwise like CplD.
LPrfx 100 0L3L2L1L0 Local TLP Prefix The sub-field L[3:0]
specifies the Local TLP Prefix type (see
Table 2-xx)
1Requests with two Fmt[2:0] values shown can use either 32 bits (the first value) or 64 bits (the
second value) Addressing Packet formats.
2Deprecated TLP Types: previously used for TCS, which is no longer supported by this
specification. If a Receiver does not implement TCS, the Receiver must treat such Requests as
Malformed Packets.
Page 7
TLP Type Fmt Type Description
[2:0] 1 [4:0]
(b) (b)
EPrfx 100 1E3E2E1E0 End-End TLP Prefix The sub-field E[3:0]
specifies the End-End TLP Prefix type (see
Table 2-yy)
All encodings not shown above are
Reserved (see Section 2.3).
Page 8
Local TLP L[3:0] Description
Prefix Type (b)
MR-IOV 0000 MR-IOV TLP Prefix Refer to the Multi-
Root I/O Virtualization and Sharing
specification for details
VendPrefixL0 1110 Vendor Defined Local TLP Prefix Refer
to section 2.2.10.1.1 for further details.
VendPrefixL1 1111 Vendor Defined Local TLP Prefix Refer
to section 2.2.10.1.1 for further details.
All other encodings are reserved
The size, routing and flow control rules are specific to each Local TLP Prefix type
It is an error to receive a TLP with a Local TLP Prefix type not supported by the
Receiver. If the Extended Fmt Field Supported bit is Set, TLPs in violation of this rule
are handled as a Malformed TLP unless explicitly stated differently in another
specification. This is a reported error associated with the Receiving Port (see Section 6.2).
If the Extended Fmt Field Supported bit is Clear, behavior is device specific.
No Local TLP Prefixes are protected by ECRC even if the underlying TLP is protected
by ECRC.
Page 9
End-End TLP E[3:0] Description
Prefix Type (b)
ExtTPH 0000 Extended TPH Refer to section <tbd> for
further details
VendPrefixE0 1110 Vendor Defined End-End TLP Prefix
Refer to section 2.2.10.2.1 for further details
VendPrefixE1 1111 Vendor Defined End-End TLP Prefix
Refer to section 2.2.10.2.1 for further details
All other encodings are reserved
Note to Editor: ExtTPH is covered by another ECN. Please correct the <tbd> reference to
point to that ECN when incorporating this change into the base specification.
The maximum number of End-End TLP Prefixes permitted in a TLP is 4;
o A Receiver supporting TLP Prefixes must check this rule. If a Receiver
determines that a TLP violates this rule, the TLP is a Malformed TLP. This is a
reported error associated with the Receiving Port (see Section 6.2)
The presence of an End-End TLP Prefix does not alter the routing of a TLP. TLPs are
routed based on the routing rules covered in section 2.2.4.
Functions indicate how many End-End TLP Prefixes they support by the Max_End_End
TLP Prefixes field in the Device Capabilities 2 register (see Section 7.8.15). TLPs received
that contain more End-End TLP Prefixes than are supported by a Function must be
handled as Malformed TLPs. AER logging (if supported) occurs as specified in section
6.2.4.4. This is a reported error associated with the Receiving Port (see Section 6.2).
Switches must support forwarding of TLPs with up to 4 End-End TLP Prefixes if the
End-End TLP Prefix Supported bit is Set.
Root Complexes must support forwarding of TLPs with up to Max End-End TLP
Prefixes if the End-End TLP Prefix Supported bit is Set in both the Ingress and Egress
Ports and forwarding of TLPs is supported between those Ports. 3
All End-End TLP Prefixes are protected by ECRC if the underlying TLP is protected by
ECRC.
It is an error to receive a TLP with an End-End TLP Prefix by a Receiver that does not
support End-End Prefixes. A TLP in violation of this rule handled as a Malformed TLP.
This is a reported error associated with the Receiving Port (see Section 6.2)
Software should ensure that TLPs containing End-End TLP Prefixes are not sent to
components that do not support them. Components where the Extended Fmt Field
Supported bit is Clear may misinterpret TLPs containing TLP Prefixes.
If one Function of an Upstream Port has the End-End TLP Prefix Supported bit Set, all
Functions of that Upstream Port must handle the receipt of a Request addressed to them
3 Root Port indication of End-End TLP Prefix Supported does not imply any particular level of
peer-to-peer support by the RC, or that peer-to-peer traffic is supported at all (see Section
2.2.10.2.2).
Page 10
that contains an unsupported End-End TLP Prefix type as an Unsupported Request. This
is a reported error associated with the Receiving Port (see Section 6.2).
If one Function of an Upstream Port has the End-End TLP Prefix Supported bit Set, all
Functions of that Upstream Port must handle the receipt of a Completion addressed to
them that contains an unsupported End-End TLP Prefix type as an Unexpected
Completion. This is a reported error associated with the Receiving Port (see Section 6.2).
For routing elements, the End-End TLP Prefix Egress Blocking bit in each Egress Port
determines whether TLPs containing End-End TLP Prefixes can be transmitted via that
Egress Port (see Section 7.8.16). If forwarding is blocked the entire TLP is dropped and a
TLP Prefix Blocked Error is reported. If the blocked TLP is a Non-Posted Request, the
Egress Port returns a Completion with Unsupport Request Completion Status. The TLP
Prefix Blocked Error is a reported error associated with the Egress Port (see Section 6.2).
For routing elements where Multicast is enabled (see Section <tbd>), End-End TLP
Prefixes are replicted in all Multicast copies of a TLP. TLP Prefix Egress Blocking of
Multicast packets is performed independently at each Egress Port.
Note to Editor: Multicast is covered by another ECN. Please correct the <tbd> reference to
point to that ECN when incorporating this change into the base specification.
2.2.10.2.1 Vendor Defined End-End TLP Prefix
As described in Table 2-yy, Types VendPrefixE0 and VendPrefixE1 are reserved for usage as
Vendor Defined End-End TLP Prefixes. To maximize interoperability and flexibility the
following rules are applied to such prefixes:
Components must not send TLPs containing Vendor Defined End-End TLP Prefixes
unless this has been explicity enabled (using vendor specific mechanisms).
It is recommended that components be configurable (using vendor specific mechanisms)
to use either of the two Vendor Defined End-End TLP Prefix encodings. Doing so
allows two different Vendor Defined End-End TLP Prefixes to be in use simultaneously
within a single PCIe topology while not requiring that every source understand the
ultimate destination of every TLP it sends.
4 An earlier version of this specification reserved the bit now defined for Fmt[2].
Page 12
In this and other sections of this specification, rules are described using conceptual
registers that a device could use in order to implement a compliant implementation. This
description does not imply or require a particular implementation and is used only to clarify
the requirements.
Flow Control information is transferred using Flow Control Packets (FCPs), which are a
type of DLLP (see Section 3.4)
The unit of Flow Control credit is 4 DW for data
For headers, the unit of Flow Control credit is one maximum-size header plus TLP digest
o The unit of Flow Control credit for Receivers that support End-End TLP Prefix
is sum of one maximum-size Header, TLP Digest and maximum number of End-
End TLP Prefixes permitted in a TLP.
o The management of Flow Control for Receivers that support Local TLP Prefix is
dependent on the Local TLP Prefix type.
Each Virtual Channel has independent Flow Control
5 For Example, errors involving MR-IOV TLP Prefixes are logged in MR-IOV structures and are not
logged in the AER Capability.
Page 13
End-End TLP Prefixes are logged in the TLP Prefix Log Register. The underlying TLP
Header is logged in the Header Log Register subject to two exceptions:
If the Extended Fmt Field Supported bit is Set (see Section 7.8.15), a Function that does
not support TLP Prefixes and receives a TLP containing a TLP Prefix will signal
Malformed TLP and the Header Log Register will contain the first 4 DWs of the TLP
(TLP Prefixes followed by as much of the TLP Header as will fit).
A Function that receives a TLP containing more End-End TLP Prefixes than are
indicated by the Functions Max End-End TLP Prefixes field must handle the TLP as a
Malformed TLP and store the first overflow End-End TLP Prefix in the 1st DW of the
Header Log register with the remainder of the Header Log register being undefined.
Modify section 6.2.6 as indicated
Page 14
Bit Location Register Description Attributes
All Ports of a Switch must have the same value for
this bit.
23:22 Max End-End TLP Prefixes Indicates the maximum RO/ HwInit
number of End-End TLP Prefixes supported by this (see
Function. TLPs received by this Function that contain description)
more End-End TLP Prefixes than are supported must
be handled as Malformed TLPs (see Section
2.2.10.2). Values are:
01b 1 End-End TLP Prefix
10b 2 End-End TLP Prefixes
11b 3 End-End TLP Prefixes
00b 4 End-End TLP Prefixes
If End-End TLP Prefix Supported is Clear, this field is
RsvdP.
This field is HwInit for Root Ports and is RO for all
other Functions. All Root Ports that have the End-End
TLP Prefix Supported bit Set must contain the same
value for this field.
For Switches where End-End TLP Prefix Supported is
Set, this field must be 00b indicating support for up to
4 End-End TLP Prefixes (see Section 2.2.10.2).
Page 15
Bit Location Register Description Attributes
cases.
Page 16
7.10.4. Uncorrectable Error Severity Register (Offset
0Ch)
Add the following fields to Figure 7-34 and Table 7-31.
Bit Location Register Description Attributes Default
25 TLP Prefix Blocked Error Severity (Optional) RWS 0b
Page 17
31 24 23 16 15 8 7 0
Header Log Register (1st DW)
Header Byte 0 Header Byte 1 Header Byte 2 Header Byte 3
Header Log Register (2nd DW)
Header Byte 4 Header Byte 5 Header Byte 6 Header Byte 7
Header Log Register (3rd DW)
Header Byte 8 Header Byte 9 Header Byte 10 Header Byte 11
Header Log Register (4th DW)
Header Byte 12 Header Byte 13 Header Byte 14 Header Byte 15
OM14549A
Page 18
Figure 7-xx TLP Prefix Log Register
Page 19