Brkcol 3224

Implementing and Troubleshooting
Secure Voice/Video on Edge Devices
Paul Anholt, Technical Leader

Felipe Garrido, Technical Leader
BRKCOL-3224
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
BRKCOL-3224 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
UC Security Sessions
BRKCOL-2014: Introduction to Cisco UC Security

• Tomorrow, Wednesday the 29th at 8:30am
BRKCOL-3224: Implementing and Troubleshooting Secure Voice/Video on Edge

Devices
• Right now!
Agenda
• Introduction
• Where to Begin
• Implementing Secure Signaling and Media
• Cisco Expressway
• Cisco Unified Border Element (CUBE)
• Interoperability between Secure and Non-Secure Networks

• Troubleshooting
• Methodology
• Tools
• Scenarios
Where to Begin
Session Prerequisites
• Working knowledge of
• SIP
• CUCM Phone and SIP trunk configuration
• IOS/CUBE configuration
• Expressway configuration
SIP
PSTN
SIP SIP
CUBE CUCM Expressway
Deploying CA-
Signed Certificates
Certificate Exchange – CA vs Self-signed
CA-Server
CUCM Self-Signed CA-Signed
• CUBE and CUCM • Client generates a

generate self- key-pair and sends
signed certificates a Certificate
Signing Request
• CUBE exports (CSR) to the
certificate to Certificate
CUCM Authority (CA).
• CUCM exports • The CA signs it
certificate to CUBE with its private key,
creating an Identity
Certificate
• Client installs the
Client installs list of
trusted CA Root
and Intermediary
Certificates and
CUBE Client Identity Certificate
Generating
Certificate Signing
Requests (CSR)
Generating a CSR in CUBE/IOS
Generating the Key-pair
For TLS 1.0, generate an RSA key-pair

ISR4KCUBE(config)#crypto key generate rsa general-key label rsakey modulus <360-4096>
For TLS 1.2, generate an EC key-pair

ISR4KCUBE(config)#crypto key generate ec keysize 256|384 label eckey
1. Create Trustpoint to store the root and identity certificates

ISR4KCUBE(config)#crypto pki trustpoint caServer
ISR4KCUBE(ca-trustpoint)#enrollment terminal
ISR4KCUBE(ca-trustpoint)#subject-name CN=ISR4KCUBE.cisco-uc.com
ISR4KCUBE(ca-trustpoint)#revocation-check none
1b. Specify the correct key pair to use

ISR4KCUBE(ca-trustpoint)#eckeypair eckey Required for TLS 1.2
or
ISR4KCUBE(ca-trustpoint)#rsakeypair rsakey
2. Generate the CSR from the Trustpoint
ISR4KCUBE(config)#crypto pki enroll caServer
% Start certificate enrollment ..
% The subject name in the certificate will include: CN=ISR4KCUBE.cisco-uc.com

% The subject name in the certificate will include: ISR4KCUBE.cisco-uc.com
% Include the router serial number in the subject name? [yes/no]: no
% Include an IP address in the subject name? [no]: no
Display Certificate Request to terminal? [yes/no]: yes
Certificate Request follows:
MIIBEzCBuwIBADA4MRwwGgYDVQQDExM0NDUxLUNVQkUuY2lzY28uY29tMRgwFgYJ
...snip...
ceZ+rgIhAN8AzHEJKdUnFovHCetMSed60qHkKjn/H+C1iOxaH4Yz
---End - This line not part of the certificate request---

Redisplay enrollment request? [yes/no]:
Generating a CSR in Maintenance > Security > Server certificate >
Generate CSR
Expressway
Maintenance > Security > Server certificate
Notes
• Only one CSR may be in progress at a

time.
• From version X8.10, you cannot select
SHA-1
Importing the
Certificate Trust
Chain
Importing the Root/Intermediate and Identity
Certificates to CUBE
Import the Root Certificate via the Terminal using the crypto pki authenticate
<trustpoint_name> command
ISR4KCUBE(config)#crypto pki authenticate caServer
Enter the base 64 encoded CA certificate.
End with a blank line or the word "quit" on a line by itself
-----BEGIN CERTIFICATE-----
...snip...
-----END CERTIFICATE-----
Certificate has the following attributes:
Fingerprint MD5: 45F42000 781A427A 152E9DBD 7C438967
Fingerprint SHA1: A8E57437 3AE1E33B 22768143 EE308B79 0A3C43E6
% Do you accept this certificate? [yes/no]: yes

Trustpoint CA certificate accepted.
Import the Identity Certificate via the Terminal using the crypto pki import
% Certificate successfully imported
<trustpoint_name> certificate command
ISR4KCUBE(config)#crypto pki import caServer certificate
Enter the base 64 encoded CA certificate.
End with a blank line or the word "quit" on a line by itself
-----BEGIN CERTIFICATE-----
...snip...
% Router Certificate successfully imported
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
Certificate Verification
show crypto pki certificate
ISR4KCUBE#show crypto pki certificate (continued..)
Certificate CA Certificate
Status: Available Status: Available
Certificate Serial Number (hex): Certificate Serial Number (hex):
10FCB16C000000000004 3A0E15FE9124C0B040801DC535DBA5C3
Certificate Usage: General Purpose Certificate Usage: Signature
Issuer: Issuer:
cn=BRKUCC_CA cn=BRKUCC_CA
Subject: Subject:
Name: ISR4KCUBE.cisco-uc.com cn=BRKUCC_CA
cn=ISR4KCUBE.cisco-uc.com
Validity Date:
Validity Date: start date: 17:44:42 EDT May 29 2017
start date: 18:34:54 EDT May 29 2017 end date: 17:54:41 EDT May 29 2022
end date: 18:44:54 EDT May 29 2019
Associated Trustpoints: caServer
Managing the Trusted CA certificates on
Expressway
Maintenance > Security > Trusted CA certificate
View and
manage trusted
server
certificates
View the CA
trust file
Add one or
more certificate
to the CA trust
file
Reset back to
default
CA-signed Certificate Trust Chain
Trust
Chain
Certificate
Authority
Let’s Encrypt
support
on Expressway-E
Let’s Encrypt Introduction
• Let’s Encrypt is a free, automated, and open Certificate Authority

• Providing X.509 certificates for TLS encryption at no charge
• Includes an automated process designed to overcome manual creation, validation,
signing, installation, and renewal of certificate
• ACME protocol (Automated Certificate Management Environment)
• More details at www.letsencrypt.org
• Let’s Encrypt signed certs are compatible with all MRA endpoints
• Compatible with both Expressway server and domain certificates
Expressway-E ACME Requirements
• TCP port 80 required to be open inbound to all Expressway-E’s from ANY

• DNS A records need to be available in public DNS for all SANs required in
Expressway-E certificate
• Admin needs to manually add Let’s Encrypt CA and Digital Signature Trust X3 root
CA certs to both Expressway C & E
• Each Expressway-E will request and maintain it’s own certificate
• The random strings required to satisfy the ACME challenges are shared across all
Expressway-E cluster peers
CSR considerations when using Let’s Encrypt
• MRA deployments should consider using the CollabEdgeDNS “format”

• collab-edge subdomain used in CSR for any configured MRA domain(s)
• This name format alternative satisfies Jabber and TelePresence endpoint
certificate requirements
• Compatible with Let’s
Encrypt DNS and HTTP
requirements
Automated Certificate Renewal and Deployment
• A new cert will be signed after 2/3 of existing certificate’s validity

• Automatic deployment of new certificate can be scheduled
• No restart of
Expressway required
• Deployment signals to
various processes to
reload the server
certificate
Implementing
Secure Signaling
Do Unsecure calls work?
TLS Session Establishment
Client Server
TCP Established
ClientHello
ServerHello
Certificate
ServerKeyExchange
Certificate CertificateRequest
ClientKeyExchange ServerHelloDone
CertificateVerify
[ChangeCipherSpec]
Finished
[ChangeCipherSpec]
Finished
TLS Established
SIP Session Establishment
Client Server
TCP Established
TLS Established
Invite (Offer)
100 Trying
180 Ringing
200 Ok (Answer)
Ack
Media/RTP
MRA CUCM Phone Device Security Profile
MRA
The UCM Device Security Profile Name must be in the

FQDN format with the enterprise domain.
This same FQDN must be present as a Subject

Alternate Name (SAN) in the Expressway-C’s server
certificate.
Use of a Universal Device Template is recommended to

avoid having to regenerate the server certificate when
new endpoint models are added.
Set the Device Security Mode to Authenticated

Expressway Business-to-Business Call Flows
Where do I apply the configuration?
To secure signaling and media end-to-end on the call flow
Ensure TLS/SRTP is enabled on
the following zones
SIP TLS SIP TLS SIP TLS
SRTP SRTP
Exp-C Exp-E
To enable secure signaling and media on a subset of the call flow

Ensure TLS/SRTP is enabled on
the following zones
SIP SIP TLS SIP TLS
SRTP SRTP
Exp-C Exp-E
Expressway Configuration
Enabling SIP TLS B2B/Hybrid
Configuration > Protocols > SIP Configuration Settings
• TLS Mode/Port – Configure the global SIP TLS listening port

• MTLS Mode/Port – Configure the global SIP MTLS Listening
Port ⚠️
• TLS Handshake timeout – Configure the number of seconds
before timing out the TLS Connection
• Certificate revocation checking mode – Enable or disable CRL
MTLS settings here may conflict with the following default zone settings
Enabling SIP TLS on Zones B2B/Hybrid
Configuration>Zones>Zone>Edit zone>SIP Configuration Settings
SIP
• Transport – (TLS) Set the zone’s outbound transport type to
TLS
• TLS Port – Peer’s SIP TLS Listening port
• TLS Verify Mode:
• On – Match the peers x.509 subject name to the zone’s
peer
• Off – Accept any useable x.509 certificate
Location
• Peer Address – FQDN or IP address of peer
Configuration>Zones>Zones>Edit zone>Location
Cipher Suites Global
Maintenance>Security>Ciphers xconfig //cipher
xConfiguration Ciphers ForwardProxyTLSCiphers Value: "HIGH:!MD5:!RC…"

xConfiguration Ciphers ForwardProxyTLSProtocol Value: "minTLSv1.0"
xConfiguration Ciphers HTTPSCiphers Value: "HIGH:!EXP:!MD5:!RC4:!…"
⚠️ xConfiguration Ciphers HTTPSProtocol Value: "minTLSv1.0"
xConfiguration Ciphers ReverseProxyTLSCiphers Value: "HIGH:!MD5:!RC4.."
xConfiguration Ciphers ReverseProxyTLSProtocol Value: "minTLSv1.0"
xConfiguration Ciphers SIPTLSCiphers Value: "ALL:!EXP:!LOW:!MD5:@…"
xConfiguration Ciphers UcClientTLSCiphers Value: "ALL"
xConfiguration Ciphers UcClientTLSProtocol Value: "minTLSv1.0"
xConfiguration Ciphers XCPTLSCiphers Value: "ALL:!ADH:!LOW:!EXP:!MD..."
xConfiguration Ciphers XCPTLSProtocol Value: "minTLSv1.0"
xConfiguration Ciphers sshd_ciphers Value: "[email protected]..."
xConfiguration Ciphers sshd_kex Value: "ecdh-sha2-nistp521,ecdh-sha2-…"
xConfiguration Ciphers sshd_macs Value: "hmac-sha2-512,hmac-sha2…"
xConfiguration Ciphers sshd_pfwd_ciphers Value: "aes256-ctr"
xConfiguration Authentication ADS CipherSuite: "HIGH:MEDIUM:!ADH:…“
⚠️TLS 1.0 is supported in X8.10 but not recommended

IOS Configuration Prerequisites
General Unsecure Voice Configuration
1. Global CUBE Configuration
voice service voip
address-hiding
mode border-element license capacity 1000
allow-connection sip to sip
3. Dial-peer Configuration – WAN (PSTN) side 2. Dial-peer Configuration – LAN side
dial-peer voice 10 voip dial-peer voice 1 voip

description to PSTN – unSecure description to CUCM Sub – unSecure
preference 1 preference 1
destination-pattern [9]1T destination-pattern 418110....
session protocol sipv2 session protocol sipv2
session target ipv4:14.50.211.31 session target ipv4:14.50.248.103
session transport tcp session transport tcp
CUBE
incoming called-number 418....... incoming called-number 9.T
dtmf-relay rtp-nte dtmf-relay rtp-nte
codec g711ulaw codec g711ulaw
no vad no vad
TCP is required to enable TLS!

IOS Configuration – Enabling TLS
1. Associate trustpoint with IOS voice process

ISR4KCUBE(config)#sip-ua
ISR4KCUBE(config-sip-ua)#crypto signaling default trustpoint caServer
2. Enable TLS transport on dial-peer

dial-peer voice 1 voip
description to CUCM Sub – Secure Signaling
preference 1
destination-pattern 418110.... 2. Enable TLS transport globally
session protocol sipv2
session target ipv4:14.50.248.101 or voice service voip
session transport tcp tls sip
incoming called-number 9.T session transport tcp tls
dtmf-relay rtp-nte
codec g711ulaw
no vad
IOS Configuration – crypto signaling
Enabling Secure Signaling
Associate CUBE trustpoint with voice process

sip-ua
crypto signaling remote-addr 14.50.248.100 255.255.255.255 trustpoint caServer
Base command Peer IP address/network association trustpoint cipher selection

association
crypto signaling default trustpoint <name>
<enter> (default)
ecdsa-cipher
strict-cipher
crypto signaling remote-addr <ip.address> <mask> trustpoint <name>
<enter> (default)
ecdsa-cipher
strict-cipher
IOS Configuration – crypto signaling
Enabling Secure Signaling and Server Identity Validation 16.11.1+
Associate CUBE trustpoint with voice process and enable Common Name (CN)/Subject Alternate Name (SAN) validation
sip-ua
crypto signaling remote-addr 14.50.248.100 255.255.255.255 trustpoint caServer cn-san-validate
• Validates that the Common Name (CN) Configure session target with SAN/CN
or Subject Alternate Name (SAN) in the
server certificate matches the dial-peer
description to CUCM Sub – Secure Signaling
session target configuration. preference 1
• If both SAN and CN are in the server destination-pattern 418110....
certificate, SAN takes precedence and
ipv4:14.50.248.101
session target dns:cucmsub.cisco-uc.com
session target must match SAN entry. session transport tcp tls
incoming called-number 9.T
dtmf-relay rtp-nte
codec g711ulaw
no vad
IOS Signaling Cipher Suites
Configuration Cipher Suites
TLS_RSA_WITH_RC4_128_MD5
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA1
Default Cipher TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA1
Strict Cipher
ECDSA Cipher
Configuring the SIP Trunk Security Profile
CUCM SIP Trunk Security Profile Configuration
Ensure Subject-name matches between IOS and CUCM

SIP trunk
crypto pki trustpoint caServer
enrollment terminal
subject-name CN=ISR4KCUBE.cisco-uc.com
revocation-check none
rsakeypair rsakey
CUCM Configuring the IOS SIP Trunk
Are Our Calls Completely Secure Now?
Signaling
Media
Implementing
Secure Media
What’s Secure RTP?
As per RFC 3711, SRTP is “a profile of the Real-time Transport

Protocol (RTP), which can provide confidentiality, message
authentication, and replay protection to the RTP traffic“
a=crypto:<tag> <crypto-suite> <key-params> [<session-params>]
SDP for RTP SDP for SRTP

m=audio 8256 RTP/AVP 0 m=audio 8264 RTP/SAVP 0
c=IN IP4 14.50.248.31 c=IN IP4 14.50.248.31
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
a=crypto:1 AES_CM_128_HMAC_SHA1_32
inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0
Secure RTP – SDP Security Descriptions (SDES)
Breaking down the crypto m-line
a=crypto:1 AES_CM_128_HMAC_SHA1_32 inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0
Tag Crypto Suite Transport Method Master Key

and Master
A decimal number Describes the encryption and authentication transforms Defines the method of Salt
used to identify the to be used in the sRTP media stream. transporting the key. Inline
particular crypto • AES_CM – AES Counter mode cipher meaning that the key and salt A 240bit base64
attribute. will follow immediately. Other encoded string
• 128 – The length of the encryption key (bits) methods (such as a URI) are containing a
• HMAC_SHA1 – (Hashed) Message not defined in rfc4568 concatenation of
Authentication Code using the SHA1 the master key
algorithm and master salt
Secure RTP – SDP Security Descriptions (SDES)
Additional Parameters
SRTP Session Parameters: Allows further session flexibility, such as unencrypted/unauthenticated RTCP
a=crypto:1 AES_CM_128_HMAC_SHA1_32 inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0 UNENCRYPTED_SRTCP
Key Lifetime and Master Key Index: Not generally used in collaboration
a=crypto:1 AES_CM_128_HMAC_SHA1_32 inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0|2^20|1:32
Secure RTP – SDP Signaling for DTLS
DTLS SDP Attributes Setup
Indicates DTLS support. The
a=setup:passive value is negotiated between
both endpoints in the offer
a=connection:new
answer model.
a=fingerprint:SHA-1 4A:AD:B9:B1:3F:82:18:3B:54:02:12:DF:3E:5D:49:6B:19:E5:7C:AB
Connection
Offer Answer
Setup Values Used to indicate whether the
active* • passive* offer/answer exchange is
active: The endpoint will initiate an outgoing *default using an existing connection
• holdconn
connection.
passive: The endpoint will accept an incoming passive • active
connection. • holdconn Fingerprint
actpass: The endpoint is willing to accept an
incoming connection or to initiate an outgoing actpass • active A cryptographic hash of the
connection. • passive x509 certificate to be used in
• holdconn the key exchange and the
holdconn: The endpoint does not want the hashing algorithm used
connection to be established for the time being.
holdconn • holdconn
Secure RTP
SRTP Packet Format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<+
|V=2|P|X| CC |M| PT | sequence number | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| timestamp | | RTP Headers
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| synchronization source (SSRC) identifier | |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | RTP Headers are authenticated, but
|
|
contributing source (CSRC) identifiers
....
| |
| |
not encrypted
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| RTP extension (OPTIONAL) | |
+>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | payload ... | |
| | +-------------------------------+ |
| | | RTP padding | RTP pad count | |
+>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<+
Encrypted Portion
| ~ SRTP MKI (OPTIONAL) ~ |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | The payload, MKI (if present), and
| : authentication tag (RECOMMENDED) : |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
authentication tag are encrypted
| | and authenticated
+- Encrypted Portion* Authenticated Portion ---+
As per RFC 3711, SRTP is “a profile of the Real-time Transport
Protocol (RTP), which can provide confidentiality, message
authentication, and replay protection to the RTP traffic“
Confidentiality
Confidentiality is achieved by encrypting the RTP payload using AES ciphers
Message Authentication
HMAC provides the authentication mechanism by reducing the packet contents to a 160 bit number using SHA1
Replay Protection
Replay attacks assume that the attacker has access to the SRTP stream and can send duplicated packets to the receiver
in order to waste resources. The SRTP cryptographic context keeps track of the number of packets signed by the master
key in a sliding window that is tolerant of loss and out of order packet delivery (up to 2^15 packets)
MRA CUCM Phone Device Security Profile
MRA
The UCM Device Security Profile Name must be in the

FQDN format with the enterprise domain.
This same FQDN must be present as a Subject

Alternate Name (SAN) in the Expressway-C’s server
certificate.
Use of a Universal Device Template is recommended to

avoid having to regenerate the server certificate when
new endpoint models are added.
Set the Device Security Mode to Encrypted

Secure Media – Expressway Configuration
B2B
Configuration>Zones>Zones>Edit Configuration Settings
zone>SIP
Force encrypted – Attempt to encrypt any unencrypted
RTP, fail the call if encryption is not available.
Force unencrypted – Decrypt any encrypted media. Force
unencrypted RTP
Best effort – Use encryption if available, otherwise fall
back to unencrypted media.
Auto – No specific media encryption policy is applied by
the VCS. Media encryption is purely dependent on
endpoint requests.
Note
Any setting other than Auto will force the call media to traverse the
VCS and thus consume a traversal call license.
Security modes explained
Global
Mode Description Expressway UCM Configuration
Configuration
Mandatory Media encryption is required. Force encrypted Mixed Mode On
Unencrypted calls should always fail Phones Security Profile(s)
no fallback is allowed. • Device Security Mode:
Encrypted
• Transport Type: TLS
Trunk Settings
SIP Trunk Security Profile
• Incoming Transport Type: TLS
• Outgoing Transport Type: TLS
SRTP Allowed Checked
Best Effort Calls that can be encrypted are Best Effort Mixed Mode On
encrypted. Phone and Trunk security settings as
If encryption cannot be established required
Calls should fall back to unencrypted Normalization script applied
media if encryption can not be
established
None No encryption in use Mixed Mode Off
Secure Media – IOS Configuration
Enabling Secure Media Pre-16.5.1
1. Enables SRTP
1. Enable SRTP on Dial-peer
2. Configure SRTP cipher suite support
description to CUCM Sub – Secure Signaling In 15.4(1), support for sha1-80
preference 1 AES_CM_128_HMAC_SHA1_80 was
destination-pattern 418110.... added
session target ipv4:14.50.248.103 3. (Optional) Configure NGE cipher
srtp suite support.
voice-class sip srtp-auth sha1-80 sha1-32
Introduced in 15.6(1)
voice-class sip srtp pass-thru
Allows for unsupported SRTP cipher
or suites to be negotiated,
1. Enable SRTP Globally • AEAD_AES_128_GCM
voice service voip • AEAD_AES_256_GCM
srtp • AEAD_AES_128_CCM
srtp pass-thru • AEAD_AES_256_CCM
sip
srtp-auth sha1-80 sha1-32 CUBE will pass-thru offered cipher
suites and keys from one call-leg to the
other call-leg.
Secure Media – IOS-XE Configuration
Enabling Secure Media 16.5.1+
Define SRTP crypto suite support 2. Enable SRTP and apply voice-class crypto-suite on Dial-peer
voice class srtp-crypto 1 dial-peer voice 1 voip
crypto 1 AEAD_AES_256_GCM description to CUCM Sub – Secure Signaling
crypto 2 AEAD_AES_128_GCM
preference 1
crypto 3 AES_CM_128_HMAC_SHA1_80
crypto 4 AES_CM_128_HMAC_SHA1_32 destination-pattern 418110....
session target ipv4:14.50.248.103
1. Create a voice class to define srtp
supported SRTP cipher suites. voice-class sip srtp-crypto 1
2. Apply the defined voice-class either or

under the dial-peer or globally
2. Enable SRTP and apply voice-class crypto-suite Globally
3. Enables SRTP voice service voip

srtp
sip
srtp-crypto 1
IOS Cipher Suite Support for Media
Version Cipher Suites

Prior to 15.4(1)T/S AES_CM_128_HMAC_SHA1_32 (default)
Starting with 15.4(1)T/S AES_CM_128_HMAC_SHA1_80
AEAD_AES_128_GCM
AEAD_AES_256_GCM
AEAD_AES_128_CCM
Starting with 15.6(1)T/S*
AEAD_AES_256_CCM
* With SRTP Passthru feature

AEAD_AES_128_GCM
AEAD_AES_256_GCM
Starting with 16.5.1*
* Native support only in IOS-XE
CUCM Configuring the IOS SIP Trunk
Are Our Calls Completely Secure Now?
Signaling
Media
Secure to Non-
Secure
Interoperability
SRTP to RTP
Interworking
CUBE-based SRTP-RTP Interworking
SIP
ISR 4000-series and CSR1000v ISR G2 - 2900/3900-series routers

• Uses built-in crypto-engine • DSP required
• No additional configuration required • Leverages DSPfarm configuration Signaling
Media
ISR G2 - DSPfarm Configuration
1. Enable DSPfarm feature on DSP

Specifies the type of
ISRG2CUBE(config)#voice-card 0 DSPfarm
ISRG2CUBE(config-voicecard)#dsp service dspfarm
Defines the supported
2. Configure DSPfarm profile codecs for audio
dspfarm profile 1 transcode security
codec g729abr8 Defines the number of total
codec g729ar8 sessions supported.
codec g711alaw
codec g711ulaw Depends on DSPs available
maximum sessions 10
associate application CUBE Associates the profile with
the Local Transcoding
no shutdown Interface (LTI) on CUBE
ISR G2 - DSPfarm Verification
show dspfarm profile
Profile ID = 1, Service = TRANSCODING, Resource ID = 1
Profile Description :
Profile Service Mode : secure
Profile Admin State : UP
Profile Operation State : ACTIVE
Application : CUBE Status : ASSOCIATED
Resource Provider : FLEX_DSPRM Status : UP
Total Number of Resources Configured : 10
Total Number of Resources Available : 10
Total Number of Resources Out of Service : 0
Total Number of Resources Active : 0
Codec Configuration: num_of_codecs:4
Codec : g711ulaw, Maximum Packetization Period : 30
Codec : g711alaw, Maximum Packetization Period : 30
Codec : g729ar8, Maximum Packetization Period : 60
Codec : g729abr8, Maximum Packetization Period : 60
TLS : ENABLED
SRTP Fallback
Secure Call Establishment
Secure
CUBE CUCM Phone
Invite
SDP: RTP/SAVP
200 OK
SDP: RTP/SAVP
SRTP
Secure Call Establishment Failure
Unsecure
CUBE CUCM Phone
Invite
SDP: RTP/SAVP
488 Not Acceptable Media
SRTP Fallback Call Establishment
Unsecure
CUBE CUCM Phone
Invite
Supported: x-cisco-srtp-fallback
SDP: RTP/SAVP
200 OK
Supported: x-cisco-srtp-fallback
SDP: RTP/AVP
RTP
SRTP Fallback - IOS Configuration
Enable on Dial-peer
description to CUCM – Secure Signaling Enable Globally
preference 1
destination-pattern 418110....
or
voice service voip
srtp fallback
sip
srtp fallback
srtp negotiate cisco
voice-class sip srtp negotiate cisco
incoming called-number 9.T
dtmf-relay rtp-nte
codec g711ulaw
no vad
Secure Media – Expressway and UCM Interop
Unified Communications
Mode Description Expressway Behavior
Manager Behavior
Media encryption is not allowed. m=RTP/AVP media description m=RTP/AVP media description
None Calls that require encryption should No crypto attributes present in No crypto attributes present in
fail. SDP SDP
Media encryption is required m=RTP/SAVP media description m=RTP/SAVP media description

Mandatory Unencrypted calls should always fail
a=crypto lines in the SDP a=crypto lines in the SDP
No fallback is allowed
Calls that can be encrypted are m=RTP/AVP In the Offer’s

encrypted. media description m=RTP/SAVP media description
Best Effort If encryption cannot be established, m=RTP/SAVP In the Answer’s a=crypto lines in the SDP
calls attempt to fall back to media description x-cisco-srtp-fallback header
unencrypted media. a=crypto lines in the SDP
Warning
Expressway will not send crypto keys without a secure signaling transport (TLS)
UCM Does not have this limitation and will send crypto keys in clear text over non secure (TCP, UDP) transport types
Secure Media – vcs-interop Lua Script
B2B
Direction Conditions Behavior Applies to
Convert media descriptions to
Inbound to m=RTP/AVP media description RTP/SAVP All requests containing
UCM a=crypto lines in the SDP Add x-cisco-srtp-fallback SDP
header
m=RTP/SAVP media description

Outbound to a=crypto lines in the SDP, or Convert media descriptions to
INVITEs only
Expressway both of the a=setup and RTP/AVP
a=fingerprint attributes
Modify the RHS of the SIP URI to

the Top Level Domain on the any All requests, including
Outbound to
All Requests and Responses of the following headers (if INVITEs with modified
Expressway
present): From, Remote-Party- media descriptions
Id, P-Asserted-Identity
Troubleshooting
Troubleshooting
Methodology
Troubleshooting
Methodology • When was it first reported?
• How often does it occur?
Identify and
• What’s the impact ?
Quantify the
• Individual user
Problem • Site
• Entire deployment
• What’s changed?
• Software versions?
Gather
• Call flow?
Information • Network topology?
• Debugs/Traces/Packet Captures
Analyze
Gathered
Information
and Narrow
the Scope
Narrowing the Scope
Is it a Signaling or Media problem?
Signaling Path
• SIP
• TLS
• Media
Negotiation
Media Path
• Media Encryption
• Media
Decryption
• Voice Quality
Identifying the Problem
• Troubleshoot and resolve
Does an unsecure call work in the same call flow? before implementing secure
configuration.
Does an unsecure call work in the same call flow?

• Indicates the TCP or TLS
Yes
connection failed to establish
Does the calling party hear ringback?
between CUCM, Expressway
and/or CUBE.
• Focus on certificate exchange
between client and server.

Yes

Yes • Possible issue with media
negotiation.
Does the call connect?
• Check SDP for codec/dtmf
advertisement

Yes

Yes

Yes • Problem with establishing
secure media
Does the lock icon appear on the phone?
• Check SDP answer and verify
secure media is being
advertised.

Yes

Yes

Yes
Does the lock icon appear on the phone?

Yes
Secure call successfully negotiated!
Troubleshooting
Tools
IOS Command Output dial-peer matched
show call [active|history] voice brief Call direction
Answer = received call
Call leg identifier Originate = placed call
ISR4KCUBE#show call active voice brief
2 : 11 572330360ms.1 +1380 pid:31 Answer 1052061 active

dur 00:00:10 tx:493/98600 rx:492/98400 dscp:0 media:0 audio tos:0xB8 video tos:0x88
IP 14.50.212.59:25482 SRTP: off rtt:0ms pl:0/0ms lost:0/0/0 delay:0/0/0ms g711ulaw
long duration call detected:n long duration call duration:n/a timestamp:n/a
LostPacketRate:0.00 OutOfOrderRate:0.00
Call identifier Calling Number

Call leg identifier Called Number
2 : 13 572330370ms.1 +1360 pid:21 Originate 4181101002 active

IP 14.50.248.150:17102 SRTP: on rtt:0ms pl:0/0ms lost:0/0/0 delay:0/0/0ms g711ulaw
IOS Command Output
show call [active|history] voice brief
Call duration
ISR4KCUBE#show call active voice brief
2 : 11 572330360ms.1 +1380 pid:31 Answer 1052061 active

IP 14.50.212.59:25482 SRTP: off rtt:0ms pl:0/0ms lost:0/0/0 delay:0/0/0ms g711ulaw
Peer IP address and RTP SRTP status Voice quality metrics

port
Tx/Rx packet counter
2 : 13 572330370ms.1 +1360 pid:21 Originate 4181101002 active

IP 14.50.248.150:17102 SRTP: on rtt:0ms pl:0/0ms lost:0/0/0 delay:0/0/0ms
IOS Command Output
show sip-ua calls
ISR4KCUBE#show sip-ua call

Call 1
SIP Call ID : [email protected] Unique SIP signaling
Calling Number : 1052061 identifier
Called Number : 4181101002
Called URI : sip:[email protected]:5061
Source IP Address (Sig ): 14.50.248.31
Destn SIP Req Addr:Port : [14.50.248.101]:5061 Call leg identifier
Destn SIP Resp Addr:Port: [14.50.248.101]:5061
Destination Name : 14.50.248.101
Media Stream 1
State of the stream : STREAM_ACTIVE
Stream Call ID : 71 IP address of media
Stream Type : voice+dtmf (1) peer
Negotiated Codec : g711ulaw (160 bytes)
Negotiated Dtmf-relay : rtp-nte
Media Source IP Addr:Port: [14.50.248.31]:8128
Negotiated SRTP
Media Dest IP Addr:Port : [14.50.248.150]:24696
Local Crypto Suite : AES_CM_128_HMAC_SHA1_32
media crypto suites
Remote Crypto Suite : AES_CM_128_HMAC_SHA1_32
Diagnostic Logs
Anatomy • Available through the Expressway Web UI (Maintenance > Diagnostics >
Diagnostic Logging)
If packet capture is needed,

be sure to select
Click to Download
the logs
Click to Start
Click to Stop after the
issue is reproduced
Diagnostic Logs
What do they capture?
• SIP/H.323 traffic
• MRA Reverse Proxy Traffic
• TCP, SSL, and DNS traffic
• Application logic, and much more
2017-09-19T14:01:46.462-04:00 amer-expressway01 tvcs: UTCTime="2017-09-19 18:01:46,461"
Module="network.tcp" Level="DEBUG": Src-ip="146.20.193.73" Src-port="40342" Dst-ip="172.16.2.2" Dst-
port="5062" Detail="TCP Connecting“

Module="network.tcp" Level="DEBUG": Src-ip="146.20.193.73" Src-port="40342" Dst-ip="172.16.2.2" Dst-
port="5062" Detail="TCP Connection Established"

Module="developer.ssl" Level="INFO" CodeLocation="ppcmains/ssl/ttssl/ttssl_openssl.cpp(1974)"
Method="::ttssl_continueHandshake" Thread="0x7f420863b700": Detail="Handshake in progress"
Reason="want read/write"
Cisco TAC Tool: Collaboration Solution Analyzer
https://cway.cisco.com/tools/CollaborationSolutionsAnalyzer/
Openssl
OpenSSL is an open-source implementation of the SSL and TLS

protocols. It is widely used in Internet web servers, serving a majority of
all web sites. The core library, written in the C programming language,
implements basic cryptographic functions and provides various utility
functions.
• s_client – A generic SSL/TLS client that can complete a TLS

connection to any remote sever:port combination
• x509 – A collection of utilities for reading, creating, and
verifying x509 certificates
• errstr - Error Number to Error String Conversion.
• ciphers - Cipher Suite Description Determination.
openssl s_client
panholt@whiskeyjack:~$ openssl s_client -connect amer-expressway01.ciscotac.net.:5061
CONNECTED(00000003)
depth=2 C=US, ST=Arizona, L=Scottsdale, O="GoDaddy.com, Inc.", CN=Go Daddy Root Certificate Authority - G2
verify return:1
depth=1 C=US, ST=Arizona, L=Scottsdale, O="GoDaddy.com, Inc.", OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
verify return:1
depth=0 OU=Domain Control Validated, CN=amer-expressway01.ciscotac.net Socket is connected and the peers server
verify return:1 cert is verified against local CA trust store
---
Certificate chain
0 s:/OU=Domain Control Validated/CN=amer-expressway01.ciscotac.net
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2 Server cert’s trust chain is
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2 displayed
---
Server certificate
-----BEGIN CERTIFICATE----- PEM encoded server cert is displayed (use -showcerts to display all the certs in the
MIIGrDCCBZSg … trust chain)
subject=/OU=Domain Control Validated/CN=amer-expressway01.ciscotac.net
issuer=/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
---
No client certificate CA names sent
Peer signing digest: SHA512 SSL/TLS Handshake is
Server Temp Key: ECDH, P-521, 521 bits complete
---
SSL handshake has read 4901 bytes and written 499 bytes
---
openssl s_client (continued)
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: A6A286C01CAE78D6F4A2F0A10E413AD121578DD01CFD1160B776E73F2E69A130
Session-ID-ctx:
Master-Key: 3EF487C8C18BCA4D10A16D4E25DB31F68238425B80E11CCC28697B0E45047FED5CD8EC3F2D885BF6D68B560280B953F6
Key-Arg : None
SSL/TLS Session PSK identity: None
Info PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 44 81 19 97 12 96 ee 0c-68 25 37 e6 ae 92 43 41 D.......h%7...CA
0010 - fc 75 c0 38 4f 35 d7 af-55 c9 a7 d7 76 75 64 a2 .u.8O5..U...vud.
0020 - ac 27 0a 5a c9 f8 80 31-4c 39 15 9f 7e e2 23 c9 .'.Z...1L9..~.#.
0030 - 36 fa 86 ce 0c 52 67 90-e2 7d ad 5b 94 93 96 89 6....Rg..}.[....
0040 - e6 be b2 d6 ac ee d3 36-d0 62 25 8d 58 93 c7 d0 .......6.b%.X...
0050 - d1 ab 5d 43 e3 59 9d bb-98 8d c2 ef 82 ac 9a 26 ..]C.Y.........&
0060 - 0c 0c bf ba 4f 88 49 55-e3 ef a6 12 d4 5d df bb ....O.IU.....]..
0070 - b7 38 3f 6d b1 b4 7e 2a-d1 c5 5a 05 0d c0 08 af .8?m..~*..Z.....
0080 - e7 3a a8 24 3a a1 12 d2-e1 d7 f0 e0 46 44 95 13 .:.$:.......FD..
0090 - 04 63 81 6a 97 7a 6f 4c-37 39 68 a7 12 00 cb 5c .c.j.zoL79h....\
Start Time: 1525379195

Timeout : 300 (sec) Session statistics and verification
---
Verify return code: 0 (ok) results
^C
openssl s_client (other usage)
CApath arg - PEM format directory of CA's
CAfile arg - PEM format file of CA's
Change the key/cert cert arg - certificate file to use, PEM format assumed
and/or CA trust file certform arg - certificate format (PEM or DER) PEM default
key arg - Private key file to use, in cert file if not specified but cert file is.
starttls prot - use the STARTTLS command before starting TLS for those protocols that support it,
where 'prot' defines which one to assume. Currently, only "smtp", "pop3", "imap", "ftp" and
"xmpp“ are supported.
Test different protocols ssl2 - just use SSLv2

or cipher suites ssl3 - just use SSLv3
tls1_2 - just use TLSv1.2
tls1_1 - just use TLSv1.1
tls1 - just use TLSv1
no_tls1_2/-no_tls1_1/-no_tls1/-no_ssl3/-no_ssl2 - turn off that protocol
mtu - set the link layer MTU

Various debugging and showcerts - show all certificates in the chain
extra output debug - extra output
msg - Show protocol messages
TLS Extension testing tlsextdebug - hex dump of all TLS extensions received
servername host - Set TLS extension servername in ClientHello
and debugging (SNI, alpn arg - enable ALPN extension, named protocols supported (comma-separated list)
ALPN)
in – input file (default stdin)
openssl x509 inform – input file format (DER, NET or PEM default: PEM)
noout – no certificate output
text - print the certificate in text form
panholt@whiskeyjack:/tmp$ openssl x509 -in server.pem -inform pem -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 8041775998364328096 (0x6f9a20a9d1c4a0a0)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com, Inc.,OU=http://certs.godaddy.com/repository/,CN=Go Daddy Secure Certificate Authority-G2
Validity
Not Before: May 31 14:48:01 2017 GMT
Not After : May 31 14:48:01 2020 GMT
Subject: OU=Domain Control Validated, CN=amer-expressway01.ciscotac.net
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (4096 bit)
Modulus:
00:8e:26:ce:19:f6:3e:a4:33:
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 CRL Distribution Points:
{snip}
X509v3 Subject Alternative Name:
DNS:amer-expressway01.ciscotac.net, DNS:www.amer-expressway01.ciscotac.net, DNS:rtp.ciscotac.net, DNS:ciscotac.net, DNS:amer-
expressway01.rtp.ciscotac.net
openssl x509 (other uses)
inform arg - input format - default PEM (one of DER, NET or PEM)
outform arg - output format - default PEM (one of DER, NET or PEM)
keyform arg - private key format - default PEM
in arg - input file - default stdin
out arg - output file - default stdout
subject - print subject DN
issuer - print issuer DN
email - print email address(es)
startdate - notBefore field
enddate - notAfter field
dates - both Before and After dates
text - print the certificate in text form
x509toreq - output a certification request object
Data to Collect
Phone Registration
Type of Problem CUCM Expressway Other
LSC Installation CAPF traces Start Log Phone console
Collection logs
Secure Phone Registration CCM traces
TFTP traces Start Log Packet capture

CTL Installation Collection
show ctl
Media Establishment
Type of
IOS Debugs IOS Command Output CUCM Expressway Other
Problem
debug voip ipipgw show dspfarm profile active
SRTP-RTP debug voip hpi show voip rtp connection Start Log
Interworking
Collection
ISR-G2 only error
debug ccsip
info
show call active|history voice brief Packet
CCM traces Start Log
Media debug ccsip media capture
show sip-ua call Collection
Data to Collect
Signaling and Call Establishment
Type of Problem IOS Debugs IOS command output CUCM Expressway Other
TCP connection transaction

debug ip tcp show tcp brief
failure packet
messages
transactions
debug crypto
validation
pki
TLS connection api
show sip-ua connection tcp tls detail
failure callback Start Log Packet
CCM traces
errors Collection capture
debug ssl
msg
openssl
states
message
SIP call
debug ccsip error show call active|history voice brief
establishment
transport
Call Routing debug voip ccapi inout show dial-peer voice summary
Troubleshooting
Scenarios
Scenario 1:
Call Setup Failure
Problem Description
Initial Problem Description
Outbound calls work but inbound calls fail.

Secure voice between CUCM and SIP-SIP CUBE on an ISR4451-X.
Unsecure between CUBE and PSTN.
SIP SIP
TCP/RTP TLS/SRTP
Outbound call
Inbound call
IOS Command Output after Outbound Call
ISR4KCUBE#show sip-ua connection tcp tls detail

Total active connections : 1 Active connections
No. of send failures : 0 counter
No. of remote closures : 0
No. of conn. failures : 0 Connection failure
No. of inactive conn. ageouts : 0 counters
TLS client handshake failures : 0
TLS server handshake failures : 0
---------Printing Detailed Connection Report--------- IP address of

signaling peer
Remote-Agent:14.50.248.103, Connections-Count:2
Remote-Port Conn-Id Conn-State WriteQ-Size Local-Address TLS-Version
=========== ======= =========== =========== ============= ===========
54715 122 Established 0 - TLSv1.2
IOS Command Output after Inbound Call Attempt
ISR4KCUBE#show sip-ua connection tcp tls detail

Total active connections : 1
No. of send failures : 0 Increase in failure
No. of remote closures : 0 counters
No. of conn. failures : 1
No. of inactive conn. ageouts : 0
TLS client handshake failures : 1
TLS server handshake failures : 0
---------Printing Detailed Connection Report---------
Remote-Agent:14.50.248.101, Connections-Count:2
Remote-Port Conn-Id Conn-State WriteQ-Size Local-Address TLS-Version
=========== ======= =========== =========== ============= ===========
54715 122 Established 0 - TLSv1.2
debug ccsip message
Troubleshooting
Inbound call
14.50.211.3 14.50.211.41
1
INVITE
Jun 22 00:16:55.003: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg: Audio codec
Received: G711ulaw
INVITE sip:[email protected]:5060 SIP/2.0
Via: SIP/2.0/TCP 14.50.211.31:5060;branch=z9hG4bKfe6e7c6463
DTMF-relay RFC2833
From: <sip:[email protected]>;tag=44900~bb0e39ab-e0bf-401d
To: <sip:[email protected]>
Date: Thu, 22 Jun 2017 00:16:55 GMT v=0
Call-ID: [email protected] o=CiscoSystemsCCM-SIP 8576 1 IN IP4 14.50.211.31
s=SIP Call
Unique call identifier c=IN IP4 14.50.212.51
Remote media IP m=audio 17714 RTP/AVP 0 101
address and RTP port a=rtpmap:0 PCMU/8000
debug voip ccapi inout
Troubleshooting
Inbound call dial-peer voice 3 voip
description inbound PSTN unsecure
14.50.211.31 14.50.211.182 translation-profile incoming switch21
session transport tcp
incoming called-number 1...
INVITE dtmf-relay rtp-nte
codec g711ulaw
Jun 22 00:16:55.021: //-1/17C53B000000/CCAPI/cc_api_call_setup_ind_common:
no vad
Interface=0x7F12BB8C3E80, Call Info(
Calling Number=1052062,(Calling Name=)(TON=Unknown, NPI=Unknown, dial-peer voice 2 voip
Screening=User,
description to CUCM Pub – Secure
Called Number=1001(TON=Unknown, NPI=Unknown),
preference 1
Incoming Dial-peer=3, Progress Indication=NULL(0), Calling IE Present=TRUE,
destination-pattern 1...
Source Trkgrp Route Label=, Target Trkgrp Route Label=, CLID Transparent=FALSE), Call sipv2
session protocol
Jun 22 00:16:55.025: //33288/17C53B000000/CCAPI/ccCallSetupRequest:
session transport tcp tls
Calling Number=1052062(TON=Unknown, NPI=Unknown, Screening=User, Passed,called-number
incoming Called 8.T
Number=1001(TON=Unknown, NPI=Unknown), dtmf-relay rtp-nte
Account Number=1052062, Final Destination Flag=TRUE, srtp
Guid=17C53B00-0001-0000-003A1FD3320E, Outgoing Dial-peer=2 codec g711ulaw
no vad
debug ccsip transport
Troubleshooting debug ip tcp transaction
debug crypto pki
Inbound call
14.50.211.31 14.50.248.100
TCP
TCP SYN
Established
Sent
Jun 22 00:16:55.053: //-1/xxxxxxxxxxxx/SIP/Transport/sipTransportPostRequestConnection:

Posting TLS conn create request for addr=14.50.248.100, port=5061, context=0x7F12BB4CB6A8
Jun 22 00:16:55.055: TCP: sending SYN, seq 1205252257, ack 0
Jun 22 00:16:55.056: TCP0: Connection to 14.50.248.100:5061, advertising MSS 1460
Jun 22 00:16:55.056: TCP0: state was CLOSED -> SYNSENT [32347 -> 14.50.248.100(5061)]
Jun 22 00:16:55.057: TCP0: state was SYNSENT -> ESTAB [32347 -> 14.50.248.100(5061)]
Jun 22 00:16:55.059: opssl_SetPKIInfo entry
Jun 22 00:16:55.059: CRYPTO_PKI: (A0050) Session started - identity selected (caServer)
Jun 22 00:16:55.059: CRYPTO_PKI(Cert Lookup) issuer="cn=BRKUCC_CA" serial number=
10 FC B1 6C 00 00 00 00 00 04
14.50.248.31 14.50.248.100
Troubleshooting
TCP Established
Client Hello
Server Hello
Server
ClientHello Certificate
Jun 22 00:16:55.062: Handshake start: before/connect initialization

Jun 22 00:16:55.062: SSL_connect:before/connect initialization
Jun 22 00:16:55.063: >>> TLS 1.2 Handshake [length 0072], ClientHello
Jun 22 00:16:55.063: 01 00 00 6E 03 03 9D 9A D4 49 99 E2 98 90 A0 2A
ServerHello
Jun 22 00:16:55.073: <<< TLS 1.2 Handshake [length 003E], ServerHello
Jun 22 00:16:55.073: 02 00 00 3A 03 03 BF 47 15 B3 D1 DD 5F 01 A4 99
Server Certificate
Jun 22 00:16:55.075: <<< TLS 1.2 Handshake [length 03C3], Certificate
Jun 22 00:16:55.075: 0B 00 03 BF 00 03 BC 00 03 B9 30 82 03 B5 30 82
14.50.248.31 14.50.248.100
Troubleshooting
TCP Established
Client Hello
Server Hello
Server
Certificate
Server Certificate analysis

Jun 22 00:16:55.101: CRYPTO_PKI: Added x509 peer certificate - (953) bytes
Jun 22 00:16:55.102: CRYPTO_PKI(Cert Lookup) issuer="l=RTP,st=NC,cn=cucmpub.cisco-
uc.com,ou=TAC,o=Cisco,c=US" serial number= 6C 47 C6 EC 38 9B D1 23 75 7F 04 65 63 AE AB 09
Jun 22 00:16:55.102: CRYPTO_PKI: Cert record not found, returning E_NOT_FOUND

Jun 22 00:16:55.102: CRYPTO_PKI: (A0051) No suitable trustpoints found
14.50.248.31 14.50.248.100
Troubleshooting
TCP Established
Client Hello
Server Hello
Server
Certificate
TLS Alert
TLS Alert
Jun 22 00:16:55.103: >>> TLS 1.2 Alert [length 0002], fatal bad_certificate
Jun 22 00:16:55.103: 02 2A
Jun 22 00:16:55.112: //33288/17C53B000000/CCAPI/ccCallDisconnect:

Cause Value=38, Tag=0x0, Call Entry(Previous Disconnect Cause=0, Disconnect Cause=0)
Jun 22 00:16:55.112: //33288/17C53B000000/CCAPI/ccCallDisconnect:
Cause Value=38, Call Entry(Responsed=TRUE, Cause Value=38)
14.50.248.31 14.50.248.100
Verifying Certificates
ISR4KCUBE#sh crypto pki certificate
CA Certificate
Status: Available
Certificate Serial Number (hex):
3A0E15FE9124C0B040801DC535DBA5C3
Certificate Usage: Signature
Issuer:
cn=BRKUCC_CA
Subject:
cn=BRKUCC_CA
Validity Date:
start date: 17:44:42 EDT May 29 2017
end date: 17:54:41 EDT May 29 2022
14.50.248.31 14.50.248.100
Verifying Certificates
Jun 22 00:16:55.101: CRYPTO_PKI: Added x509 peer certificate - (953) bytes

Jun 22 00:16:55.102: CRYPTO_PKI(Cert Lookup) issuer="l=RTP,st=NC,cn=cucmpub.cisco-
uc.com,ou=TAC,o=Cisco,c=US" serial number= 6C 47 C6 EC 38 9B D1 23 75 7F 04 65 63 AE AB 09
Summary
• CUBE only has a dial-peer pointing to CUCM Pub for inbound

PSTN calls.
• CA-signed identity certificate was not imported into CUCM Pub
Root Cause • Working outbound calls were sent by the CUCM Subscriber.
• Import CA-signed identity certificate to CUCM Pub and restart

CallManager service.
Solution • Create dial-peer pointing to CUCM Subscriber for redundancy.
Scenario 2:
Secure Mobile and
Remote Access
Call Issues
Problem Description
Initial Problem Description
We secured our existing and working MRA solution for end to end security. Now calls
establish, but devices cannot end calls and users are reporting call drops after
around 15 minutes
SIP TCP
TLS SIP TCP
SIP TLS SIP TLS SIP TLS
RTP
SRTP SRTP
RTP SRTP SRTP
Exp-C Exp-E
Verify Call is Encrypted
CSA Ladder Diagram
Expressway-E Expressway-C CUCM
CSA Analysis
Packet Capture analysis
Expressway-C Trust Store
Subject: CN=rtp12-tpdmz-118-ucmpub.rtp.ciscotac.net
First Entry Serial Number: 7c:dc:3c:62:5c:f7:27:e9:7e:01:9a:41:98:0b:15:48
Second Entry Subject: CN=rtp12-tpdmz-118-ucmpub.rtp.ciscotac.net

Serial Number: 7f:65:f7:b0:8f:ae:96:7b:c3:a8:a4:a2:2c:0f:d0:f5
Verify CUCM Certificates
Matches first entry in Expressway Trusted CA Matches second entry in Expressway Trusted CA
Summary
• Customer is using self signed certificates on the CUCM, and has

uploaded both the Tomcat.pem and Callmanager.pem to the
Expressway-C’s Trust Store.
• Both of these certificates share the same common name (CN).
• Openssl, within the Expressway, parses the trust store from top-down
looking for a match on the CN, and finds the Tomcat.pem entry.
• The Callmanager.pem is used for SIP and Media thus the Tomcat.pem
certificate is unacceptable for this scenario and the TLS connection
fails to establish.
Solutions
1. Use CA-signed
certificates, generate a
new CSR on the CUCM
and have a trusted CA
sign them. Configuration > Unified Communications > Unified CM servers > Edit
(Recommended)
2. Delete the Tomcat.pem
certificate from the
Expressway-C’s trust
store and turn TLS
Verify Mode to Off for
the UCM Server.
(Workaround)
Additional UC Security Sessions
BRKCOL-3501: Implementing and Troubleshooting Secure IP Phones and Endpoints

• Watch the video from CLUS 2019 online at www.ciscolive.com
BRKCOL-2014: Introduction to Cisco UC Security

• Tomorrow, Wednesday the 29th at 8:30am
LABCCT-2984 : Secure Voice Communication - Configuration and Troubleshooting

• Walk-In Lab. Available all week
Complete your
online session
survey • Please complete your session survey
after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events
Mobile App or by logging in to the Content
Catalog on ciscolive.com/emea.
Cisco Live sessions will be available for viewing on

demand after the event at ciscolive.com.
Continue your education
Demos in the
Walk-in labs
Cisco campus
Meet the engineer

Related sessions
1:1 meetings
Thank you

Brkcol 3224

Uploaded by

Brkcol 3224

Uploaded by

Implementing and Troubleshooting

Secure Voice/Video on Edge Devices

Paul Anholt, Technical Leader

BRKCOL-2014: Introduction to Cisco UC Security

BRKCOL-3224: Implementing and Troubleshooting Secure Voice/Video on Edge

• Interoperability between Secure and Non-Secure Networks

• CUBE and CUCM • Client generates a

For TLS 1.0, generate an RSA key-pair

For TLS 1.2, generate an EC key-pair

1. Create Trustpoint to store the root and identity certificates

1b. Specify the correct key pair to use

% The subject name in the certificate will include: CN=ISR4KCUBE.cisco-uc.com

---End - This line not part of the certificate request---

• Only one CSR may be in progress at a

% Do you accept this certificate? [yes/no]: yes

• Let’s Encrypt is a free, automated, and open Certificate Authority

• TCP port 80 required to be open inbound to all Expressway-E’s from ANY

• MRA deployments should consider using the CollabEdgeDNS “format”

• A new cert will be signed after 2/3 of existing certificate’s validity

The UCM Device Security Profile Name must be in the

This same FQDN must be present as a Subject

Use of a Universal Device Template is recommended to

Set the Device Security Mode to Authenticated

SIP TLS SIP TLS SIP TLS

To enable secure signaling and media on a subset of the call flow

• TLS Mode/Port – Configure the global SIP TLS listening port

Configuration>Zones>Zone>Edit zone>SIP Configuration Settings

xConfiguration Ciphers ForwardProxyTLSCiphers Value: "HIGH:!MD5:!RC…"

⚠️TLS 1.0 is supported in X8.10 but not recommended

3. Dial-peer Configuration – WAN (PSTN) side 2. Dial-peer Configuration – LAN side

dial-peer voice 10 voip dial-peer voice 1 voip

TCP is required to enable TLS!

1. Associate trustpoint with IOS voice process

2. Enable TLS transport on dial-peer

Associate CUBE trustpoint with voice process

Base command Peer IP address/network association trustpoint cipher selection

Configuration Cipher Suites

Ensure Subject-name matches between IOS and CUCM

As per RFC 3711, SRTP is “a profile of the Real-time Transport

a=crypto:<tag> <crypto-suite> <key-params> [<session-params>]

SDP for RTP SDP for SRTP

Tag Crypto Suite Transport Method Master Key

a=crypto:1 AES_CM_128_HMAC_SHA1_32 inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0 UNENCRYPTED_SRTCP

a=crypto:1 AES_CM_128_HMAC_SHA1_32 inline: L5+zq2AXJxLk+058lu/XRQWJZiK0c0D0|2^20|1:32

The UCM Device Security Profile Name must be in the

This same FQDN must be present as a Subject

Use of a Universal Device Template is recommended to

Set the Device Security Mode to Encrypted

None No encryption in use Mixed Mode Off

2. Apply the defined voice-class either or

3. Enables SRTP voice service voip

Version Cipher Suites

* With SRTP Passthru feature

ISR 4000-series and CSR1000v ISR G2 - 2900/3900-series routers

1. Enable DSPfarm feature on DSP

488 Not Acceptable Media

Media encryption is required m=RTP/SAVP media description m=RTP/SAVP media description

Calls that can be encrypted are m=RTP/AVP In the Offer’s

m=RTP/SAVP media description

Modify the RHS of the SIP URI to

Does an unsecure call work in the same call flow?

Does an unsecure call work in the same call flow?

Does the calling party hear ringback?

Does an unsecure call work in the same call flow?

Does the calling party hear ringback?

Does the call connect?

Does an unsecure call work in the same call flow?

Does the calling party hear ringback?

Does the call connect?

Does the lock icon appear on the phone?

Secure call successfully negotiated!

2 : 11 572330360ms.1 +1380 pid:31 Answer 1052061 active

Call identifier Calling Number

2 : 13 572330370ms.1 +1360 pid:21 Originate 4181101002 active

ISR4KCUBE#show call active voice brief

2 : 11 572330360ms.1 +1380 pid:31 Answer 1052061 active