Common Cisco Expressway Implementation Pitfalls and How to Troubleshoot Them (updated)

Since its introduction, the Cisco Expressway Series has made it possible for users outside your firewall to have simple, highly secure access to all your collaboration workloads. And although it remains a solid addition to your Cisco UC stack, there are some common pitfalls and challenges organizations encounter within your organization’s own network you should be aware of.  

In this article, we will focus on these issues and potential troubleshooting options. We will not be listing all the firewall ports nor DNS records that need to be configured with a typical deployment, as that is already well documented by Cisco.

Instead, we will guide you through the details and steps that are especially difficult or need special attention. We will also provide methods of troubleshooting during or after your deployment process.

What is Cisco Expressway?

Cisco Expressway offers highly secure remote access for users to all collaboration tools and services inside your organization, including voice, content, IM & Presence, and video. It can also unite your organization with other organizations with similar services. 

The advanced Cisco Expressway collaboration gateways are a significant enhancement for your unified communications environment and should be considered for organizations using Cisco for their UC. The firewall-traversal technology can benefit both your business-to-business (B2B) and business-to-consumer (B2C) collaborations, extending organizational reach.

Internal and External DNS A and SRV records:

You have to configure multiple DNS SRV records, internal and external for your organization. It’s important to remember that your Internal A record for the Edge should point towards the internal interface IP address (in a dual NIC deployment).

You’ll also need to configure an internal DNS server reference for your core servers and an external (public) DNS server reference for your Edge servers.

Cisco provides a handy tool that can help with your Expressway deployment. The Collaboration Solutions Analyser (SRV checker section) will verify if all External DNS records are configured properly. You can filter the records per feature; see the below image as an example (all features selected):

Make sure to also deploy the following SRV records explicitly in your internal network. _Cisco-uds and _cuplogin should not resolve on the internet:

Firewall Design Considerations

Should you use a Single or Dual Network Interface Controller (NIC) on the Cisco Expressway EDGE for your deployment? We recommend using Dual NIC.

A Single NIC can have many drawbacks, including:

• An asymmetric routing design when used with NAT Reflection, can be a security concern.
• Hair-pinned media isn’t supported by all firewalls
• Excessive bandwidth consumption.
• Risk of public IP exposure in SIP signaling to B2BUA.

Next, disable ALG and/or SIP inspections on the Firewall. The ALG is also referred to as the Inspection or Application Awareness, where the firewall would identify SIP and H.323 traffic and will modify the payload by replacing Public with the Private IP. In some instances, with SIP TLS the firewall will not be able to apply the inspection rules. Therefore the best course of action is to disable Inspection. Then the external interface of the EDGE server will be configured in Static NAT mode (Advanced networking option key required).

If your organization is using an IDS or an IPS, make sure to accurately identify SIP TLS traffic (SIP TLS uses port 5061). We have seen cases where SIPs are identified under SSL traffic, and TCP session timeout creates a race condition with the SIP session refresh timer which will result in SIPs sessions dropped. The workaround is to increase the TCP session timeout.  SIP Session refresh interval in Expressway is the maximum time allowed between session refresh requests for SIP calls. The default value is 1800; the range is from 90 to 86400.

Automatic Protection Service

You can use the automated protection service to detect and block malicious traffic and to help protect the Cisco Expressway from dictionary-based attempts to breach login security.

It works by parsing the system log files to detect repeated failures to access specific service categories, such as SIP, SSH, and web/HTTPS access. When the number of failures within a specified time window reaches the configured threshold, the source host address (the intruder) and destination port are blocked for a specified period of time.

ZIRO has seen cases where regular users are blocked because the “HTTP proxy resource access requests” or “HTTP proxy authorization requests” failed more than the configured threshold within the configured detection window. That could happen if a user has numerous devices that are requesting registration or device information from CUCM and are behind the same IP address outside your corporate network. ZIRO recommends that either you configure a reasonable “Trigger level” or turn off “HTTP proxy resource access failures” and “HTTP proxy authorization requests” from the System Protection, Automated detection configuration page.

Below are the default settings for HTTP proxy resource access failure:

Cisco Expressway Licensing

If your organization has already purchased CUWL Std or CUWL Pro licenses, Expressway Mobile and Remote Access (MRA) comes with no additional cost. All you need is to request from your Cisco Account Manager the Expressway activation keys and option keys for MRA deployment. On the other hand, if you will wish to implement a B2B feature, you will have to purchase additional RMS licenses for the traversal calls. There is an additional cost for Telepresence or Room kit unit registration; you will have to buy licenses to be able to use the Expressway for call control.

Cisco Expressway Troubleshooting Tools

The first thing that you will be asked when opening a TAC case for Cisco Expressway-related problems is to upload Diagnostic logs from both servers, Edge and Core. From the Expressway web administration page, go to Maintenance, Diagnostics, Diagnostic logging, select the “Take tcpdump while logging” option, and start the logging. Once the problem is replicated, you can stop the logging and download the logs.

As mentioned earlier in this blog, Cisco has developed helpful tools for validating Expressways configuration during deployment as well as troubleshooting problems with different expressway features.

The Collaboration Solutions Analyser has four sections you should be aware of:

• The first tool is the “SRV checker” which allows verifying all necessary DNS records per feature you want to deploy.
• The second tool is the “CollabEdge validator” which can verify your MRA deployment; you need to provide the domain, username, and password of your test user.
  The third tool is the “B2B call tester” which can generate a testing space for you that would be used to make and receive test calls with both SIP and H.323 protocols.
• The final tool is the Log analyzer; you can use it with Diagnostic logs previously downloaded from your Expressway servers. The tool will execute a deep analysis of your system based on the logs uploaded and will provide you with a report on the features deployed, along with the bugs and problems discovered.

Final Thoughts

Like any other solution, you may experience many pitfalls or challenges with Cisco Expressway. But you don’t have to go it alone.

Our UC experts can take the hassle out of implementing and troubleshooting Cisco UC challenges with no-nonsense managed services and easy-to-use provisioning software.

Contact us today or call +1-844-940-1600.

_

Editor’s Note: This post was originally published on June 6, 2016.