ADP's Software Defined Network Journey

By Vincent Patrizio, VP-Head of Global Network Services & Distributed Database Services, ADP

Vincent Patrizio, VP-Head of Global Network Services & Distributed Database Services, ADP

ADP is one of the world’s largest providers of human capital management services, including comprehensive payroll solutions, human resources, benefits, time and labor and talent management. The company is a long-time leader in the cloud computing industry and the Software-as-a-Service (SaaS) business model. Today more than 18 million users leverage ADP’s cloud-based platforms.

ADP is the world’s largest provider of human capital management services, including comprehensive payroll solutions, human resources, benefits and retirements administration, and talent management. The company is a long-time leader in the cloud computing industry and the Software-as-a-Service (SaaS) business model. Today more than 18 million users leverage ADP’s cloud-based platforms.

With the evolution to cloud-based platforms, traditional operational methods no longer allowed the company to keep up with business demands for provisioning and supporting SaaS offerings. Over the past four years, we have undergone an internal transformation to support efficiency and innovation. Infrastructure and operations were simplified to allow on demand services. Agile development and rapid deployment methods were implemented to move towards self-service delivery. After significant automation and virtualization of its systems, we now operate on Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS) models. Benefits include lower costs and risk, less maintenance, and program ability allowing focus on innovation and meeting user expectations for enterprise application delivery.

“Our approach to software defined networking is to ensure agility and efficiency with 100 percent availability ”

ADP’s clients, business units, and software developers all depend upon the company’s global network of interconnecting data centers located around the world. It is at the core of our infrastructure and operations. Thus, the transition to the software defined network (SDN) has become a key enabler to scaling the company’s rapidly growing SaaS offerings.

ADP’s SDN framework begins with the underlay and is followed by the overlay. Application performance management tools sit upon these layers. Automation and orchestration is a key ingredient for success across the SDN stack.

Our approach to software defined networking is to ensure agility and efficiency with 100 percent availability. Exposure to risk, including fault containment, technology maturity, workforce skill sets, and operational tool sets must be in constant equilibrium with the value derived from on-demand service delivery, rapid scalability, as well as capital and operational expenditure reductions.

ADP is able to achieve a balance by creating tight standards and reference architectures for fixed compute workloads through the SDN framework. Consistency runs through all layers, directly contributing to system availability.

The formation of an open standards underlay network marked the start of ADP’s software defined networking journey four years ago. The first step was to reconcile technology requirements in rack architecture. Building an open standards environment capable of Application Program Interface (API) integration laid the foundation for a programmable and orchestration enabled network. This enhanced visibility, monitoring, and telemetry. A standard rack structure with a top of rack (TOR) switch pair and multi-chasses link aggregation for servers allowed better management of fault domains and zero-downtime maintenance. Then began the adoption of a modular approach to the data center, connecting ADP data centers as a single virtual data center. This approach facilitated horizontal scaling and fault containment. It also allowed rapid release management and back-out capabilities. Any agility challenges were addressed with capacity planning. Having achieved the goal of simplifying, standardizing, and orchestrating the network foundation, we plan to continue improving the data center fabric by moving away from a traditional 3-tier model to a layer 3 Equal Cost Multi path (ECMP) spine-leaf design.

This past year, we focused on building the overlay by implementing L2 switching via virtual extensible LAN. This approach supports multi-rack virtual machine clusters and mobility. It also provides security and service segmentation. Next, scaling of the virtual extensible LAN will begin with the inclusion of VXLAN Tunnel Endpoints (VTEP) at virtual switches, incorporation of the SDN controller and integration of Open Stack. This will result in greater correlation and visibility between the underlay and overlay networks. In order to manage and monitor this level of scaling, ADP will conduct an assessment on operational capabilities and the workforce will be re-educated. In the next several years and beyond, ADP will have a fully virtualized network with open flow, policy-based routing, transport agnostic branch connectivity, Service Level Agreement (SLA) driven programmable connectivity, hardware agnostic networking, and network functions virtualization. In summary, the ADP network will be agile, efficient, and always available.

Alongside these efforts, we improved visibility and application performance management by implementing passive taps at critical junctions, Switched Port Analyzer (SPAN) ports on all access switches, NetFlow and SFlow collection and reporting, 10G line rate packet capture, real-time data analysis, a historical repository of captured data, and IP flow construction and playback. ADP built a network infrastructure for tap aggregation allowing pro-active always-on data capture for PaaS, databases, and application delivery controllers. We implemented virtual machine mobility tracking and on-demand packet capture capabilities. Voice analytics are collected through Real Time Streaming Protocol(RTSP) packets and Session Initiation Protocol(SIP) control protocol connection times. ADP also has the ability to slice and dice packets to specific tool destinations so mean time to recovery or resolution is achieved. In this year and beyond, the company will extend visibility and orchestration in s everal w ays. W e w ill c ontinue t o strengthen correlation and visibility between the underlay and overlay and will also include the vSwitch span. Plus, we will further automate the tap aggregation infrastructure to steer span traffic from all switches via one front e nd. C omprehensive n etwork intelligence and analytics will be collected and forensic capabilities will be enhanced.

ADP’s network life cycle orchestration is conducted through an internally developed configuration and automation tool named RapidCA that utilizes a range of technologies including Python, Perl-CGI, REST, JSON, JQUERY, Apache, HTML5/ CSS, SNMP/SYSLOG, and SVN. The tool allows zero-touch provisioning, monitoring systems integration, zero-touch lifecycle maintenance, and zero-touch failure management and replacement. Configurations are always consistent with standards, human error and downtime is eliminated, and time for deployment and upgrades is reduced from h ours t o m minutes. A Additional benefits include better inventory management, asset tracking, financial tracking, configuration management, and audit and compliance.