Applying A Software Design Pattern To Network Automation

I will be talking about network automation—a lot—in a forthcoming series of posts! If you have no experience or familiarity with the subject, let me assure you that I will do my very best to deliver my message and materials about automation in a friendly conversational tone.

I started my network automation journey with an “infrastructure as code” approach and four of the five years of my journey were spent learning and mastering Ansible.

More recently, in an effort to improve my Python skills, and to have a “real-world” project to help me study for my DevNet Associate (which I passed, thank you), I started an open-source project called Merlin that uses a Python web-framework called Django. This new approach, which follows a software design pattern, really feels more like infrastructure as software—with a real database and web engine—versus infrastructure as code (typically “just” YAML files).

I had incredible success with Ansible and infrastructure as code, but there were clearly some significant downsides to the approach. One problem was a sprawling number of both individual scripts and the output files generated by these scripts. It was incredibly challenging to gain traction and adoption because there was still a lot of friction involved.

Another problem was that operations still had to face Ansible and Python learning curves in order to consume and deploy automation solutions. And possibly most importantly, we had collected all of this incredibly rich network state and configuration data and had nowhere to put it and no way to effectively present it to the enterprise stakeholders.

Speaking of Merlin, and network automation projects in general, I was finding more and more the difficulty was no longer in solving infrastructure problems programmatically, either with a domain-specific language like Ansible, or with my own Python code; but in transitioning these solutions into a consumable, production-ready state. And even though I have transitioned most of my solutions away from Ansible to Python using Cisco’s pyATS framework and libraries, I still had several problems:

1. Where and how to store network state data captured from the network at scale. Raw JSON files? In the Git repository or a network share?
2. How to present the data to users in business-ready documentation. Cumbersome customized Jinja2-templated comma separated files? Hand-crafted HTML pages?
3. How to manage a large number of devices.

A YAML File? Really?

During my recent studies for the Cisco DevNet Associate exam I learned about common design patterns, specifically the Model View Controller (MVC) Pattern.

The MVC is a modular design pattern built on the separation of concerns principle. There are several implementations including JavaScript frameworks like Angular, Express, and Backbone, as well as Python frameworks like Django, Flask, and web2py.

The MVC design works incredibly well. In fact, I bet the website or application you are using right now was constructed using this approach. But how does it apply to network infrastructure? And specifically, how can this simple design pattern help us solve the inherit problems of YAML-driven network automation?

First, does this idea of MVC and separation of concerns and a modular three-tier approach sound vaguely familiar? In my mind there is a direct parallel between MVC and a traditional, three-tier application design.

Django – “All Batteries Included”

As mentioned, you have many choices for a framework. There are significant differences among Django, Flask, and web2py that you should be aware of. Flask, which takes a micro-services approach, allows you to develop a leaner solution without the full database. Web2py may give you the full tiered framework and even more control over your code. I selected Django for multiple reasons:

• A full backend with an SQL database
• Easily port over my “stand-alone” pyATS automation solutions without throwing anything away
• Easily port over my Jinja2 templates into the Django Templating Language (DTL)
• The ability to serialize and present network state JSON as a REST API with the Django REST Framework (DRF)
• The ability to schedule periodic tasks (automate the automation!) with Celery and Redis integrations

Here are how all of the parts fit and how I transformed my pyATS jobs from a collection of isolated, independent, YAML-driven, Python scripts into friction-free on-demand web services using the MVC Pattern.

Model

The Model is responsible for the data manipulation and is directly tied to an SQL table in the database. In fact, after you create your model, you have to perform a two-step process to transform it into a database table. First, you use Django’s manage.py script (the main script Django provides to perform administrative functions) to makemigrations from your models. Then, after you’ve made a migration, you migrate and create your database table.

Django includes all of the above capabilities – and more. Django comes with a built in Administrator page (you do not have to build this) where administrators can work with the Models (populate, update, or delete database tables directly in the backend) or even schedule the Celery Beats (periodic tasks). I mention this under the Models section because a great way to validate your migration success and investigate your model-as-a-database-table is to register the model with your admin URL (a View).

Controller

The controller is almost like a proxy or an intermediate device between what is presented to the user in Views and the logic in the backend manipulating the data. In my particular use case of network automation, I use the Controller layer to handle user requests, from the Views, to trigger pyATS jobs that manipulate the Models in the database, typically by populating the database with new records from the network state.

View

A pretty self-explanatory layer, Views are what the user sees and interacts with. It is also our opportunity to present Model data processed at the Controller level or trigger automation in the Controller with buttons or input boxes.

I like to conceptually break down Views into three sub-components: the Views themselves, where the logical interaction with the Controller takes place; the URLs, or the unique location where the View itself will be presented; and finally the Template used to generate the HTML view.

A final thought on Views, which we will explore further, is that we can connect the HTML templates to very powerful Cascading Style Sheets (CSS) and JavaScripts (JS) to really enhance the user experience and front end of our Django application.

In the next part of the series, we will go deeper into Django, Python, SQL, and how to build up a web-based, MVC Pattern-driven, YAML-free, infrastructure as a software solution. Until then, stay safe and feel free to reach out and connect with me!