After a few months of configuring my test network using one-off scripts i have now realized that perhaps this was not entirely proper way of doing things.
In one of my previous blog posts i mentioned that configuring the router using that long change.txt wasn’t ”network as code” but rather network as a long messy text file. This was true, although it was a small step forward.
From now on I will try to combine all the things i’ve done before, that is Unicon, jinja templates but using the OOB approach and modifying the network using code, so that I have the object-oriented app logic in one file, change execution in another (importing the logic), jinja templates in another file (.j2) and jinja attributes in another file (.yml).
The endgoal will be to have the following workflow:
1. prepare the new config,
2. get current network configuration and state,
3. implement the config,
4. get new network configuration and state
5. get change notification with config delta in Slack.
6. Optional (start the workflow from Slack, the ”chatops” way)
First let’s create some basic logic.
class Router:
def __init__(self, sship, sshuser, sshpass):
self.config = []
self.sship = sship
self.sshuser = sshuser
self.sshpass = sshpass
def setIPaddress(self, intname, IP, mask):
self.intname = intname
self.ip = IP
self.mask = mask
commandlist = []
commandlist.append(self.intname)
commandlist.append('ip address ' + str(self.ip) + ' ' + str(self.mask))
self.config.append(commandlist)
def configure(self):
#todo
router = Router()
router.setIPaddress("int fa0/0","192.168.0.1","255.255.255.0")
print(router.config)
Now we have the OOB basic, I will take advantage of all the cool bits and pieces from the previous blog posts to create this full script including Unicon, jinja etc.
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys
import shutil
import diffios
import os
import time
import logging
from gtts import gTTS
import re
class Router:
def __init__(self):
self.config = ''
def setIPaddress(self, intname, IP, mask):
self.intname = intname
self.ip = IP
self.mask = mask
commandlist = []
commandlist.append("interface " + str(self.intname))
commandlist.append('ip address ' + str(self.ip) + ' ' + str(self.mask))
self.config += commandlist
def prepareACLconfig(self, datafile, template):
self.datafile = datafile
self.template = template
accesslists = yaml.load(open(self.datafile), Loader=yaml.SafeLoader)
env = Environment(loader = FileSystemLoader('.'), trim_blocks=True, autoescape=True)
template = env.get_template(self.template)
acl_config = template.render(data=accesslists)
self.config += acl_config
def configure(self):
testbed = loader.load('testbed.yml')
c = testbed.devices['R1']
c.connect()
output = c.configure(router.config)
c.destroy()
#todo
#and now let's test an actual config change with acl data in the .yml object and template in the .j2 object, with the GNS3 routers described in the testbed.yml object.
router = Router()
router.prepareACLconfig('vpnacl.yml', 'acl.j2')
router.configure()
while all the data files look like this:
tode@ubuntu:~/perfectscripts/generator$ cat vpnacl.yml
accesslists:
aclname: permit_www
aclcommand: permit
protocol: ip
source: 192.168.122.100
sourcewildcard: 0.0.0.0
destination: 8.8.8.8
destwildcard: 0.0.0.0
tode@ubuntu:~/perfectscripts/generator$ cat acl.j2
{% for accesslist in data.accesslists %}
ip access-list extended {{accesslist.aclname}}
{{accesslist.aclcommand}} {{accesslist.protocol}} {{accesslist.source}} {{accesslist.sourcewildcard}} {{accesslist.destination}} {{accesslist.destwildcard}}
{% endfor %}
tode@ubuntu:~/perfectscripts/generator$ cat testbed.yml
devices:
R1:
connections:
cli:
ip: 192.168.122.100
port: 23
protocol: telnet
credentials:
default:
password: cisco
username: cisco
enable:
password: cisco
os: ios
type: ios
Decorators
Let’s add a simple function here that will do ‚show run’ before and after the change. To do that, i’ve implemented a simple decorator.
def diffdecorator(func):
def wrapper(self):
testbed = loader.load('testbed.yml')
c = testbed.devices['R1']
c.connect(log_stdout=False)
output = c.execute('show run')
f = open('R1.bak', "w")
f.write(output)
f.close()
c.destroy()
func()
c = testbed.devices['R1']
c.connect(log_stdout=False)
output = c.execute('show run')
f = open('R1.txt', "w")
f.write(output)
f.close()
c.destroy()
diff = diffios.Compare('R1.bak', 'R1.txt')
deltaoutput = diff.delta()
filename = 'R1_change_delta.bak'
f = open(filename, "w")
f.write(deltaoutput)
f.close()
return wrapper
@diffdecorator
def configure():
testbed = loader.load('testbed.yml')
c = testbed.devices['R1']
c.connect()
output = c.configure(router.config)
c.destroy()
Nested Jinja template
Going further, i created an ospf.j2 file and a corresponding ospf.yml file. Now because the ospf can have multiple network statements, a nested jinja template was needed.
tode@ubuntu:~/perfectscripts/generator$ cat ospf.yml
ospf:
- processnumber: 100
routerid: 1.1.1.1
network:
- address: 192.168.122.0 0.0.0.255 area 0
- address: 1.1.1.1 0.0.0.0 area 0
tode@ubuntu:~/perfectscripts/generator$ cat ospf.j2
{% for element in data.ospf%}
router ospf {{element.processnumber}}
router-id {{element.routerid}}
{% for entry in element.network%}
network {{entry.address}}
{% endfor %}
{% endfor %}
This has produced the following config:
tode@ubuntu:~/perfectscripts/generator$ python3 ng_oob.py
router ospf 100
router-id 1.1.1.1
network 192.168.122.0 0.0.0.255 area 0
network 1.1.1.1 0.0.0.0 area 0
Next steps:
- using pyats to check network state before change (another decorator)
- decision about folder structure with .ymls, templates and script change history
- how to implement idempotency (file-based, or code-based)
- using Slack to trigger changes (reverse bot)
Adding Slack notification
Let’s rewrite the decorator again:
def diffdecorator(func):
def wrapper(self):
testbed = loader.load('testbed.yml')
c = testbed.devices['R1']
c.connect(log_stdout=False)
output = c.execute('show run')
f = open('R1.bak', "w")
f.write(output)
f.close()
c.destroy()
func()
c = testbed.devices['R1']
c.connect(log_stdout=False)
output = c.execute('show run')
f = open('R1.txt', "w")
f.write(output)
f.close()
c.destroy()
diff = diffios.Compare('R1.bak', 'R1.txt')
deltaoutput = diff.delta()
p = get_notifier('slack')
p.notify(webhook_url='https://hooks.slack.com/services/xxxxx/yyyyy/zzzzzz', message='config on this router has been changed!')
p.notify(webhook_url='https://hooks.slack.com/services/xxxxx/yyyyy/zzzzzz', message=diff.delta())
filename = 'R1_change_delta.bak'
f = open(filename, "w")
f.write(deltaoutput)
f.close()
return wrapper
Adding Genie magic
A while ago I was tasked with creating a report based on the uptime taken from output of ‚show version’ command plus software version. Preparing the regex was funny, and i was well proud of the way i did the ”screenscrubbing”, until I saw how you can do this easier.
The regex way for uptime doesn’t look that bad but what if Cisco changed the output of the show version command?
endresult = re.findall(r'control processor .*', result)
And here comes Genie. Initialising the testbed looks very similar to the unicon way ( Genie.init instead of loader.load )
testbed = Genie.init('testbed.yml')
r1 = testbed.devices['R1']
r1.connect()
shver = r1.parse("show version")
print(shver['version']['version_short'])
print(shver['version']['uptime'])
The output of this is:
12.4 5 hours, 8 minutes
Now another thing we can do with Genie is compare the state of the network. This is different than a configuration diff. A Genie diff can for example show you which OSPF LSAs you received in between running two ”Genie learn commands”. Let me show you how it’s done:
genie learn ospf --testbed testbed.yml --device R1 --output learn earning '['ospf']' on devices '['R1']' 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:03<00:00, 3.04s/it] +==============================================================================+ | Genie Learn Summary for device R1 | +==============================================================================+ | Connected to R1 | - Log: learn/connection_R1.txt Learnt feature 'ospf' - Ops structure: learn/ospf_ios_R1_ops.txt - Device Console: learn/ospf_ios_R1_console.txt ==============================================================================
Then I run my python script which generates and implements the ospf config:
tode@ubuntu:~/perfectscripts/generator$ python3 ng_oob.py
router ospf 100
router-id 1.1.1.1
network 192.168.122.0 0.0.0.255 area 0
network 1.1.1.1 0.0.0.0 area 0
(...) here i didn't include the unicon's screen output
Now i run the Genie learn again, this time i redirect the output to another folder:
tode@ubuntu:~/perfectscripts/generator$ genie diff learn learn2 1it [00:00, 127.13it/s] +==============================================================================+ | Genie Diff Summary between directories learn/ and learn2/ | +==============================================================================+ | File: ospf_ios_R1_ops.txt | - Diff can be found at ./diff_ospf_ios_R1_ops.txt
Inside the diff file is the following:
tode@ubuntu:~/perfectscripts/generator$ cat diff_ospf_ios_R1_ops.txt --- learn/ospf_ios_R1_ops.txt +++ learn2/ospf_ios_R1_ops.txt +info: + vrf: + default: + address_family: ipv4: instance: 100: areas: 0.0.0.0: area_id: 0.0.0.0 area_type: normal database: lsa_types: 1: lsa_type: 1 lsas: 1.1.1.1 1.1.1.1: adv_router: 1.1.1.1 lsa_id: 1.1.1.1 ospfv2: body: router: links: 1.1.1.1: link_data: 255.255.255.255 link_id: 1.1.1.1 topologies: 0: metric: 1 mt_id: 0 type: stub network 192.168.122.0: link_data: 255.255.255.0 link_id: 192.168.122.0 topologies: 0: metric: 1 mt_id: 0 type: stub network num_of_links: 2 header: adv_router: 1.1.1.1 age: 42 checksum: 0xE54B length: 48 lsa_id: 1.1.1.1 option: None seq_num: 80000001 type: 1 interfaces: FastEthernet0/0: bfd: enable: False cost: 1 dead_interval: 40 demand_circuit: False dr_ip_addr: 192.168.122.100 dr_router_id: 1.1.1.1 enable: True hello_interval: 10 hello_timer: 00:00:03 interface_type: broadcast lls: True name: FastEthernet0/0 passive: False priority: 1 retransmit_interval: 5 state: dr transmit_delay: 1 Loopback1: bfd: enable: False cost: 1 demand_circuit: False enable: True interface_type: loopback name: Loopback1 mpls: te: enable: False statistics: area_scope_lsa_cksum_sum: 0x00E54B area_scope_lsa_count: 1 spf_runs_count: 1 bfd: enable: False graceful_restart: cisco: enable: False type: cisco ietf: enable: False type: ietf mpls: ldp: autoconfig: False autoconfig_area_id: 0.0.0.0 nsr: enable: False preference: single_value: all: 110 router_id: 1.1.1.1 spf_control: paths: 4 throttle: spf: hold: 10000 maximum: 10000 start: 5000 stub_router: always: always: False external_lsa: False include_stub: False summary_lsa: False
What i’m going to do is include this learn process before and after each change, on top of the diffios change, so that we can have more proof that our change has been significant.
However, we can use this Genie diff also if no change has taken place, or if the change has been external to our administrative domain. Something breaks down even if we didn’t change anything, so how about running a Genie diff to check if we now have fewer ospf routes than before? or fewer bgp routes? It’s never been easier to troubleshoot a network problem.
…to be continued…
My name is Tomek De Wille. Welcome to my network blog. 