Hello
Today I wanted to see how multiple nested commands (e.g. archive > log config > notify syslog, archive > log config > logging enable) will be configured using my script.
I’ve discovered an interesting thing. With a config like this:
R1, archive, log config, notify syslog,
R1, archive,log config, logging enable,
R1, archive, log config , hidekeys,
Only one of the commands (the last one) would be implemented on the router. It seems that while the defaultdict can have a list of values under a key, its keys still need to be unique. There are a few ways to work around this:
- adding randomized empty spaces in the subkeys (_log config, log config_, log config, ). This is actually quite nice because spaces have no impact on the ios,
- cutting the last letter of the key in each command set (archive, archiv, archi)
- trying to change the logic: having just Archive word as the key, and then [log config, notify syslog] as the values. But this would require a heavy rework of the code. Alas, it’s probably the best solution. So this would be just a defaultdict+defaultdict, rather than a defauldict+defaultdict+defaultdict. On a second thought even a single defaultdict would do: {R1: [archive, log config, notify syslog]}. Why I never thought about this before is beyond me. I made my code (and life) so much more complicated with this triple nested defaultdict.
Anyways, let’s do some mass implementations using the ’empty spaces’ workaround
This is the input:
tode@ubuntu:~/perfectscripts$ cat change.txt
R1, interface FastEthernet 1/0, ip address 100.100.0.2 255.255.255.0,
R1, interface FastEthernet 1/0, no shut,
R1, crypto isakmp policy 10, encr aes,
R1, crypto isakmp policy 10, hash sha,
R1, crypto isakmp policy 10, lifetime 3600,
R1, crypto isakmp policy 10, group 5,
R1, crypto isakmp policy 10, auth pre-share,
R1, crypto isakmp key cisco address 0.0.0.0 0.0.0.0, ,
R1, crypto ipsec transform-set MYSET esp-des esp-sha-hmac, ,
R1, crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.1.2,
R1, crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET,
R1, crypto map MYMAP 10 ipsec-isakmp, match address 101,
R1, ip access-list extended 101, permit ip host 1.1.1.1 host 2.2.2.2,
R1, interface FastEthernet 1/0, crypto map MYMAP,
R1, ip route 2.2.2.2 255.255.255.255 100.100.0.1, ,
R1, int loopback 1, ip address 1.1.1.1 255.255.255.255,
R1, ip route 100.100.1.2 255.255.255.255 100.100.0.1, ,
R1, archive, log config, notify syslog,
R1, archiv, log config, logging enable,
R1, archi, log config, hidekeys,
R2, interface FastEthernet 1/0, ip address 100.100.1.2 255.255.255.0,
R2, interface FastEthernet 1/0, no shut,
R2, crypto isakmp policy 10, encr aes,
R2, crypto isakmp policy 10, hash sha,
R2, crypto isakmp policy 10, lifetime 3600,
R2, crypto isakmp policy 10, group 5,
R2, crypto isakmp policy 10, auth pre-share,
R2, crypto isakmp key cisco address 0.0.0.0 0.0.0.0, ,
R2, crypto ipsec transform-set MYSET esp-des esp-sha-hmac, ,
R2, crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.0.2,
R2, crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET,
R2, crypto map MYMAP 10 ipsec-isakmp, match address 101,
R2, ip access-list extended 101, permit ip host 2.2.2.2 host 1.1.1.1,
R2, interface FastEthernet 1/0, crypto map MYMAP,
R2, ip route 1.1.1.1 255.255.255.255 100.100.0.2, ,
R2, int loopback 1, ip address 2.2.2.2 255.255.255.255,
R2, ip route 100.100.0.2 255.255.255.255 100.100.1.1, ,
R2, archive, log config, notify syslog,
R2, archiv, log config, logging enable,
R2, archi, log config, hidekeys,tode@ubuntu:~/perfectscripts$
And the output is…
tode@ubuntu:~/perfectscripts$ python3 MassChangeWithAmanda.py critical R1%R2
interface FastEthernet 1/0
ip address 100.100.0.2 255.255.255.0
no shut
crypto map MYMAP
crypto isakmp policy 10
encr aes
hash sha
lifetime 3600
group 5
auth pre-share
crypto isakmp key cisco address 0.0.0.0 0.0.0.0
crypto ipsec transform-set MYSET esp-des esp-sha-hmac
crypto map MYMAP 10 ipsec-isakmp
set peer 100.100.1.2
set transform-set MYSET
match address 101
ip access-list extended 101
permit ip host 1.1.1.1 host 2.2.2.2
ip route 2.2.2.2 255.255.255.255 100.100.0.1
int loopback 1
ip address 1.1.1.1 255.255.255.255
ip route 100.100.1.2 255.255.255.255 100.100.0.1
archive
log config
notify syslog
log config
logging enable
log config
hidekeys
I am trying to implement this change on router R1 Are you sure? yes
interface FastEthernet 1/0
ip address 100.100.1.2 255.255.255.0
no shut
crypto map MYMAP
crypto isakmp policy 10
encr aes
hash sha
lifetime 3600
group 5
auth pre-share
crypto isakmp key cisco address 0.0.0.0 0.0.0.0
crypto ipsec transform-set MYSET esp-des esp-sha-hmac
crypto map MYMAP 10 ipsec-isakmp
set peer 100.100.0.2
set transform-set MYSET
match address 101
ip access-list extended 101
permit ip host 2.2.2.2 host 1.1.1.1
ip route 1.1.1.1 255.255.255.255 100.100.0.2
int loopback 1
ip address 2.2.2.2 255.255.255.255
ip route 100.100.0.2 255.255.255.255 100.100.1.1
archive
log config
notify syslog
archiv
log config
logging enable
archi
log config
hidekeys
I am trying to implement this change on router R2 Are you sure? yes
I can see one big problem here: because commands are grouped, the script is trying to insert the map on interface fa1/0 when the map doesn’t exist yet. This will not be implemented. It will be implemented if the same script runs again. Oh bother. Diffios change log confirms this:
tode@ubuntu:~/perfectscripts$ cat R1_change_delta.bak
--- baseline
+++ comparison
1: Current configuration : 2239 bytes
2: interface FastEthernet1/0
no ip address
shutdown
1: Current configuration : 2767 bytes
2: access-list 101 permit ip host 1.1.1.1 host 2.2.2.2
3: archive
logging enable
4: crypto ipsec transform-set MYSET esp-des esp-sha-hmac
5: crypto isakmp key cisco address 0.0.0.0 0.0.0.0
6: crypto isakmp policy 10
encr aes
authentication pre-share
group 5
lifetime 3600
7: crypto map MYMAP 10 ipsec-isakmp
set peer 100.100.1.2
set transform-set MYSET
match address 101
8: interface FastEthernet1/0
ip address 100.100.0.2 255.255.255.0
9: interface Loopback1
ip address 1.1.1.1 255.255.255.255
10: ip route 100.100.1.2 255.255.255.255 100.100.0.1
11: ip route 2.2.2.2 255.255.255.255 100.100.0.1
tode@ubuntu:~/perfectscripts$ cat R2_change_delta.bak
--- baseline
+++ comparison
1: Current configuration : 1225 bytes
2: interface FastEthernet1/0
no ip address
shutdown
1: Current configuration : 1791 bytes
2: access-list 101 permit ip host 2.2.2.2 host 1.1.1.1
3: archive
logging enable
notify syslog contenttype plaintext
4: crypto ipsec transform-set MYSET esp-des esp-sha-hmac
5: crypto isakmp key cisco address 0.0.0.0 0.0.0.0
6: crypto isakmp policy 10
encr aes
authentication pre-share
group 5
lifetime 3600
7: crypto map MYMAP 10 ipsec-isakmp
set peer 100.100.0.2
set transform-set MYSET
match address 101
8: interface FastEthernet1/0
ip address 100.100.1.2 255.255.255.0
9: interface Loopback1
ip address 2.2.2.2 255.255.255.255
10: ip route 1.1.1.1 255.255.255.255 100.100.0.2
11: ip route 100.100.0.2 255.255.255.255 100.100.1.1
tode@ubuntu:~/perfectscripts$
After small edits, let’s run the script twice and read the change log from the 2nd run.
tode@ubuntu:~/perfectscripts$ cat R1_change_delta.bak
--- baseline
+++ comparison
tode@ubuntu:~/perfectscripts$ cat R2_change_delta.bak
--- baseline
+++ comparison
This is nice, nothing changes during the 2nd run. Let’s now check if the script is actually trying to implement config during the 2nd (and every subsequent) run.
It is. Oh boy. The reason is that the input file needs to be sanitized. If it’s not (if change commands don’t reflect the existing config 100%) the script will not realize these are the same commands and will try to implement them again.
This is the sanitized input file:
tode@ubuntu:~/perfectscripts$ cat change.txt
R1,crypto map MYMAP 10 ipsec-isakmp, match address 101,
R1,access-list 101 permit ip host 1.1.1.1 host 2.2.2.2, ,
R1,interface FastEthernet1/0, ip address 100.100.0.2 255.255.255.0,
R1,interface FastEthernet1/0, no shut,
R1,interface FastEthernet1/0, crypto map MYMAP,
R1,ip route 2.2.2.2 255.255.255.255 100.100.0.1, ,
R1,interface Loopback1, ip address 1.1.1.1 255.255.255.255,
R1,ip route 100.100.1.2 255.255.255.255 100.100.0.1, ,
R1,archive, log config, notify syslog,
R1,archiv, log config, logging enable,
R1,archi, log config, hidekeys,
R2,crypto isakmp policy 10, encr aes,
R2,crypto isakmp policy 10, hash sha,
R2,crypto isakmp policy 10, lifetime 3600,
R2,crypto isakmp policy 10, group 5,
R2,crypto isakmp policy 10, authentication pre-share,
R2,crypto isakmp key cisco address 0.0.0.0 0.0.0.0, ,
R2,crypto ipsec transform-set MYSET esp-des esp-sha-hmac, ,
R2,crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.0.2,
R2,crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET,
R2,crypto map MYMAP 10 ipsec-isakmp, match address 101,
R2,access-list 101 permit ip host 2.2.2.2 host 1.1.1.1, ,
R2,interface FastEthernet1/0, ip address 100.100.1.2 255.255.255.0,
R2,interface FastEthernet1/0, no shut,
R2,interface FastEthernet1/0, crypto map MYMAP,
R2,ip route 1.1.1.1 255.255.255.255 100.100.0.2, ,
R2,interface Loopback1, ip address 2.2.2.2 255.255.255.255,
R2,ip route 100.100.0.2 255.255.255.255 100.100.1.1, ,
R2,archive, log config, notify syslog,
R2,archiv, log config, logging enable,
R2,archi, log config, hidekeys,tode@ubuntu:~/perfectscripts$
and the live output for R2:
crypto isakmp policy 10
hash sha
interface FastEthernet1/0
no shut
archiv
log config
logging enable
archi
log config
hidekeys
I am trying to implement this change on router R2 Are you sure?
Something is still left. Hash sha is the automatic choice, so doesn’t appear in the config file. This is not something that we need to handle, because sha should no longer be used for VPNs (obsolete).
No shut needs a separate code fix, because if a port is open, there is no command ‚no shut’ on the port.
For the nested commands i don’t have a good solution yet. I believe the whole code needs to be reworked (a single defaultdict with a list of values) for idempotency to work properly.
RAW CODE
Here’s the code with the ‚no shut’ fix:
#!/usr/bin/python3
from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse
from unicon import Connection
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys
import shutil
import diffios
import os
import time
import logging
from gtts import gTTS
import re
LEVELS = {'debug': logging.DEBUG,
'info': logging.INFO,
'warning': logging.WARNING,
'error': logging.ERROR,
'critical': logging.CRITICAL}
level_name = sys.argv[1]
level = LEVELS.get(level_name, logging.NOTSET)
logging.basicConfig(level=level)
unicondevicelist = sys.argv[2].split("%")
#loading the testbed file with router management IPs
testbed = loader.load('testbed.yml')
#downloading router configuration files and storing them away as .bak files
#x = 0
for x in range(0,len(unicondevicelist)):
devicename = unicondevicelist[x]
c = testbed.devices[devicename]
c.connect(log_stdout=False)
output = c.execute('show run')
f = open(devicename, "w")
f.write(output)
f.close()
newdevicename = devicename + '.bak'
shutil.copyfile(devicename, newdevicename)
c.destroy()
with open('change.txt', 'r') as reader:
configchangetext = reader.read()
splittext = configchangetext.split('\n')
my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for line in splittext:
dataline = line.split(',')
my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
logging.info('\n')
for k, v in my_dict.items():
logging.info('\ni am showing now how I want router config to be modified for router ' + str(k) + ' ' + '\n\n')
my_new_dict = my_dict[k].items()
logging.info('\n')
for item in my_new_dict:
x = 0
for x in range (x, len(item)):
newstringordict= item[x]
if isinstance(newstringordict, str):
logging.info(newstringordict)
else:
for k in newstringordict.keys():
logging.info(k)
realdevicelist = []
for i in my_dict.keys():
realdevicelist.append(i)
for j in my_dict[i].keys():
pass
#let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
m = 1
while True:
try:
x = 0
for x in range(x,len(realdevicelist)):
with open(realdevicelist[x], 'r') as reader:
config = reader.read()
my_config_as_list = config.split("\n")
except FileNotFoundError as e:
if m==1:
text = "I found " + str(m) + " change for a router that is not in the testbed."
if m > 1:
text = "I found another error. "
logging.critical('\n')
logging.critical('*******************************ERROR***************************************************************************')
logging.critical(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
logging.critical('*******************************ERROR***************************************************************************')
logging.critical('\n')
del my_dict[realdevicelist[x]]
realdevicelist.remove(realdevicelist[x])
m = m + 1
continue
else:
for k, v in my_dict.items():
my_new_dict = my_dict[k].items()
for item in my_new_dict:
#item is a tuple consisting of string and default dict
x = 0
for x in range (x, len(item)):
newstringordict= item[x]
if isinstance(newstringordict, str):
logging.info(newstringordict)
else:
for z in newstringordict.keys():
logging.info(z)
break
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
with open(router, 'r') as reader:
config = reader.read()
my_config_as_list = config.split("\n")
parse = CiscoConfParse(my_config_as_list)
my_new_dict = my_dict[router].items()
for tpl in my_new_dict:
interface = tpl[0]
cfgs = tpl[1]
for key in cfgs.keys():
answer = parse.find_objects('^' + interface)
if len(answer) > 0:
#logging.info('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ')
second_answer = parse.find_objects_w_child(interface, key)
if len(second_answer) > 0:
#logging.warning('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change ' + interface + ' ' + key)
pass
else:
if key == ' ':
pass
elif key == ' no shut':
findshut = ' shut'
shutanswer = parse.find_objects_w_child(interface, findshut)
if len(shutanswer) > 0:
pass
else:
pass
else:
logging.warning(' no match in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
my_final_dict[router][interface][key] = cfgs[key]
#connect to router and implement interface, key
else:
logging.warning(interface + ' not found on router ' + router + ' but i will try to create this ' + interface + ' with the value ' + key)
my_final_dict[router][interface][key] = cfgs[key]
#connect to router and try to implement
finalstatement = ''
for k, v in my_final_dict.items():
my_newfinal_dict = my_final_dict[k].items()
globlist = []
for item in my_newfinal_dict:
x = 0
for x in range (x, len(item)):
newstringordict= item[x]
if isinstance(newstringordict, str):
globlist.append(newstringordict)
logging.info(newstringordict)
else:
for j in newstringordict.keys():
globlist.append(j)
n = newstringordict.get(j)
globlist.append(n)
c = testbed.devices[k]
c.connect(log_stdout=False)
for line in globlist:
print(line)
decision = input("I am trying to implement this change on router " + k + " Are you sure? " )
if "yes" in decision:
output = c.configure(globlist)
if 'Invalid' in output:
logging.critical(output)
else:
pass
c.destroy()
unicondevicelist = realdevicelist
x = 0
for x in range(x,len(unicondevicelist)):
afterchanges = unicondevicelist[x]
beforechanges = afterchanges + '.bak'
c = testbed.devices[afterchanges]
c.connect(log_stdout=False)
output = c.execute('show run')
#time.sleep(1)
f = open(afterchanges, "w")
f.write(output)
f.close()
diff = diffios.Compare(beforechanges, afterchanges)
deltaoutput = diff.delta()
filename = afterchanges + '_change_delta.bak'
f = open(filename, "w")
f.write(deltaoutput)
f.close()
os.remove(afterchanges)
os.remove(beforechanges)