Implementing idempotency in Unicon a.k.a do the desired changes already exist on the router? Part 6: Nested configs and mass changes.

Hello

Today I wanted to see how multiple nested commands (e.g. archive > log config > notify syslog, archive > log config > logging enable) will be configured using my script.

I’ve discovered an interesting thing. With a config like this:

R1, archive, log config, notify syslog,
R1, archive,log config, logging enable,
R1, archive, log config , hidekeys,

Only one of the commands (the last one) would be implemented on the router. It seems that while the defaultdict can have a list of values under a key, its keys still need to be unique. There are a few ways to work around this:

  • adding randomized empty spaces in the subkeys (_log config, log config_, log config, ). This is actually quite nice because spaces have no impact on the ios,
  • cutting the last letter of the key in each command set (archive, archiv, archi)
  • trying to change the logic: having just Archive word as the key, and then [log config, notify syslog] as the values. But this would require a heavy rework of the code. Alas, it’s probably the best solution. So this would be just a defaultdict+defaultdict, rather than a defauldict+defaultdict+defaultdict. On a second thought even a single defaultdict would do: {R1: [archive, log config, notify syslog]}. Why I never thought about this before is beyond me. I made my code (and life) so much more complicated with this triple nested defaultdict.

Anyways, let’s do some mass implementations using the ’empty spaces’ workaround

This is the input:

tode@ubuntu:~/perfectscripts$ cat change.txt
 R1, interface FastEthernet 1/0, ip address 100.100.0.2 255.255.255.0, 
 R1, interface FastEthernet 1/0, no shut, 
 R1, crypto isakmp policy 10, encr aes, 
 R1, crypto isakmp policy 10, hash sha, 
 R1, crypto isakmp policy 10, lifetime 3600, 
 R1, crypto isakmp policy 10, group 5, 
 R1, crypto isakmp policy 10, auth pre-share, 
 R1, crypto isakmp key cisco address 0.0.0.0 0.0.0.0, , 
 R1, crypto ipsec transform-set MYSET esp-des esp-sha-hmac, , 
 R1, crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.1.2,
 R1, crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET, 
 R1, crypto map MYMAP 10 ipsec-isakmp, match address 101, 
 R1, ip access-list extended 101, permit ip host 1.1.1.1 host 2.2.2.2, 
 R1, interface FastEthernet 1/0, crypto map MYMAP, 
 R1, ip route 2.2.2.2 255.255.255.255 100.100.0.1, , 
 R1, int loopback 1, ip address 1.1.1.1 255.255.255.255, 
 R1, ip route 100.100.1.2 255.255.255.255 100.100.0.1, , 
 R1, archive,  log config,   notify syslog,
 R1, archiv,  log config,   logging enable,
 R1, archi,  log config,   hidekeys,
 R2, interface FastEthernet 1/0, ip address 100.100.1.2 255.255.255.0, 
 R2, interface FastEthernet 1/0, no shut, 
 R2, crypto isakmp policy 10, encr aes, 
 R2, crypto isakmp policy 10, hash sha, 
 R2, crypto isakmp policy 10, lifetime 3600, 
 R2, crypto isakmp policy 10, group 5, 
 R2, crypto isakmp policy 10, auth pre-share, 
 R2, crypto isakmp key cisco address 0.0.0.0 0.0.0.0, , 
 R2, crypto ipsec transform-set MYSET esp-des esp-sha-hmac, , 
 R2, crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.0.2,
 R2, crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET, 
 R2, crypto map MYMAP 10 ipsec-isakmp, match address 101, 
 R2, ip access-list extended 101, permit ip host 2.2.2.2 host 1.1.1.1, 
 R2, interface FastEthernet 1/0, crypto map MYMAP, 
 R2, ip route 1.1.1.1 255.255.255.255 100.100.0.2, , 
 R2, int loopback 1, ip address 2.2.2.2 255.255.255.255, 
 R2, ip route 100.100.0.2 255.255.255.255 100.100.1.1, , 
 R2, archive, log config, notify syslog,
 R2, archiv,  log config,   logging enable,
 R2, archi,  log config,   hidekeys,tode@ubuntu:~/perfectscripts$

And the output is…

tode@ubuntu:~/perfectscripts$ python3 MassChangeWithAmanda.py critical R1%R2
  interface FastEthernet 1/0
  ip address 100.100.0.2 255.255.255.0
 no shut
 crypto map MYMAP
 crypto isakmp policy 10
  encr aes
 hash sha
 lifetime 3600
 group 5
 auth pre-share
 crypto isakmp key cisco address 0.0.0.0 0.0.0.0
 crypto ipsec transform-set MYSET esp-des esp-sha-hmac
 crypto map MYMAP 10 ipsec-isakmp
  set peer 100.100.1.2
 set transform-set MYSET
 match address 101
 ip access-list extended 101
  permit ip host 1.1.1.1 host 2.2.2.2
 ip route 2.2.2.2 255.255.255.255 100.100.0.1
 int loopback 1
  ip address 1.1.1.1 255.255.255.255
 ip route 100.100.1.2 255.255.255.255 100.100.0.1
 archive
  log config
  notify syslog
 log config
  logging enable
  log config 
  hidekeys
 I am trying to implement this change on router R1 Are you sure? yes
  interface FastEthernet 1/0
  ip address 100.100.1.2 255.255.255.0
 no shut
 crypto map MYMAP
 crypto isakmp policy 10
  encr aes
 hash sha
 lifetime 3600
 group 5
 auth pre-share
 crypto isakmp key cisco address 0.0.0.0 0.0.0.0
 crypto ipsec transform-set MYSET esp-des esp-sha-hmac
 crypto map MYMAP 10 ipsec-isakmp
  set peer 100.100.0.2
 set transform-set MYSET
 match address 101
 ip access-list extended 101
  permit ip host 2.2.2.2 host 1.1.1.1
 ip route 1.1.1.1 255.255.255.255 100.100.0.2
 int loopback 1
  ip address 2.2.2.2 255.255.255.255
 ip route 100.100.0.2 255.255.255.255 100.100.1.1
 archive
  log config
  notify syslog
  archiv
  log config
  logging enable
  archi
  log config
  hidekeys
 I am trying to implement this change on router R2 Are you sure? yes

I can see one big problem here: because commands are grouped, the script is trying to insert the map on interface fa1/0 when the map doesn’t exist yet. This will not be implemented. It will be implemented if the same script runs again. Oh bother. Diffios change log confirms this:

tode@ubuntu:~/perfectscripts$ cat R1_change_delta.bak 
 --- baseline
 +++ comparison
 1: Current configuration : 2239 bytes
 2: interface FastEthernet1/0
 no ip address
 shutdown
 1: Current configuration : 2767 bytes
 2: access-list 101 permit ip host 1.1.1.1 host 2.2.2.2
 3: archive
 logging enable
 4: crypto ipsec transform-set MYSET esp-des esp-sha-hmac
 5: crypto isakmp key cisco address 0.0.0.0 0.0.0.0
 6: crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 5
 lifetime 3600
 7: crypto map MYMAP 10 ipsec-isakmp
 set peer 100.100.1.2
 set transform-set MYSET
 match address 101
 8: interface FastEthernet1/0
 ip address 100.100.0.2 255.255.255.0
 9: interface Loopback1
 ip address 1.1.1.1 255.255.255.255
 10: ip route 100.100.1.2 255.255.255.255 100.100.0.1
 11: ip route 2.2.2.2 255.255.255.255 100.100.0.1
 tode@ubuntu:~/perfectscripts$ cat R2_change_delta.bak 
 --- baseline
 +++ comparison
 1: Current configuration : 1225 bytes
 2: interface FastEthernet1/0
 no ip address
 shutdown
 1: Current configuration : 1791 bytes
 2: access-list 101 permit ip host 2.2.2.2 host 1.1.1.1
 3: archive
 logging enable
 notify syslog contenttype plaintext
 4: crypto ipsec transform-set MYSET esp-des esp-sha-hmac
 5: crypto isakmp key cisco address 0.0.0.0 0.0.0.0
 6: crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 5
 lifetime 3600
 7: crypto map MYMAP 10 ipsec-isakmp
 set peer 100.100.0.2
 set transform-set MYSET
 match address 101
 8: interface FastEthernet1/0
 ip address 100.100.1.2 255.255.255.0
 9: interface Loopback1
 ip address 2.2.2.2 255.255.255.255
 10: ip route 1.1.1.1 255.255.255.255 100.100.0.2
 11: ip route 100.100.0.2 255.255.255.255 100.100.1.1
 tode@ubuntu:~/perfectscripts$  

After small edits, let’s run the script twice and read the change log from the 2nd run.

tode@ubuntu:~/perfectscripts$ cat R1_change_delta.bak 
 --- baseline
 +++ comparison
 tode@ubuntu:~/perfectscripts$ cat R2_change_delta.bak 
 --- baseline
 +++ comparison

This is nice, nothing changes during the 2nd run. Let’s now check if the script is actually trying to implement config during the 2nd (and every subsequent) run.
It is. Oh boy. The reason is that the input file needs to be sanitized. If it’s not (if change commands don’t reflect the existing config 100%) the script will not realize these are the same commands and will try to implement them again.
This is the sanitized input file:

tode@ubuntu:~/perfectscripts$ cat change.txt 
 R1,crypto map MYMAP 10 ipsec-isakmp, match address 101, 
 R1,access-list 101 permit ip host 1.1.1.1 host 2.2.2.2, , 
 R1,interface FastEthernet1/0, ip address 100.100.0.2 255.255.255.0, 
 R1,interface FastEthernet1/0, no shut, 
 R1,interface FastEthernet1/0, crypto map MYMAP, 
 R1,ip route 2.2.2.2 255.255.255.255 100.100.0.1, , 
 R1,interface Loopback1, ip address 1.1.1.1 255.255.255.255, 
 R1,ip route 100.100.1.2 255.255.255.255 100.100.0.1, ,
 R1,archive, log config,  notify syslog,
 R1,archiv,  log config,   logging enable,
 R1,archi,  log config,   hidekeys,
 R2,crypto isakmp policy 10, encr aes, 
 R2,crypto isakmp policy 10, hash sha, 
 R2,crypto isakmp policy 10, lifetime 3600, 
 R2,crypto isakmp policy 10, group 5, 
 R2,crypto isakmp policy 10, authentication pre-share, 
 R2,crypto isakmp key cisco address 0.0.0.0 0.0.0.0, , 
 R2,crypto ipsec transform-set MYSET esp-des esp-sha-hmac, , 
 R2,crypto map MYMAP 10 ipsec-isakmp, set peer 100.100.0.2,
 R2,crypto map MYMAP 10 ipsec-isakmp, set transform-set MYSET, 
 R2,crypto map MYMAP 10 ipsec-isakmp, match address 101, 
 R2,access-list 101 permit ip host 2.2.2.2 host 1.1.1.1, ,  
 R2,interface FastEthernet1/0, ip address 100.100.1.2 255.255.255.0, 
 R2,interface FastEthernet1/0, no shut,  
 R2,interface FastEthernet1/0, crypto map MYMAP, 
 R2,ip route 1.1.1.1 255.255.255.255 100.100.0.2, , 
 R2,interface Loopback1, ip address 2.2.2.2 255.255.255.255, 
 R2,ip route 100.100.0.2 255.255.255.255 100.100.1.1, , 
 R2,archive, log config,  notify syslog,
 R2,archiv,  log config,   logging enable,
 R2,archi,  log config,   hidekeys,tode@ubuntu:~/perfectscripts$ 

and the live output for R2:

crypto isakmp policy 10
  hash sha
 interface FastEthernet1/0
  no shut
 archiv
   log config
    logging enable
 archi
   log config
    hidekeys
 I am trying to implement this change on router R2 Are you sure?

Something is still left. Hash sha is the automatic choice, so doesn’t appear in the config file. This is not something that we need to handle, because sha should no longer be used for VPNs (obsolete).
No shut needs a separate code fix, because if a port is open, there is no command ‚no shut’ on the port.
For the nested commands i don’t have a good solution yet. I believe the whole code needs to be reworked (a single defaultdict with a list of values) for idempotency to work properly.

RAW CODE

Here’s the code with the ‚no shut’ fix:

#!/usr/bin/python3

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse
from unicon import Connection
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys
import shutil
import diffios
import os
import time
import logging
from gtts import gTTS
import re

LEVELS = {'debug': logging.DEBUG,
          'info': logging.INFO,
          'warning': logging.WARNING,
          'error': logging.ERROR,
          'critical': logging.CRITICAL}

level_name = sys.argv[1]
level = LEVELS.get(level_name, logging.NOTSET)
logging.basicConfig(level=level)
unicondevicelist = sys.argv[2].split("%") 

#loading the testbed file with router management IPs
testbed = loader.load('testbed.yml')

#downloading router configuration files and storing them away as .bak files
#x = 0
for x in range(0,len(unicondevicelist)):
    devicename = unicondevicelist[x]
    c = testbed.devices[devicename]
    c.connect(log_stdout=False)
    output = c.execute('show run')
    f = open(devicename, "w")
    f.write(output)
    f.close()
    newdevicename = devicename + '.bak'
    shutil.copyfile(devicename, newdevicename)
    c.destroy()
with open('change.txt', 'r') as reader:
          configchangetext = reader.read()
          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            dataline = line.split(',')
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
          
logging.info('\n')
for k, v in my_dict.items():
  logging.info('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  my_new_dict = my_dict[k].items()
  logging.info('\n')
  for item in my_new_dict:
    x = 0
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        logging.info(newstringordict)
      else:
        for k in newstringordict.keys():
          logging.info(k)
realdevicelist = []
for i in my_dict.keys():
  realdevicelist.append(i)
  for j in my_dict[i].keys():
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
m = 1
while True:  
  try:
    x = 0       
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    if m==1:
      text = "I found " + str(m) + " change for a router that is not in the testbed."
    if m > 1:
      text = "I found another error. "
    logging.critical('\n')
    logging.critical('*******************************ERROR***************************************************************************')
    logging.critical(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
    logging.critical('*******************************ERROR***************************************************************************')
    logging.critical('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    m = m + 1
    continue
  else:
    for k, v in my_dict.items():
      my_new_dict = my_dict[k].items()
      for item in my_new_dict:
      #item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            logging.info(newstringordict)
          else:
            for z in newstringordict.keys():
              logging.info(z)       
    break        
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      my_new_dict = my_dict[router].items()
      for tpl in my_new_dict:
        interface = tpl[0]
        cfgs = tpl[1]
        for key in cfgs.keys():
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #logging.info('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #logging.warning('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change ' + interface + ' ' + key)
                  pass
                else:
                  if key == ' ':
                    pass
                  elif key == ' no shut':
                     findshut = ' shut'
                     shutanswer = parse.find_objects_w_child(interface, findshut)
                     if len(shutanswer) > 0:
                        pass
                     else:
                     	pass
                  else:
                    logging.warning(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                    my_final_dict[router][interface][key] = cfgs[key]
                    #connect to router and implement interface, key
          else:
                logging.warning(interface + ' not found on router ' + router + ' but i will try to create this ' + interface + ' with the value ' + key)
                my_final_dict[router][interface][key] = cfgs[key]
                #connect to router and try to implement
finalstatement = ''
for k, v in my_final_dict.items():
  my_newfinal_dict = my_final_dict[k].items()
  globlist = []
  for item in my_newfinal_dict:
    x = 0
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        globlist.append(newstringordict)
        logging.info(newstringordict)
      else:
        for j in newstringordict.keys():
          globlist.append(j)
          n = newstringordict.get(j)
          globlist.append(n)
  c = testbed.devices[k]
  c.connect(log_stdout=False)
  for line in globlist:
  	print(line)
  decision = input("I am trying to implement this change on router " + k + " Are you sure? " )
  if "yes" in decision:
    output = c.configure(globlist)
    if 'Invalid' in output:
      logging.critical(output)
  else:
    pass
  c.destroy()

unicondevicelist = realdevicelist
x = 0
for x in range(x,len(unicondevicelist)):
    afterchanges = unicondevicelist[x]
    beforechanges = afterchanges + '.bak'
    c = testbed.devices[afterchanges]
    c.connect(log_stdout=False)
    output = c.execute('show run')
    #time.sleep(1)
    f = open(afterchanges, "w")
    f.write(output)
    f.close()
    diff = diffios.Compare(beforechanges, afterchanges)
    deltaoutput = diff.delta()
    filename = afterchanges + '_change_delta.bak'
    f = open(filename, "w")
    f.write(deltaoutput)
    f.close()
    os.remove(afterchanges)
    os.remove(beforechanges)




Skomentuj

Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:

Logo WordPress.com

Komentujesz korzystając z konta WordPress.com. Wyloguj /  Zmień )

Zdjęcie na Google

Komentujesz korzystając z konta Google. Wyloguj /  Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Wyloguj /  Zmień )

Zdjęcie na Facebooku

Komentujesz korzystając z konta Facebook. Wyloguj /  Zmień )

Połączenie z %s