Implementing idempotency in Unicon a.k.a do the desired changes already exist on the router? Part 4: implementation of logic fixes

This time the script connects to routers in GNS3, downloads the config, parses the change file, gets the delta (makes a list of commands from it for each router) and implements it using Unicon configure.

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse
from unicon import Connection
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys


unicondevicelist = ['R1', 'R2', 'R3']
print(unicondevicelist[0])
testbed = loader.load('testbed.yml')
x = 0
for x in range(x,len(unicondevicelist)):
    devicename = unicondevicelist[x]
    c = testbed.devices[devicename]
    c.connect()
    output = c.execute('show run')
    f = open(devicename, "w")
    f.write(output)
    f.close()
    c.destroy()

with open('change.txt', 'r') as reader:
          configchangetext = reader.read()

          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            
            dataline = line.split(',')
            #for l in dataline:
            print(dataline)
              #print(l)
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
            


#print(my_dict)
print('\n')
print('***************************************************************************************************************************')
print('*******************************IMPLEMENTATION DRAFT PLAN*******************************************************************')
print('***************************************************************************************************************************')
for k, v in my_dict.items():
  #print (str(k) + '\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_new_dict = my_dict[k].items()
  print('\n')
  for item in my_new_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        print(newstringordict)
      else:
        for k in newstringordict.keys():
          print(k)
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('\n\n\n')        	
        
print('***************************************************************************************************************************')
print('***************************END OF IMPLEMENTATION DRAFT PLAN****************************************************************')
print('***************************************************************************************************************************')
print('\n')

print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('****************************CONFIG SANITIZATION IN PROGRESS****************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
realdevicelist = []
for i in my_dict.keys():
      #i add the devices to the list from dict main keys
  realdevicelist.append(i)
  for j in my_dict[i].keys():
      	#i don't do anything here
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
    #if a config is not found, an exception is caught
while True:
  try:
    x = 0
        #print(str(realdevicelist) + '  ..this is the list ')
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    print('\n')
    print('*******************************ERROR***************************************************************************')
    print(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
    print('*******************************ERROR***************************************************************************')
    print('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    print('*******************************INFO****************************************************************************')
    print('An error has been found and removed. ')
    print('*******************************INFO****************************************************************************')
    print('\n')
    continue
  else:
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE FINISHED **************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE RESULTS PHASE**********************************************************************')
    print('***************************************************************************************************************************')
    print('After removing all errors from the implementation plan, what follows is the sanitized implementation plan. \n This is not the final implementation plan yet, because i still need to compare the plan with actual configs.  ')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************IMPLEMENTATION PLAN BEFORE COMPARE PHASE****************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    for k, v in my_dict.items():
    #print (str(k) + '\n')
      print('\n i am showing now how I want router config to be modified for each router: \n\n' + str(k))
      my_new_dict = my_dict[k].items()
      
      for item in my_new_dict:
      #print(item)#item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            print(newstringordict)
          else:
            for z in newstringordict.keys():
              print(z)
    print('\n')          
    print('***************************************************************************************************************************')
    print('*******************************END OF PLAN*********************************************************************************')
    print('***************************************************************************************************************************')
    
    break 

print('\n')          
print('***************************************************************************************************************************')
print('*******************************COMPARE PHASE*******************************************************************************')
print('***************************************************************************************************************************')
    

print('***************************************************************************************************************************')
print('*******************************FINAL CONFIG DELTA FOR IMPLEMENTATION*******************************************************')
print('***************************************************************************************************************************')
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
    #print (str(router) + '\n')
    #print('i am on router ' + router + '\n')
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      #print('\n final config for ' + str(router) + ':' + ' \n\n' + str(router))

      my_new_dict = my_dict[router].items()
      
      for tpl in my_new_dict:
      #print(item)#cfg is a tuple consisting of interface string and defaultdict config values: 'shut' : ' '
        
        interface = tpl[0]
        cfgs = tpl[1]
        
        for key in cfgs.keys():
          #print('this is a key ' + key)

          #print('this is ' + interface + ' and this is its config ' + key)
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #print('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #print('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change')
                  pass
                else:
                  #print(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                  
                  my_final_dict[router][interface][key] = ' '
                  #print(router + '\n ' + interface + '\n ' + key + '\n')
                  #connect to router and implement interface, key


for k, v in my_final_dict.items():

  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I FINALLY want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_newfinal_dict = my_final_dict[k].items()
  print('\n')
  globlist = []
  for item in my_newfinal_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        globlist.append(newstringordict)
        print(newstringordict)
      else:
        for j in newstringordict.keys():
          globlist.append(j)
          #print(j)
  print(globlist)
  print('i am in ' + str(k) + '\n')
  c = testbed.devices[k]
  c.connect()
  output = c.configure(globlist)
  c.destroy()

There is some interesting stuff that’s happening here, and still the script keeps implementing one-line commands, such as logging buffered informational, which needs to be fixed. A slight change in the code, then:

for tpl in my_new_dict:
      #print(item)#cfg is a tuple consisting of interface string and defaultdict config values: 'shut' : ' '
        
        interface = tpl[0]
        cfgs = tpl[1]
        
        for key in cfgs.keys():
          #print('this is a key ' + key)

          #print('this is ' + interface + ' and this is its config ' + key + ' and this is value ' + cfgs[key])
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #print('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #print('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change ' + interface + ' ' + key)
                  pass
                else:
                  if key == ' ':
                    #print('parent found but detailed match parent and key not found because this is a dummy value. not doing anything (cases like logging buffered informational')
                    pass
                  else:
                    print(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                  
                    my_final_dict[router][interface][key] = cfgs[key]
                   
          else:
                print(interface + ' not found on router ' + router)
                my_final_dict[router][interface][key] = cfgs[key]
                
for k, v in my_final_dict.items():

  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\nHere i am showing now how I FINALLY want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_newfinal_dict = my_final_dict[k].items()
  print('\n')
  globlist = []
  for item in my_newfinal_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        globlist.append(newstringordict)
        print(newstringordict)
      else:
        for j in newstringordict.keys():
          globlist.append(j)
          #print(j)
  print(globlist)
  print('i am in ' + str(k) + '\n')
  c = testbed.devices[k]
  c.connect()
  output = c.configure(globlist)
  c.destroy()

I’ve changed two things here: first i added a condition where the parent (like logging buffered informational command) was found in the config but then the value of the tuple is empty, because there can be no subcommand here. Secondly, i’ve added a general Else statement, because sometimes the parent is not found at all (like a missing interface, or a missing general command), and it will have to be implemented.

The dictionary my_final_dict ”gathers” all the data for which implementation conditions are met.

The next step was to log all the changes resulting from the change.txt and remove the interim files. To this end, i copy the files to .bak files at the beginning of the script, then I download the config at the end of the script once again from all routers and do a final diffios compare.

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse
from unicon import Connection
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys
import shutil
import diffios
import os
import time



unicondevicelist = ['R1', 'R2', 'R3']
testbed = loader.load('testbed.yml')
x = 0
for x in range(x,len(unicondevicelist)):
    devicename = unicondevicelist[x]
    c = testbed.devices[devicename]
    c.connect()
    output = c.execute('show run')
    f = open(devicename, "w")
    f.write(output)
    f.close()
    newdevicename = devicename + '.bak'
    shutil.copyfile(devicename, newdevicename)
    c.destroy()

with open('change.txt', 'r') as reader:
          configchangetext = reader.read()

          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            
            dataline = line.split(',')
            #for l in dataline:
            print(dataline)
              #print(l)
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
            


#print(my_dict)
print('\n')
print('***************************************************************************************************************************')
print('*******************************IMPLEMENTATION DRAFT PLAN*******************************************************************')
print('***************************************************************************************************************************')
for k, v in my_dict.items():
  #print (str(k) + '\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_new_dict = my_dict[k].items()
  print('\n')
  for item in my_new_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        print(newstringordict)
      else:
        for k in newstringordict.keys():
          print(k)
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('\n\n\n')        	
        
print('***************************************************************************************************************************')
print('***************************END OF IMPLEMENTATION DRAFT PLAN****************************************************************')
print('***************************************************************************************************************************')
print('\n')

print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('****************************CONFIG SANITIZATION IN PROGRESS****************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
realdevicelist = []
for i in my_dict.keys():
      #i add the devices to the list from dict main keys
  realdevicelist.append(i)
  for j in my_dict[i].keys():
      	#i don't do anything here
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
    #if a config is not found, an exception is caught
while True:
  try:
    x = 0
        #print(str(realdevicelist) + '  ..this is the list ')
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    print('\n')
    print('*******************************ERROR***************************************************************************')
    print(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
    print('*******************************ERROR***************************************************************************')
    print('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    print('*******************************INFO****************************************************************************')
    print('An error has been found and removed. ')
    print('*******************************INFO****************************************************************************')
    print('\n')
    continue
  else:
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE FINISHED **************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE RESULTS PHASE**********************************************************************')
    print('***************************************************************************************************************************')
    print('After removing all errors from the implementation plan, what follows is the sanitized implementation plan. \n This is not the final implementation plan yet, because i still need to compare the plan with actual configs.  ')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************IMPLEMENTATION PLAN BEFORE COMPARE PHASE****************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    for k, v in my_dict.items():
    #print (str(k) + '\n')
      print('\n i am showing now how I want router config to be modified for each router: \n\n' + str(k))
      my_new_dict = my_dict[k].items()
      
      for item in my_new_dict:
      #print(item)#item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            print(newstringordict)
          else:
            for z in newstringordict.keys():
              print(z)
    print('\n')          
    print('***************************************************************************************************************************')
    print('*******************************END OF PLAN*********************************************************************************')
    print('***************************************************************************************************************************')
    
    break 

print('\n')          
print('***************************************************************************************************************************')
print('*******************************COMPARE PHASE*******************************************************************************')
print('***************************************************************************************************************************')
    

print('***************************************************************************************************************************')
print('*******************************FINAL CONFIG DELTA FOR IMPLEMENTATION*******************************************************')
print('***************************************************************************************************************************')
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
    #print (str(router) + '\n')
    #print('i am on router ' + router + '\n')
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      #print('\n final config for ' + str(router) + ':' + ' \n\n' + str(router))

      my_new_dict = my_dict[router].items()
      
      for tpl in my_new_dict:
      #print(item)#cfg is a tuple consisting of interface string and defaultdict config values: 'shut' : ' '
        
        interface = tpl[0]
        cfgs = tpl[1]
        
        for key in cfgs.keys():
          #print('this is a key ' + key)

          #print('this is ' + interface + ' and this is its config ' + key + ' and this is value ' + cfgs[key])
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #print('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #print('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change ' + interface + ' ' + key)
                  pass
                else:
                  if key == ' ':
                    #print('parent found but detailed match parent and key not found because this is a dummy value. not doing anything (cases like logging buffered informational')
                    pass
                  else:
                    print(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                  
                    my_final_dict[router][interface][key] = cfgs[key]
                    #print(router + '\n ' + interface + '\n ' + key + '\n')
                    #connect to router and implement interface, key
          else:
                print(interface + ' not found on router ' + router + ' but i will try to create this ' + interface + ' with the value ' + key)
                my_final_dict[router][interface][key] = cfgs[key]
                #connect to router and try to implement

finalstatement = ''
for k, v in my_final_dict.items():

  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\nHere i am showing now how I FINALLY want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_newfinal_dict = my_final_dict[k].items()
  print('\n')
  globlist = []

  for item in my_newfinal_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        globlist.append(newstringordict)
        print(newstringordict)
      else:
        for j in newstringordict.keys():
          globlist.append(j)
          #print(j)
 
  c = testbed.devices[k]
  c.connect()
  finalstatement += ' On ' 
  finalstatement += k
  finalstatement += ' i have tried to configure '
  finalstatement += str(globlist)
  finalstatement += ' \n\n '
  

  output = c.configure(globlist)
  c.destroy()

print(finalstatement + '\n\n')
#now download new config again and diffios against old config to see what has really been configured.

print('***************FINAL COMPARE PHASE, FOR A DETAILED CHANGE LOG FILE SEE CHANGE_DELTA.BAK FILES*******************')


unicondevicelist = ['R1', 'R2']
x = 0
for x in range(x,len(unicondevicelist)):
    afterchanges = unicondevicelist[x]
    beforechanges = afterchanges + '.bak'

    c = testbed.devices[afterchanges]
    c.connect()
    output = c.execute('show run')
    time.sleep(2)
    f = open(afterchanges, "w")
    f.write(output)
    f.close()
    #time.sleep(10)


    baseline = beforechanges
    
    time.sleep(3)
    comparison = afterchanges
    
    time.sleep(3)
    #ignore = 'ignore.txt'
    diff = diffios.Compare(baseline, comparison)
    time.sleep(1)
    #print('****************THIS IS DIFFIOS FOR DELTA ' + afterchanges + '***********************************************')
    deltaoutput = diff.delta()
    filename = afterchanges + '_change_delta.bak'

    f = open(filename, "w")
    f.write(deltaoutput)
    f.close()
    #print('****************THIS IS END OF DIFFIOS FOR ' + afterchanges + '***********************************************')
    os.remove(afterchanges)
    os.remove(beforechanges)


print('*******************************THIS IS THE END OF ALL OUTPUT**********************************************')

So if my change input is the following command batch (which includes fake routers, fake interfaces, bad commands (like service-policy input QOS which will not be implemented because policy QOS doesn’t exist):

tode@ubuntu:~$ cat change.txt
R1,interface FastEthernet2/0, shut,
R1,interface FastEthernet0/0, description LAN,
R1,interface FastEthernet1/0, shut,
R1,interface FastEthernet1/0, service-policy input QOS,
R2,interface FastEthernet2/0, ip address 192.168.0.1 255.255.255.0,
R2,interface FastEthernet2/0, description LAN,
R2,interface FastEthernet0/2, description WAN,
R2,interface FastEthernet0/0, description DMZ,
R8,interface FastEthernet4/0, description LAN,
R9,interface Loopback1, description LOOP,
R9,interface Loopback1, ip address 1.1.1.1 255.255.255.0,
R2,interface Loopback1, description LOOP,
R1,logging buffered errors, , 

Then my real change output is this:

tode@ubuntu:~$ cat R1_change_delta.bak
--- baseline
+++ comparison


--- Current configuration : 2239 bytes
--- logging buffered informational

+++ Current configuration : 2249 bytes
+++ interface FastEthernet0/0
+++  description LAN
+++ logging buffered errors

and this:

tode@ubuntu:~$ cat R2_change_delta.bak
--- baseline
+++ comparison


--- Current configuration : 1225 bytes
    interface FastEthernet2/0
--- no ip address

+++ Current configuration : 1337 bytes
    interface FastEthernet0/0
+++ description DMZ
    interface FastEthernet2/0
+++ description LAN
+++ ip address 192.168.0.1 255.255.255.0
    interface Loopback1
+++ description LOOP
+++ no ip address

Next steps

  • I should probably create an error.log file as well to collect all the stuff that is wrong with the input (non-existing routers, failed attempts to create interfaces, configs which were already there on the router so were not implemented etc.).
  • I should also think about deletion commands (no ip address etc.), and multiple nested commands (like QoS).
  • Maybe the script should also ask the user if he agrees to implement config whenever there is a need to do it (interactive change).
  • I should get rid of all hardcoded values


By the way the script breaks if there is an end of line character at the end of change.txt
This can be fixed with the following commands in vi before saving.

set noeol
set nofixendofline

This is now a version with the logging implemented and some hardcoded values removed.

#!/usr/bin/python3

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse
from unicon import Connection
from jinja2 import Environment, FileSystemLoader
import yaml
from pyats.topology import loader
import sys
import shutil
import diffios
import os
import time
import logging

LEVELS = {'debug': logging.DEBUG,
          'info': logging.INFO,
          'warning': logging.WARNING,
          'error': logging.ERROR,
          'critical': logging.CRITICAL}


level_name = sys.argv[1]
level = LEVELS.get(level_name, logging.NOTSET)
logging.basicConfig(level=level)

unicondevicelist = sys.argv[2].split("%")
print(unicondevicelist) 


testbed = loader.load('testbed.yml')


x = 0
for x in range(x,len(unicondevicelist)):
    devicename = unicondevicelist[x]
    c = testbed.devices[devicename]
    c.connect(log_stdout=False)
    output = c.execute('show run')
    f = open(devicename, "w")
    f.write(output)
    f.close()
    newdevicename = devicename + '.bak'
    shutil.copyfile(devicename, newdevicename)
    c.destroy()

with open('change.txt', 'r') as reader:
          configchangetext = reader.read()
          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            dataline = line.split(',')  
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
            
time.sleep(1)
logging.info('\n')
logging.info('***************************************************************************************************************************')
logging.info('*******************************IMPLEMENTATION DRAFT PLAN*******************************************************************')
logging.info('***************************************************************************************************************************')
for k, v in my_dict.items():
  logging.info('-------------------------------------------------------------------------------------------------------------------------')
  logging.info('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  logging.info('-------------------------------------------------------------------------------------------------------------------------')
  my_new_dict = my_dict[k].items()
  logging.info('\n')
  for item in my_new_dict:
    #item is a tuple consisting of string and default dict
    x = 0
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        logging.info(newstringordict)
      else:
        for k in newstringordict.keys():
          logging.info(k)
  time.sleep(2)
logging.info('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
logging.info('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
logging.info('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
logging.info('\n\n\n')        	      
logging.info('***************************************************************************************************************************')
logging.info('***************************END OF IMPLEMENTATION DRAFT PLAN****************************************************************')
logging.info('***************************************************************************************************************************')
logging.info('\n')
logging.info('***************************************************************************************************************************')
logging.info('***************************************************************************************************************************')
logging.info('***************************************************************************************************************************')
logging.info('****************************CONFIG SANITIZATION IN PROGRESS****************************************************************')
logging.info('***************************************************************************************************************************')
logging.info('***************************************************************************************************************************')
logging.info('***************************************************************************************************************************')

realdevicelist = []
for i in my_dict.keys():
      #i add the devices to the list from dict main keys
  realdevicelist.append(i)
  for j in my_dict[i].keys():
      	#i don't do anything here
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
    #if a config is not found, an exception is caught
while True:
  try:
    x = 0        
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    logging.critical('\n')
    logging.critical('*******************************ERROR***************************************************************************')
    logging.critical(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
    logging.critical('*******************************ERROR***************************************************************************')
    logging.critical('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    logging.info('*******************************INFO****************************************************************************')
    logging.info('An error has been found and removed. ')
    logging.info('*******************************INFO****************************************************************************')
    logging.info('\n')
    continue
  else:
    logging.info('************************************************************************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('****************************SANITISE FINISHED **************************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('\n\n\n')
    logging.info('************************************************************************************************************************')
    logging.info('*******************************SANITISE RESULTS PHASE**********************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('After removing all errors from the implementation plan, what follows is the sanitized implementation plan. \n This is not the final implementation plan yet, because i still need to compare the plan with actual configs.  ')
    logging.info('\n\n\n')
    logging.info('************************************************************************************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('*******************************IMPLEMENTATION PLAN BEFORE COMPARE PHASE****************************************************')
    logging.info('************************************************************************************************************************')
    logging.info('************************************************************************************************************************')
    for k, v in my_dict.items():
      logging.info('\n i am showing now how I want router config to be modified for each router: \n\n' + str(k))
      my_new_dict = my_dict[k].items()
      for item in my_new_dict:
      #item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            logging.info(newstringordict)
          else:
            for z in newstringordict.keys():
              logging.info(z)
    logging.info('\n')          
    logging.info('************************************************************************************************************************')
    logging.info('*******************************END OF PLAN*********************************************************************************')
    logging.info('************************************************************************************************************************')
    break 
logging.info('\n')          
logging.info('***************************************************************************************************************************')
logging.info('*******************************COMPARE PHASE*******************************************************************************')
logging.info('***************************************************************************************************************************')
    
logging.info('***************************************************************************************************************************')
logging.info('*******************************FINAL CONFIG DELTA FOR IMPLEMENTATION*******************************************************')
logging.info('***************************************************************************************************************************')
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      my_new_dict = my_dict[router].items()
      for tpl in my_new_dict:
        interface = tpl[0]
        cfgs = tpl[1]
        for key in cfgs.keys():
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #logging.info('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #logging.warning('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change ' + interface + ' ' + key)
                  pass
                else:
                  if key == ' ':
                    #logging.warning('parent found but detailed match parent and key not found because this is a dummy value. not doing anything (cases like logging buffered informational')
                    pass
                  else:
                    logging.warning(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                    my_final_dict[router][interface][key] = cfgs[key]
                    #connect to router and implement interface, key
          else:
                logging.warning(interface + ' not found on router ' + router + ' but i will try to create this ' + interface + ' with the value ' + key)
                my_final_dict[router][interface][key] = cfgs[key]
                #connect to router and try to implement
finalstatement = ''

for k, v in my_final_dict.items():
  logging.info('-------------------------------------------------------------------------------------------------------------------------')
  logging.info('\nHere i am showing now how I FINALLY want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  logging.info('-------------------------------------------------------------------------------------------------------------------------')
  my_newfinal_dict = my_final_dict[k].items()
  logging.info('\n')
  globlist = []
  for item in my_newfinal_dict:
    #item is a tuple consisting of string and default dict
    x = 0
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        globlist.append(newstringordict)
        logging.info(newstringordict)
      else:
        for j in newstringordict.keys():
          globlist.append(j)
          #logging.info(j)
  c = testbed.devices[k]
  c.connect(log_stdout=False)
  finalstatement += ' On ' 
  finalstatement += k
  finalstatement += ' i have tried to configure '
  finalstatement += str(globlist)
  finalstatement += ' \n\n '
  #here implement interactivity
  output = c.configure(globlist)
  logging.critical(output)
  #here implement check if weird return from router, e.g. invalid something
  c.destroy()

logging.info(finalstatement + '\n\n')
logging.warning('***************COMPARE IN PROGRESS****FOR A DETAILED CHANGE LOG FILE SEE CHANGE_DELTA.BAK FILES*******************')


unicondevicelist = realdevicelist
x = 0
for x in range(x,len(unicondevicelist)):
    afterchanges = unicondevicelist[x]
    beforechanges = afterchanges + '.bak'
    c = testbed.devices[afterchanges]
    c.connect(log_stdout=False)
    output = c.execute('show run')
    time.sleep(2)
    f = open(afterchanges, "w")
    f.write(output)
    f.close()
    #time.sleep(10)
    baseline = beforechanges
    time.sleep(3)
    comparison = afterchanges
    time.sleep(3)
    #ignore = 'ignore.txt'
    diff = diffios.Compare(baseline, comparison)
    time.sleep(1)
    deltaoutput = diff.delta()
    filename = afterchanges + '_change_delta.bak'
    f = open(filename, "w")
    f.write(deltaoutput)
    f.close()
    os.remove(afterchanges)
    os.remove(beforechanges)
logging.info('*******************************THIS IS THE END OF ALL OUTPUT**********************************************')


Now let’s execute this script. After the modifications i’m using a logging arg and a list of routers that I want to execute the change on. The output is much less verbose by default

tode@ubuntu:~$ python3 testtest.py critical R1%R2%R3
['R1', 'R2', 'R3']
CRITICAL:root:
CRITICAL:root:ERROR**
CRITICAL:root:[Errno 2] No such file or directory: 'R8'.This may mean that in the change file there is a router mentioned for which i do
not have a config. This error appeared in R8
CRITICAL:root:ERROR**
CRITICAL:root:
CRITICAL:root:
CRITICAL:root:ERROR**
CRITICAL:root:[Errno 2] No such file or directory: 'R9'.This may mean that in the change file there is a router mentioned for which i do
not have a config. This error appeared in R9
CRITICAL:root:ERROR**
CRITICAL:root:
CRITICAL:root:interface FastEthernet0/0
description LAN
interface FastEthernet1/0
service-policy input QOS
% policy map QOS not configured
logging buffered errors
CRITICAL:root:interface FastEthernet2/0
ip address 192.168.0.1 255.255.255.0
description LAN
interface FastEthernet0/2
interface FastEthernet0/2
^
% Invalid input detected at '^' marker.
description WAN
description WAN
^
% Invalid input detected at '^' marker.
interface FastEthernet0/0
description DMZ
interface Loopback1
description LOOP

----------------------------------------

tode@ubuntu:~$ cat R1_change_delta.bak
--- baseline
+++ comparison
1: Current configuration : 2239 bytes
2: logging buffered informational
1: Current configuration : 2249 bytes
2: interface FastEthernet0/0
description LAN
3: logging buffered errors
tode@ubuntu:~$ cat R2_change_delta.bak
--- baseline
+++ comparison
1: Current configuration : 1225 bytes
2: interface FastEthernet2/0
no ip address
1: Current configuration : 1337 bytes
2: interface FastEthernet0/0
description DMZ
3: interface FastEthernet2/0
description LAN
ip address 192.168.0.1 255.255.255.0
4: interface Loopback1
description LOOP

I will continue this in Implementing idempotency in Unicon a.k.a do the desired changes already exist on the router? Part 5: negation commands and interactive changes

Skomentuj

Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:

Logo WordPress.com

Komentujesz korzystając z konta WordPress.com. Wyloguj /  Zmień )

Zdjęcie na Google

Komentujesz korzystając z konta Google. Wyloguj /  Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Wyloguj /  Zmień )

Zdjęcie na Facebooku

Komentujesz korzystając z konta Facebook. Wyloguj /  Zmień )

Połączenie z %s