Implementing idempotency in Unicon a.k.a do the desired changes already exist on the router? Part 3: Rewriting the dictionary and getting the delta

Continuing from Part 2, where i found the bug in my ”input dictionary), which eliminated commands if they applied to the same interface. This is because a normal dictionary only allows unique keys, whereas with the defaultdict one can add a list of values to one key.
After a considerable amount of cursing, I rewrote the dictionary method:

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter



with open('change.txt', 'r') as reader
 configchangetext = reader.read()
 splittext = configchangetext.split('\n')
 my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
 for line in splittext:
        dataline = line.split(',')
        my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]

where the input file looks like this:

R1,interface fa0/1, shut,
R1,interface fa0/0, shut,
R1,interface fa0/0, shut,
R1,interface fa0/0, shut,
R1,interface fa0/0, hsrp 10,
R1,interface fa0/0, service-policy QOS,
R2,interface fa0/0, ip address 192.168.0.1 255.255.255.0,
R2,interface fa0/4, desc LAN,
R2,interface fa0/2, desc WAN,
R2,interface fa0/0, desc DMZ,

Now the question is: how to iterate through this dictionary so that we get a config block?

for k, v in my_dict.items():
 print (str(k) + '\n')

 items = my_dict[k].items()
 
 for item in items:
   #print(item)#each item is a tuple consisting of a string and a defaultdict which in turn consists of the command and an empty space...
   x = 0
   for x in range (x, len(item)):
     new_string_or_dict= item[x]
     if isinstance(new_string_or_dict, str):
       print(new_string_or_dict)
     else:
       for k in new_string_or_dict.keys():
         print(k)

So now I have both the dictionary and i know how to loop through this dictionary to generate a config block. Job almost done.

Now some small rewrite of the ”sanitize” method from part 2 blog entry, plus I insert a fake R8 config into the input text file.

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter

with open('change.txt', 'r') as reader:
          configchangetext = reader.read()

          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            
            dataline = line.split(',')
            #for l in dataline:
              #print(l)
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
            


#print(my_dict)
print('\n')
print('***************************************************************************************************************************')
print('*******************************IMPLEMENTATION DRAFT PLAN*******************************************************************')
print('***************************************************************************************************************************')
for k, v in my_dict.items():
  #print (str(k) + '\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_new_dict = my_dict[k].items()
  print('\n')
  for item in my_new_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        print(newstringordict)
      else:
        for k in newstringordict.keys():
          print(k)

        	
        
print('***************************************************************************************************************************')
print('***************************END OF IMPLEMENTATION DRAFT PLAN****************************************************************')
print('***************************************************************************************************************************')
print('\n')

print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('****************************CONFIG SANITIZATION IN PROGRESS****************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
realdevicelist = []
for i in my_dict.keys():
      #i add the devices to the list from dict main keys
  realdevicelist.append(i)
  for j in my_dict[i].keys():
      	#i don't do anything here
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
    #if a config is not found, an exception is caught
while True:
  try:
    x = 0
        #print(str(realdevicelist) + '  ..this is the list ')
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    print('\n')
    print('*******************************ERROR***************************************************************************')
    print(str(e) + '.This may mean that in the change file there is a router mentioned for which i do not have a config.\n This error appeared in ' + realdevicelist[x])
    print('*******************************ERROR***************************************************************************')
    print('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    print('*******************************INFO****************************************************************************')
    print('An error has been found and removed. ')
    print('*******************************INFO****************************************************************************')
    print('\n')
    continue
  else:
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE FINISHED **************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE RESULTS PHASE**********************************************************************')
    print('***************************************************************************************************************************')
    print('After removing all errors from the implementation plan, what follows is the sanitized implementation plan. \n However, I still need to check if it makes sense to implement these changes because these may already be found in existing configs. ')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************FINAL IMPLEMENTATION PLAN*******************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    for k, v in my_dict.items():
    #print (str(k) + '\n')
      print('\n i am showing now how I want router config to be modified for each router: \n\n' + str(k))
      my_new_dict = my_dict[k].items()
      
      for item in my_new_dict:
      #print(item)#item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            print(newstringordict)
          else:
            for k in newstringordict.keys():
              print(k)
    print('*******************************END OF PLAN***********************************************************************')
    break 
    #print('This is the list again ' + str(realdevicelist))  

And the output is quite magnificent, you can see how the changes for the non-existent R8 router are eliminated from the actual change config.

IMPLEMENTATION DRAFT PLAN
***
i am showing now how I want router config to be modified for router R1

interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for router R2

interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN
i am showing now how I want router config to be modified for router R8

interface fa0/8
desc LAN

END OF IMPLEMENTATION DRAFT PLAN***




CONFIG SANITIZATION IN PROGRESS



ERROR**
[Errno 2] No such file or directory: 'R8'.This may mean that in the change file there is a router mentioned for which i do not have a config.
This error appeared in R8
ERROR**
INFO* An error has been found and removed. INFO*


***SANITISE FINISHED **




SANITISE RESULTS PHASE***

After removing all errors from the implementation plan, what follows is the sanitized implementation plan.
However, I still need to check if it makes sense to implement these changes because these may already be found in existing configs.


FINAL IMPLEMENTATION PLAN


i am showing now how I want router config to be modified for each router:
R1
interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for each router:
R2
interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN
END OF PLAN

So now that we have the ‚sanitized’ input config block, it is time to compare this with the actual existing config and only select those commands for implementation that are not in the existing config.

print('\n')          
print('***************************************************************************************************************************')
print('*******************************COMPARE PHASE*******************************************************************************')
print('***************************************************************************************************************************')
    

print('***************************************************************************************************************************')
print('*******************************FINAL PHASE*******************************************************************************')
print('***************************************************************************************************************************')
for router, config in my_dict.items():
    #print (str(router) + '\n')
    #print('i am on router ' + router + '\n')
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      #print('\n final config for ' + str(router) + ':' + ' \n\n' + str(router))

      my_new_dict = my_dict[router].items()
      
      for tpl in my_new_dict:
      #print(item)#cfg is a tuple consisting of interface string and defaultdict config values: 'shut' : ' '
        
        interface = tpl[0]
        cfgs = tpl[1]
        
        for key in cfgs.keys():
          #print('this is a key ' + key)

          #print('this is ' + interface + ' and this is its config ' + key)
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #print('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #print('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change')
                  pass
                else:
                  #print(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                  
                  print(router + ' ' + interface + ' ' + key + '\n')
                #connect to i and implement config
              
              #print(answer)

And the final output of the whole script:

IMPLEMENTATION DRAFT PLAN
***
i am showing now how I want router config to be modified for router R1

interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for router R2

interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN
i am showing now how I want router config to be modified for router R8

interface fa0/8
desc LAN

END OF IMPLEMENTATION DRAFT PLAN***




CONFIG SANITIZATION IN PROGRESS



ERROR**
[Errno 2] No such file or directory: 'R8'.This may mean that in the change file there is a router mentioned for which i do
not have a config. This error appeared in R8
ERROR**
INFO* An error has been found and removed. INFO*


***SANITISE FINISHED **




SANITISE RESULTS PHASE***

After removing all errors from the implementation plan, what follows is the sanitized implementation plan.
This is not the final implementation plan yet, because i still need to compare the plan with actual configs.


IMPLEMENTATION PLAN BEFORE COMPARE PHASE*


i am showing now how I want router config to be modified for each router:
R1
interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for each router:
R2
interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN

END OF PLAN


COMPARE PHASE


FINAL CONFIG DELTA FOR IMPLEMENTATION

R1 interface fa0/0 hsrp 10
R1 interface fa0/0 service-policy QOS
R2 interface fa0/0 ip address 192.168.0.1 255.255.255.0
R2 interface fa0/0 desc DMZ

I can also create another (final) dictionary that will only gather valid, implementable items.
Here’s the change.txt file

R1,interface fa0/1, shut,
R1,interface fa0/0, desc LAN,
R1,interface fa0/0, shut,
R1,interface fa0/0, hsrp 10,
R1,interface fa0/0, service-policy QOS,
R2,interface fa0/0, ip address 192.168.0.1 255.255.255.0,
R2,interface fa0/4, desc LAN,
R2,interface fa0/2, desc WAN,
R2,interface fa0/0, desc DMZ,
R8,interface fa0/8, desc LAN,
R9,interface loop1, desc LOOP,
R2,interface loop1, desc LOOP,

here’s the R1 file

hostname R1
!
interface loop1
ip address 1.1.1.1 255.255.255.255
shut
!
interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc LAN
shut
!
logging buffered

Here’s the R2 file

hostname R2
!
interface loop1
ip addr 2.2.2.2 255.255.255.255
interface fa0/0
ip addr 192.168.1.1 255.255.255.0

And here’s the (perhaps) latest form of the script.

from collections import defaultdict
import json
import pprint
import collections
from operator import itemgetter
from ciscoconfparse import CiscoConfParse

with open('change.txt', 'r') as reader:
          configchangetext = reader.read()

          splittext = configchangetext.split('\n')
          my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
          for line in splittext:
            
            dataline = line.split(',')
            #for l in dataline:
              #print(l)
            my_dict[dataline[0]][dataline[1]][dataline[2]] = dataline[3]
            


#print(my_dict)
print('\n')
print('***************************************************************************************************************************')
print('*******************************IMPLEMENTATION DRAFT PLAN*******************************************************************')
print('***************************************************************************************************************************')
for k, v in my_dict.items():
  #print (str(k) + '\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_new_dict = my_dict[k].items()
  print('\n')
  for item in my_new_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        print(newstringordict)
      else:
        for k in newstringordict.keys():
          print(k)
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^')
print('\n\n\n')        	
        
print('***************************************************************************************************************************')
print('***************************END OF IMPLEMENTATION DRAFT PLAN****************************************************************')
print('***************************************************************************************************************************')
print('\n')

print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('****************************CONFIG SANITIZATION IN PROGRESS****************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
print('***************************************************************************************************************************')
realdevicelist = []
for i in my_dict.keys():
      #i add the devices to the list from dict main keys
  realdevicelist.append(i)
  for j in my_dict[i].keys():
      	#i don't do anything here
    pass
    #let's now use this list of keys (R1, R2, R3, R4 in this case) and using those names let's open the configs from desktop.
    #if a config is not found, an exception is caught
while True:
  try:
    x = 0
        #print(str(realdevicelist) + '  ..this is the list ')
    for x in range(x,len(realdevicelist)):
      with open(realdevicelist[x], 'r') as reader:
        config = reader.read()
        my_config_as_list = config.split("\n")
  except FileNotFoundError as e:
    print('\n')
    print('*******************************ERROR***************************************************************************')
    print(str(e) + '.This may mean that in the change file there is a router mentioned for which i do \nnot have a config. This error appeared in ' + realdevicelist[x])
    print('*******************************ERROR***************************************************************************')
    print('\n')
    del my_dict[realdevicelist[x]]
    realdevicelist.remove(realdevicelist[x])
    print('*******************************INFO****************************************************************************')
    print('An error has been found and removed. ')
    print('*******************************INFO****************************************************************************')
    print('\n')
    continue
  else:
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE FINISHED **************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('*******************************SANITISE RESULTS PHASE**********************************************************************')
    print('***************************************************************************************************************************')
    print('After removing all errors from the implementation plan, what follows is the sanitized implementation plan. \n This is not the final implementation plan yet, because i still need to compare the plan with actual configs.  ')
    print('\n\n\n')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    print('*******************************IMPLEMENTATION PLAN BEFORE COMPARE PHASE****************************************************')
    print('***************************************************************************************************************************')
    print('***************************************************************************************************************************')
    for k, v in my_dict.items():
    #print (str(k) + '\n')
      print('\n i am showing now how I want router config to be modified for each router: \n\n' + str(k))
      my_new_dict = my_dict[k].items()
      
      for item in my_new_dict:
      #print(item)#item is a tuple consisting of string and default dict
        x = 0
        for x in range (x, len(item)):
          newstringordict= item[x]
          if isinstance(newstringordict, str):
            print(newstringordict)
          else:
            for z in newstringordict.keys():
              print(z)
    print('\n')          
    print('***************************************************************************************************************************')
    print('*******************************END OF PLAN*********************************************************************************')
    print('***************************************************************************************************************************')
    
    break 

print('\n')          
print('***************************************************************************************************************************')
print('*******************************COMPARE PHASE*******************************************************************************')
print('***************************************************************************************************************************')
    

print('***************************************************************************************************************************')
print('*******************************FINAL CONFIG DELTA FOR IMPLEMENTATION*******************************************************')
print('***************************************************************************************************************************')
my_final_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
for router, config in my_dict.items():
    #print (str(router) + '\n')
    #print('i am on router ' + router + '\n')
    with open(router, 'r') as reader:
      config = reader.read()
      my_config_as_list = config.split("\n")
      parse = CiscoConfParse(my_config_as_list)
      #print('\n final config for ' + str(router) + ':' + ' \n\n' + str(router))

      my_new_dict = my_dict[router].items()
      
      for tpl in my_new_dict:
      #print(item)#cfg is a tuple consisting of interface string and defaultdict config values: 'shut' : ' '
        
        interface = tpl[0]
        cfgs = tpl[1]
        
        for key in cfgs.keys():
          #print('this is a key ' + key)

          #print('this is ' + interface + ' and this is its config ' + key)
          answer = parse.find_objects('^' + interface)
          if len(answer) > 0:
                #print('a subkey from the input config dictionary matches a parent object from the actual config. This does not tell us anything yet. Maybe the value will be modified, maybe not ') 
                second_answer = parse.find_objects_w_child(interface, key)
                if len(second_answer) > 0:
                  #print('i am on ' + router + ' in subkey ' + interface + ' in value ' + key + ' and I have found a match in the existing configuration of ' + router + ' so i am not going to implement this change')
                  pass
                else:
                  #print(' no match  in config of ' + router + ' found when on ' + router + ' in subkey ' + interface + ' in value ' + key + ' so i will implement this change ')
                  
                  my_final_dict[router][interface][key] = ' '
                  #print(router + '\n ' + interface + '\n ' + key + '\n')
                  #connect to router and implement interface, key


for k, v in my_final_dict.items():

  print('-------------------------------------------------------------------------------------------------------------------------')
  print('\ni am showing now how I FINALLY want router config to be modified for router ' + str(k) + '  ' + '\n\n')
  print('-------------------------------------------------------------------------------------------------------------------------')
  my_newfinal_dict = my_final_dict[k].items()
  print('\n')
  for item in my_newfinal_dict:
    #print(item)#item is a tuple consisting of string and default dict
    x = 0
    
    for x in range (x, len(item)):
      newstringordict= item[x]
      if isinstance(newstringordict, str):
        print(newstringordict)
      else:
        for k in newstringordict.keys():
          print(k)

This now produces the following output:

IMPLEMENTATION DRAFT PLAN
***
i am showing now how I want router config to be modified for router R1

interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for router R2

interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN
interface loop1
desc LOOP
i am showing now how I want router config to be modified for router R8

interface fa0/8
desc LAN
i am showing now how I want router config to be modified for router R9

interface loop1
desc LOOP
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

END OF IMPLEMENTATION DRAFT PLAN***




CONFIG SANITIZATION IN PROGRESS



ERROR**
[Errno 2] No such file or directory: 'R8'.This may mean that in the change file there is a router mentioned for which i do
not have a config. This error appeared in R8
ERROR**
INFO* An error has been found and removed. INFO*
ERROR**
[Errno 2] No such file or directory: 'R9'.This may mean that in the change file there is a router mentioned for which i do
not have a config. This error appeared in R9
ERROR**
INFO* An error has been found and removed. INFO*


***SANITISE FINISHED **




SANITISE RESULTS PHASE***

After removing all errors from the implementation plan, what follows is the sanitized implementation plan.
This is not the final implementation plan yet, because i still need to compare the plan with actual configs.


IMPLEMENTATION PLAN BEFORE COMPARE PHASE*


i am showing now how I want router config to be modified for each router:
R1
interface fa0/1
shut
interface fa0/0
desc LAN
shut
hsrp 10
service-policy QOS
i am showing now how I want router config to be modified for each router:
R2
interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface fa0/4
desc LAN
interface fa0/2
desc WAN
interface loop1
desc LOOP

END OF PLAN


COMPARE PHASE


FINAL CONFIG DELTA FOR IMPLEMENTATION**
***
i am showing now how I FINALLY want router config to be modified for router R1

interface fa0/0
hsrp 10
service-policy QOS
i am showing now how I FINALLY want router config to be modified for router R2

interface fa0/0
ip address 192.168.0.1 255.255.255.0
desc DMZ
interface loop1
desc LOOP

The next step is to test this in GNS3: I will change the code so that the script downloads actual configs of GNS routers, then I will add the part where the script should push out the compare delta to execute the changes.

Skomentuj

Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:

Logo WordPress.com

Komentujesz korzystając z konta WordPress.com. Wyloguj /  Zmień )

Zdjęcie na Google

Komentujesz korzystając z konta Google. Wyloguj /  Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Wyloguj /  Zmień )

Zdjęcie na Facebooku

Komentujesz korzystając z konta Facebook. Wyloguj /  Zmień )

Połączenie z %s