Exporting inventory from Prime to Netbox part 1. Creating per-site inventory .yaml files from Prime inventory.


It’s a simple scenario – you have all your devices in Prime and you want to start playing with your devices using Genie or Cisco PyATS but you don’t want to start creating .yaml inventories from scratch.

Let’s assume that your device names follow a pattern xxxWROyyy where the middle three letters stand for a site name and that you have a number of sites.

First, let’s create a set with all your site (abbreviation) names:

def get_all_devices_csv():
    url = "https://prime.yourcompany.com/webacs/api/v4/data/Devices?.full=true"
    response = requests.get(url, auth=primeCreds, verify=False)
    obj=xmltodict.parse(response.text)
    internal_obj = obj['queryResponse']['entity']
    unique_site_set = set()

    for item in internal_obj:
        name = item['devicesDTO']['deviceName']
        fixed_name = name[3:6]
        unique_site_set.add(fixed_name)

    return unique_site_set

We will use that set later in a loop that will go through all sites.

Now let’s create a prime query in the following way: we will try to search for all device names that contain a given abbreviation (e.g WRO):

def query_prime(filterword, searched_thing):
         url="https://yourcompany.com/webacs/api/v4/data/Devices?.full=true&.sort=ipAddress&deviceName=" + filterword + "(%22"+searched_thing+"%22)"
     response=requests.get(url,auth=primeCreds,verify=False)
     obj=xmltodict.parse(response.text)
     internal_obj=obj['queryResponse']
     return internal_obj

Now for the most important part: a function that creates a dictionary with device ip addresses and os type.

def parse_prime_query_response(response_object):
    
    device_topology = {}

    if 'entity' in response_object.keys():
        query_response = response_object['entity']
        if isinstance(query_response, list):
           for item in query_response:
               device=item['devicesDTO']
               device_name=device['deviceName']
               device_name_short=device_name.replace('.yourcompany.com', '')
               device_type=device['deviceType']
               device_ip=device['ipAddress']
               #sometimes the field location is not there if someone forgot to fill it in on Prime
               if 'location' in device.keys():
                  device_location=device['location']
               else:
                  device_location='default location'
               try:
                  manuf_nrs_dict=device['manufacturerPartNrs']
                  manuf_list=manuf_nrs_dict['manufacturerPartNr']
               except:
                  print(f'error creating entry for {device_name_short}')
                #we look here at a specific model
               if device['deviceType']=='Cisco Catalyst 29xx Stack-able Ethernet Switch' or device['deviceType']=='Cisco Catalyst 68xx Virtual Switch' or device['deviceType']=='Cisco 3750 Stackable Switches' or device['deviceType']=='Cisco 2821 Integrated Services Router' or device['deviceType']=='Cisco Catalyst 6500 Virtual Switching System' or device['deviceType']=='Cisco Catalyst 2960CX-8PC-L Switch':
                  device_topology[device_name_short]={}
                  device_topology[device_name_short]['ip_address'] = device_ip
                  device_topology[device_name_short]['device_os'] = 'ios'
               elif device['deviceType']=='Cisco Catalyst38xx stack-able ethernet switch' or device['deviceType']=='Cisco Catalyst 4500 Virtual Switching System' or device['deviceType']=='Cisco Catalyst 9300 Switch':
                  device_topology[device_name_short]={}
                  device_topology[device_name_short]['ip_address'] = device_ip
                  device_topology[device_name_short]['device_os'] = 'iosxe'
               elif device['deviceType']=='Cisco 5508 Wireless LAN Controller':
                   try:
                     device_topology[device_name_short]={}
                     device_topology[device_name_short]['ip_address'] = device_ip
                     device_topology[device_name_short]['device_os'] = 'other'
                   except Exception as e:
                     print('something went wrong here')
               else:
                   try:
                     device_topology[device_name_short]={}
                     device_topology[device_name_short]['ip_address'] = device_ip
                     device_topology[device_name_short]['device_os'] = 'other'
                   except Exception as e:
                       print('something went wrong if type is not listed in code')

        else:
                   device=entity['devicesDTO']
                   if device['deviceType']=='Cisco Catalyst 29xx Stack-able Ethernet Switch':
                    print('oooh there is one device on this site only')
                     #TODO #if no results for this query meaning a dummy site with zero devices but this should never happen
    #else:
        #print('no results for this query')
        #quit()
    return device_topology

Now we need a function that takes the dictionary with device data and creates a topology file for each site.

def create_topology_file(filename, device_topology):
  
     filepath = os.path.join('./topologies', filename)
     f = open(filepath, 'w')
     writer = csv.writer(f)
     header = ['hostname','ip' , 'username' ,'password' , 'protocol' , 'os']
     writer.writerow(header)
     try:
         for key,value in device_topology.items():
             ip_address = device_topology[key]['ip_address']
             device_os = device_topology[key]['device_os']
             row = [key, ip_address, user, passkey, 'ssh', device_os]
             writer.writerow(row)
     except:
         print('the query did not return any results')
         quit()
     print('PHASE TWO: GENERATING A TOPOLOGY FILE FOR A LOCATION. PLEASE WAIT.')

Now we need a function that glues it all together:

def create_site_csv_for_yaml(filterword, searched_thing, user, passkey):
         internal_obj=query_prime(filterword, searched_thing)
         device_topology= parse_prime_query_response(internal_obj)
         filename = 'devices_' + filterword + '-' + searched_thing + '.csv'
         create_topology_file(filename, device_topology)

Finally, let’s loop through all the sites to create separate .csv files for each site:

for x in get_all_devices_csv():
  create_site_csv_for_yaml(filterword, x, user, passkey)

Then I created a bash script that I will use to control the script flow (there will be much more to do in the future).

#!/bin/bash

while [[ "$#" -gt 0 ]]; do
            case $1 in
                -l|--location) location="$2"; shift ;;
                -f|--filterword) prime_filterword="$2"; shift ;;
                -q|--query_content) query_content="$2"; shift ;;
                -n|--netbox) netbox='true' ;;
                *) echo "Unknown parameter passed: $1"; exit 1 ;;
            esac
            shift
done
echo "Please enter your admin username"

read user
echo "Please enter your password"
read -s passkey

echo "Please enter your netbox token"
read -s token

python3 first_python.py --filterword $prime_filterword --user $user --passkey $passkey
FILES="./topologies/*"
for f in $FILES
do
  y=${f%.*}
  pyats create testbed file --path $f --output $y.yaml --encode-password
done
rm ./topologies/*.csv

For now, I will only use part of the available options with
./do_all_bash.sh -f ‚contains’

The result is:

with each file containing my devices per site:

 r01wro001-vss:
    connections:
      cli:
        ip: 172.16.0.1
        protocol: ssh
    credentials:
      default:
        password: '%ENC{hashedkey}'
        username: myadminuser
      enable:
        password: '%ENC{hashedkey}'
    os: iosxe
    type: iosxe
  r01wro002:
    connections:
      cli:
        ip: 172.16.0.2
        protocol: ssh
    credentials:
      default:
        password: '%ENC{hashedkey}'
        username: myadminuser
      enable:
        password: '%ENC{hashedkey}'
    os: iosxe
    type: iosxe
  r01wro003:
    connections:
      cli:
        ip: 172.16.0.3
        protocol: ssh
    credentials:
      default:
        password: '%ENC{hashedkey}'
        username: myadminuser
      enable:
        password: '%ENC{hashedkey}'
    os: iosxe
    type: iosxe

And the little bits and pieces that are necessary to run this:

import pandas as pd
import os
import xlrd
import argparse
import pynetbox
import requests
import pprint
import json
import warnings
import re
import sys
import xmltodict
import pprint
import getopt
import csv
from csv import reader, writer
from requests.auth import HTTPBasicAuth
from requests.packages.urllib3.exceptions import InsecureRequestWarning
warnings.filterwarnings("ignore")
from genie.testbed import load

p = argparse.ArgumentParser()
p.add_argument('--filterword', type=str)
p.add_argument('--user', type=str)
p.add_argument('--passkey', type=str)
args = p.parse_args()

user = args.user
passkey = args.passkey
filterword = args.filterword


primeServer="prime.yourcompany.com"
primeCreds = HTTPBasicAuth(user, passkey)

Skomentuj

Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:

Logo WordPress.com

Komentujesz korzystając z konta WordPress.com. Wyloguj /  Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Wyloguj /  Zmień )

Zdjęcie na Facebooku

Komentujesz korzystając z konta Facebook. Wyloguj /  Zmień )

Połączenie z %s

%d blogerów lubi to: