This was an interesting project I worked on. Unfortunately much of the code around accessing devices depends on our own in-house inventory system for device info. I wanted to document the project with enough code snippetts to be useful to someone else who might look to do this in their environment without having to wade through all of the custom modules we have for inventory. In the end, we're able to have installation teams run through an area simply replacing access points at pretty much the speed it takes them to print a new label and swap the AP. The AP gets installed in the proper ap group and most importantly for our RTLS customers, gets automatically placed in the right location on the map. All with no need for the installers to carry around an electronic device, or for my team to need to do any custom work in Prime.
Because CopyAndReplace is not an API endpoint, there is some risk this could change on us in the future. If it becomes a supported API this sort of effort would be even easier. And if it gets modified we'll have to take what we know and attempt to figure out how to retool to support it.
Goal Summary:
- Allow the installers to pull an unconfigured access point (AP) out of the box and swap an older model for a new model without needing to search for a specific 1:1 replacement.
- Automate as much of the AP replacement routine as possible including
- Configuring the new AP
- same or similar name as the AP it was replacing.
- same AP group as the AP it was replacing.
- Placing the new AP in the same spot on the Prime maps as the AP it was replacing.
- Coordinating timing with the installers to minimize the time that an unconfigured AP was in service.
- Configuring the new AP
Use Story:
Our customers use Stanley Mobileview to sync our PRIME/MSE data for Real Time Location Services. Two of these services include refrigerated medical storage temperature monitoring and staff duress incident response. Because of this, significant numbers of APs not properly located on maps creates impact to critical services. We needed to be able to replace APs and locate them on the map in a relatively timely fashion. Both the installation team and the networking team were low on resources for any more manual work than necessary.
We found that the CopyAndReplace function in Cisco Prime could do the work we wanted in one step. However, it was not an API exposed endpoint. And it would still require some coordination between the installers and the networking team.
We could perform individual CopyAndReplace functions but we'd need a way of knowing old and new AP. There is also a bulk CopyAndReplace tool that could load CSVs but we didn't want to wait for multiple APs worth of info from the install team.
We set out to automate the coordination of events with just a webhook that would tell my team that an AP had been replaced and provide the needed info to manually execute a CopyAndReplace. We ended up finding a way to automate the whole process by looking at the CopyAndReplace URL in Fiddler and figuring out how to make the call ourselves.
Process Overview:
Query Prime API for details of any AP in an unreachable state.
From the details, grab the last known CDP Neighbor switch/port for each unreachable AP.
Using napalm, get LLDP neighbor from that switch/port.
If a new AP is learned on that switch port:
Query Prime API for the new AP and ensure it is unconfigured by finding it in the "default-group" AP group
Use CopyAndReplace AP function to create a Prime job to replace the AP - This is not an API call. Details below on how we managed to access this function.
Query Prime API for status of the job and log when the job is completed successfully or use the Teams webhook to announce when a job has failed.
Results:
Cronning this script to run every 5 minutes a New AP is detected and if everything completes successfully the new AP is copied from the old one within 15-20 minutes.
Programming overview:
Get from prime the AccessPointDetails for any UNREACHABLE or UNKNOWN AP
aps=conn.Get('data/AccessPointDetails', full='true',filter='reachabilityStatus=in("UNREACHABLE","UNKNOWN")')Read the cdp neighbor from the accessPointDetailsDTO
neighborIP=ap['cdpNeighbors']['cdpNeighbor'][0]['neighborIpAddress']['address'] neighborPort=ap['cdpNeighbors']['cdpNeighbor'][0]['neighborPort']
Use Napalm to get the LLDP neighbors from the swtich
lldp_neighbors=device.get_lldp_neighbors_detail()
Find any neighbor on the same neighbor port and see if the 'remote_system_description' contains 'Cisco AP Software". If it does, and the new AP isn't the same as the AP that was reported down by Prime. Kick off the copy replace job
try:
apGroupName=newap['unifiedApInfo']['apGroupName']
except Exception as e:
log.error('noNewAPGroupName newap={}'.format(newap['name']))
if apGroupName!='default-group' :
return False
macDmacAString='{},{},{},{}'.format(oldap['name'],oldapmac,newap['name'],newapmac)
data={
"macDmacAString":macDmacAString,
"copyLocation":True
}
headers = {
'Authorization': 'Basic BASICAUTHSTRING=',
'Content-Type': 'application/json',
}
try:
with requests.Session() as s:
AuthHeaders={'Content-Type': 'application/x-www-form-urlencoded'}
AuthPayload={'j_username':AUTHUSER,'j_password':AUTHPASSWORD,
'spring-security-redirect':'/pages/common/index.jsp?&selectedCategory=en',
'action':'login'}
AuthResponse=s.post('<https://prime/webacs/j_spring_security_check>',
headers=AuthHeaders,
data=AuthPayload,
verify=False,
allow_redirects=True)
AuthFollow=s.get('<https://prime/webacs/loginAction.do?action=login&product=wcs&selectedCategory=en>',
verify=False)
CopyReplaceresponse = s.post("<https://prime/webacs/rs/rfm-config/CopyReplace/singleCopy>",
verify=False,
headers=headers,
data=json.dumps(data))
Logout=s.get('<https://prime/webacs/j_spring_security_logout>')
except Exception as e:
log.error("APReplacementFailed macDmacAString={} error={}".format(macDmacAString,e))
payload = {
"text":
'FAILED CopyReplace [PrimeCopyReplaceURL](<https://prime/webacs/loginAction.do?action=login&product=wcs&selectedCategory=en#pageId=copyreplace_AP_pageId&forceLoad=true>) {}'
.format(macDmacAString)
}
response = requests.post(webhookURL, headers=None, data=json.dumps(payload))
if 'loginAction.do' in CopyReplaceresponse.text:
#This means our Login failed for some reason and we have to abort
log.error('APReplacementFailed macDmacAString={} error=PrimeLoginError'.format(macDmacAString))
payload={
"text":'APCopyReplaceFailed macDmacAstring={} error=PrimeLoginError'.format(macDmacAString)
}
response = requests.post(webhookURL, headers=None, data=json.dumps(payload))
return False
elif 'JOBID' in CopyReplaceresponse.text:
# SUCCESS Check the status of the JOBID in 1 minute
jobResult=conn.Get('data/JobSummary', full='true',filter='id={}'.format(jobid))
jobStatus=jobResult['entity'][0]['jobSummaryDTO']['jobStatus']
resultStatus=jobStatus=jobResult['entity'][0]['jobSummaryDTO']['resultStatus']
if jobStatus!='COMPLETED' or resultStatus !='SUCCESS':
log.error('CopyReplaceJobIssue jobid={} jobstatus={} resultstatus={}'.format(jobid,jobStatus,resultStatus))
payload={
"text":'CopyReplaceJobIssue jobid={} jobstatus={} resultstatus{}'.format(jobid,jobStatus,resultStatus)
}
response = requests.post(webhookURL, headers=None, data=json.dumps(payload)) The API calls are fairly well documented. The initial plan was to simply use API calls and alert the network team that an AP had been replaced via webhook and then require the network team to log into prime and use the copy replace functionality manually. However with a little digging we were able to find a way to call that command remotely as if it were an API call.
Learning how to POST the CopyReplace/singleCopy using requests
Fiddler to decrypt HTTPS
I installed fiddler on my workstation and enabled HTTPS decrypting. https://docs.telerik.com/fiddler/configure-fiddler/tasks/decrypthttps With that I was able to see the POST call to the CopyReplace/singleCopy endpoint. However I found I could not call that endpoint directly using requests. The response from Prime would indicate that I was not logged in and could not make a non-API call without logging in. So I decrypted more from fiddler, and looked at some github examples from before Prime had much of an API exposed to find the sequence I needed to do.
Login
First step is to POST to webacs/j_spring_security_check. You need to use provide the fields j_username and j_password in your payload, and you need to use a header for content-type of 'application/x-www-form-urlencoded' This is what is passed by the prime login page when you type a username and password.. You'll get back a Cookie with a JSESSIONID. by using a requests.Session the cookie will persist across the section which is required. Requests did not seem to follow the redirect so the next step is to issue a GET to webacs/loginAction.do. Now you have convinced Prime that you're a logged in web user.
(Please forgive that all the JSESSIONIDs below are different they all came from efforts to document the process at different times)
CopyReplace/singleCopy
using fiddler I was able to find the json format for the CopyReplace/singleCopy endpoint. since my JSESSIONID endpoint was persisted, I could provide that json as payload in the post. the response is a Jobname and jobid "SingleApCopy_AIR1-217-TEL_to_APAC7A.56D0.F3A8JOBID47162146103"
Logout
necessary or I would hit my maximum number of logins for a web user (5)
Cut-sheets
An additional step done to try to make the job easier on the installation team was to provide cut-sheets. For this project we are replacing 3502 APs. We could not find a way to create a clean map in prime with only 3502 APs displayed. For that we used the MSE API, the Python Image Library (Pillow), and xhtml2pdf (to generate a printable PDF)
Get the maps and info.
The first request gets the map image the second one gets a json of the "info" for the map which includes all the AP and their XY cordinates
headers = {
'Authorization': 'Basic BASICAUTH=',
'Accept': 'image/*',
}
image=requests.get(url='{}{}'.format(mapsimageurl,Floor), verify=False, headers=headers,stream=True)
headers['Accept']='application/json'
mapsinfo=requests.get(url='{}{}'.format(mapsinfourl,Floor), verify=False, headers=headers)Load the map image and info
the MSE reports the mapinfo and the AP locations in terms of "FEET". However Pillow is referring to an image with pixels so we need to create a ratio of pixels per "FOOT". (the number of Feet in a map is given to Prime at the building creation and may not be accurate.) (apimage is a small wireless icon)
floormap=Image.open(image.raw)
mapsinfojson=json.loads(mapsinfo.text)
apimage=Image.open('./R_apclip.png')
floormap=floormap.convert('RGB')
image_editable=ImageDraw.Draw(floormap)
xratio=floormap.width/mapsinfojson['Floor']['Dimension']['width']
yratio=floormap.height/mapsinfojson['Floor']['Dimension']['length']Position the APs on the map
for ap in mapsinfojson['Floor']['AccessPoint']:
if '350' in ap['ApInterface'][0]['antennaPattern']:
apdict={}
apdict['apname']=ap['name']
apx=int(xratio*ap['MapCoordinate']['x'])
apy=int(yratio*ap['MapCoordinate']['y'])We call the Prime API to get the AP Serial Number and CDP neighbor for some additional info for the installers.
Place the AP icon and the apName on the map at the xy coordinates reported by the MSE (after being multiplied by the ratio). Export a 300x300 crop of the image with the AP location in the center
floormap.paste(apimage,(apx,apy),apimage)
image_editable.text((apx,apy),apdict['apname'],font=fnt, fill=(255,0,0))
cropx=int(croppct*floormap.width)
cropy=int(croppct*floormap.height)
im1=floormap.crop((apx-cropx,apy-cropy,apx+cropx,apy+cropy))
im1=im1.resize((300,300))
im1.save('./{}.png'.format(apdict['apname']),'PNG')save a full-sized map with all the added icons and labels
floormap.save("./{}.png".format(filename))We write all of this out to an HTML cutsheet with links to the pdf version of the cutsheet and the full-sized map...
There's a fair amount of variability in the full-sized images from the MSE... so in the case of smaller buildings the cut sheets may be zoomed in too close, but those buildings are usually no more than 5 APs per floor so it's not really necessary.
Final Instructions for installers
This allows the instructions to the installers to be fairly simple and efficient
Pull new AP from box
Label new AP with the same label as the existing AP.
Replace AP.
Place one of the 3 stickers that come with the AP on to the cutsheet next to that AP. This allows for a double-check or recovery if the automated CopyReplace fails
Indicate on the map or cutsheet if the AP location is inaccurate by more than a few feet. This allows us to update the map to be more accurate
2 Comments
