Background   : As a system administrator, you want to plan for potential hardware or VNF software failures. After a service chain is deployed, you can create a backup. In the event of hardware or software failure, you can restore from the the backup copy and recover the site Hardware and Software versions used ENCS 54xx hardware NFVIS 3.11.2FC2 or above Prerequisites NFVIS 3.11.2FC2 installed on hardware with access to the on-box web GUI. VNF Service chain deployed and in active state.      Step-by-Step Instructions Based on the Backup option chosen(choices in step 2), restore can be executed.         Login to designated ENCS using Google chrome Create a backup of deployed service chain. Backup can be       Config ONLY /  NFVIS config+ALL VNF virtual disk / Select VNF disk. You can select either option but one at a time.  Select VNF virtual disk Delete the deployed service chain Restore from Backup. Picture below show import of a single VNF. Note1 : When restoring ALL VNFs, there is an option of restoring deployed VNFs without impacting the management connectivity, this is important when dealing with remote sites. Note 2 : management connectivity is intact when restoring individual VNFs. Frequently Asked Questions 1. During restore, is having the same platform a best practice or is it enforced in software? It is software enforced. Different platforms have different architectures. Not all features are supported for all platforms. Ex.: Dual wan and DPDK are only supported for ENCS 5400. A backup from ENCS 5400 should not be restored to UCS.   2. If checked in software, what exactly is checked for, and how strict is it? It is a strict check for PID. (nfvis# show platform-details hardware_info PID)   3. Could you restore a 5406 to a 5408? No, we cannot restore a backup from ENCS 5406 to a ENCS 5408. If we add a new feature specific to ENCS 5408 in future, we do not want it to conflict with ENCS 5408   4. Does restored NFVIS need to have the exact same amount of memory, disks, etc? No. As of now, we do not check for disk space. Also, there is no need for restored NFVIS to have exact same amount of memory, disks. However, if any VM is deployed on external disk, restored NFVIS expects an external disk in the same slot to redeploy that VM Please elaborate on what is meant by "VM configurations”. From the context of conf-only backup, VM configurations are NFVIS related VM configurations. In deeper technical terms, it means domain-xml for kvm deployments which included number of CPU and memory required, disk-space, I/O controllers, etc. From conf-and-vms backup, as the VM disks are also backed up, all configurations including configurations within VMs are backed up 5. "If a VM is in an invalid state.." – according to what? nfvis# show vm_lifecycle opdata tenants tenant admin deployment <deployment-name> shows current state of the VM. A VM can be in following states: UNDEFINED_STATE, DEPLOYING_STATE, MONITOR_UNSET_STATE (valid state), MONITOR_DISABLED_STATE, STOPPING_STATE, SHUTDOWN_STATE (valid state), STARTING_STATE, REBOOTING_STATE, INERT_STATE (valid state for unmonitored VMs), ALIVE_STATE (valid state), UNDEPLOYING_STATE, ERROR_STATE (valid state) Explain the format of the backup file - what files are in it and how you can view contents of it? The backup package is a gzipped tarball. It can be inflated using any tar utility. It contains config.xml, manifest file and list of VM export files with their SHA1 checksums About how long does a backup take? The answer to this question would be a relative answer. There is no time bound for backup. It depends on type of backup, size of VM disks, number of VM disks, and storage used for backup. A simple conf-only backup to intdatastore completes within seconds. A conf-and-vms backup with multiple large VM disks to a USB takes 9-10 hours. 6. How/where do you download the backup file via GUI? CLI? The portal shows a list of backup files available on system on Backup-Restore page (Host à Utilities à Backup - Restore). It can be downloaded from there. For CLI, nfvis# show system file-list shows backup files. These can be SCP’ed out of NFVIS or copied to USB drive if disk space permits it.   7. What logs capture backup progress, where should we start with troubleshooting if a problem is encountered? All logs for backup and restore are logged in nfvis_config.log. As usual, debug logs show a detailed information and proves helpful in troubleshooting Any way to run scheduled backups? No. As of now, there is no way of doing this. How "clean" does a system have to be to run a restore? Can you run it on a fully configured system with running VMs, etc? If so, how does config get merged? As long as confd, esc-lite and nfvos-confd daemons are running, a restore can be performed. Yes, it can be performed on fully configured system. In this case, all previous configurations are erased and overwritten by configurations from backup package. (Except connectivity settings when ‘except-connectivity’ option is used) 8. How long does it take to run a restore once the backup is uploaded? This is also relative question. Restore is also not time bound. It depends on the time taken to deploy VMs, number of VMs deployed, storage option used. Can you restore on to same platform with a different IP/hostname? Yes. ’except-connectivity’ option in restore is provided just for this purpose. A restore can be performed on to same platform (different device) with different IP. Hostname will be overwritten. Can you restore on to completely different platform  (5412->CSP) ? No Are there any config paramsthat must match? No. For a restore to start, only PID and NFVIS version should match. 9. Because vm restoration requires registered image, how does “hostaction restore“ recreate this registered image, if there is none during the restoration?   Registered image is not restored from any backup files (vm backup or host backup), it is restored from original downloaded image.   10. Before vmExport/hostaction_backup, the deployment disk is thin-provisioned_link-clone disk (e.g., small size but needs backingStore file), does restored VM from backup (vmImport or hostaction_restore) will restore this kind of disk? [Ans] No. The backup (by vmExport or host backup) will merge *backingStore disk* and this small *link-clone disk* into *thin-provision full-clone disk*. This process is not reversible. The restored VM disk is a full-clone disk ( == backingStore + link-clone disk). The only way to restore a link-clone disk is: backup feature is created by copying this disk into backup files, not merging backingStore with link-clone file.   11. Can we change datastore for restored VM? [Ans] path is hard-coded inside vm backup file, changing datastore is not possible.   12. What disks included in my calculation for end-to-end host restoration? [Ans]  registered_image + backup_file + restore_vm_disk, where:                         Backup_file <= gzipped ( registered_image + link_clone disk)                         Restore_vm_disk = merged (registered_image + link_clone disk) > registered_image We tried the host restoration after deleting the registered image, due to the downloaded image is not deleted, so host restore process was successful. If I deleted both download file and registered image, the host restore process will fail. So to me, that does not make any differences on the calculation above.   13. There are four files we concern about for VM restoration: a. Downloaded package. This can be deleted after registration. b. Register image. Required by deployment and restore/import process. c. Backup files. d. Deployment disk, either disk created from fresh installation or disk restored from backup. Put vmImport/vmExport aside, if Registered image is not existed when doing host restored, disk space need to be bigger than 1+2+3+4 for successful restoration. Registered image is existed when doing host restored, disk space need 2+3+4 for successful restoration.   ... View more

This document provides step by step instruction to deploy vedge-cloud and vm-series paloalto firewall with optimal service chain performance.   Software versions used in this Guide NFVIS version 3.10.1 on ENCS platform vedge-cloud 18.4.0 paloalto vm-series 18.3.1   Login to NFVIS local portal and First step is to upload the vedge-cloud and vm-series-paloalto-firewall image packages into image repository. Image packages can be built using the following instructions. how-to-create-a-vedge-cloud-package-for-nfvis how-to-create-a-paloalto-firewall-package-for-nfvis   CPU type, number of vcpu's, type of networking (SRIOV, OVS, OVS-DPDK) will influence the throughput performance through service chain. For optimal networking, use SRIOV and OVS-DPDK. Enable OVS-DPDK system-wide on NFVIS Enable OVS DPDK Deploy vedge-cloud with site specific information and point the sdwan router to  vManage orchestrator Deploy vedge Deploy paloalto firewall and point to panorama server   Deploy vm-series paloalto firewall Verify vedge in site 122 is discovered in vManage vedge cloud in site 122   Verify vm-series in site 122 is discovered in Panorama server vmseries deployed and licensed   ... View more

 This is a Howto step-by-step instruction for creating a vedge-cloud VNF package for deployment in NFVIS.  What is a VNF package?  VNF package is essentially a set of files bundled for ease of distribution and automation of deployment. VNF package typically contains the disk image(s), bootstrap configuration(s), meta-data that represents the capability, manifest file that confirms integrity.  Why package? Creating a VNF package is not mandatory however it has it advantages Provides a way to scale out deployments.  Support for Day 0 configuration for Cisco and 3rd party VNFs The packaging utility creates a tar.gz file which contains The raw QCOW2 file. Note: ISO, vmdk formats would require conversion to qcow2 before package can be created.  Image properties file Supported and default profiles Day 0 configs Image properties file is created by using either the GUI or using the packaging utility provided with every release. Note: DNAC NFV app is a recommended solution for Automation and Assurance of VNF service chain across 100's of Enterprise sites. VNF package created by NFVIS packaging tool would be imported to DNAC Image repository. How to Package? There are at least 2 common ways to build a package. Method 1. GUI approach, available via NFVIS local portal Method 2. Python script based tool downloaded from NFVIS local portal Method 1   : NFVIS Local Portal based Image Packaging Image Packaging for NFVIS Part 1 Image packaging for NFVIS part 2 Image packaging for NFVIS part 3   Next step is to register the package that was built for use in the local system. Alternately, the package can be downloaded for use via an orchestrator like DNA Center. Note :   For file sizes larger than 1GB, SCP based copy would be a faster and reliable method. By default SCP is disabled in NFVIS system. SCP can be enabled from specific source IP using the following CLI.   nfvis(config)# system settings ip-receive-acl 0.0.0.0/0 service scpd priority 2 action accept nfvis(config-ip-receive-acl-0.0.0.0/0)# commit Method 2 :   Download and Execute python script based tool Refer Method 2 here https://community.cisco.com/t5/network-architecture-documents/how-to-create-a-vm-package-for-nfvis/ta-p/3732265 ... View more

This document describes step by step procedure to Design and Deploy SDWAN Router and Firewall on NFVIS platform.  Hardware and Software used in this guide DNAC 1.2.5 NFVIS Version 3.9.2 FC4 ENCS5408/K9 vedge-cloud Prerequisites NFVIS installed on (supported) hardware and access to the local management Web-GUI. DNAC installed on appliance and access to the management GUI. Step-by-step instructions This document assumes that Sites, Network Settings are created and available in the DNAC system. Upload Images into image repository Import vedge-cloud sdwan and paloalto firewall packages onto DNAC image repository   upload vedge-cloud upload paloalto firewall package for NFVIS Select version for use in design and deployment Create vBranch Network Design   Platform and WAN Select Network Functions   Add Custom Network   Switch Config Custom Config Templates Design Complete Associate Design and Site(s)   Configure vManage access info in DNAC vManage info in DNAC settings Discover device and add to Inventory Discover Device Provision vBranch Provision workflow start Provision Router WAN Address   Complete bootstrap configuration for VNFs   Verify design and Deploy   DNAC starts deploying design with configuration   Provision complete Success ... View more

This is a Howto step-by-step instruction for creating a VNF package for deployment in NFVIS.  What is a VNF package?  VNF package is essentially a set of files bundled for ease of distribution and automation of deployment. VNF package typically contains the disk image(s), bootstrap configuration(s), meta-data that represents the capability, manifest file that confirms integrity.  Why package? Creating a VNF package is not mandatory however it has it advantages Provides a way to scale out deployments.  Support for Day 0 configuration for Cisco and 3rd party VNFs The packaging utility creates a tar.gz file which contains The raw QCOW2 file. Note: ISO, vmdk formats would require conversion to qcow2 before package can be created.  Image properties file Supported and default profiles Day 0 configs Image properties file is created by using either the GUI or using the packaging utility provided with every release. Note: DNAC NFV app is a recommended solution for Automation and Assurance of VNF service chain across 100's of Enterprise sites. VNF package created by NFVIS packaging tool would be imported to DNAC Image repository. How to Package? There are at least 2 common ways to build a package. Method 1. GUI approach, available via NFVIS local portal Method 2. Python script based tool downloaded from NFVIS local portal Method 1 : NFVIS Local Portal based Image Packaging Image Packaging for NFVIS     Image Packaging for NFVIS (Contd.)   Next step is to register the package that was built for use in the local system. Alternately, the package can be downloaded for use via an orchestrator like DNA Center. Note : For file sizes larger than 1GB, SCP based copy would be a faster and reliable method. By default SCP is disabled in NFVIS system. SCP can be enabled from specific source IP using the following CLI.   nfvis(config)# system settings ip-receive-acl 0.0.0.0/0 service scpd priority 2 action accept nfvis(config-ip-receive-acl-0.0.0.0/0)# commit Image Packaging for NFVIS Complete Method 2 : Download and Execute python script based tool Untar the downloaded file and the following files are part of the package. Python 2.7+ environment is required for executing the script. tar -xvf nfvisvmpackagingtool.tar nfvpt.py image_properties_template.xml tool-usage-examples.txt     Example 1 : Usage for ASAv NOTE: --bootstrap1 day0-config (The bootstrap filename **must** be day0-config as ASAv only looks for this filename nfvpt.py -o asav961-201 -i asav961-201.qcow2 -n ASAv -t firewall -r 961-201 --monitored true --bootstrap day0-config:filename1 --min_vcpu 1 --max_vcpu 4 --min_mem 1024 --max_mem 8192 --min_disk 8 --max_disk 16 --vnic_max 8 --optimize true --profile ASAv5,"ASAv5 profile",1,1024,8192 --profile ASAv10,"ASAv10 profile",1,4096,8192 --profile ASAv30,"ASAv30 profile",4,8192,16384 --default_profile ASAv5   Example 2 : Usage for ISRv NOTE: --bootstrap1 ovf-env.xml (The bootstrap filename **must** be ovf-env.xml as ISRv only looks for this filename nfvpt.py -o isrv.16.03.01 -i isrv-universalk9.16.03.01.qcow2 -n ISRv.16.03.01 -t ROUTER -r 16.03.01 --monitored true --privileged true --bootstrap ovf-env.xml:file1,ios-xe.txt:file2 --min_vcpu 2 --max_vcpu 8 --min_mem 4096 --max_mem 8192 --min_disk 8 --max_disk 8 --vnic_max 8 --optimize true --profile ISRv-small,"ISRv small profile",2,4096,8192 --profile ISRv-medium,"ISRv medium profile",4,4096,8192 --default_profile ISRv-small --sriov_list igb,igbvf,i40evf --custom tech_package,ax   Example 3  : Usage for 3rd Party VM with config drive (ISO) mounted at specific path on the VM: nfvpt.py -o test.1.0 -i test-1.0.qcow2 -n TEST -t OTHER -r 1.0 --monitored true --privileged true --bootstrap /:bootstrap.xml,/license/lic.txt:license.txt --min_vcpu 2 --max_vcpu 8 --min_mem 4096 --max_mem 8192 --min_disk 8 --max_disk 8 --vnic_max 8 --optimize true --profile small,"small profile",2,4096,8192 --profile medium,"medium profile",4,4096,8192 --default_profile small in this case test.1.0.pkg : bootstrap.xml get mounted as bootstrap.xml at root and license.txt get mounted as /license/lic.txt   Example 4 : Usage for vedge-cloud  nfvpt.py -o vedge18.3.1 -i viptela-edge-genericx86-64.qcow2  -n vedge.18.03.01 -t ROUTER -r 18.03.01 --monitored false --privileged true --bootstrap /openstack/latest/user_data:cloudinit.cfg,/openstack/latest/meta_data.json:meta_data,/openstack/latest/vendor_data.json:vendor_data --min_vcpu 2 --max_vcpu 8 --min_mem 4096 --max_mem 8192 --min_disk 8 --max_disk 8 --vnic_max 8 --optimize true --nocloud true --profile vEdge-small,"vEdge small  profile",2,4096,8192 --profile vEdge-Standard,"vEdge Standard profile",4,4096,8192 --default_profile vEdge-Standard --custom ORGNAME, --custom OTP, --custom UUID, --custom SYSTEM_IP, --custom VBOND, --custom SITE_ID,   There are 3 files to be passed as bootstrap config files, please follow the information below and create the 3 files and make them available in the same directory as the nfvpt.py File 1 : cloudinit.cfg      Action : save the text below in file, name it cloudinit.cfg     Content-Type: multipart/mixed; boundary="===============8815267485200512281==" MIME-Version: 1.0 --===============8815267485200512281== Content-Type: text/cloud-config; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="cloud-config" #cloud-config vinitparam: - otp : ${OTP} - vbond : ${VBOND} - uuid : ${UUID} - org : ${ORGNAME} --===============8815267485200512281== Content-Type: text/cloud-boothook; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="config-6978c2a4-8f5e-4489-8500-80e8048f60ad.txt" #cloud-boothook system personality vedge device-model vedge-cloud chassis-number ${UUID} ztp-status success config-template-name "vEdge-cloud-template" pseudo-confirm-commit 300 ! system personality vedge device-model vedge-cloud host-name vedgecloud1 system-ip ${SYSTEM_IP} domain-id 1 site-id ${SITE_ID} no route-consistency-check organization-name "${ORGNAME}" vbond ${VBOND} port 12346 aaa auth-order local radius tacacs usergroup basic task system read write task interface read write ! usergroup netadmin ! usergroup operator task system read task interface read task policy read task routing read task security read ! user admin password $6$siwKBQ==$wT2lUa9BSreDPI6gB8sl4E6PAJoVXgMbgv/whJ8F1C6sWdRazdxorYYTLrL6syiG6qnLABTnrE96HJiKF6QRq1 ! ! logging disk enable ! ! ! omp no shutdown graceful-restart advertise connected advertise static ! security ipsec authentication-type sha1-hmac ah-sha1-hmac ! ! vpn 0 name "Transport VPN" interface ge0/0 ip dhcp-client tunnel-interface encapsulation ipsec no allow-service bgp allow-service dhcp allow-service dns allow-service icmp no allow-service sshd no allow-service netconf no allow-service ntp no allow-service ospf no allow-service stun ! no shutdown ! ! vpn 1 interface ge0/1 no shutdown ! ! vpn 512 name "Transport VPN" interface eth0 ip dhcp-client no shutdown ! ! ! ! --===============8815267485200512281==--   File2 : meta_data    Action : save the text below in file, name it meta_data     {"uuid":"myinstance.mydomain.com"}   File3: vendor_data  Action : save the text below in file, name it vendor_data     {"test":"sample"}     Example 5 :  Usage for Ubuntu Linux nfvpt.py -o Ubuntu16.04 -i ubuntu.qcow2  -n ubuntu.16.04 -t OTHER -r 16.04 --monitored false --privileged true --bootstrap user-data:user-data, meta-data:meta_data --min_vcpu 1 --max_vcpu 4 --min_mem 2048 --max_mem 4096 --min_disk 8 --max_disk 20 --vnic_max 4 --optimize true --nocloud true --profile ubuntu-small,"ubuntu small profile",1,2048,8192 --profile ubuntu-medium,"ubuntu medium  profile",2,4096,20480 --default_profile ubuntu-small --custom IP_ADDRESS, --custom NETMASK, --custom UUID, --custom GATEWAY,   There are 2 files to be passed as bootstrap config files, please follow the information below and create the 2 files and make them available in the same directory as the nfvpt.py File 1 : user-data       Action : save the text below in file, name it user-data   #!/bin/bash passwd root << EOF cisco123 cisco123     Y EOF echo "Cloud-init running user-data off config drive (/dev/sr0)" echo Setting up interfaces and addresses ifconfig ens3 down ifconfig ens3 $NICID_0_IP_ADDRESS netmask $NICID_0_NETMASK ifconfig ens4 down ifconfig ens4 $IP_ADDRESS netmask $NETMASK route add default gw $GATEWAY ens3 netstat -rn adduser lab sudo ifconfig ens3 up ifconfig ens4 up sed -i 's/PermitRootLogin no/PermitRootLogin yes/' /etc/ssh/sshd_config sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config service ssh restart File 2 : meta-data      Action : save the text below in file, name it meta-data instance-id: ubuntu-bionic hostname: ubuntu-bionic local-hostname: ubuntu-bionic ... View more

Background : There is growing interest for SDWAN capability on NFVIS based virtual branch. SDWAN XE code enables SDWAN with key features that are part of IOS XE codebase, including WAN diversity option with T1/E1, LTE, etc.  This Howto document provides the required information for creating a virtual branch with SDWAN XE image.    Hardware and Software versions used ENCS 54xx hardware NFVIS 3.8.1 or above SDWAN XE ISRv 16.9.2 or above vManage suite 18.3.1 or above   Prerequisites NFVIS installed on a hardware with access to the on-box web GUI Access to vManage management, path for control connections from SDWAN devices. Device serial file with ISRv specific UUID and Tokens available in vManage.    Step-by-Step Instructions Step 1 : Prepare vmanage, vsmart, vbond to authenticate root-certificate and deploy device certificate after verification of one time use token. Create and attach template to Device ID with site-specific parameters populated. This allows for vmanage to generate bootstrap config for use in step 3. Step 2 : Upload Image and create/deploy the SDWAN XE virtual instance on NFVIS/ENCS. In this step, bootstrap config is generated from vmanage for DeviceID and passed on to the virtual instance deployed at remote site with this specific file name ciscosdwan_cloud_init.cfg Step 3 : Verify that the the SDWAN XE virtual instance on-boarded is accepted by vManage suite.   Step 1 Associate Template in vManage to a device   Confirm Attaching Device to Template is Successful   Generate Bootstrap Configuration   Select and Generate Bootstrap config in Default format   copy-n-paste or download bootstrap configuration  Step 2 : Upload SDWANXE image onto device and Deploy the Router instance   Upload SDWAN XE image into Image repository Image Registration Successful  Deploying an SDWAN instance with day 0 bootstrap configuration.   Deploy XE-SDWAN and ASAv Firewall Verify registration of router to vManage Token is validated and device certificate is pushed ... View more