XRv 9000 on ESXi 6.5

[fsebera]

Looking for assistance in getting XRv 9000 operational on ESXi 6.5

I have tried different versions and feature sets of XRv 9000 version 6.5.3, to include the .ova and .iso.

VRR, with VGA, non-VRR. Although the Cisco CSR 1000v installs without issues and runs fine.

The Cisco install guide indicates the XRv 9000 is supported in ESXi 6.5 but doesn't specify any specific Enterprise or Standard versions or vCenter requirement. I run VMware 6.5 Standard version with free license.

I have followed Cisco’s Install Guide. On Install, I add the serial over network server telnet option: telnet://ESXi_Host_IP:2000 for both of the VGA and non-VGA versions.

The VGA version of XRv 9000 vm starts, provides a message saying loading and after a few minutes presents a Welcome to the Cisco IOS XRv9k Platform message. The message also say Please wait for the Cisco IOS XR to start.   Still no response in the vm window ever after 2 days of waiting.  The non VGA version loads with the final message saying to use the serial port to access the cli but the specified port is not open on the host and cannot access via telnet over serial.

The serial port as specified in the Cisco's Install document in Putty to set to 115200, n, 8, 2 stop bits. In Putty, select telnet, ESXi Host IP address and port 2000. I have double checked the serial over network in ESXi firewall is enabled. The ESXi cli command esxicli network ip connection list doesn’t show my specified port is open or even listening on port 2000 but shows other active, listening and established ports. Choosing a different port doesn't change anything.

nc –z ESXi_Host_IP 2000 does not respond but using port 80, 443, 23 and various other open ports response is succeeded.  CLI packet capture on the ESXi host confirms my telnet traffic is reaching the physical interface vmnet0 (management interface) but no response from the XRv 9000 vm. 

The host ESXi server has 383G of Ram, 4TB of free disk space, dual CPU running @ 2.9 GHz. I have shut down all VMs except the XRv 9000 but no go. I have also reloaded the host as well as the XRv vm.

At a complete loss.

Anyone have an idea of what I am doing wrong??

Thanks Frank

[smilstea]

Sorry if this covers what you already checked but are you using vCenter via vSphere client or vCloud Director?

Are you using E1000 data ports as vNICs or VMXNET3?

Most importantly how many vCPUs are allocated? We recommend a minimum of 4.

Thanks,

Sam

[fsebera]

Yes, I have 4 vCPUs and 32GB of ram dedicated and have tried E1000, E1000e, VMXNET2 and VMXNET3.

I only have a single ESXi 6.5 Standard version of VMware. If we ever get this thing operational, we will consider spending the money for additional features and functions but at this point it doesn't work at all.

As I understand it, the VGA version does not require the Serial over Network option and allows CLI access via the VMware VM window the XRv booted into. But have tried both [serial over net and non-serial] options. Both fail.

Thanks

Frank

[fsebera]

.................. as an FYI, this is the screen I'm stuck on.

!###################################################

!#     Welcome to the Cisco IOS XRv9k platform    !#

!#                                                !#

!#   Please wait for the Cisco IOS XR to start.   !#

!#                                                !#

!# Copyright (c) 2014-2017 by Cisco System, Inc.  !#

!###################################################

You can type in the this window but no response it given.

Thanks

Frank

[manmp · Accepted solution]

Hi Frank. Have you created a network port-group in management vlan? XRV9K uses 3 management ports on VM for internal communication and first 3 network ports of the VM need to be assigned to management port-group. 4th one will be used for data port.

Try using OVA VGA file, without serial port configuration. You should be able to directly console using the VM console option. Also, have HDD with minimum 45GB as that is the min supported HDD.

After this is done, XRv9K takes a bit longer time for first time to boot, around 30-45 minutes, and you should get XR prompt to set username/pwd. Let me know how it goes. Initially the console response will be slower, till you login to XR prompt and unshut the management port.

[fsebera]

YES, using the .ova file version 6.5.3 in ESXi 6.5 Standard version [no vCenter, no Enterprise, no vCloud just using the standard free version of VMware ESXi) with standard vSwitches works.  Not sure what was broken before -[perhaps too many moving parts] but it is operational now.   The XRv vm boots within the VMware 6.5 vm console window within 3 minutes with the cli prompt ready to go and accessible.

Wow, what a task.

THANK YOU

Frank

[manmp]

Great to know it worked!

[fsebera]

FYI to anyone else attempting to load this software - After installing the .ova file I could not get the XRv router to load to where I could access the cli even after running/loading overnight. My fix was to reload the ESXi server (of course powering down all running VMs first however in my case the single XRv was the only VM on the box). After the ESXi server reloaded, it took several minutes for the XRv router to become active and accessible. Added a username and password and all is good to go!!!!

XRv software version I am using:  xrv9k-fullk9-x-vga-6.5.3.ova

VMware ESXi 6.5 upgrade 2, free Standard version with standard vSwitch (no vCenter, vCloud, no other add ons).

Thank you again for your assistance.

Frank

[fsebera]

FYI to anyone else attempting to load this software - After installing the .ova file I could not get the XRv router to load to where I could access the cli even after running/loading overnight. My fix was to reload the ESXi server (of course powering down all running VMs first however in my case the single XRv was the only VM on the box). After the ESXi server reloaded, it took several minutes for the XRv router to become active and accessible. Added a username and password and all is good to go!!!!

XRv software version I am using:  xrv9k-fullk9-x-vga-6.5.3.ova

VMware ESXi 6.5 upgrade 2, free Standard version with standard vSwitch (no vCenter, vCloud, not other add ons).

Thank you again for your assistance.

Frank

[mamikhai]

I've been using XRv 9000 since it started. I have dozens in topologies running, and supported customers getting significant amounts of traffic through it.

It should be easy. For the serial access configure as server with telnet://:22000 for instance. Then you can watch the boot process over the serial if you like. May take up to 20 minutes. Then close the serial and reconnect, it will connect to the cli with the banner and asks you to configure a root password, ...

No hard requirements. It consumes a bit of memory. Any tuning will be specific to amounts of control/forwarding needed.

[fsebera]

in VMware 6.5 (Management address 192.168.1.1) for the XRv vm, I add the Serial port 1, serial over network option, choose Use Network, check box connect at power on is checked, choose server option and port URI: box enter telnet://192.168.1.1:22000.

vm serial port connected over network is enabled under the network firewall rules section.  Incoming ports shown for vm serial port connected over network are 1024,23. 

I assume the XRv vm with the server telnet://192.168.1.1:22000 will open this port when loaded and accept traffic though the VMnic0 management interface. 

When I try to connect from a locally connected PC (PC ip is 192.168.1.2/24) I receive a rejected message immediately.

I use Putty with serial port COM1 set at 115200,8 data bits, 2 stop bits, Parity none and Flow control XON/XOFF.

The port 22000 is not open or listening as seen by the ESXi cli command esxcli network ip connection list. Many ports are shown open, established and or listening but not 22000.

When the XRv vm loads, I can see what is happening in the vm console window, I can use the Action button, Guest OS, send keys and send reload with control-Alt-Delete but that is about it.

Thanks

Frank

[mamikhai]

Please add a serial to the VM, with settings as server telnet://:22000 . Not tied to IP (or should be the host IP not management IP). Safer to use exact as telnet://:22000

If needed, adjust ESXi firewall rules to allow incoming connections.

for the client: use telnet not serial client. For me I use Mac OS native:

MAMIKHAI-M-827T:~ mamikhai$ telnet vm1 63107

Trying 10.82.66.121...

Connected to vm1.

Escape character is '^]'.

Username: cisco

Password:

RP/0/RP0/CPU0:P107#show version   

Tue Sep 24 16:55:22.854 UTC

Cisco IOS XR Software, Version 6.4.2

Copyright (c) 2013-2017 by Cisco Systems, Inc.

Build Information:

Built By     : radharan

Built On     : Fri Jul 27 02:39:09 PDT 2018

Build Host   : iox-lnx-090

Workspace    : /auto/srcarchive17/prod/6.4.2/xrv9k/ws

Version      : 6.4.2

Location     : /opt/cisco/XR/packages/

cisco IOS-XRv 9000 () processor

System uptime is 2 weeks, 6 days, 2 hours, 0 minutes

[fsebera]

Mamikhai,

I will attempt your option shortly, THANK YOU for your assistance too

Thanks

Frank