Re: Strange behavior of Business Dashboard

PacketFilter · ‎06-09-2021

Hello all,

we are installing several SX350 devices and configure them with Business Dashboard via SNMP and http(s).

We had to restart our Hypervisor to install Updates and Hardwareupgrades, after each restart of the Dashboard VM a Job, created by System, run to configure every Port as Trunk with all VLANs. How can I stop this?

And how can I update the present installation to the new release (June)?

Thanks and regards,

PacketFilter

David Harper · ‎06-09-2021

Hi PacketFilter,

Can I get you to call the support center and log a case for this problem. What you have described is not expected behaviour, but it is going to be easier to troubleshoot live as there are a number of different things that should be looked at to try and get to the bottom of what is happening. Dashboard will add VLANs and set ports to trunk for devices in a group that has a VLAN profile assigned, but it should only be the VLANs in the profile and it should only be for ports connecting to other network devices in the group. It also should not be happening repeatedly. So there is some troubleshooting needed here.

On the second question, Now that the 2.3.0 release has been published, you should see a green badge on the About icon in the top right of the GUI. If you open the About screen, you will then have the option to click to upgrade. It can take up to a week after the software is published for the badge to appear, but it has been long enough now that it should be there, so if it is not, I would get the support engineer to look at that too.

Alternatively, if you want to skip the troubleshooting and just apply the update, you can download the Linux installer from https://cisco.com/go/cbd-sw and update the system using the steps in the Linux installation guide that you can find at https://cisco.com/go/cbd-docs.

Cheers,

Dave.

PacketFilter · ‎06-09-2021

Hi Dave,

thanks for your reply.

we use Dashboard to add our VLANs to the devices, but I guess this feature is not suitable for this case? If so I need to know how to add VLANs on all devices in our network (we have to add another 20-24 devices).

Is there any way to force the upgrade via cli?

I'll raise a ticket later this day.

Regards,

PacketFilter

David Harper · ‎06-10-2021

There shouldn't be any issue using CBD to add/modify/delete VLANs in the network. The issue here is that the behaviour you have described is not how it is supposed to work. So we need to figure out why this is happening and get it fixed for you.

As for the upgrade, there is no CLI command to trigger the dashboard to download the update and apply it. But you can copy the installer to the Linux VM using scp or similar and then execute it at the bash prompt. Does that work for you?

Cheers,

Dave.

PacketFilter · ‎06-10-2021

I tried to copy the installer into the VM (we use the ova file provided by Cisco), but WinScp reports that the server denies the connection.

Today I added a new Switch (SX350X-52-K9) and added the default configuration via PNP, but under "Inventory" the switch is grayed out (Version and Model is missing). And I don't see any Login attempts from the Dashboard IP.

David Harper · ‎06-10-2021

The SSH server is turned off by default in the VM images. To turn it on temporarily, use the command 'sudo systemd start ssh', and to make the change permanent so that the SSH server will start at boot time, use the command 'sudo systemd enable ssh'.

As for the new switch, it does need a mechanism to discover the dashboard address so it can connect and be provisioned through PnP. There are several options for doing this, including the use of a DHCP option, using DNS lookup, using a Cisco service called PnP Connect, or even using manual config to create the initial connection. Have a look at the Plug and Play Server Discover section of the PnP design guide at the link below. Also pay attention to the section titled Setting up the Server Identity, as the PnP protocol has some fairly stringent security requirements to ensure that the device configuration process can't be compromised. The design guide can be found here:

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/cisco-business-dashboard/technical-reference/PnP_Guide_2_2.html

Cheers,

Dave.

PacketFilter · ‎06-15-2021

Hi Dave,

sorry for the late response. I was able to switch on ssh via "sudo systemctl start ssh", "sudo systemend start ssh" gave me "Excess arguments".

Update ran fine, but after restart as 2.3 the same issue (config VLANs on Devices) occurred again.

On the other note: Are two NICs supported? We have a "Management" VLAN and a "Office" VLAN with Access to www. We would like to add a 2nd NIC for Internet access.

Will raise the ticket after vacation

David Harper · ‎06-15-2021

Sorry about that. My brain must have been a bit broken when I wrote the reply. You are correct of course - the command is systemctl, not systemd.

Two NICs should not be an issue. We don't explicitly test multiple NICs for the dashboard, but there is nothing in the way the application is built that should cause any problems. We *do* test multiple NICs for the probe application as that has implications for discovery, and that is definitely supported.

Can you PM me the case number once you have it open? I'd like to be able to track this one.

Cheers,

Dave.

PacketFilter · ‎06-16-2021

No worries about that, but it might be helpful for some users asking the same about manual updating the Server.

Thanks for your information about two NICs, it is working fine.

About the behavior:

I created a Job to ad our VLANs to the devices which is fine. Is the default behavior to add (all) VLANs to trunk Ports? After setting some Ports to Access the issue was gone for those Ports. Tbh i'm not 100% sure and it needs some more testing.

I'll send you the Case # once its open.

Regards,

PacketF

David Harper · ‎06-19-2021

Hi PacketFilter,

I had a chat to the engineering team about the VLAN behaviour, and it turns out this is actually by design. There was a change made relatively recently to rerun the configuration jobs when a device comes online. This was done to address some problems we had seen with device configurations getting out of sync due to the configurations not being saved to startup before a reboot. In the majority of cases, the jobs do not actually have any impact because the device configuration has been saved, but in cases where the config was not saved it gets repaired automatically as the device comes online.

Having said that, please doublecheck whether the VLAN config is being applied to *all* ports or just to ports connecting to other network devices. The VLANs should only be added to ports where it is needed - not to all ports. So if you are seeing it being added to all ports, then we will still need a case opened. But if the VLANs are only being added to ports connected to other network devices, then we should all be good.

Of course, if the VLAN jobs are causing other problems for you, please provide details of the problem and we will see how best to solve it.

Cheers,

Dave.

PacketFilter · ‎07-05-2021

Hi Dave,

sorry for my late response, but I was on vacation.

However, as far as I see it only affectes Ports where network devices are connected. But how do we stop/prevent working this "feature"? E.g. if a Firewall or AP has several dedicated VLANs configured and Business Dashboard want's to configure all VLANs on those specific Ports, how we can prevent this? A Popup or Notification Mail a lá "Hey Network Guy, Switch XY has now a different config. Please check" should be enough.

We consider to setup a new Server for this, does this affect our previous installed devices?

Thanks and regards

David Harper · ‎07-05-2021

There should not be any configuration applied that was not previously applied when you created the VLAN profile and assigned it to the device group. The VLANs are only being (re-)applied to devices and ports that should already have those VLANs enabled. The only time these jobs should result in an actual change to device config is if that VLAN config has been removed - either by someone using the device GUI/CLI, or as a result of a device being rebooted without having the running configuration saved.

If the behaviour you are seeing is different to this, then that would be a problem that needs to be fixed. But if what you are seeing is the same as what I have described, then I'd like to better understand the reason why you would prefer this didn't happen. Can you give me an example of how this causes problems for you?

Cheers,

Dave.

PacketFilter · ‎07-06-2021

Hi Dave,

if I change a configuration on one of our devices via cli or WebIf, CBD should accept this. Yesterday I configured a SX350X-08 Switch to use all Ports as Access and only one Port as Trunk.

I attached you my config from yesterday (org_fs.txt), today the config has been reverted back to "cbd_fs1.txt". Config was accepted by Switch, no errors occured during copy.

We have dedicated Switches and want those clean without unnecessary stuff like additional VLANs on (e.g.) their Uplink Ports + it's a security consideration.

PS: We are switchting from an old Catalyst environment, that's the main reason why we expect an other behavior

David Harper · ‎07-07-2021

The reason CBD cannot just accept the device config is because VLAN profiles are applied to a device group, and if changes are made at the individual device level then the policy could be applied inconsistently across the group. This could be handled using a notification rather than reapplying the config and that is something we can consider as a future enhancement.

However, I don't think that is the root of the problem here. Based on what you have sent through and some more testing we have done locally, it seems we have a bug. When the VLAN profile is being applied, it should only be automatically enabled on ports connecting network devices in the same device group. However, what is actually happening is that the VLAN is being enabled on all active ports. This isn't correct, and it almost certainly what is overwriting the changes you are trying to make.

I'll continue to work this through with engineering, and will update the thread when we have a fix, or at least, a timeframe for a fix. Would you be interested in trying a test build of CBD with the fix in it to confirm whether this is the root cause of the problem or if there is more to it?

Cheers,

Dave.

PacketFilter · ‎07-07-2021

Hi Dave,

If I understand you right, I should add VLANs during initial config via Network Plug & Play -> Configurations and later, till a Bugfix was provided, manual via CLI or Webinterface? Personally I think it's not a big deal for us because we do not create or delete VLANs regular.

Do you know why VLANs are added in interface config even the interface has been configured as access?

E.G.

interface TenGigabitEthernet1/0/1
 description "Hello Dave"
 switchport access vlan 10
 switchport trunk native vlan 10
 switchport trunk allowed vlan 1,10
!

I would expect (like as I know from Catalyst)

interface TenGigabitEthernet1/0/1
 description "Hello Dave"
 switchport access vlan 10
!

I'm always happy to help engineering, if you have a build for us we'll test it. Thanks for your help (and regards to the engineering team)

Looking forward to hear from you,

PacketF