Dual homed trunks from C9300 to C9606R one 10G and one 1G

[wickedevo]

I have a question pertaining to trunks with C9300 and C9600. The C9600s are my core nodes and C9300 is an end building node. I was upgrading the 1G trunks to 10G trunks that I have dual homed from the C9300 to two separate C9600s. I did one at a time and after swapping in the 10G SFPs on the first trunk link it came right up and was passing L2 and L3 traffic no problem but then I noticed on the 1G trunk that is going to my second C9600 that my OSPF Point-to-Point went down. So I SSH'd into both switches and started looking around. I did a "sh int trunk" and no VLANs were being fwd out of either side and like I said the OSPF link was down. When I did a "sho int status" both sides showed as connected and looking at the switches both still had green lights. It seemed as soon as that 10G trunk came up the 1G trunk just stopped passing traffic. With that in mind I ran out and swapped out the 1G SFPs with 10G SFPs so now both are 10G trunks and when I did that all traffic started passing again and OSPF link came right up. It is probably my ignorance and not understanding the network module that is in the C9300 (C9300-NM-8X) and how the 10G and 1G ports work on it.

So, my question is once I change one trunk to 10G do all others after that have to be 10G? Or is it something to the effect that once I bring up a 10G port on that network module the first 4 ports are used for 10G and the last 4 are for 1G?

I have tried googling and finding an answer but to no avail. Any wisdom from the gurus out there would be greatly appreciated. The more I learn the better!

[MHM Cisco World]

Can you draw topolgy 

MHM

[marce1000]

  - In general and for starters I would say configure a central syslog server first , for all  switches. Then when doing those operations check the logs on the syslog server as they are arriving. Usually you will get lots of insights from switches , if they complain about wrong stuff or other matters.

 M.

[Reza Sharifi]

Hi,

First, the C9300-NM-8X supports both 1 Gig and 10 Gig on all ports. So, it should not matter what port is 10 and what port is 1.

a couple of questions and confirmations

Can you confirm that the 9600s are not configured with StackWise Virtual and that they are separate switches?

If yes, is there an interlink between the 9600s? If yes, is the link a layer-3 link with OSPF enabled?

Are you running OSPF between the 9300 and the 9600s if they are separate? Is the default gateway for all the vlan located on the 9300?

HTH

[wickedevo]

Copy all, give me a few and I will draw it all out to hopefully make it a little easier to understand.

[wickedevo]

I added topology. Hopefully it is legible.

[MHM Cisco World]

your design is not clear why you run ospf between three SW if you have VRRP ??

MHM

[wickedevo]

Sorry I didn’t show my entire topology. I have 4 core nodes (complete mesh together) and several hybrid distro/access switches (all dual homed to 2 different CNs) that do L3/L2 that are set up just like the pic I posted. Those distro/access switches are in buildings with a lot of users so there are user/printer subnets created just for those buildings which have SVIs and then we let OSPF advertise those subnets across the rest of the network. We also have several access switches in buildings with few users that are just L2 to the core nodes which is why I have those vlans with VRRP on the core nodes. I didn’t design it this way someone else did. I am just trying to fix things here and there.

[MHM Cisco World]

Still not clear to me why ospf for vlan use in vrrp.

The vrrp is for vlan in all three SW' ospf need only for retrun traffic' and it config between only core SW .

Anyway check my previous comment 

MHM

[MHM Cisco World]

You have three SW triangle topolgy

This design sure one link must BLK by STP' if you run pvst then not all vlan will blk but only vlan that allow in three link trunk will blk.

As I see it normal there is no issue why you see some change now because you change 10g to 1g' the stp fwd 10g before now after change it to 1g the stp dont prefer it anymore.

So just do some show stp and see the difference before abd after 10g change 

MHM

[Leo Laohoo]

The link was upgraded from 1- to 10 Gbps, right?  

So what is the firmware of the 9300?  Was the configuration of the 1 Gbps interface "migrated" to the 10 Gbps interface?

[wickedevo]

Yes I upgraded one side from 10G to 1G and then the other 1G stopped passing traffic. All my switches are currently on 17.9.5 this includes the C9300s and C9600s. When the switch was running as a 1G interface in the config it was under the Te interface. Both links were so I simply swapped the 1G SFP with the 10G SFP. In my configuration those links were never on int Gx/x/x, it was always Tex/x/x since it was first set up. After that second 1G interface went down I did try to go into the config and create gx/x/x for that interface but it did not come back up. I could see it as gx/x/x when I did a show run but when I did a sho int status only the Te interfaces 1-8 show up.

[Leo Laohoo]

If the 1 Gbps optics are thrown back in does the link go up? 

If they do, what is the type of patch cable?  Is it OM3/OM4?

[wickedevo]

I’ll have to give it a go on Monday and report back what happens. I appreciate yall taking interest!

[Joseph W. Doherty]

Your routing across two gig links, you change one to 10g, and all traffic stops flowing on the remaining gig link.

Well, as both STP and routing protocol can consider bandwidth in choosing a path, is it possible the newly changed 10g link was "better" and the gig link was logically stopped being used?  When you set the other gig link, to 10g, you reported it started passing traffic again, but then again, the two paths would be "equal" again, too.

Possible I misunderstand, because when you note OSPF went down, it dropped the neighbor?  What did the syslog show?