C9300 RSPAN not working

[vsurresh]

Hello. I'm trying to setup RSPAN to capture all the traffic come in and out of a switch port. I double checked all the configs but the traffic is not getting mirrored properly. Please see the attached diagram. 

1. The server is connected to a 9300 switch port which is configured with access vlan 10. 
2. The 9300 is then connected to another 9500 switch via trunk port where both VLAN 10 and 100 (remote-span) are allowed.
3. I have wireshark running on a PC connected to the 9500 with default configuration. 

c9300(config)#vlan 100
c9300(config)#remote-span
c9300(config)#monitor session 10 source interface Gi1/0/2
c9300(config)#monitor session 10 source interface Gi1/0/3
c9300(config)#monitor session 10 destination remote vlan 100 

c9500(config)#vlan 100
c9500(config)#remote-span
c9500(config)#monitor session 10 source remote vlan 100
c9500(config)#monitor session 10 destination interface Twe1/0/10

On wireshark, I can only see a very few packets such as broadcast and some L2 traffic between the servers. What am I missing? If I take a capture directly on the switch using 'monitor capture xxx' command, I can see all the traffic. 

Appreciate any response. 

[Georg Pauwen]

Hello,

does Vlan 100 (the RSPAN Vlan) exist on the 9500 switch (sh vlan) ?

[vsurresh]

Thanks for the response. Yes, the vlan exists on both switches and allowed on the trunk (can verify using 'show interface trunk')

[balaji.bandi]

Can you post below output : ( from both the switches?)

#show monitor session 10
#show vlan

i will also try this see if that can capture only Server vlan 10 traffic. :

c9300(config)#vlan 100
c9300(config)#remote-span
c9300(config)#monitor session 10 source interface Gi1/0/2 both
c9300(config)#monitor session 10 source interface Gi1/0/3 both
c9300(config)#monitor session 10 filter vlan 10
c9300(config)#monitor session 10 destination remote vlan 100 

c9500(config)#vlan 100
c9500(config)#remote-span
c9500(config)#monitor session 10 source remote vlan 100
c9500(config)#monitor session 10 destination interface Twe1/0/10 ingress vlan 10

[vsurresh]

Thanks for the response. I tried your config (including filter vlan and ingress) but it didn't make a difference. 

Here is the output:

9300#show vlan
100  Remote-Span_VLAN                 active 

9300#show monitor session 10
Session 10
----------
Type                     : Remote Source Session
Source Ports             : 
    Both                 : Gi1/0/2,Gi1/0/3
Dest RSPAN VLAN          : 100




9500#show vlan
100  VLAN0100                         active   

9500#show monitor session 10
Session 10
----------
Type                     : Remote Destination Session
Source RSPAN VLAN        : 100
Destination Ports        : Twe1/0/10
    Encapsulation        : Native
          Ingress        : Disabled

[balaji.bandi]

What is the Server VLAN belong to? i do not see VLAN 10 in your show VLAN ?

Can you post below to look at what is configured on the interface?

Cat 9300 

show run interface gi 1/0/2

show run interface gi 1/0/3

Cat 9300 

show run interface twe1/0/10 

[vsurresh]

Sorry, I truncated the config as the switches have a large amount of VLANs. Interfaces where the servers are connecting configured as access ports with VLAN-10. The destination port only has a default config as suggested by Cisco.

Both VLAN 10 and 100 are allowed on the trunk between the switches. 

9300 - source switch

interface GigabitEthernet1/0/2
 description server-12
 switchport access vlan 10
 switchport mode access
 spanning-tree portfast

interface GigabitEthernet1/0/3
 description server-13
 switchport access vlan 10
 switchport mode access
 spanning-tree portfast

10  SERVER-LAN                      active    Gi1/0/2, Gi1/0/3


9500 - destination switch

interface TwentyFiveGigE1/0/10
 description CAPTURE-PC

10  SERVER-LAN                      active   

[balaji.bandi]

Sure i can understand how your setup and VLAN exists, what is the version of code and license you to have in this kit

show version

can you try source as VLAN 10 ( instead of gi 1/0/2 and gi 1/03 ) - what is the results?

[vsurresh]

I just tried adding VLAN 10 as the source instead of the physical interface but still not seeing the traffic I'm after. I can only see broadcast and SSDP traffic. SSDP was not present when using the physical interfaces instead of VLAN 10. I'm running continuous ping to the server so, expected to see the ICMP traffic from my PC. I can only see SSDP traffic from my PC even though the ICMP was successful. 

If I run the capture directly from the switch using 'monitor capture xxx', I can see all the traffic, very odd. 

9300 - 17.03.03 

9500 - 16.12.05b

both switches have network-advantage license. 

[balaji.bandi]

Looks odd to me, until we see any bug, do you have any other session running in the same switches?

can you post  from both the switches :

show session 

[vsurresh]

There are other 'SPAN' sessions configured on the source switch. These were configured a long time ago so, not sure whether these have worked or not. My understanding is that these local SPAN sessions shouldn't have an impact on the remote session. 

I tried everything but the packets are not getting captured, I'm going to try local SPAN instead but obviously I will need to go to the site to plug the laptop in.  

9300#show monitor session all
Session 1
---------
Type                     : Local Session
Source Ports             : 
    Both                 : Gi1/0/16
Destination Ports        : Gi1/0/22
    Encapsulation        : Native
          Ingress        : Disabled


Session 2
---------
Type                     : Local Session
Source Ports             : 
    Both                 : Gi1/0/48
Destination Ports        : Gi1/0/23
    Encapsulation        : Native
          Ingress        : Disabled

Session 10
----------
Type                     : Remote Source Session
Source Ports             : 
    Both                 : Gi1/0/2,Gi1/0/3
Dest RSPAN VLAN          : 100

9500#show monitor session all 
Session 10
----------
Type                     : Remote Destination Session
Source RSPAN VLAN        : 100
Destination Ports        : Twe1/0/10
    Encapsulation        : Native
          Ingress        : Disabled

[balaji.bandi]

I can not remember how long, this may be 3750 generation time,  i have seen some issues before with other sessions configured.

if those sessions were not used, suggest removing that session and make it only session 10 and check (this is just a suggestion if possible ?)

Also as suggested try source VLAN 10 instead of the interface (did that work).  

[vsurresh]

Thanks for the suggestion. I removed all the monitor sessions and re-configured just the 10 but still no luck. I can only see broadcast, multicast and some NTP traffic. I also tried VLAN10 as the source instead of the physical interface but still the same. Appreciated all your response. 

[balaji.bandi]

Ok now time for us to look your complete config here ( please remove any confidential information and post both the switches complete config to understand why this was not working ?

[vsurresh · Accepted solution]

Hello. 

I created a new VLAN for RSPAN (200) and that fixed the issue. It doesn't make any sense how a new VLAN made a difference but it works now. Thanks again for your responses.