With Native IOS the TCP Reset Interface is hidden from the user in the switch configuration.
In traditional Cat OS the TCP Reset Interface is port 1 in the switch configuration.
The TCP Reset Interface in both Cat OS and Native IOS is a dot1q trunk port that trunks all of the vlans in the switch.
In Cat OS you can then clear vlans from the trunk, but I generally do not recommend doing this, and just leaving the default configuration.
In Native IOS you can't change the configuration.
The TCP Resets will be sent out the TCP Reset Interface when a signature fires that has been configured for TCP Reset actions.
So in MOST cases all you have to do is configure a signature for TCP Resets, and it should just start working without having to do anything to the switch configuration.
Then there are the rare cases where TCP Resets aren't working with the default configuration.
If you believe the TCP Resets are not working then there are a few things to try:
1) Look at the alarm itself through the "show events" command on the CLI. Look to make sure that it lists TCP Reset as an action that should have taken place on that alarm. If it does not state it in the alarm, then the sensor is not configured to TCP Reset that signature.
Recheck your configuration.
2) Check the Vlan listed in the alarm. If the vlan is a number other than 0 then the TCP Resets will have been sent out on that Vlan.
If the Vlan is being listed as 0 then the IDSM-2 did not receive a dot1q header on the packets. This happens when the packet is from the Native Vlan of the IDSM-2 trunk port monitoring the packet. (The Native Vlan is the vlan used in the command "set vlan /7or8".
In Native IOS, the Native Vlan for the TCP Reset Interface and for the 2 monitoring Interfaces is set automatically by the system to vlan 1 and is not modifiable by the user.
BUT In traditional Cat OS, the Native Vlan on port 1(TCP Reset) and ports 7 and 8 (monitoring) are configurable. If you change the Native Vlan on one port, you should make the same change to the other 2 ports. This is because the Native Vlan of the TCP Reset Port must match the Native Vlan of ports 7 and 8 to be able to send the TCP Reset packets to the right vlan when no trunk port encapsulation is seen.
OTHER POSSIBILITIES THAT COULD BE HAPPENING:
A) If that Vlan is an RSPAN vlan then the TCP Resets will be sent out on the RSPAN Vlan and will never reset the connection. The IDSM-2 can not TCP Reset connections for traffic being monitored through RSPAN.
B) In some special cases, a device between the sensor and the end machine may be dropping the Resets.
We have seen a few situations, where a Firewall between the sensor and the end machine was able to detect that the packets were being spoofed (by the sensor) and automatically dropped these spoofed packets. Because the packets were dropped at the Firewall they never made it to the end machine and the connection was never reset.
The IDSM-2 won't be able to Reset the connections in these environments.
C) The connection is happening too fast. In some cases like connections to mail servers or web servers the connections are very short. In these cases it is often near the end of the connection where the sensor sees the packet that triggers the alert and causes the resets.
When the sensor sees the packet, the server is receiving it also so the attack is already happening when the TCP Reset gets sent. The TCP Reset just closes down the connection on which the attack had ALREADY taken place and in many cases the connection was already being closed anyway. In many cases like web server and mail server connections, the client immediately detects the connection went down and wuickly connects a second time. The sensor analyzes this as a brand new connection.
D) In some rare cases the TCP Resets may not work when the client and server are very close in proximity on a busy network. The sensor and server receive the packet at the same time. BUT if the sensor is busy analyzing other packets, the server may continue the connection onto to the next sequence of packets before the sensor can analyze the original packet. Once the sensor analyzes the packet it alarms and fires the Reset, but by this time the connection is actually further along, and the sensor must guess at the new sequence numbers that the connection has progressed to.
The sensor will attempt 100 guesses, but in very very very rare cases it doesn't guess write and the connection continues.
