No connectivity from TE interface on a C3750E-24TD-E

[richard]

Hello

I'm testing a newly received 3750E switch with just a fresh configuration...meaning..all ports are Vlan1..no trunking...nothing....

Gi1/0/1 is connected to my home network 10/16 with gateway 10.0.0.1..

Gi1/0/8 has a PC connected with 10.0.100.20/16

Te1/0/2 has a FreeBSD box connected with 10.0.1.2/16

Vlan1 has 10.0.20.200/16

From the switch I can ping 10.0.0.1, 10.0.1.2 and 10.0.100.20...so all is good...

I can ping from Gi1/0/8 (10.0.100.20) Vlan1 IP, the gateway 10.0.0.1...but not 10.0.1.2 on Te1/0/2

I can ping from 10.0.1.2 on Te1/0/2 the Vlan1 IP 10.0.20.200....but nothing else...

So what is wrong with this switch???

Image installed is:  c3750e-ipbasek9-mz.150-2.SE.bin

[Reza Sharifi]

Hi,

Can you make sure all devices have the correct subnet mask (/16)?

I know all these devices are in the same subnet and you should not need IP routing enabled, but just for test purpose can you try the following config on the switch and test again

config t

ip  routing

HTH

[richard]

Hello

Doesn't help in any way....

I should correct the subject...as the switch doesn't forward any packet between any 2 interfaces...

I see the MAC addresses from each device on the corresponding port....and each device can ping vlan1 interface....and switch can ping each devices ip....but not between devices...

Something to do with this console log message?

%ILET-1-AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco

[Mark Malone]

did you buy that switch of a certified reseller , do you have access to download new IOS version for that switch  , I would start with that this is the current recommended version c3750e-ipbasek9-mz.150-2.SE10a.bin

%ILET-1-AUTHENTICATION_FAIL: ILET Platform Authentication Failure The software has detected that this switch may not be a genuine Cisco product and that software loaded on the switch may have been copied in violation of Cisco's end user license agreement. Cisco licensing, warranties and support programs only apply to genuine Cisco products. Cisco may deny operation of this product, or support under your warranty or under a Cisco support program such as SmartNet. If you receive this message, please contact your Cisco Sales Representative for assistance.

This message means that an ILET authentication failure occurred.

Recommended Action:Contact your Cisco sales representative for assistance.

Related documents- No specific documents apply to this error message

[richard]

Are you implying that due to this message it stops forwarding packets between interfaces and turns a 3750E into a piece of unusable scrap metal?

[Mark Malone]

No im not implying that its an unusable scrap metal , im providing exactly what Ciscos response is to that alert from there documentation if seen and they recommend to contact them , one of the fixes is to get off the IOS version that its seen on if you don't have support to contact them , your on an early release IOS version ED which can be more susceptible to bugs as there not tested as much as the MD releases before being put into release.

If your already seeing one bug you could be hitting more that version is 3 years old , upgrade and see if it fixes the issues

routing should work by default on a L3 switch once its in the same subnet or if multiple subnets ip routing must be enabled which allows inter vlan routing between subnets , unlike an actual router where its already enabled by default.

So something's not working right either in software or hardware level , software something's you can rule out by upgrading the IOS version, if its failing in hardware youo will need to RMA the switch and return it

[richard]

Installed now c3750e-universalk9-mz.150-2.SE10a.bin....but to no joy...

And from the boot log I don't see any suspicious message...dunno what FIPS is though (o;

Cisco IOS Software, C3750E Software (C3750E-UNIVERSALK9-M), Version 15.0(2)SE10a, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Thu 03-Nov-16 13:36 by prod_rel_team
HULC:Collect Entropy thread starting on box...
Initializing flashfs...
Using driver version 3 for media type 2
mifs[4]: 10 files, 1 directories
mifs[4]: Total bytes : 1835008
mifs[4]: Bytes used : 614400
mifs[4]: Bytes available : 1220608
mifs[4]: mifs fsck took 0 seconds.
mifs[4]: Initialization complete.

mifs[5]: 0 files, 1 directories
mifs[5]: Total bytes : 3932160
mifs[5]: Bytes used : 1024
mifs[5]: Bytes available : 3931136
mifs[5]: mifs fsck took 0 seconds.
mifs[5]: Initialization complete.

mifs[6]: 5 files, 1 directories
mifs[6]: Total bytes : 262144
mifs[6]: Bytes used : 9216
mifs[6]: Bytes available : 252928
mifs[6]: mifs fsck took 0 seconds.
mifs[6]: Initialization complete.

mifs[7]: 5 files, 1 directories
mifs[7]: Total bytes : 262144
mifs[7]: Bytes used : 9216
mifs[7]: Bytes available : 252928
mifs[7]: mifs fsck took 0 seconds.
mifs[7]: Initialization complete.

mifs[8]: 1 files, 1 directories
mifs[8]: Total bytes : 57409536
mifs[8]: Bytes used : 20606976
mifs[8]: Bytes available : 36802560
mifs[8]: mifs fsck took 1 seconds.
mifs[8]: Initialization complete.

...done Initializing flashfs.
Checking for Bootloader upgrade..
Boot Loader upgrade not required (Stage 2)

FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found, FIPS Mode Not Enabled

POST: CPU MIC register Tests : Begin
POST: CPU MIC register Tests : End, Status Passed

POST: MA BIST : Begin
POST: MA BIST : End, Status Passed

POST: TCAM BIST : Begin
POST: TCAM BIST : End, Status Passed

POST: SF ASIC BIST : Begin
POST: SF ASIC BIST : End, Status Passed

SmartChip Authentication Failed

POST: Switch Fabric Memory Tests : Begin
POST: Switch Fabric Memory Tests : End, Status Passed

POST: CPU MIC interface Loopback Tests : Begin
POST: CPU MIC interface Loopback Tests : End, Status Passed

POST: PortASIC RingLoopback Tests : Begin
POST: PortASIC RingLoopback Tests : End, Status Passed

extracting front_end/front_end_ucode_info (309 bytes)
Waiting for Stack Master Election...
POST: Thermal, Fan Tests : Begin
POST: Thermal, Fan Tests : End, Status Passed

POST: PortASIC Port Loopback Tests : Begin
POST: PortASIC Port Loopback Tests : End, Status Passed

POST: EMAC Loopback Tests : Begin
POST: EMAC Loopback Tests : End, Status Passed

Election Complete
Switch 1 booting as Master
Waiting for Port download...Complete

[Mark Malone]

FIPS is just a standard for compliance used globally on communication devices for security its standard log to see during bootup

This alert is now gone from your device logs yes  --%ILET-1-AUTHENTICATION_FAIL: ILET  

Can you post the running config and the show ip arp and show ip route   , just remove the passwords if there cleartext in your config

[richard]

No..the log message is still there...around 6 minutes after boot:

*Jan  2 00:06:05.676: %ILET-1-AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco or with Cisco's aut.

Switch#sh ip route     

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP

       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 

       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

       E1 - OSPF external type 1, E2 - OSPF external type 2

       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

       ia - IS-IS inter area, * - candidate default, U - per-user static route

       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP

       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks

C        10.0.0.0/16 is directly connected, Vlan1

L        10.0.20.201/32 is directly connected, Vlan1

Switch#sh ip arp       

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

Internet  10.0.0.1                0   000d.b940.83e8  ARPA   Vlan1

Internet  10.0.1.2                0   0002.c929.30f0  ARPA   Vlan1

Internet  10.0.20.201             -   0021.d7fb.cb40  ARPA   Vlan1

10.0.0.1 being local gateway..and 10.0.1.2 being the host connected to Te1/0/2...

Both also appear in sh mac add:

1 0002.c929.30f0 DYNAMIC Te1/0/1
1 000d.b940.83e8 DYNAMIC Gi1/0/1

And now the simple configuration:

!
version 15.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Switch
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
switch 1 provision ws-c3750e-24td
system mtu routing 1500
ip routing
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
!
!
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
!
!
!
!
interface FastEthernet0
no ip address
no ip route-cache
shutdown
!
interface GigabitEthernet1/0/1
!
interface GigabitEthernet1/0/2
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
!
interface GigabitEthernet1/0/15
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet1/0/19
!
interface GigabitEthernet1/0/20
!
interface GigabitEthernet1/0/21
!
interface GigabitEthernet1/0/22
!
interface GigabitEthernet1/0/23
!
interface GigabitEthernet1/0/24
!
interface GigabitEthernet1/0/25
!
interface GigabitEthernet1/0/26
!
interface GigabitEthernet1/0/27
!
interface GigabitEthernet1/0/28
!
interface TenGigabitEthernet1/0/1
!
interface TenGigabitEthernet1/0/2
!
interface Vlan1
ip address 10.0.20.201 255.255.0.0
!
!
ip http server
ip http secure-server
!
!
!
!
!
!
line con 0
speed 115200
line vty 0 4
login
line vty 5 15
login
!
end

[Mark Malone]

Have  you contacted Cisco they will replace this switch with that alert after an upgrade has not fixed it , something's not right with that alert still being there, these switches X series have lifetime warranty based on their serial its worth a call see what they say , there are reports on other forums for that alert where upgrade did not work TAC replaced the switch instantly as there is a known issue with some batches of 2/3 series models reporting this fault even when bought from a legitimate Cisco reseller

If the arp is there , interfaces are up and routings enabled and macs are present but you still cant ping then there is something up with the switch itself and how its operating in the control plane in hardware that processes routing traffic between interfaces  

The only other thing I would try is move off the 10GB interfaces and see if the same problem is there on the standard 1GB ports as these are on different ASICs in the switch at hardware level but I would be calling Cisco or the reseller I bought it off to replace it as its sending out a hardware fault to logs which specifies call Cisco if seen and also the fact its not working correctly

[richard]

Well actually I bought it off Ebay....and to 200% this guy is not an official Cisco reseller....

I already moved on box from Te to a Gi interface..same results...also attached devices don't get an IP address from DHCP....what puzzles me is the fact that it shows the same behavior on two identical boxes....maybe serial numbers are close to each other...

So I am stuck with those two heavy paper weights (o;

[richard]

Hmm...

Is there a doc saying which ASIC handles which switch port so to rule out if a specific ASIC is broken or packet forwarding in general is broken?

[richard]

Just out of curiosity I removed switchport from Gi1/0/1 and Te1/0/1...

So now I have:

Internet Gateway (10.0.0.1/16) <-> Gi1/0/1 (10.0.20.200/16) <-> Te1/0/1 (172.16.0.1/16) <-> FreeBSD Box (172.16.1.2/16)

Now I can ping from the FreeBSD box the internet router, also from all my home LAN (10/16) the 172.16.1.2...

So Layer3 is working on this switch....weird (o;

[Mark Malone]

Ok not so weird but least now you know switch is not fxxked , as I was saying there is blocks of ASICs and 10GBs or fibre ports should be on a separate ASIC , I don't have the doc to hand and they very per patform I just know from working on these usually the blocks are in 8-16 ports per ASIC depending on platform/chassis

One of the commands is --show platform pm platform-block

This allows you to see which ASIC the port may be bound to

....

So now lets see why is the 10GB not working , are you using an SFP  , or are these 10gb copper ? If using SFP what type

[richard]

On Te1/0/1 sits a X2-10GB-SR connected via SC/LC MM fibre to a SFP+ module plugged into a ProLabs SFP-10G-SR-C  which in turn sits on the Mellanox ConnectX-2 10GB PCIe card  in the Proliant DL360 G7 running FBSD 11...

Out to the home LAN I go via Gi1/0/1 with plain copper...

From the FBSD box I can reach now the internet via the 3750e just fine...

Well..the 10GB is working now in Layer3 mode....