Before I put down my weird problem, I would like to ask if direct attaching storage using twinax cables with VIC1227 or 1225 (10GbE) is supported or not?
We are trying to "Direct Attach" NetApp FAS3240 (cdot 8.3.1) with cisco UCS c240-m4 VIC1227 (mlom, slot 7) and 1225 (slot 4) using twinax SFP-H10GB-CU3M. We have installed xenserver 6.5 on the server and given ip addresses to one port on xenserver (xenbr3) and one port on FAS3240 (node 1 port e1b)
Even though the links are up we cannot reach either side. Pinging from NetApp shows network unreachable and looking at packet traces from Xen side we do see communications of this sort:
RECEIVE Frames/second: 0 | Bytes/second: 0 | Errors/minute: 0 Discards/minute: 0 | Total frames: 1997 | Total bytes: 963k Total errors: 0 | Total discards: 0 | Multi/broadcast: 1232 No buffers: 0 | Non-primary u/c: 0 | L2 terminate: 0 Tag drop: 1997 | Vlan tag drop: 0 | Vlan untag drop: 0 Vlan forwards: 0 | CRC errors: 0 | Runt frames: 0 Fragment: 0 | Long frames: 0 | Jabber: 0 Error symbol: 0 | Illegal symbol: 0 | Bus overruns: 0 Queue drop: 0 | Xon: 0 | Xoff: 0 Jumbo: 0 | JMBuf RxFrames: 0 | JMBuf DrvCopy: 0 TRANSMIT Frames/second: 0 | Bytes/second: 0 | Errors/minute: 0 Discards/minute: 0 | Total frames: 136 | Total bytes: 5712 Total errors: 0 | Total discards: 0 | Multi/broadcast: 136 Queue overflows: 0 | No buffers: 0 | Xon: 0 Xoff: 0 | Jumbo: 0 | TSO non-TCP drop: 0 Split hdr drop: 0 | Pktlen: 0 | Timeout: 0 Timeout1: 0 | Stray Cluster Pk: 0 DEVICE Rx MBuf Sz: Large (3k) LINK_INFO Current state: up | Up to downs: 2 | Speed: 10000m Duplex: full | Flowcontrol: full
Looks like the server and the filer initiate the communication and exchange the MAC addresses but then stop, ICMP is not blocked and we have also tried to access the ip addresses through other protocols. We can see from interface statistics on both sides that the bytes (probably broadcast arp requests only) are flowing in.