9372-TX Configuration For Oracle RAC Interconnect Network

[dohogue]

I am deploying a 3 node Oracle RAC. I plan to use 2 Cisco 9372-TX switches for the Public  network. Each node will have 2 10 Gb NICs and one NIC will plug into each switch for redundancy. I will use VLANS for this. My Interconnect Private network for the RAC will be 2 2960-X 1 GB switches. Each Node will have 2 Gb NICs and one NIC will plug into each 2960-X for switch redundancy. Are these just access ports? I have heard you should turn off spanning tree in this network. What will keep the nodes from looping? Does any one know of a configuration example of how the ports should be configured? I am going to enable Jumbo Frames in both the 9372-TX and 2960-X switches.

Thanks

[nazimkha]

A network topology diagram would be helpful.

I dont have a lot of information of Oracle RAC but from a server's perspective if the NICs are teamed together then you can use vPC (virtual port-channel) to form a port-channel with the two Nexus 9372 switches.

Are you using the 2960X switches for management network ??

[dohogue]

Here is an overview. RAC needs 3 networks. 1) Public - Network which users and application severs connect to access data on the database server. This will be a simple VLAN. The servers will be attached to 2 9372TX. 2) Private (Interconnect) - Oracle RAC requires a cluster interconnect configured for inter-node communication. This cluster interconnect communication is fundamentally of two types. 1) Inter-node cluster heartbeat and messaging for cluster topology (node and group) management. Inter-node cluster heartbeats and messages tend to be small, ~200 bytes. 2) Inter-node global cache buffer transfers for applications, such as Real Application Clusters (RAC), that exploit active/active inter-node, in-memory buffer access across nodes in the cluster. Inter-node global cache buffer transfers may vary from 2k-16k block sizes depending on the application and the data access patterns. The Oracle Clusterware interconnect is a non-traditional interconnect and should be considered, in shared global cache applications, such as Oracle RAC, to be an I/O subsystem for inter-node global cache buffer transfers with the latency and bandwidth requirements of high performance cluster computing. For this reason, Oracle Clusterware for RAC requires a highly reliable, low-latency, high-bandwidth transport medium to ensure highly available and efficient global cache buffer processing. This requirement traditionally has been satisfied by stand alone, dedicated switches where isolation guarantees control over latency and throughput. To achieve complete isolation and dedicated bandwidth, dedicated switches and NICs are required. The 2 - 2960X switches will be the switches for this network. 3) SAN (Storage Area Network) - For the purpose of this network the existing Storage Area Network can be used. The existing Storage Area Network is made up of 6 Brocade Fiber Channel switches. It is believed these switches and the network will meet requirements for RAC. Oracle Flex ASM used to manage volumes. I will also attach a high level topology.

[dohogue]