6248UP - Fabric Interconnect [id], HA functionality not ready

Neil Stewart · ‎03-01-2013

Hello

Having an HA issue with a FI cluster and have not able to find much info.

The error I get (in the subject) leads me the the below URL with 5 steps to check.

http://www.cisco.com/en/US/docs/unified_computing/ucs/ts/faults/reference/2.0/UCS_SEMs.html

Step 1 - Not sure how to check this - Any input on this ?

Step 2 - Have checked and even replaced the cables

Step 3 - Checked and confirmed under Equipments -> Rack-Mounts -> Fex ( 2 x N2K-C2232PP-10GE)

Step 4 - Checked and confirmed under Equipments -> Rack-Mounts -> Servers ( 4 x UCSC-C220-M3)

Step 5 - New installation no active contract as yet.

Anyone had similar issues, any assistance would be appreciated.

Thanks

Andreas Linde · ‎03-01-2013

Step 1 - Have you configured Management port IP´s for FI A and B as well as Cluster IP correctly? Refer to this document for Initial System Setup:

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/cli/config/guide/2.1/b_UCSM_CLI_Configuration_Guide_2_1.pdf

Step 2 - Make sure the links are not cross-connected. L1 must go to L1 and L2 to L2.

//

Neil Stewart · ‎03-01-2013

Step 1

Management all setup with initial setup-script, confirmed in extract from running config below

scope fabric-interconnect a

set out-of-band ip 172.16.1.4 netmask 255.255.255.0 gw 172.16.1.1

scope fabric-interconnect b

set out-of-band ip 172.16.1.5 netmask 255.255.255.0 gw 172.16.1.1
scope system

set virtual-ip 172.16.1.6

Double confirmed via show commands below

CTCPFIHA-A# scope fabric-interconnect a

CTCPFIHA-A /fabric-interconnect # show detail

Fabric Interconnect:
    ID: A
    Product Name: Cisco UCS 6248UP
    PID: UCS-FI-6248UP
    VID: V01
    Vendor: Cisco Systems, Inc.
    Serial (SN): SSI163308YF
    HW Revision: 0
    Total Memory (MB): 16232
    OOB IP Addr: 172.16.1.4
    OOB Gateway: 172.16.1.1
    OOB Netmask: 255.255.255.0
    Operability: Operable
    Thermal Status: Ok
    Current Task 1:
    Current Task 2:
CTCPFIHA-A /fabric-interconnect # scope fabric-interconnect b
CTCPFIHA-A /fabric-interconnect # show detail

Fabric Interconnect:
    ID: B
    Product Name: Cisco UCS 6248UP
    PID: UCS-FI-6248UP
    VID: V01
    Vendor: Cisco Systems, Inc.
    Serial (SN): SSI16330FY1
    HW Revision: 0
    Total Memory (MB): 16232
    OOB IP Addr: 172.16.1.5
    OOB Gateway: 172.16.1.1
    OOB Netmask: 255.255.255.0
    Operability: Operable
    Thermal Status: Ok
    Current Task 1:
    Current Task 2:

CTCPFIHA-A /fabric-interconnect # scope system

CTCPFIHA-A /system # show detail

Systems:
    Name: CTCPFIHA
    Mode: Cluster
    System IP Address: 172.16.1.6
    System Owner:
    System Site:
    Description for System:

Step 2

L1<>L1 L2<>L2 confirmed and new cables tried.

padramas · ‎03-01-2013

Hello Neil,

Can you please SSH into FI and share the output of following command ?

show system firmware

show cluster extended-state

Padma

Neil Stewart · ‎03-01-2013

CTCPFIHA-A# show system firmware
UCSM:
    Running-Vers: 2.1(1a)
    Package-Vers: 2.1(1a)A
    Activate-Status: Ready

CTCPFIHA-A# show cluster extended-state
Cluster Id: 0xb72022187f3511e2-0x9307547feeed8b24

Start time: Mon Feb 25 14:15:07 2013
Last election time: Tue Feb 26 14:40:01 2013

A: UP, PRIMARY
B: UP, SUBORDINATE

A: memb state UP, lead state PRIMARY, mgmt services state: UP
B: memb state UP, lead state SUBORDINATE, mgmt services state: UP
heartbeat state PRIMARY_OK

INTERNAL NETWORK INTERFACES:
eth1, UP
eth2, UP

HA NOT READY
No device connected to this Fabric Interconnect

Andreas Linde · ‎03-01-2013

Could it be that You haven´t connected any chassis yet? The FI´s look fine to me.

padramas · ‎03-01-2013

Hello Neil,

The output is as expected as we do not have any chassis or rack server connected to FI. SEEPROM in chassis or file in rack server management flash serves to decide active FI during spilt brain scenario.

The message would disappear once you have chassis or rack server connected to FI.

A good blog about this topic

http://ucsguru.com/2012/11/07/ha-with-ucsm-integrated-rack-mounts/

HTH

Padma

Neil Stewart · ‎03-01-2013

Hey Padma

We have 4 x UCSC-C220-M3 server connected which are discovered and displayed in the Manager Topology diagram.

Setup is as per below Document

http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/cisco_ucs_vmware_ethernet_flexpod.html#wp451279

Thanks

padramas · ‎03-01-2013

Hello Neil,

Can you please share the output of following command

show server status

Are you able to connect to CIMC of the servers ? ( connect cimc )

Padma

Neil Stewart · ‎03-01-2013

Hello Padma

I have left site and will be back next week wednesday day,

I will update the dicussion then.

Thanks

Neil Stewart · ‎03-01-2013

CTCPFIHA-A# show service status

Service Profile Name Server Assignment Association Power State Op State

-------------------- ------- ---------- -------------- -------------- --------

CTCPVM01 1 Assigned Associated On Ok

CTCPVM02 2 Assigned Associated On Ok

CTCPVM03 3 Assigned Associated On Ok

CTCPVM04 4 Assigned Associated On Ok

Old Unassigned Unassociated Unassociated

VM-Host-Infra Unassigned Unassociated Unassociated

padramas · ‎03-01-2013

Hello Neil,

We need more logs to investigate the issue.

Please open a TAC service request with UCSM and one rack server tech support bundle.

Thanks

Padma

drhallahan · ‎03-16-2013

Was it just that shared storage wasn't connected or configured?

Neil Stewart · ‎03-18-2013

A Cabling Issues caused the problem.

We had the servers cabled as below.

CIMC -> FI A

LAN1 -> FI_B

Strangely, FI_B coulds see the CIMC, FI_A could not.

We recabled the servers the (as below) and re-acknowledged the servers.

LAN1 -> FI_A

LAN2 -> FI_B

Both IFs see the CIMC and HA up - all is good.