In the event of a kernel panic or freeze, critical logs may not be written to /var/log/messages making it difficult to troubleshoot the root cause. Serial console logging can be configured to capture any kernel messages dumped to the console.
Before beginning this procedure, stop the HDB processes on all hosts. A reboot will be required to complete the procedure.
Configure SUSE for console access
Modify /etc/inittab
Edit /etc/inittab to set the default runlevel to 3 (disables x-windows on startup).
change
id:5:initdefault:
to
id:3:initdefault:
Uncomment and modify the S0 line to match:
S0:12345:respawn:/sbin/agetty -L 115200 console vt102
Modify grub boot menu
Edit /boot/grub/menu.lst
Comment out gfxmenu
#gfxmenu (hd0,0)/message
Add these lines near the top of the file (before the grub entries):
serial --unit=0 --speed=115200
terminal --timeout=8 serial console
Copy the default boot entry, creating a new one before the default entry. Add `console=tty0 console=ttyS0,115200` to the end of the kernel options. Changes to the default grub entry are highlighted in red here:
###Don't change this comment - YaST2 identifier: Original name: linux###
title Serial -- SUSE Linux Enterprise Server 11 SP1 - 2.6.32.12-0.7
kernel (hd1,0)/vmlinuz-2.6.32.12-0.7-default root=/dev/rootvg/lv_root resume=/dev/rootvg/lv_swap splash=silent crashkernel=256M-:128M showopts console=tty0 console=ttyS0,115200
initrd (hd1,0)/initrd-2.6.32.12-0.7-default
Modify /etc/securetty
Add the following lines to the end of the file to allow root login via console:
console
ttyS0
xvc0
Configure UCS
Create Serial-over-LAN Policy
Under Servers->Policies->root->Serial over LAN Policies: Create a new policy.
Enable the State and choose Speed: 115200.
Create IPMI Policy
Under Servers->Policies->root->IPMI Access Profiles: Create a new policy.
Create a new user with Admin role. This will be the username/password you use to ssh to the console port.
Create BIOS Policy
Under Servers->Policies->root->Sub-Organizations->T01->BIOS Policies:
Create or edit a BIOS Policy. Under the Server Management tab, modify the console redirection settings:
- Console Redirection: serial-port-a
- Flow Control: none
- BAUD Rate: 115200
- Terminal Type: Platform Default
- Legacy OS Redirect: enabled (or Platform Default)
Apply the Policies to the Service Profile Template
Under Servers->Service Profile Templates->root->Sub-Organizations->T01->Service Template T01-ServiceProfileTmpl:
Under Policies tab:
Click `Change Serial over Lan Policy`
Select: Use an Existing Serial over LAN Policy
Serial over LAN Policy: sol-115200 (the one created in an earlier step).
Under BIOS Policy, select the BIOS Policy created in an earlier step
Under IPMI Access Profile Policy, select the IPMI Policy created in an earlier step.
Identify the Blade Management IPs:
Under Admin->Communication Management->Management IP Pool->IP Address:
Record the IPs (192.168.76.x) associated with each blade. Check the service profiles to identify which host is running on each blade.
Reboot
Reboot the nodes to apply the changes to the service profiles.
Setup Console Logging Session
Open a console session to begin logging output. SSH using the username/password created in the IPMI access profile to the blade mgmt IP of the server.
To log the output to a file, here are a few options:
- Open the ssh session from an ssh client with logging enabled (putty, securecrt, etc.)
- Start a script session from a linux machine and then connect via ssh
