09-30-2018 12:03 AM - edited 03-08-2019 04:16 PM
my old technology was working as l2 and we decided to change from L2 to L3 And we configure our ospf between Distribution to the core and after 3 days I lose the connectivity between the core, Distribution so I thought the issue is in the Distribution so I configure the Distribution as L2 and it was working Fine for more than one month last week I configure the Distribution as L3 and after 3 days I loss the connectivity so when I chick the Distribution I found the Distribution is not working As VSS
VSSDS-# show switch virtual role
xecuting the command on VSS member switch role = VSS Active, id = 1
RRP information for Instance 1
--------------------------------------------------------------------
Valid Flags Peer Preferred Reserved
Count Peer Peer
--------------------------------------------------------------------
TRUE V 0 0 0
Switch Switch Status Preempt Priority Role Local Remote
Number Oper(Conf) Oper(Conf) SID SID
--------------------------------------------------------------------
LOCAL 1 UP FALSE(N ) 100(100) ACTIVE 0 0
Peer 0 represents the local switch
Flags : V - Valid
In dual-active recovery mode: No
I in L2 and it was working for more
VSSDS-# show switch virtual role
xecuting the command on VSS member switch role = VSS Active, id = 1
Switch mode : Virtual Switch
Virtual switch domain number : 2
Local switch number : 1
Local switch operational role: Virtual Switch Active
Cannot execute command on peer supervisor because it is not in SSO Standby state
VSSDS-# show switch virtual link
Executing the command on VSS member switch role = VSS Active, id = 1
VSL Status : DOWN
VSL Uptime : -
VSL Control Link : -
VSL Encryption : Configured Mode - Off, Operational Mode - Off
Cannot execute command on peer supervisor because it is not in SSO Standby state
The IOS for the 2 Distribution is : cat4500e-universalk9.SPA.03.05.00.E.152-1.E.bin
And also I chick the Debug I found
*Sep 26 04:28:04.428: %C4K_REDUNDANCY-5-CONFIGSYNC: The private-config has been successfully synchronized to the standby supervisor
*Sep 26 04:28:06.025: %C4K_REDUNDANCY-5-CONFIGSYNC: The startup-config has been successfully synchronized to the standby supervisor
*Sep 26 04:31:49.463: %C4K_REDUNDANCY-5-CONFIGSYNC: The private-config has been successfully synchronized to the standby supervisor
*Sep 26 04:31:50.894: %C4K_REDUNDANCY-5-CONFIGSYNC: The startup-config has been successfully synchronized to the standby supervisor
*Sep 27 21:42:17.243: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Spanning Tree", ipl= 0, pid= 201
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+16EFE7C :1F6A5000+16EB6CC :1F6A5000+3F97448 :1F6A5000+3F97BF8 :1F6A5000+3F61CC4
*Sep 27 21:42:47.314: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Spanning Tree", ipl= 0, pid= 201
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+16EFE7C :1F6A5000+16EB6CC :1F6A5000+3F97448 :1F6A5000+3F97BF8 :1F6A5000+3F61CC4
*Sep 27 21:43:17.384: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Spanning Tree", ipl= 0, pid= 201
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+16EFE7C :1F6A5000+16EB6CC :1F6A5000+3F97448 :1F6A5000+3F97BF8 :1F6A5000+3F61CC4
*Sep 27 21:43:47.398: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:44:17.406: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:44:47.459: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:45:17.475: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:45:47.487: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:46:17.519: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:46:47.559: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:47:17.583: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:47:47.595: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:48:17.599: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:48:47.612: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:49:17.619: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:49:47.647: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:50:17.703: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+4182EC8 :1F6A5000+41830A8
*Sep 27 21:50:47.712: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
*Sep 27 21:51:17.747: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8
*Sep 27 21:51:47.778: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 8’
When I restart the Distribution every think work Fine can you Please Advise
09-30-2018 12:50 AM
Hello,
post a schematic drawing of your topology. Which platforms are you using ? Also post the configs of both OSPF enabled neighbors...
09-30-2018 01:31 AM
interface Port-channel44
description connected to DIS- port-channel 44
no switchport
ip address 10.x.x.x 255.255.255.252
no ip redirects
no ip unreachables
no ip proxy-arp
interface TenGigabitEthernet2/3/3
description To | SWITCH |
no switchport
no ip address
no ip redirects
no ip unreachables
no ip proxy-arp
channel-group 44 mode on
!
interface TenGigabitEthernet2/3/4
description To | SWITCH |
no switchport
no ip address
no ip redirects
no ip unreachables
no ip proxy-arp
channel-group 44 mode on
!
router ospf 1
router-id 2.2.2.2
ispf
log-adjacency-changes
auto-cost reference-bandwidth 100000
area 50 nssa
redistribute connected subnets
network 10.x.x.x 0.0.0.0 area 50
This configuration from Distribution side
interface Port-channel44
description connected to port-channel 44
ip address 10.x.x.x 255.255.255.252
no ip redirects
no ip unreachables
no ip proxy-arp
ip ospf 1 area 50
bfd interval 750 min_rx 750 multiplier 3
interface TenGigabitEthernet1/2/3
description To | SWITCH
no switchport
no ip address
channel-group 44 mode on
!
interface TenGigabitEthernet2/2/3
description To | SWITCH |
no switchport
no ip address
channel-group 44 mode on
!
router ospf 1
router-id 5.5.5.5
ispf
auto-cost reference-bandwidth 100000
area 50 nssa
redistribute connected subnets
distribution model: WS-C4507R+E
IOS:cat4500e-universalk9.SPA.03.05.00.E.152-1.E.bin
CORE MODEL: WS-C6506-E
CORE: s2t54-ipservicesk9-mz.SPA.122-50.SY.bin
09-30-2018 01:43 AM
I configure the 4 area in the same configuration but 3 areas are working Fine and 1 area I have issue with this area also I get observe one issue the memory so I chick the memory this morning
show processes memory sorted
System memory : 2008840K total, 1205181K used, 803659K free, 88312K kernel reserved
Then after one hour I chick this is was the output
show processes memory sorted
System memory : 2008840K total, 1209284K used, 799556K free, 88312K kernel reserved
I Did This trablshouting when the issue happen yesterday I found in the debug
*Sep 27 21:42:47.314: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Spanning Tree", ipl= 0, pid= 201
-Traceback= 1#49176b00b95a50f3145e3825de17d470 :1F6A5000+2CFD660 :1F6A5000+2CFDA24 :1F6A5000+4160EA0 :1F6A5000+4168078 :1F6A5000+16F0E3C :1F6A5000+16EFE7C :1F6A5000+16EB6CC :1F6A5000+3F97448 :1F6A5000+3F97BF8 :1F6A5000+3F61CC4
*Sep 27 21:43:17.384: %SYS-2-MALLOCFAIL: Memory allocation of 276 bytes failed from 0x20D95E38, alignment 8
Pool: I/O Free: 9096 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Spanning Tree", ipl= 0, pid= 201
Hope this information help
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide