Disciplined engineer with over 20 years of experience, highly trained, certified and motivated with a broad spectrum of proficiencies in all aspects of Enterprise Networking. Currently focusing on Data Center consolidation and virtualization for networks, storage fabrics, as well as compute. Possessing the industry's top certifications from Cisco, VCE, VMware, and NetApp I have consistently designed and implemented networks for converged Voice & Data and leading edge LAN/SAN Fabrics using Cisco Nexus 7k, 5k, 2k, 1000v, Cisco MDS Directors, and UCS B-Series and C-Series compute bleeding edge products along with the full Cisco portfolio of networking routers and switches.
Well versed from a pre-sales engineering capacity to post sales design and implementation in all relevant technologies and compatible product sets from many vendors. Looking to focus these skill sets in a productive pre-sales position where expertise in design and implementation of proper products, features, and technologies is appreciated. Currently holding a "Hybrid" role within CDW as both Field Solutions Architect in Pre-Sales and Principal Consulting Engineer post sales.
Current Specialties & Certifications:
- Cisco CCIE #6746 R/S (Active)
- Cisco Unified Computing (UCS) Design & Implementation
- Cisco Unified Computing (UCS) Technology Design & Support
- Cisco Unified Fabric Design & Implementation
- Cisco Storage Network Design & Implementation
- Cisco Data Center Networking Design & Implementation
- Cisco Data Center Storage Networking Design & Support
- Data Center Virtualization
- VCE/Vblock Architecture/Implementation
- NetApp Technologies & Architectures for DataONTAP 7, 8, and Clustered DataONTAP
- VMware Certified Professional - Data Center Virtualization (VCP-5)
- End to End QoS Policy design and implementation for VoIP, FCoE, and iSCSI
- UCS Director Data Center work flow automation & provisioning of virtualized environments
- Cisco CCIE Voice Written
- Cisco CCIE Datacenter Written & many more.
Disciplined engineer with over 20 years of experience, highly trained, certified and motivated with a broad spectrum of proficiencies in all aspects of Enterprise Networking. Currently focusing on Data Center consolidation and virtualization for networks, storage fabrics, as well as compute. Possessing the industry's top certifications from Cisco, VCE, VMware,
For those having the same issues its a defect in the code but a simple work around. I had HSRP configured despite VSS in 6880X core due to doing migration from old core to new core. When HSRP was configured the "no ip redirects" that automatically gets configured in IOS does not get added in that version of 6800 IOS and has a bug id to track and fix it. The slow traffic was due to ip redirects constantly happening. Once I saw the "no ip redirects" actually didn't exist I added it to all SVI interfaces and traffic normalized immediately. Once the migration was completed the HSRP was then removed of course but despite having VSS configured HSRP was in use between old and new cores in order to gracefully swing routing off old core and onto new by changing priority on a per vlan basis until we were sure all we moved and doing well. Due to 24/7 operations this was required and requested by customer as graceful method of moving back and forth to test. I overlooked the ip redirects not getting disabled automatically simply being used to it happening for so long :) If you have this issue, this code, "no ip redirects" should be added to each SVI, especially and specifically if using HSRP
... View more
Yes, you are correct, and sorry you're right but I forgot to update this posting. I found the bug ID but it was closed stating fixed in 2.1(1b) however I am running 2.2(3d) and still hitting it. Did this come up again, or get a new bug ID ?? What's the work around ?? Anything you can forward me to read up would be appreciated as I'm finding nothing beyond the bug id confirming this, but mismatch on the firmware version. This is actually production impacting, as I said despite them being "identical" in appearance via GUI and nx-os cli commands, they show to duplicated, existing side by side and when they do, I cannot see my LUN's... when they dont' exist I see the LUN fine. thanks for confirming Walter, long time no talk :) now need to find new bug id or current "fixed" version since I've never hit this before, rarely doing local zoning anymore.. d-
... View more
UCS 2.2(3d) new system in customer lab for us to automate with Director and test workflows before cutting loose on production UCS clusters. Customer has a standard in which the mac-addresses and wwxn's must be statically assigned based upon location of data center, rack, chassis, slot, etc. Without ability to use pools, vNIC and vHBA templates are out and since entire address is being dictated with no variables we've found it easiest to create this way. UCSD workflow prompts for values mentioned above, customer responds and using those mac's and wwn's the workflow clones an existing service profile we have there acting like a template. The service profile I created has local UCS zoning done using single-to-single san connection policies. During profile creation I created the vHBA Intiator groups, tied vHBA0 to primary and secondary A side targets, replciated on fabric b of course. When associated the zones go active, 2 on each fabric, LUN is seen and director builds bare metal esxi server. Now, if UCS Director does entire workflow and associates the service profile it created va cloning the "template" version of a service profile we created prior, and tested to be known working. After the clone process takes place saving it with the new name derived from customer input prompts it then changes both mac-addresses, then the WWNN followed by the WWPN's again according to their standards the script properly generated based upon the values they entered. The problem is every profile created has FOUR ZONES per fabric, it has the two correct vHBA0 plus the primary and secondary targets, then those are mirrored, identically ! FI reports 4 zones, NX-OS complies, all matches despite them being identical and you'd think if created twice one would over write the other. Using the original it works fine, using the cloned by UCS DIrector despite the zones looking proper it doens't see the LUN so something is messing it up. We've done this many time, over and over the problem keeps recreating itself consistently. Any one seen or heard of these duplicated zones in UCS ?? on webex with customer, time is of the essence if anyone can shed some light.. thanks in advance, Dave
... View more
Nothing, its too simple as I eluded to being that they have such a small network. They have never had jumbo mtu's enabled and they're not now. We have never been able to turn the old router off so no real traffic is traversing just our pings and such during windows we're allowed to bring down the port-channel or change hsrp priorities allowing the 6800 to take over as ".1" gateway so to speak. They do have EIGRP but only for the four wan routers directly connected into IA stack 3 for their branches. The provider when adding a branch network manages the router so the core gets the new subnet via eigrp. otherwise both routers have the same small set of static routes with one being the default gateway. The static routes distribute between the firewall and a wan router, pretty much only those two destinations but they are identical and again reach-ability has never been an issue per se. We can always "reach" but its the performance that makes it appear to be unreachable at times. I was mostly opening seeking others who may have seen or experienced a bug as this version was the only image available at time of install that supported greater than 3 switches in a IA stack, they required 4. I now see 5 maintenance releases have come out since, and may layer 3 forwarding and routing issues both multicast and unicast are documented. While nothing identical to what I see here several that exhibit the same type of behavior. My hang up is and reason for going public is that if all layer 3 traffic was being process switched then first I would think it would still be faster than it is given today's cpu's but more so that the CPU would go up and its not. I couldn't see it while in production the night they first attempted shutting down the 4500 thats doing the routing, but since then I have recreated many times at will and cpu always sticks at 22% never higher, never lower. that said during our windows nothing was happening so 22% should be high as far as I am concerned... tac case opened now but no intrusive testing until weekend allowed, hoping they can bug scrub seeing more than I have access to outside Cisco anymore..
... View more
As I eluded, core has 3 IA stacks, that IS the topology with exception of the 4500 as a router on a stick that's going away when layer 3 performs "normal" in 6800. Legacy and remaining network has been moved, everything is in this core via IA's. We are talking from IA 3 on one vlan, to another vlan also on IA 3 for testing to isolate.
... View more
I agree, yes I checked all that first as well, I've checked EVERYTHING. The VSS Core is the STP Root. With the legacy 4500 gone, the VSS core is pretty much all there is with everything terminating into it. From the switch I see no legacy going directly to pc1 or to pc2. But pc1 speaking to pc2 has a great deal of latency as its being routed from one vlan to another. Two devices on the same vlan dont experience latency, nor do they if the 4500 is hsrp primary and doing all the routing. There's no topology to draw out beyond 3 IA stacks connected to this vss core consisting of 2x6880X's with supervisors only, no line cards as the IA's are their "line cards". IA switch stacks 1 and 2 contain users only, IA switch stack 3 contain some users and all wan routers, firewall, etc. The VSS Core is STP root whether the 4500 is online routing or not, it doesn't change but yes I went thru STP as I typically do when I design a datacenter.
... View more
Installed VSS 6800 pair with 3 IA groups for a customer in a pretty small network. This 6800 is core and four wan routers terminate into IA's for branches, Internet etc. Transitioned them off a pair of 4507's with a trunk between the two networks and using HSRP migrated everyone over. Now that EVERY device is running on the 6800's and 4500's to totally empty its time to allow hsrp values to change and move the active gateway to 6800's to take over layer 3 and turn down the 4500's. When we do so the network comes to a crawl. I put the cable back in for the 4507 router on a stick and speed comes right back. I can find no errors, no reasons for this to happen checking everything so many times. They have a few static routes and rest are eigrp neighbor routes from the wan routers mentioned. Traceroutes show same paths as computers are all in IA's going to one router or another for destination. From switch I see no latency to computer 1, nor to computer 2, but computer 1 and computer 2 can hardly speak to each other its so slow. At any time routing goes back to 4507 where all traffic must hairpin out and back in the speed is very fast, lose that hairpin "router on a stick" and business crashes. This is final step to get 4500's out of production and cpu remains around 22%, no bad stats on interfaces, no drops etc. no syslog errors whatsoever, eigrp routes and statics, infact config matches 4507 but for IA's in place. Very small network with 4 routers, plus core, 3 IA "IDF" stacks, 2 for users 1 is on core for data center routers. No acl's and only default qos is enabled, traffic numbers are very very low but when hsrp standby goes active on 6800X it crawls despite how little traffic. It all dies at single vlan 4, but that vlan is where ALL layer 3 happens as its home to all routers. No line cards in 6800's so IA ports are used which is not what I like to do but Cisco markets and sells them as fully capable so customer ran with that. All inf ports are dual 10GB port-channels, dual 10GB to each of 3 fex stacks, dual 10GB VSL links, all other ports are 1Gb IA terminated pc's and routers. Anyone aware of layer3 forwarding issues ?? tonight was last straw, rebooted vss pair and still does same thing, can recreate at will, just have to allow a hsrp standby IP to go active... and yes also removed hsrp standby's and changed IP to .1 on SVI as its not needed once the 4500 is offline.. leaving as without 4500 business cannot run. thanks in advance, dave
... View more
New customer install has issues with single blade. After running a few days we powered down all devices, upon powering back on 1 server wouldn't boot. B200 M4 2.2(5A) Booting from SAN, no changes done to it all others are identical and powered up fine. Boot up shows no vNICs no vHBAs, boot policy configured reflects policy of remote cd and 2 vhba's, actual shows them all missing. Loaded ISO in virtual and says its not there but boots from it, goes directly into bios where boot shows legacy matching policy, but only 1 option which is EFI Shell, cannot get around shell, boot from vNIC, never see them in KVM window again. Have acknowledged, unassociated and ressociated and now server says it has correct # of vNICs and vHBA's and all looks good. However boot up in KVM exactly the same goes directly to blue screen stating no devices exist to boot from, then to EFI Shell. Its like since powering off and back on even tho graceful, the bios will no longer accept the pre-existing policies. All the UCS B Series is new and installed last week. Only powered down after UC engineers were on site to add VoIP vNIC templates to SP Template, what can I do to recover this one blade to get CUCM installed and customer happy ?? On webex with customer now, trying all I can. Thanks, dave
... View more
Walter that is correct and I and Marcus have both posted the versions, multiple times. The system was upgraded to 2.2(3d) and the result of that upgrade has caused all of these errors: ERROR: Service Profiles failed to associate to blade - this is on every blade in the system REASON: Local Storage Controller firmware is stuck in 'activating' state. As it says, its 98% complete and then hangs there on every blade. They did indeed boot up since they are FC SAN Boot and it did say FC uplink down for vsan but its not. I've done this same code upgrade on other systems without this issue, but now its continuing to throw out errors customer wants fixed and I dont blame them. All other firmware updates completed without issue, so you know all the versions. Anyone familiar with this or aware of issue ??? thanks in advance, Dave
... View more
Hey Walter Marcus is tied up so I'm looking into this issue... No the disk is not local, it has no local drives and boots from FC SAN. As Marcus stated the blade boots up via FC despite the error stating FC link is down on respective VSAN. Blades are all up and running now as they were prior to this upgrade with versions in original post. The only difference is these errors continually running out for every service profile stating that it cannot associate due to firmware in activating state. Post upgrade this error and firmware of local storage controller stuck in activating state is for EVERY Service Profile, affecting every blade in production, with these "errors". Firmware for the local storage controller appears to be stuck in 'activating' state for some reason. I have never saw this and did this upgrade many times, and gave the version to Marcus to use in this instance.. Blades are all B200-M2's controller is LSI MegaRAID SAS 2004 ROMB rev B2, all normal stuff I see each and every week.. Customer is running in production with upgrade completed they requested however they want the errors cleaned up and for them to stop before moving forward. Not finding anything on this online, hoping someone here/tac can assist. Screenshots attached of each status/error and versions grouped below Upgraded 2.1(2a) to 2.2(3d) Both running and startup version for controller are 20.11.1-0135 bootloader version 4.35.00_4.12.05.00_0x05270000 Activate Status: Activating
... View more
This is new data center setup for a customer, cisco 3850 stack x2 for management with all Nexus and UCS beneath it. I do this quite often without issue but fear I am hitting a bug or something in 3850-48T-S with 3.2.3SE code. Setting up stack of 2 new 3850-48T-S switches using out of band g0/0 ports. I cannot ping these ports nor terminate anything on them using 03.02.03SE code. I have both hooked up and using them to manage my Nexus devices which are all responding. I can ping everything on the subnet except the 3850 g0/0 ports for some reason. I have configured a default route for the vrf Mgmt-vrf as well as assigned the IP to the automatically configured g0/0 port in the Mgmt-vrf. Using ping vrf Mgmt-vrf I cannot ping anything on the subnet, but everything else can ping everything else on this basic mgmt subnet. I have searched but not finding a direct bug in 3850 code for this but others with mgmt port, anyone seen this or know of known bug in this release ?? Every buy I find says I dont have permissions to view it. thanks in advance, Dave
... View more
Having issues with new core on 7710's upgraded from 6.2(8a) to 6.2(10) running ACL's on all SVI's, Netflow, and DHCP Snooping with one VDC added along side the admin VDC with 2 F348 line cards and dual-sups. The 6.2.10 upgrade was due to TCAM errors before finishing the entry of all ACL's. After the upgrade I entered the command for hardware access-list resource feature bank-mapping. Remaining ACL entries entered without issue, Netflow and DHCP snooping started. During redundancy testing I can reload vdc 2 and while everything performs flawlessly when the vdc returns back online module 1 cannot be allocated back into the vdc due to tcam allocation error. Checking the tcam allocation resources I see tcam 1 bank0 is half full, tcam 1 bank 1 is empty as is all others. Can this be reallocated, re-shuffled around for better distribution ?? Do I have to REMOVE everything and reconfigure to do this ?? Dont like having to reset the entire switch to recover modules if a vdc reloads but at least the constant errors seem to have stopped unless trying to configure or unconfigure the bank-mapping to try resource pooling, even with atomic updates disabled. Please advise, dont have time to call Tac right now, hope someone has been thru this already, thanks in advance. dave
... View more
Does UCSM REQUIRE that the CIMC obtain an IP address from the ext-mgmt pool in order to fully discovery the blade and place in inventory ?? or can/does it use an internal 127. address in absence of ext-mgmt yet being created in order to complete discovery of the blade inventory ?? My thought was that until I populated the ext-mgmt pool with a block of ip addresses to be handed out to the CIMC cards the discovery process didn't kick off. I'm now being told that's not the case and with newer versions having more insight to see CIMC sessions etc on the 127. network need to know exact process. thanks in advance, dave
... View more
I got it working by just adding the Flexflash Scrub policy to the existing service profile and re-acknowledged the blades, removed the scrub policy after the Raid was completed and installed properly. I think the first time I just didn't wait long enough as it took awhile for the raid to complete and show error free. thanks
... View more
Thanks, I read and followed that but doesn't appear disassociation of the service profile created a mirror and further on one server the SD card booted up as if the scrub had no effect. I'll add the scrub policy to the service profile and re-acknowledge the server and see if that process helps fix the raid issue. -d
... View more