Hi Michael, Thanks again for taking a look. I had thought there might be a simple solution or way to reinstate the missing files from the ISO image. But is seems not. To answer your last set of questions: @micgarc2 wrote: Is this the admin user or some type of TACACS ? This is the local admin user the ONLY user on a system directly after a complete erase and restart. I see the 403/404 return code. Is the fabric fully fit? Yes. The summary page clearly states "Fully Fit" on both Fabrics Do you see any core files generated? No core files at all generated So I guess I am no closer. ... View more

Hi Michael, Thanks for giving this a shot. On the APIC giving the (403, 404), I see the following when I tail the log and try to access the docs apic1# tail -f /var/log/dme/log/access.log | grep doc/html ::ffff:172.16.1.102 - - [02/Aug/2019:15:52:25 +1000] "GET /doc/html/ HTTP/1.1" 404 262 "https://apic1.fab2.hl.dns/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"   On the other APIC, the one that gives me text but no pictures, I see the following when I open the docs apic1# tail -f /var/log/dme/log/access.log | grep doc/html ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:04 +1000] "GET /doc/html/ HTTP/1.1" 200 709 "https://apic1.fab1.hl.dns/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/header.html HTTP/1.1" 200 452 "https://apic1.fab1.hl.dns/doc/html/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/LeftSummary.html HTTP/1.1" 200 351321 "https://apic1.fab1.hl.dns/doc/html/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/Summary.html HTTP/1.1" 200 850 "https://apic1.fab1.hl.dns/doc/html/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/footer.html HTTP/1.1" 200 484 "https://apic1.fab1.hl.dns/doc/html/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/styles/images/bg_fade.jpg HTTP/1.1" 404 262 "https://apic1.fab1.hl.dns/doc/html/styles/stylesheet.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/SpryAssets/SpryAccordion.css HTTP/1.1" 200 1221 "https://apic1.fab1.hl.dns/doc/html/LeftSummary.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:07 +1000] "GET /doc/html/styles/tabcontent.css HTTP/1.1" 200 1040 "https://apic1.fab1.hl.dns/doc/html/Summary.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" Then in this case I clicked on aaa:ADomainRef - just like in the picture in my question to get: ::ffff:172.16.1.102 - - [02/Aug/2019:16:01:17 +1000] "GET /doc/html/MO-aaaADomainRef.html HTTP/1.1" 200 5540 "https://apic1.fab1.hl.dns/doc/html/LeftSummary.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" Curiously, when I right click on the image icon and copy the link, it gives me: http://ifc-eng.insieme.local/uml/index.php?data=rZPPTsMwDMbveQqLXjaxVoIjmqZ164SQ2IQqXsCk3ohIkyoNYhWMZydpun9l3OglsfPZ38%2BWOq0tGvteSlYhf8MNwdXj4n6xyq5gPM6J28kEoqz94JNxiXUNc624IUtLXZBMXfoasCiFiiujKxaDKCspuLAhjkChe9zEhrDQSjYh%2FQ26IhPuO4YvtTXILQSHtAtbhxk4i93eu733CJJk0i%2B5A9oGCjAk0QqtWE%2Fiq%2BbglI6PiqNscK4bOdEQEsi79zbb0Zzl3MJgkI%2B04a%2BiGIJbnN%2BMH%2FCmO29%2FT4qISZrpEoXKaQ1RQ1Lqj3bIC8qM1kIJ7%2BgUkecO%2BG69ycpFvupcNv6K454JO%2Bl3MHaoqJS27TiuaamT9BCzmLZ2qTbFrAlPi3343FTE4lKrJy0zX2doTYYUp2QWzP6mPBn7QHkJ8uGU8h%2B9jn3ZlFTh%2F4Ef I don't know if this helps you figure out a next step.  I have considered booting off the apic image iso if I can find the instructions on how to install from scratch - but my (poor) memory tells me that that really screws around with certificates ans seial numbers, so am reluctant to do that.     ... View more

Hi @WiliamRodrigues7077 , I think you have small misconception when you say  I understand that the final scenario should be: Active APIC Nodes (APIC1 and APIC2 on POD-1) and APIC3 and APIC4 as stand-by on POD-2 Now it could be a gramatical/parentheses misplacement error. The final scenario should actually be: Active APIC Nodes (APIC 1 and APIC 2 on POD-1 and APIC3 on POD-2) and APIC4 as stand-by on POD-2 - which is pretty much how you explain it nearer the end of your question. The key point being that you need three active nodes, and although the 4th APIC is classed as "stand-by", it is really a "potential replacement" for a failed node. Now my first advice is that you raise a TAC case and make use of their expertise. But in regard to your question, I think you logic is sound as far as Step4.  At which point I think you should stop. (I'm happy to be corrected on this point) where your new APIC has taken the role of APIC3, and the removed and cleaned APIC3 becomes the standby APIC4 at DC-2.     ... View more

Hi @Mohamed.Tawfeeq , I'm curious about your comment  "but I was astonished to know that version HX 4.0 (1a) can segregate clusters even it's on the same FI" because I have not seen this option in v4.0(1a).  Where did you read this? I would LOVE to see your reference   So I suspect that the way you achieve what you want to do (have a VSI cluster and a VDI CLuster) is the same as all previous versions, which is (tailored for the design below from your post): Step#1: Install the VDI Cluster (Standard install) but with a couple of variations: Make sure you allocate at least 8 IP addresses for the CIMC (or KVM addresses in the installer) when slecting the nodes to deploy, choose only nodes 1-3. When you get to the Advanced Configuration options in the Cluster Configuration Screen, select the [x] Optimize for VDI Only Deployment [See note below] When the cluster is operational and you have saved your configuration from the installer, and then choosen to Start Over,move on to... Step #2: Install the VSI Cluster. Using the "I know what I'm doing" options of: [x] Run Hypervisor Configuration [x] Deploy the HX Software [x] Create HX Cluster and then continue with a normal install, this time: selecting nodes 5-8 for the deployment, use the same VLAN IDs, and the same vCenter Datacenter name, but... ..use a DIFFERENT vCenter Cluster Name (and a different HX Cluster Name - recommend making the HX CLuster name the same as the vCenter Cluster Name) Note: I have not had a chance to actually test this, and I have a sneaking suspicion that you may not see the [x] Optimize for VDI Only Deployment option unless you choose the I know what I am doing path and just select all options. I hope this helps   Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem ... View more

Hi, True, Cisco do use the word "quorum" in regards to the two-node Hyperflex Edge, but that it the only place I've seen Cisco use that term in relation to Hyperflex, and is appropriate becuase the stripe units for ALL VMs are present on each node.  We could use the same logic and probably refer to a quorum with a three node cluster using RF3 as well. When it comes to LAZ, you need at least 8 nodes before LAZ kicks in.  With less than 8 nodes, you can consider a cluster with n nodes to have n Logical Availability Zones, each with one node. In either case, loose a drive or a whole node or even a whole zone, three strikes and you are out. But the stikes MUST be in different zones. ... View more

HI @smailmilak , OK. I think your problem is that you know too much about distributed databases and the concept of a quorum.  In Hyperflex there is no quorum concept because, unlike a database that will have each shard replicated three times, each VM is split into many stripe units, which are distributed evenly across the cluster - and each stripe unit is replicated three times. Without LAZ, the loss of data on any three nodes (be it disk drive or complete node) will put the system offline, because potentially at least one stripe unit of one VM is no longer available. With LAZ, those three losses have to be across three different Zones.  Two losses in the same zone mean at most one one stripe unit of any VM has been lost - in fact the entire zone can be lost and at most one one stripe unit of any VM has been lost. So it's not about how many nodes we need to form a quorum, but about how many nodes can we loose without the potential of loosing data.  And without LAZ, it makes no difference if you have 5 or 30 nodes, you can't tolerate three losses.  With LAZ, you can't tolerate losses in three different zones. ... View more

Hi @smailmilak , The answer is one of those "It depends" answers. If you have two nodes in the same zone fail, it is no worse than having one fail. In fact you can loose ALL the servers in one zone with the loss of onl one copy of any data - should you be so lucky. With replication Factor 3, you could loose ALL the servers in TWO zones, and you'd still have one good copy of all the data somewhere. So, in your case with 8 servers, you could potentially loose up to four servers and till have a good copy of all the data, so long as those four servers were contained to two LAZs.  On the other hand, you could loose three servers in three different LAZs, and you'd be in read-only mode. The idea of LAZ is to increase your chances of NOT loosing data if multiple servers fail.  If you have four servers, data is stiped across four servers. If you have 4 Zones, data is striped across 4 zones, so if yo uare lucky enough to loose two servers in the same zone (and statistically you will be 50% of the time) then the hit is less painful. I hope this helps   Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem ... View more

OK - perhaps I mis-read yoru last reply. I though you had it sorted. It might be time to contact the TAC ... View more

Glad to see you got it sorted. ... View more

@supportgns , I see your point.  And your suggestion: it's perfectly possible to send this OSPF traffic through the current uplink by adding the routed network as a VLAN to the trunk, and adding another physical wire to the port-channel. is a great idea. Exactly what I would reommend, just remember that if you alrady have two ports in your port-channel on each FI, you'll want to keep the same number of ports on each FI.  In other words (looking at the diagram below), if your design is like the one on the left, and you add another link to PO211, I'd advise you add another to PO212 in case of fail-over.  If your design is like the one on the right, it is a bit more complicated. You can't really add ONE link to PO201, you'd want to add two, one to each upstream Nexus switch. And then, you should also add two more links to PO202 as well for balance, so what starts out looking like adding one cable may turn out to be adding four!   I hope this helps Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem   ... View more