Welcome to the party! Nexus Dashboard (ND) is a platform for hosting multiple services such as Insights, NDFC, Data Broker in addition to Orchestrator (NDO) - which is what you'll need for provisioning ACI Multisite policies.
For your questions:
What would be the difference between the APIC and the ND?
[Robert] APIC is the controller a single-fabric fabric (collection of Spine & Leaf switches running a shared management and availability domain). An APIC cluster can manage a single/multi-pod fabric. The APIC is responsible for pushing local configuration to the fabric as well as collecting run-state information (stats, faults, policy-resolution results etc). Nexus Dashboard (ND) as stated above is a hardened app hosting platform (based on Kubernetes) which supports the apps mentioned above. What you're probably more interested in is APIC vs. NDO (Orchestrator). Where APIC provides single fabric management & monitoring, Orchestrator sites a level above, to allow Tenant policies to be distributed to multiple sites. NDO pushes config to the APIC, and the APIC in-turn deploys it to it's fabric. When you talk about ACI multisite you may want to have some of your Workloads spread across sites (ie. Web endpoints for the same app distributed across sites). Orchestrator creates all the policy-translation to allow L2 or L3 connectivity between endpoints across any site. Orchestrator does NOT collect run-state level information & fault details as APIC does. So for pushing policies across one or more Sites you'll use Orchestrator (NDO). For monitoring fabrics you'd leverage APIC.
What kind of connection should I have for the interconnection between sites? Is it leased-line, VPN, or MPLS?
[Robert] For ACI multisite design, you typically have one or more Edge devices which we refer to as ISN (Inter-site network) devices. These aren't anything special - they can be any make/model of switch/router that supports OSPF or BGP, Jumbo MTU, and Sub Interfaces. Doesn't even have to be a Cisco device (but hopefully it is!). These ISN devices are not managed/controlled by ACI. You will simply push policy to the spines to create that first hop connection using the appropriate L3 configuration. Now, between your ISN devices (the DCI) you can use whatever IP connectivity option you want. Again, only really supportability concern is that Jumbo frames support min. 1600B. Customers us MPLS, DWDM, Dark Fiber, WAN Links etc. If you happen to be using a shared/public WAN link between your ISN devices, ACI can encrypt the Site-to-Site traffic using multi-hop MacSec (aka CloudSec) assuming your Spine model supports it.
What is the ND use for basically?
[Robert] Per above its a highly available, hardened, network-team-owned, appliance for hosting Cisco DC Day2 Apps (Orchestrator, Databroker, Insights, Fabric Controller etc.
Is it possible to have same IP addressing scheme in each sites then I will form an L2 extension between the sites?
[Robert] Stretching IP Subnets across sites - yes! One of biggest values ACI Multisite provides is simple L2 or L3 extension between sites. I'd suggest you grab a big coffee/tea and read through our ACI Multisite Whitepaper written by my good friend Max Ardica. It's a long read, but its an easy to follow overview for all the types of questions you'll have. https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739609.html
Cheers,
Robert
