You don't need to assign a AAA Server to each NDG. NDG's can be groups of just AAA Clients (the normal method), combinations of AAA Clients and Servers, or just groupsd of AAA Servers.
As I said, the "normal" way to define an NDG is just as a bunch of AAA Clients, you don't need to assign these clients to a server specifically.