It is always handy if you say a bit more about your deployment, like MCU Hardware/Software version,
how the conferences are dialed up (AA, e164, uri, sip, h323), if its integrated with the VCS and so on.
What I would try is to assign two numbers to the conference (under conferences > conference > configuration),
the "Numeric ID" and the "Guest Numeric ID" and either let the "master" know as the only one the "Numeric ID" to dial up (all others dial the "Guest ID") or define the master as the master.
Be aware that the conference does not get connected unless a master is present, untill all participants would see
a black screen and hear (if enabled) a "waiting for the conference chair person"
Please rate the answers using the stars below!