In layer 2 switching usually we don't speak about ports instead of VLANS. So you can only limit the access based on VLANS. If you assign the first port to the dedicated VLAN, only from the first port you can connect to the switch for admin purpose. But if this VLAN is not only locally significant and between the other switches the management VLAN is published,from other switches PC which are in the same VLAN can access to the switch....
Try to use something like ACL for limitation, I don't know in Cat500 how you can set it via web interface....
bye
FCS
Please rate me if I helped.