Is there a way to configure the 2504 WLC so that its internal DHCP only services the LAPs?
I don't want the controller to give out IP addressed to wireless clients.
DHCP on the WLC is never intended to offer DHCP services to any "wired" side devices, which includes your APs. These DHCP pools are specifically to hand addresses out to wireless clients attached to WLANs of the specific WLC it is configured on. Even then, unless you absolutely cannot offer DHCP somewhere else, DHCP on the WLC is not a suggested practice.
DHCP Pool Configuration and Restrictions.
But don't I need DHCP enable so that the LAP gets an IP address in order to join the WLC.
My LAP wouldn't join the WLC until I enabled DHCP.
No, that is not the intended use of the internal DHCP server on the WLC. I have seen some versions of code/platforms that will leak and offer DHCP to "wired" devices (such as your APs), but that is incorrect behavior.
Your APs need to get a DHCP address from somewhere other than the WLC.