Hi Azi,
What type of LAN device are you using? Does it support NBAR for application identification? If yes, then you can match protocols in the ACL for classification
If you don't want all the packets destined to NMS to be classified, you can use a combination of ports with the IP addresses for classification.
E.g. If NMS IP is <NMS_IP> and the managed device in LAN has an IP <LAN_IP>. Then the ACL that will be used for classification of a SSH packet coming from the LAN side, and destined to the NMS, can be as follows
access-list extended MATCH_NMS
10 permit tcp host <LAN_IP> host <NMS_IP> eq 22
This can be matched in a class-map to match the relevant traffic. The on the LAN port, in the inbound direction, you can have a policy that sets the appropriate DSCP value to this traffic. With this DSCP marking, the downstream router(s) and network components can provide appropriate priority in the network, provided these routers have appropriate QoS policies.