Siva:
You can configure MTU on both the physical interfaces as well as the SVI for a vlan. The thing is that the MTU of a packet originating on a host on a vlan is not checked at the ingress point to the vlan interface. It is checked, however, upon egress to the vlan. Therefore, if a packet requires L3 switching (which is why it was forwarded to the SVI in the first place, right?), the packet will be fragmented if the MTU is smaller than the MTU of the physical port. Unless, of course, the DF bit is set. With fragmentation comes poor performance, latency, overhead, and applications that can time out and reset.
As an example, if you enable jumbo frames on a physical port that belongs to a certain vlan, it is not enough if you expect that functionality to be extended to the L3 switching domain. You must configure jumbo frame support on the SVI also.
HTH
Victor