you may have bursty traffic. Taht mean even if with show int you see normal load. you may have a peak of data that is received in a short period - les then 1 sec - that serial interface can not handle then it is dropped.
Please issue the sh int command several times with may be a 1 sec interval, then when you see drops increased check from last show int, the output bytes and compare to output bytes in the previous command
the difference should be output in bytes/sec and compare it to port speed.
Hope that is clear if not let me know
regards,
cisand