Check to see what kind of utilization you are getting on that link . Any traffic that has to be routed between vlans travels up that link to be routed and then back down that same link to the other vlan along with any traffic going off the router to the wan etc.. With the errors you sseeing you probably seeing at least bursts that may be tieing up that 100 meg link . Check nic settings on the devices.
One simple issue you might be bumping up against, a 3825 is not a high performance LAN router. I don't recall whether 3825 interfaces support jumbo Ethernet, but if they don't, MTU fragmentation can also drag down a router's performance. What does the router's CPU usage look like when it's busy?
Performance with the same VLAN being better is likely due to the router being bypassed.
If the router is overstressed, you might consider obtaining a L3 switch, the 8 port 3560 might be ideal for you.