Thats because the transmission media is not synchonous. Transit times from A to B and to A can differ greatly in these days of client-server applications, thats why QoS is becomming so important in most networks. The amount of information flowing across a link is usually skewed in favor of one side of the link or the other.
Jitter is a variation in packet transit delay caused by queuing, contention and serialization effects on the path through the network.
