This has to do with the AGC functionality in 3.1(3). It tried to raise all the recording to the specified target dB level (i.e. -26 or there abouts), including silence. In it's efforts to do that it magnifies the background hiss (or comfort noise) which is what you're hearing there.
In Unity 3.1(4) the AGC stuff was enhanced to be smart enough not to touch it if the volume is lower than a set level (-40 by default) and to not raise it more than a set amount (10 dB I think by default) to avoid this problem.
If you can't upgrade to 3.1(4) then you'll have to turn off AGC on this system if you want that problem to go away... however this will introduce other problems that I'm sure you're familiar with involving varrying recording volumes coming from different internal/external sources and the like.
