sounds to me like monitoring is turned on, but that would be odd since at least with me, reaper doesn't start with any tracks or fx monitoring...
sometimes I've had weird issues where moving my mouse would cause noise through my speakers, it was only when I used my onboard audio though...