it's definitely a perceptual thing. it happens to me, too.
that's also why live performances are generally at a slightly higher tempo than studio albums (where the tempo is usually monitored).
unless the drummer is cheating with a click track, that is!