Frankly, I have been guilty of this more often than not, and I always wondered why some images were just a bit fuzzy. I always assumed I had jiggled the setup during the exposure. I decided to create an example as I was photographing in Redwoods National Park in Northern California.
I mounted my Canon Rebel T2i body with 24-105mm IS lens on my tripod and photographed a redwood trunk from about eight feet away. I used no filters, and because I was shooting in shade, adjusted the white balance to "Cloudy". I was very careful to avoid shaking the setup during exposure. Then I exposed a shot with IS turned ON and another with IS turned OFF. Once I returned home, I cropped out a small area in order to magnify the surface of the bark. No processing or sharpening has been performed - both images are straight out of the camera. See how much sharper the second image looks? Because digital camera include an infrared (IR) filter which increases blur slightly, it would look even better with a bit of post-process sharpening.
Cropped image as shot with IS turned ON.
Cropped image with IS turned OFF.