No matter how fine the surface looks, there are always pits scratches bumps and hollows. I'd guess that the hot one is more so than the other, but the difference could be only microns or it could be a tiny burr that's become the only point of contact. Regardless, you need TIM for a reliable thermal interface.