OK it seems to be some sort of "memory leakage" bug - If I don't reboot my machine for like 48 hours or so, the GPU load of the first card would stay at 100%, and any UAC prompt for user privilege elevation of Win7, or alt+tab switching into/out of game would cause seconds of lag.

I hope they fix this bug in the final release.