Try running the x264 HD Benchmark and watch the CPU usage on the 2nd pass encodes. It should be >99 %.

Hmmm.. which BIOS are you using? My DFI board has an option in there to disable the other two cores but I'm pretty sure that the P5B (at least my old P5B-Deluxe) didn't have such an option. If you're using the latest or a likely a beta BIOS, it might be a problem. Did you try asking in the Asus support forums?