Tbh if nvidia had a gf104 chip with some tweks and just plain stacking up of cuda cores I dont see the problem of calling it 5xx series. Both amd and nvidia had problems with 32nm transition and its normal that these next batch of cards be not revolutionary.
As long as they can keep up with Cayman they will be fine and Im quite certain they can provide a competitor to Cayman if they work on their gold gf104 chip.