Well, if this wasn't meant to be controversial!
http://news.bbc.co.uk/2/hi/science/nature/7371645.stm
A study (in Australia, one of the biggest uranium exporters) tells that extracting ore from the soil and uranium from the ore consumes Diesel-fuel, and more so as we need to grasp at deeper and poorer deposits as the easiest ones are depleted.
I consider we should account the form of energy this activity COULD use, not the one it DOES use today - that is, if excavators and lorries can run on nuclear-produced electricity, we shouldn't account them as CO2 producers.
Just as we shouldn't decide over biofuels based on the gasoline used to transport them, as we can transport them using biofuel as well.
In this view, the only limit is the amount of energy needed to extract the uranium that will produce energy. Which isn't said to be a very easy limit neither, from what I read before - making claims of huge available deposits less interesting, as these poor ores consume more (and maybe too much) energy to grasp.