Yeah, the environmental issues that are orders of magnitude less problematic than literally pumping the toxic chemicals into the atmosphere like with fossil fuels, vs comparatively miniscule amount of solid waste to store inert.

these rocks are long terms with the consequences on humans and the environment thousands of years later.

You bury them in concrete, done. Nuclear waste isn't an issue and hasn't ever been

Isn't nuclear energy like super safe and have killed incredibly few people compared to all the other energy sources?

Or are you talking about destilling the magic rocks very much and putting them in a bomb?

Or to put it another way, we almost ruined a large swath of land and learned from that mistake, but chose not to use it so when we do have to switch to nukes because destroyed our planet we will have forgotten all those lessons and do it again.