it all depends on the specific piece of wood.
generally elm isnt known for its stability - but if you find a piece and the grain suggests it could work, and after a while settling in it seems dimensionally stable then its worth giving it a go
same with the neck woods, yes they can work and they can be stable - if you get the right piece (which applies just as much to maple as it does to rosewood, pau ferro or paduak)
wood is too variable to be any more specific than that.
...
also, why obviously with a separate fingerboard? 1-piece rosewood necks with a rear mounted truss rod can be awesome.