I think its ok for your toadstool to touch your birdnest, shouldn't have much issue. Mine is touching too.
For xenia, if u have a scrapel or small penknife, u can try scrape it & peel it off from the base where its attached to the rock slowly in order not to damage it. If u can't remove it without hurting it then guess u have to destroy it.
Better to do at nite when its polyps all closed up.