Well, if it's sps, I add pretty much anything that catches my eye. The condition is that its got to be small. And now that I'm running out of space, any new additions have to be tiny.
As for LPS, shrooms and softies, I only keep a few that are generally non-aggresive. And these I keep near the bottom and on the sandbed. I recently moved my torch to my other nano as it was getting too large and too close to "my precious".
In short, yes, I have to be careful.