Well, it’s 100% linked to your contacts in one way or another because when you install it Signal will happily alert you to which ones of your contacts are already using Signal. I can’t see how they could manage that without slurping up your contact information.

Presumably if there’s a switch to turn it on then there’s also a switch to turn it back off again.