Not usually. Even the combat and assassination models are under strict control of their owners, so the only people they are dangerous to it their enemy.

Incidents are rare. But then again, so is manumission - uprisings are infrequent and easily put down, in general.