An event manager would have the rights to remove someone as the role manager for a particular, in a particular event.
This can be done on the Role & Qualifications tab.
1. Click on Take Action on the right of the page
2. Click on Edit Role Manager
3. Remove the Role Manager(s) by un-checking the box against the name of the respective Role Manager
Note: Removing someone as a Role Manager does not remove the person as a staff from the role.