Users with `read` permission of pull requests can be assigned too (#27263) (#29372)
Backport #27263 by @lunny
This PR will also keep the consistent between list assigned users and
check assigned users.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>