Does a Peer score mean my peers are "right"?

No. A Peer score is how a group of people experience your behavior. It is real and useful, but it is a perception, shaped by who you invited and how well they know you.

Treat it as information about impact, alongside your own view.