Right now, it seems you cannot reply to extension reviews unless you have the Contributor (or Owner) role, which also entitles you to publish/update extensions.
Consequently, our publisher has an uncomfortably large number of people who can publish/update extensions--while most of them only need to be able to reply to reviews. Ideally, we would like to remove publishing permission from all but a few managed identities that we use for publishing.