Just a guess but is the secondary replica set to asynchronous and/or manual failover? What do you get when you run the following:

SELECT

ag.name,

r.replica_server_name,

r.availability_mode_desc,

r.failover_mode_desc

FROM

sys.availability_replicas r

INNER JOIN sys.availability_groups ag

ON r.group_id = ag.group_id

ORDER BY

ag.name,

r.replica_server_name;