The correct server name should be the NETBIOS\INSTANCE of the host machine, not the listener.  As AOAG use separate instances and the Listener is just a DNS alias which controls where to route the connections its not actually a server.

Is there a reason your trying to loop back using 4 part naming, and not using 3 part naming instead?