In 2014 environment, when I tried that once it didn't work. In that case, the secondary database showed in "restoring state". I ended up getting rid of the secondary and then added the db (backup and restore through ssms) to AG. So, wanted to verify if that is what needs to be done. Jeffrey suggests that a regular join won't work in 2014 and has a process outlined above.
When testing, I removed the db from AG. The db showed as "not synchronized" in the secondary. But I was able to add back to AG with join only (ssms skipped the verification process).
Perhaps, when it is in "restoring" state (say the db gets kicked out of AG for some reason) then the join wouldn't work?