OK, found the problem.
This command helped:
Get-AzVMUsage -Location "East US"
The issue was not the zone... the issue was that the limit seems to be 4 vCPU, which honestly, it is ridiculously low. I had to use 3 VMs with 1 vCPU each and that fixed my issue.