According to what I found on the internet and tests, the lowest permission is CONTROL on a certificate and only REFERENCES on a symetric key. Then user can't drop or modify the certifacte and key but still is able to turn off keys management on the certificate and set a password, but maybe I missed something.
"Overview of encryption and types of keys, the key hierarchy, key management, encrypting data, encrypting a database (TDE) and Extensible Key Management (EKM)."