In SQL server you have limitation with varchar(max) column and want to encrypt it and as well ENCRYPTBYPASSPHRASE has a limitation that it can only encrypt data up to 8000 bytes.
You can also use another approach:
1. Use a Symmetric Key: SQL Server gives CREATE SYMMETRIC KEY statement to create a symmetric key that you can use for encryption and decryption.
CREATE SYMMETRIC KEY MySymmetricKey
WITH ALGORITHM = AES_256
ENCRYPTION BY PASSWORD = 'YourStrongSymmetricKeyPassword';
2. Using a certificate as well can help you to encrypt and decrypt data with SQL server instances.
I hope it works for you!!