I restored msdb from other server for hardware migration. The SQLAgent Service account was not in Systemadmin server role and caused the SQL Server Agent service couldn't start due to lack of permission to access master database. I added the SQLServerAgentService account to SysAdmin server role.

SQLServer Agent Service was able to start right away. Just thought about to share.