Sure, you can break down the merge into its constituent INSERTs, UPDATEs and DELETEs.
Once you have done that, you will be in a better position to work out which statement(s) are the more time-consuming.
You will also be able to get execution plans which should help with your troubleshooting. I am assuming that you have taken the time to ensure that the columns you are matching on are appropriately indexed.
If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.