By: Ron L'Esteve | Comments | Related: > DevOps
Problem
In my previous article, Azure DevOps CI/CD using GitHub Repo and Visual Studio Azure SQL Database Project, I demonstrated how to deploy an Azure SQL Database from Visual Studio using Azure DevOps using the Azure SQL Database deployment release task. However, with this process, the SQL Database Username and Password was exposed in the DevOps Azure Pipeline. This can be a concern for a number of reasons including security, compliance, audits and more. How can we secure secrets in Azure DevOps pipelines?
Solution
Azure DevOps tasks are the building blocks for defining automation in pipelines. Tasks are packaged scripts or procedures that have been abstracted with a set of inputs. Often times, these inputs contain sensitive credentials and secrets that need to be appropriately secured. There are a few options for securing secrets in AzureDevOps pipelines including pipeline variables, variable groups, and Azure Key Vault tasks. In this article, I will demonstrate how to implement these options. As a pre-requisite, ensure that you have completed the steps outlined in my previous article, Azure DevOps CI/CD using GitHub Repo and Visual Studio Azure SQL Database Project.
Option 1: Using Pipeline Variables
Pipeline Variables provide a convenient way to get key data elements into various parts of the pipeline. These variables can be used to store key pipeline secret credentials. By defining pipeline variables within the AzureDevOps pipeline, we are exclusively authorizing the associated Azure pipeline to access these secret variables within the tasks of the pipeline. Note that this method of storing and retrieving secrets by using pipeline variables does not utilize or link to Azure Key Vault secrets since the secrets are stored within the AzureDevOps as a pipeline variable.
Let's get started within implementing pipeline variables to store our Azure SQL Database login secret credentials.
To begin, open the Azure SQL Database deployment release pipeline task containing the Login and Password secrets.
Next, click variables to access pipeline variables.
Within the pipeline variables tab, add the administratorLoginUser and administratorLoginPassword and values. Notice the lock icon to the right of the values. Click the lock icon to change the variable type to a secret.
The secrets are now secured and associated with the release pipeline scope.
Next, within the pipeline task, replace the actual secret values with $(administratorLoginUser) and $(administratorLoginPassword). This parameterizes the values and links it to the secret values defined in the pipeline variables tab.
Click save and then create a new release.
As expected, we can see that the database deployment task succeeded which means it was able to successfully utilize the secrets provided in the pipeline variables.
Upon drilling into the Agent job details, we can see all the detailed steps that have been successfully completed.
Now that we have successfully tested this method of securing secrets in the Azure DevOps pipeline let's delete the pipeline variables and save to prepare for the next option.
Option 2: Using Variable Groups
Variable groups are slightly different from pipeline variables in that their scope can be expanded to be shared between pipelines. Additionally, variable groups can be linked to specific stages in the pipeline, build or release along with the capability of linking to secrets withing Azure Key Vault. In short, this option of variable groups is more robust that pipeline variables.
Since we will be linking Key Vault secrets to the variable group we will create, let's start by adding our SQL Database administratorLoginUser and administratorLoginPassword secrets within Azure Key Vault in the portal.
Now that we have created the key vault secrets in the portal, lets navigate to ‘Library' within the Azure DevOps project and select the Variable groups tab.
Configure the following Variable group details and ensure to toggle Link secrets from an Azure Key vault as variables. You'll then be prompted to select your Azure subscription and associated key vault name and click Authorize to grant the variable group access to your Azure Key Vault.
Next, click Add to add the key vault secrets as variables.
When the ‘Choose secrets' UI appears, select the secrets that we created in the key vault and click Ok.
Finally, click Save.
Next, navigate to the release pipeline and select Variables, variable groups and then select ‘Link variable group'.
Select the Variable group that we created in the previous steps, associate it with the release scope, and click Link.
Confirm that the release pipeline variable group contains the linked SQL DB Credentials and click Save and then create the release.
Once the release pipeline successfully completes running, we can confirm that the Variable groups containing the linked key vault secrets authenticated the SQL Server as expected.
From the deployment process Agent job details, we can see that the key vault secrets we downloaded and utilized within the pipeline.
Now that we have successfully tested this method of securing secrets in Azure DevOps, we can unlink the variable group from the release pipeline to prepare for Option 3.
Additionally, delete the Variable group from the Azure DevOps Library.
Option 3: Using Azure Key Vault Task
So far, we have tested two methods of securing secrets in Azure DevOps: pipeline variables and variable groups. There is one last option that is left to test and that is the Azure Key Vault Task within the pipeline task.
Let's begin by searching for Azure Key Vault in the task search section.
Next, add the Azure Key Vault task to the release pipeline.
Ensure that the task is placed before the task requiring the key vault secrets. In this case, we will place the Azure Key Vault task before the Azure SQL Database deployment task containing the SQL secret credentials.
When configuring the Azure Key Vault task there is the option to either enter the desired secrets in the Secrets filter section or leave a * to download all secrets from the Azure Key Vault.
To only retrieve the desired secrets, I have only entered the administratorLoginUser and administratorLoginPassword secrets in the filter section.
Finally, notice the option to ‘Make secrets available to the whole job'. By enable this option, the key vault secrets will be exposed to all subsequent tasks in the job and not just the single task that follows this one. Since we only need this secret for one task in the pipeline, let's leave this unchecked for now.
As expected, when the release completes running successfully, we can confirm that the Azure Key Vault Task containing the secret values were successfully authenticated by the Azure SQL Database deployment task.
Lastly, let's take a look at the Agent job details to once again confirm that the Azure Key Vault task successfully complete and passed the secret values to the Azure SQL deployment task.
Next Steps
- For more information on setting up an Azure Key Vault, read Quickstart: Set and retrieve a secret from Azure Key Vault using the Azure portal.
- Read more about Using secrets from Azure Key Vault in Azure Pipelines.
- Read more about the Azure DevOps Release pipeline Azure Key Vault Task.
- For more information on pipeline variables and variable groups, read Define Variables.
About the author
Ron L'Esteve is a trusted information technology thought leader and professional Author residing in Illinois. He brings over 20 years of IT experience and is well-known for his impactful books and article publications on Data & AI Architecture, Engineering, and Cloud Leadership. Ron completed his Master’s in Business Administration and Finance from Loyola University in Chicago. Ron brings deep tecThis author pledges the content of this article is based on professional experience and not AI generated.
View all my tips
