Focus on CI/CD pipelines
Introduction
This is the fourth part of Azure mini-series Azure explained deep enough
Part1: Azure explained deep enough: learn and get certified
Part2: Azure explained deep enough: Containers
Part3: Azure explained deep enough: Azure PaaS
Part4: Azure explained deep enough: Azure DevOps <- this blog
In this blog, we are going to look into Azure DevOps, CI/CD pipelines and Infrastructure as Code.
Before we take a closer look at Azure DevOps, let’s try to understand first what is DevOps
According to Microsoft, it is:
A compound of development (Dev) and operations (Ops), DevOps is the union of people, process, and technology to continually provide value to customers.
What does DevOps mean for teams? DevOps enables formerly siloed roles — development, IT operations, quality engineering, and security — to coordinate and collaborate to produce better, more reliable products. By adopting a DevOps culture along with DevOps practices and tools, teams gain the ability to better respond to customer needs, increase confidence in the applications they build, and achieve business goals faster.
I like to use this shorter description:
DevOps is a way to ensure a continuous, automated, uninterrupted flow of change. For software, this flow of change will be of course new features and products.
First look at Azure DevOps
Azure DevOps is a set of collaborative development tools. Azure DevOps features are similar to GitHub, GitLab, Bitbucket and many others. In fact, Microsoft acquired Github in 2018 and I speculate that the roadmaps of Azure DevOps and GitHub will move closer with time.
It uses Organization as a top grouping level. You can use an organization to separate business units, departments or distinct physical offices, it is up to you.
Each organization (a default one is created automatically), comes with the following:
Each organization gets its own free tier of services (up to five users for each service type) as follows. You can use all the services, or choose just what you need to complement your existing workflows.
- Azure Pipelines: One hosted job with 1,800 minutes per month for CI/CD and one self-hosted job
- Azure Boards: Work item tracking and Kanban boards
- Azure Repos: Unlimited private Git repos
- Azure Artifacts: Package management
- Unlimited Stakeholders
- Five Azure DevOps users (Basic)
- Free tier of Microsoft-hosted CI/CD (one concurrent job, up to 30 hours per month) 2 GiB of Azure Artifacts storage
- One self-hosted CI/CD concurrent job
Each organization can have one or more projects. Organizations are only for grouping and billing purposes, a project is where all interaction with Azure DevOps happens from the user point of view.
Once deployed Azure DevOps Project can contain the following components.
For an in-depth review of all services, please refer to the official Azure DevOps Documentation.
Sign Up for Azure DevOps
There are two ways to sing up for Azure DevOps. Today we are going to use GitHub integration. Please follow the steps outlined here to set up the Azure DevOps service.
CI/CD
Out of all the Azure DevOps Services, I would like to focus a bit more on CI/CD process. CI/CD stands for Continuous Integration/Continuous Deployment/Delivery. From a very high level perspective, CI/CD looks like this:
Let’s take a closer look at what is happening on the diagram and map it to Azure DevOps components.
Parts that are not depicted on the diagram are how the work gets assigned to a developer, here is one common scenario from a SCRUM Process.
In Azure, DevOps work comes from User Stories refined from PBIs which stands for Product Backlog Item, so let’s assume that this is also the case here.
Once a developer starts working on a user story, they will usually create a separate branch, often called Feature Branch and connect it with the User Story.
From this point, the diagram provides a high-level overview of a generic CI/CD pipeline.
This is of course a very simplistic pipeline, in real-life scenarios, pipelines are usually much more elaborate. Here is an example of a more mature pipeline:
Pipelines as Code
Traditionally pipelines in Azure DevOps and its predecessor Team Foundation Service (TFS) were created and managed via GUI. One of the main reasons, why this was a “click-based” process was the fact that Dev and Ops teams were separate silos.
The beginning of DevOps movement and Shift Left philosophy called for inventing new ways of describing build, deployment and infrastructure requirements. Developers should be fully empowered to design and execute pipelines. Based on those requirements Pipelines as Code were introduced.
Demo
For demo purposes, we are going to deploy a sample static web page to an Azure Storage Account with Static Website Hosting option enabled.
Create storage account
Here are the steps If you would like to follow along with the demo:
Log in to the Azure portal and select Cloud Shell. Follow this tutorial to activate cloud shell if you are login for the first time on a fresh account. Once you are in the cloud shell, make sure to select the bash environment and follow the steps below.
# Clone the exercise repository
git clone https://github.com/ilearnazuretoday/azure-devops.git# Switch to right directory
cd azure-devops/terraform# Initialize terraform with Azure Provider
terraform init# Validate terraform scripts
terraform plan# Create infrastructure, confirm with "yes"
terraform apply
The output should look similar to this:
Create service connection to allow deployment
In order to be able to deploy form our Azure DevOps service to our Azure Subscription, we need to create a service connection to enable it in Project Settings.
This action will create a service principal account on our behalf to establish the connectivity.
Good practice is to narrow permissions scope!
Note down your service connection name, it will be needed for the pipeline setup.
Deploy sample static web page
We will deploy a sample app directly from GitHub, but for demo purposes we are going to clone the repository into our Azure DevOps first.
It is very easy to do it in Azure DevOps. Click on the repository and use this link to import.
Once the repository has been imported into Azure DevOps, we can set up CI/CD pipeline. The CI/CD pipeline YAML file is already in the repository, but please remember to adjust the change
- azureSubscription: serviceConnectionName from previous step
- storage: name of the storage account to deploy to from terraform output
Here are the steps to set up the initial build:
Let’s look at the pipeline file and see what it contains.
The pipeline file describes steps that need to be performed in sequence to build and deploy our software:
- choose trigger (can be branch, tag, etc)
- make sure to exclude files which you don’t want to trigger the pipeline
- choose VM image for the build agent
- perform build and deployment steps (specific to what you are deploying)
- check-in the YAML file into the repository called
azure-pipelines.yaml - trigger a change in your code that satisfies trigger criteria and push changes to remote
- build should trigger automatically
- under the Pipelines -> Pipelines menu, you can observe pipeline logs live
If you are interested in learning more about PWA web technology, check out 5 Options to deploy static web sites
You might be wondering where do the tasks come from? Like AzureFileCopy or DotNetCoreCLI etc? Those tasks are mostly TypeScript programs or PowerShell scripts and you can see their source code on GitHub!
Tasks are generic and reusable and you could write your own as well.
If everything went well, navigate to the URL outputted by terraform and you should see a static page deployed! experiment with the CI/CD process.
- Try to change any file (other than README.md), did the pipeline trigger?
- What will happen if there is an error and dotnet cannot build the artefacts?
Destroy the Infrastructure
This step is IMPORTANT if you have followed along with the demo. To avoid unnecessary charges, let’s remove the resources, it is very easy to do this with terraform.
# Destroy all resources, confirm with yes
terraform destroyConclusion
We have covered a lot of ground and some of the concepts might be new of confusing especially if you are new to Azure. Please don’t be discouraged, instead try to review everything step by step and deepen your knowledge in the areas that you are least familiar with. I have tried to recreate a scenario that is close to real flow, that’s why there is IaC and deployments mixed with core Azure DevOps concepts.
You can check out the rest of my blogs to learn more.
