Skip to main content

CI/CD Pipelines

CI/CD pipelines, or Continuous Integration/Continuous Deployment pipelines, function as an automated process that triggers on a determined action and automatically builds and hosts a project. This process is implemented to achieve better testing on a 'live'/'hosted' environment while also improving efficiency by removing the repeated effort associated with deployment.

Overview & Architecture​

There are three major portions of the CI/CD pipeline: GitHub workflow files, an internal Azure Linux virtual machine, and two or more (likely external) managed identities.

The process starts when some action on GitHub, such as a push to a branch or a release being made, triggers one of the GitHub actions workflows. The workflow leverages the Azure VM to build the project, and then hosts it to the Azure resource using the managed identities for authentication.

Setup & Implementation​

Within this folder are guides to adding a basic CI/CD pipeline to any given repo for an Azure Web App or Azure Static Web App. Workflows can be further developed to handle an array of tasks like testing, adding comments to PRs, creating documentation, sending notifications, managing tickets, and more.

Anatomy of Process​

This graphic shows the process in whole (click to open full in new tab). Below each piece is explained in detail.

FAQ​

The VM is on our tenant, but the web app is on the client's - will CI/CD still work?

  • Yes! The VM only handles building the project image. GitHub then spins up its own ubuntu instance and the deployment authentication is done between GitHub and the managed identities on the client's tenant. Review the 'GitHub-Actions' Virtual Machine section above for more specifics on this.

Are workflow files branch specific?
  • Yes! If changes are made to the workflow files in Main and not pulled to your current branch, the current branch's workflow files will be used when a trigger occurs on that branch (such as a PR or commit push).
How does role-based security actually work?

  • Two major portions protect the dev and staging branches (using the setup in this guide as example).
    • GitHub environment rules are protected by org/repo owner permissions.
    • Azure managed identities are protected by explicitly assigned Azure permissions.
  • Ex. a staging environment can be set up to only allow GitHub releases -> the managed identity can be set up to only federate on the staging GitHub environment -> the staging deployment slot can be set up to use the staging managed identity as its authentication for image pushes and hosting changes.
    • So, changing a workflow to push a branch to staging will always fail since it is not a tag and does not match the environment rules, meaning you must have permissions to make a release to push to the staging deployment slot.
      • Configuring GitHub rules and preventing developers with 'write' permissions from making releases can be found under other (currently TBA) documentation.

Why does it matter if secrets are repo-level or environment-level in GitHub?

  • Neither repo-level nor environment-level secrets can be read by any user. However, Repo-level secrets can be changed or deleted by any developer with 'write' permission or higher on the GitHub repo, while environment-level secrets can only be changed or deleted by org/repo owners. A small guide on how to migrate repo-level secrets to environment-level secrets can be found at the end of the guide below.

GitHub​

Workflow Files

Generated initially by Azure (if following the guide below), workflow files are written in YAML and are stored in GitHub repositories under .github/workflows. They function as the procedures that GitHub actions follow when triggered. They specify what trigger begins the flow and what all occurs in the GitHub action. They're highly customizable and can be made to handle testing, add comments to PRs, create documentation, send notifications, manage tickets, and more.

Environments

GitHub natively allows the creation of 'Environments' under Settings -> Environments which can be used as permission-controlled layers for deploying since only org admins and repo owners can edit environments. When referenced by Azure managed identities and GitHub workflows, Environments can be used to ensure any trigger of the workflow or use of the identity follows the branch/tag name protection rules set by the environment. Additionally, environment-level encrypted secrets may be defined in environments and used in workflows.

FortisureIT Azure Tenant​

'GitHub-Actions' Virtual Machine

Currently hosted on Azure, under the [Fit-Internal] resource group, is a Linux x64 virtual machine named 'GitHub-Actions'. It is configured to have a dedicated ip that is allowed through our GitHub org ip restriction rules, allowing it to run build operations on any FortisureIT repo. This virtual machine is prepped with an org-wide GitHub runner that can be used to handle one GitHub action at a time, but is organized so repository-specific runners can be added for concurrent actions.

IMPORTANT NOTE: The VM only handles 'building' the image. Once it is built, no further step in the workflow requires access to the codebase, so GitHub spins up it's own ubuntu instance to communicate with Azure and handle hosting. The hosting step CAN be handled by the VM as well, but, following the guide below, has been left as a free GitHub job to save resources on our VM.

Client Web App Azure Tenant (Web App Specific)​

Managed Identities

Azure handles the authentication of non-user entities through managed identities. Typically these are connected to a resource, but they can be made as their own resource as well. Each managed identity may be given federations which function as an authentication method for the identity (A GitHub org-repo-environment combo as example). These identities may also be given any set of permissions ('website contributor', which is needed to deploy to a web app, as example).

Deployment Slots

Azure Web Apps, at premium scale (P0V3) or higher, allow for the creation of deployments slots. These slots are usually configured as a pair of two: 'dev' for internal testing and 'staging' for client testing. The guide below follows this setup, but can be tweaked to fit any setup. Each deployment slot, and the prod web app, can be configured to use 'GitHub actions' as a source under their respective deployment center.

File Preservation​

Here are full versions of important files contained within our VM in case we ever need a backup.

sudoers.tmp
#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
#
Defaults        env_reset
Defaults        mail_badpass
Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"

# This fixes CVE-2005-4890 and possibly breaks some versions of kdesu
# (#1011624, https://bugs.kde.org/show_bug.cgi?id=452532)
Defaults        use_pty

# This preserves proxy settings from user environments of root
# equivalent users (group sudo)
#Defaults:%sudo env_keep += "http_proxy https_proxy ftp_proxy all_proxy no_proxy"

# This allows running arbitrary commands, but so does ALL, and it means
# different sudoers have their choice of editor respected.
#Defaults:%sudo env_keep += "EDITOR"

# Completely harmless preservation of a user preference.
#Defaults:%sudo env_keep += "GREP_COLOR"

# While you shouldn't normally run git as root, you need to with etckeeper
#Defaults:%sudo env_keep += "GIT_AUTHOR_* GIT_COMMITTER_*"

# Per-user preferences; root won't have sensible values for them.
#Defaults:%sudo env_keep += "EMAIL DEBEMAIL DEBFULLNAME"

# "sudo scp" or "sudo rsync" should be able to use your SSH agent.
#Defaults:%sudo env_keep += "SSH_AGENT_PID SSH_AUTH_SOCK"

# Ditto for GPG agent
#Defaults:%sudo env_keep += "GPG_AGENT_INFO"

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root    ALL=(ALL:ALL) ALL

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL

# See sudoers(5) for more information on "@include" directives:

@includedir /etc/sudoers.d

# Example repo _work directory cleaner - Runnable by GH action to remove temp files created by ORYX
ALL ALL=(root) NOPASSWD: /home/GitHubAdmin/repo-runners/[example repo]-runner/wipe_work.sh
wipe_work.sh
#!/bin/bash
set -euo pipefail

TARGET_DIR="/home/GitHubAdmin/repo-runners/[example repo]-runner/_work"

# Safety check: refuse to run if directory is empty or /
if [[ -z "$TARGET_DIR" || "$TARGET_DIR" == "/" ]]; then
echo "Refusing to wipe unsafe directory"
exit 1
fi

rm -rf "${TARGET_DIR:?}/"*
Last updated on by Chris Miller