Using OIDC to secure your CI/CD Pipelines - No more long lived access keys!

Long-lived, static access keys are a commonality in many companies cloud infrastructure these days. Access-keys are often not rotated which poses a security risk, especially if the access keys are used for deploying applications or any utility functions (e.g building AMIs / Golden images, deploying resources with Terraform)

When authenticating to AWS the best practice is to use a role with short-term access keys / session tokens - Thanks to AWS IAM OIDC Providers, we can now remove the need for access keys in CI/CD making our pipelines and infrastructure more secure and less fragile as we don’t need to worry about access key secrets being accidentally updated, or deleted.

Here's some of the primary benefits of using this method of authentication for your workflows:

• No access keys are required, increasing security
• Far more granular control of how the credentials are used in comparison to access keys - you can limit the role to only be able to be used on a single branch, in a single repository as an example
• No need to rotate credentials as the credentials provided by your cloud provider are short-lived

#Table of Contents

• Introduction
  • How does it actually work?
    • GitHub to AWS
    • GitHub to Azure
• Implementation
  • Azure Setup
    • Provider Setup
    • Variables Setup
    • Configuring the OIDC Provider
  • AWS Setup
    • Provider Setup
    • Variables Setup
    • Configuring the OIDC Provider
• Configuring the GitHub Actions Workflow
  • Required Secrets
  • Workflow Configuration
  • Authentication Examples
• Next Steps

#Introduction

This blog post covers how we can remove the need for access keys by using OIDC (OpenID Connect) to authenticate to AWS or Azure, as well as creating the required resources via Terraform.

Note: While this blog post is specifically catered to GitHub Actions, but you can use the same principals to do the same for most CI/CD providers, such as:

• CircleCI - https://circleci.com/docs/openid-connect-tokens/
• BitBucket - https://support.atlassian.com/bitbucket-cloud/docs/deploy-on-aws-using-bitbucket-pipelines-openid-connect/
• GitLab - https://docs.gitlab.com/ee/ci/cloud_services/aws/
• You probably could also use it for Jenkins, but I think you’d need to host your own OIDC provider.

In the example (and tutorial) below, I’m using GitHub Actions, Terraform, Packer and Ansible to create a Golden Image which I’ll use as a base image for Kubernetes nodes (This won't be covered - but you can read This blog post about it on AWS here ) but this concept can be applied to many different things such as application deployment, building application assets and pushing them to S3, etc. Literally anything you do in GitHub Actions that uses your Azure/AWS credentials will heavily benefit from this.

However, I'll only be covering the following:

• Setting up the required resources on AWS And Azure to so GitHub Actions can authenticate via OIDc
• Granting the required permissions for my use case, as an example.

We'll also be creating a Terraform module to create the resources, so it can be re-used by you/your company in the future - as well as configuring GitHub Actions be able to access AWS/Azure via OIDC.

#How does it actually work?

This diagram below from GitHub's documentation explains it quite well.

And to further quote GitHub's documentation, :

1. In your cloud provider, create an OIDC trust between your cloud role and your GitHub workflow(s) that need access to the cloud. I'll be going into this with more detail for both AWS and Azure below.
2. Every time your job runs, GitHub's OIDC Provider auto-generates an OIDC token. This token contains multiple claims to establish a security-hardened and verifiable identity about the specific workflow that is trying to authenticate.
3. You could include a step or action in your job to request this token from GitHub's OIDC provider, and present it to the cloud provider.
4. Once the cloud provider successfully validates the claims presented in the token, it then provides a short-lived cloud access token that is available only for the duration of the job.

#GitHub to AWS

An IAM Identity provider is used to create a trust between GitHub's OIDC provider and our AWS Account. Then, the role that's being used in the workflow will have conditions in it's trust policy to lock it down to specific GitHub repositories. See this example IAM Role policy form GitHub's documentation below:

oidc-role-trust-policy.json
1{
2    "Version": "2012-10-17",
3    "Statement": [
4        {
5            "Effect": "Allow",
6            "Principal": {
7                "Federated": "arn:aws:iam::123456123456:oidc-provider/token.actions.githubusercontent.com"
8            },
9            "Action": "sts:AssumeRoleWithWebIdentity",
10            "Condition": {
11                "StringLike": {
12                    "token.actions.githubusercontent.com:sub": "repo:octo-org/octo-repo:*"
13                },
14                "StringEquals": {
15                    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
16                }
17            }
18        }
19    ]
20}

Here's a simplified diagram of how it works:

Note: In this diagram sp is "Service Principal" - the equivalent of an IAM role on Azure.

#GitHub to Azure

While the technologies are still the same, it's a bit different when using Azure.

Firstly, an Azure AD Application is created as well as a Service Principal. This Service principal is the Principal (or IAM role, if you're familiar with AWS) and it is what permissions will be assigned to.

Secondly, a set of Federated Identity Credentials is created for the Azure AD Application. While these credentials are long-lived and are would usually be considered a secret, it's not an issue in this case as the 'client secret' isn't required to authenticate to github actions from our repository, only the Client ID, Tenant ID and Subscription ID.

This diagram from Azure's documentation displays the process very well:

In this case, the external workload is GitHub Actions and the external IdP is Github's OIDC Provider.

#Implementation

Since we're using Terraform we're going to create a module so it can be reused by you or your company.

Since I'm using this pipeline to build VM Images with Packer, there will be a few things that are specific to my use case (such as creating a secret that's used in the pipeline, permissions, etc) but feel free to modify this to fit your use case.

#Azure

Provider setup

create the following in providers.tf:

1terraform {
2  required_providers {
3    azurerm = {
4      source  = "hashicorp/azurerm"
5      version = "3.48.0"
6    }
7    azuread = {
8      source = "hashicorp/azuread"
9      version = "2.36.0"
10    }
11  }
12  required_version = ">=1.4.0"
13}
14
15provider "azurerm" {
16  features {}
17}

Variables Setup

Before we look at creating any resources, let's setup some variables so this can be re-used in the future!

Create the following in variables.tf:

1variable "gh_username" {
2  type        = string
3  description = "GitHub Username, used to specify which repo/org for GH Actions permissions."
4}
5
6variable "repo" {
7  type        = string
8  description = "Name of GitHub Repository that GitHub Actions role will use."
9}
10
11variable "az_rg_name" {
12  type        = string
13  description = "Name of Azure Resource Group"
14  default     = "golden-image-build"
15}
16
17variable "az_region" {
18  type        = string
19  description = "Name of Azure region"
20  default     = "Australia Southeast"
21}

Configuring the OIDC Provider

In this section, we'll create the following:

• Azure Resource group to keep everything clean and organized in one 'solution'
• AAD (Azure AD) Application Federated Identity Credential
• AAD Service Principal (Which is used by GitHub Actions)

We'll also create these resources, which are OPTIONAL and only required if you need to access some form of secret in your pipeline such as an ansible vault password.

• Key vault (to store secret used in to encrypt ansible-vault secrets)
• Key vault secrets themselves
• AAD groups to grant permissions on who can access the secret
• Access policies granting permissions to those who are in those groups

First, create a resource group (to contain the resources)

1resource "azurerm_resource_group" "github_oidc" {
2  name     = var.az_rg_name
3  location = var.az_region
4}

Then create an Azure AD Application, Service Principal and Federated Identity Credential. In Azure AD, a service principal is the equivalent of an IAM role - and the Azure AD Application is equivalent to an IAM identity provider.

Here's a brief overview of how those resources tie in together:

• Permissions are assigned to the Service Principal
• The Service Principal is the identity that the Azure AD Application uses to access resources
• The Federated Identity credential is what actually allows GitHub Actions to authenticate to Azure

1resource "azuread_application" "github_oidc" {
2  display_name = "${var.repo}-gh-actions"
3
4  api {
5    requested_access_token_version = 2
6  }
7
8}
9
10resource "azuread_service_principal" "github_oidc" {
11  application_id = azuread_application.github_oidc.application_id
12}
13
14// Actual OpenID Connect connection
15
16resource "azuread_application_federated_identity_credential" "github_oidc" {
17  application_object_id = azuread_application.github_oidc.object_id
18  display_name          = "${var.repo}-gh-actions"
19  description           = "Deployments for ${var.gh_username}/${var.repo} for Production Environment"
20  audiences             = ["api://AzureADTokenExchange"]
21  issuer                = "https://token.actions.githubusercontent.com"
22  // NOTE: WILDCARDS IN SUBJECT DON'T WORK
23  subject               = "repo:${var.gh_username}/${var.repo}:ref:refs/heads/main"
24}

Note: I’ve chosen to only allow jobs on the ‘main’ branch to be able to authenticate as that’s the only branch I want this job to run on. If desired, you configure your github actions jobs to run in a specific environment, then grant the federated identity permissions to run on ALL jobs in that environment, eg:

1resource "azuread_application_federated_identity_credential" "github_oidc" {
2  application_object_id = azuread_application.github_oidc.object_id
3  display_name          = "${var.repo}-gh-actions"
4  description           = "Deployments for ${var.gh_username}/${var.repo} for Production Environment"
5  audiences             = ["api://AzureADTokenExchange"]
6  issuer                = "https://token.actions.githubusercontent.com"
7  subject               = "repo:${var.gh_username}/${var.repo}:environment:your-environment"
8}

Now, let's assign the service principal permissions:

1data "azurerm_subscription" "primary" {}
2
3resource "azurerm_role_assignment" "github_oidc" {
4  scope                = data.azurerm_subscription.primary.id
5  role_definition_name = "Contributor"
6  principal_id         = azuread_service_principal.github_oidc.id
7}

If you're only needing to setup the connection and permissions between Azure and GitHub actions, this is all that's required - you can deploy this with terraform and you'll be good to go - you'll only need to modify your workflow as described in the section Configuring the GitHub Actions workflow

However, if you're going to use Azure Key vault to store ansible vault passwords then keep following along.

First, create a key vault and secret - this is used to store an Ansible vault Password

1resource "azurerm_key_vault" "ansible_vault" {
2  name                       = "${var.repo}-image-build"
3  location                   = azurerm_resource_group.github_oidc.location
4  resource_group_name        = azurerm_resource_group.github_oidc.name
5  tenant_id                  = data.azurerm_client_config.current.tenant_id
6  sku_name                   = "standard"
7  soft_delete_retention_days = 7
8
9  network_acls {
10    default_action = "Allow"
11    bypass         = "AzureServices"
12  }
13}

Add the following to variables.tf:

1variable "upn" {
2  type = string
3  description = "User principal name"
4}

We need this value to add ourselves (in Azure AD) to a group that we're creating for granting access to the Key Vault. To get the user principal name, login to the Azure Console - then search for Users, and you'll see a list of each user and their User Principal Name.

Now, create a group and add your Azure user to it - this is primarily so we can still manage the resource with Terraform.

1resource "azuread_group" "kv-full" {
2  display_name  = "${local.kv_name}-full"
3  security_enabled = true
4}
5
6resource "azuread_group_member" "me" {
7  group_object_id = azuread_group.kv-full.id
8  member_object_id = data.azuread_user.me.object_id
9}
10
11data "azuread_user" "me" {
12  user_principal_name = var.upn
13}
14

Now, let's create some permissions on the key vault.

1resource "azuread_group" "kv-read" {
2  display_name  = "${local.kv_name}-read"
3  security_enabled = true
4}
5
6// Add GH Actions SP to READ access grp
7resource "azuread_group_member" "sp_read" {
8  group_object_id = azuread_group.kv-read.id
9  member_object_id = azuread_service_principal.github_oidc.object_id
10}
11
12
13################################################
14# AZURE KEY VAULT - STORE ANSIBLE VAULT SECRET #
15################################################
16
17resource "azurerm_key_vault_access_policy" "read" {
18  key_vault_id = azurerm_key_vault.ansible_vault.id
19  tenant_id = data.azurerm_client_config.current.tenant_id
20  object_id = azuread_group.kv-read.object_id
21
22  lifecycle {
23    create_before_destroy = true
24  }
25
26  key_permissions = [
27    "Get",
28  ]
29
30  secret_permissions = [
31    "Get",
32  ]
33}
34
35resource "azurerm_key_vault_access_policy" "full" {
36  tenant_id = data.azurerm_client_config.current.tenant_id
37  object_id = data.azurerm_client_config.current.object_id
38  key_vault_id = azurerm_key_vault.ansible_vault.id
39
40  lifecycle {
41    create_before_destroy = true
42  }
43
44  key_permissions = [
45    "Create",
46    "Get",
47  ]
48  secret_permissions = [
49    "Set",
50    "List",
51    "Get",
52    "Delete",
53    "Purge",
54    "Recover"
55  ]
56}
57
58resource "azurerm_key_vault_access_policy" "full_grp" {
59  tenant_id = data.azurerm_client_config.current.tenant_id
60  object_id = azuread_group.kv-full.object_id
61  key_vault_id = azurerm_key_vault.ansible_vault.id
62
63  lifecycle {
64    create_before_destroy = true
65  }
66
67  key_permissions = [
68    "Create",
69    "Get",
70  ]
71  secret_permissions = [
72    "Set",
73    "Get",
74    "Delete",
75    "Purge",
76    "Recover"
77  ]
78}
79
80resource "azurerm_key_vault_secret" "ansible_vault_pass" {
81  name         = "ansible-vault-pass"
82  value        = var.vault_pass_secret_value
83  key_vault_id = azurerm_key_vault.ansible_vault.id
84
85  depends_on = [
86    azurerm_key_vault_access_policy.full,
87    azurerm_key_vault_access_policy.full_grp,
88    azurerm_key_vault_access_policy.read
89  ]
90}

Now that that's all done, run the following to create the resources:

terraform plan
terraform apply

#AWS

Providers setup

Create the following in providers.tf:

1terraform {
2  required_providers {
3    aws = {
4      source = "hashicorp/aws"
5      version = "4.59.0"
6    }
7  }
8
9  required_version = ">=1.4.0"
10}

Variables Setup

Create the following in variables.tf:

1variable "repo" {
2  type        = string
3  description = "Name of GitHub Repository that GitHub Actions role will use."
4}
5
6variable "gh_username" {
7  type        = string
8  description = "Name of GitHub user that owns the repository that gh actions will run on"
9}
10
11variable "gh_role_name" {
12  type        = string
13  description = "IAM Role name for GitHub Actions"
14}
15
16variable "tags" {
17  default = {}
18  type    = map(string)
19}

Configuring the OIDC Provider

Since initially i created this module for setting up the required permissions for an image-build workflow using Packer (with AWS) on GitHub Actions, i opted to use an existing module for the OIDC side of things.

See their documentation here

First, let's create the IAM OpenID Connect Identity Provider. You can omit this if you already have one created in your account as there's a limit of 1 per AWS account. Add the following to main.tf

1resource "aws_iam_openid_connect_provider" "github_actions" {
2  url             = "https://token.actions.githubusercontent.com"
3  client_id_list  = ["sts.amazonaws.com"]
4  // GitHub's OIDC Thumbprint
5  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
6}
7
8Then, create the role and required permissions.
9```tf
10module "github_oidc" {
11
12  source                      = "philips-labs/github-oidc/aws"
13  version                     = "0.6.0"
14  role_name                   = var.gh_role_name
15  repo                        = "${var.gh_username}/${var.repo}"
16  openid_connect_provider_arn = aws_iam_openid_connect_provider.github_actions.arn
17  role_policy_arns            = [module.gh_role_policy_packer.arn,module.gh_role_policy_secretsmanager.arn]
18  default_conditions          = ["allow_main"]
19  conditions = [{
20    test     = "StringLike"
21    variable = "token.actions.githubusercontent.com:sub"
22    values   = ["repo:${var.gh_username}/${var.repo}:pull_request"]
23  }]
24}

If you're not setting up an image build pipeline, then change the role_policy_arns parameter above to your desired IAM policies (so the IAM role you created can access resources) and you'll be off to the races once you deploy.

Otherwise, add the following to main.tf (or use the below as an example), which will grant our workflow permissions to do the following once deployed:

• Build AMIs with Packer
• Access a specific Secrets Manager secret
• Update an SSM parameter (which is used to get the latest AMI)

1
2data "aws_iam_policy_document" "gh_role_permissions_packer" {
3  statement {
4    sid = "RatherSafeActions"
5    actions = [
6      "ec2:CopyImage",
7      "ec2:CreateImage",
8      "ec2:CreateSnapshot",
9      "ec2:CreateTags",
10      "ec2:CreateVolume",
11      "ec2:DescribeImages",
12      "ec2:DescribeImageAttribute",
13      "ec2:DescribeInstanceStatus",
14      "ec2:DescribeInstances",
15      "ec2:DescribeRegions",
16      "ec2:DescribeSecurityGroups",
17      "ec2:DescribeSnapshots",
18      "ec2:DescribeSubnets",
19      "ec2:DescribeVolumes",
20      "ec2:DescribeTags",
21      "ec2:RegisterImage",
22      "ec2:RunInstances",
23      "ec2:GetPasswordData",
24      "ec2:CreateKeyPair",
25      "ec2:DeleteKeyPair",
26      "ec2:CreateSecurityGroup",
27      "ec2:AuthorizeSecurityGroupIngress",
28      "ec2:DeleteSecurityGroup"
29    ]
30    resources = ["*"]
31  }
32  statement {
33    sid = "DangerousActions"
34    actions = [
35      "ec2:AttachVolume",
36      "ec2:DeleteSnapshot",
37      "ec2:DeleteVolume",
38      "ec2:DeregisterImage",
39      "ec2:DetachVolume",
40      "ec2:ModifyImageAttribute",
41      "ec2:ModifyInstanceAttribute",
42      "ec2:ModifySnapshotAttribute",
43      "ec2:StopInstances",
44      "ec2:TerminateInstances"
45    ]
46    resources = ["*"]
47    condition {
48      test     = "StringEquals"
49      variable = "ec2:ResourceTag/Creator"
50      values   = ["Packer"]
51    }
52  }
53}
54
55resource "aws_secretsmanager_secret" "ansible_vault_pass" {
56  name = "${var.repo}-ansible-vault-pass"
57}
58
59data "aws_iam_policy_document" "gh_role_permissions_secretsmanager" {
60  statement {
61    actions = [
62      "secretsmanager:GetResourcePolicy",
63        "secretsmanager:GetSecretValue",
64        "secretsmanager:DescribeSecret",
65        "secretsmanager:ListSecretVersionIds"
66      ]
67      resources = [
68        aws_secretsmanager_secret.ansible_vault_pass.arn,
69        "${aws_secretsmanager_secret.ansible_vault_pass.arn}/*",
70        "${aws_secretsmanager_secret.ansible_vault_pass.arn}*",
71      ]
72  }
73  statement {
74    actions   = ["secretsmanager:ListSecrets"]
75    resources = ["*"]
76  }
77  statement {
78    actions = [
79      "secretsmanager:GetResourcePolicy",
80      "secretsmanager:GetSecretValue",
81      "secretsmanager:DescribeSecret",
82      "secretsmanager:ListSecretVersionIds",
83      "secretsManager:PutSecretValue",
84      "secretsManager:CreateSecret",
85      "secretsManager:UpdateSecret"
86    ]
87    resources = [
88      aws_secretsmanager_secret.ansible_vault_pass.arn,
89      "${aws_secretsmanager_secret.ansible_vault_pass.arn}/*",
90      "${aws_secretsmanager_secret.ansible_vault_pass.arn}*"
91    ]
92  }
93  statement {
94    actions = [
95      "ssm:DescribeParameters",
96      "ssm:PutParameter",
97      "ssm:GetParameters",
98      "ssm:GetParameter",
99      "ssm:GetParametersByPath",
100      "ssm:GetParameterHistory",
101    ]
102    resources = [
103      aws_ssm_parameter.golden_image_id.arn
104    ]
105  }
106}
107
108resource "aws_ssm_parameter" "golden_image_id" {
109  name  = "golden-image-id"
110  type  = "String"
111  value = "changeme"
112}
113
114
115// IAM policy that grants permissions to access SecretsManager
116module "gh_role_policy_secretsmanager" {
117  source  = "terraform-aws-modules/iam/aws//modules/iam-policy"
118  version = "~> 3.0"
119
120  name        = "${var.gh_role_name}-secretsmanager"
121  path        = "/"
122  description = "Grants permissions to GitHub OIDC Role (tf-aws repo) to read secretsmanager secret for img build pipeline"
123  policy      = data.aws_iam_policy_document.gh_role_permissions_secretsmanager.json
124}
125
126// Policy to grant permissions required for building AMIs
127module "gh_role_policy_packer" {
128  source  = "terraform-aws-modules/iam/aws//modules/iam-policy"
129  version = "~> 3.0"
130
131  name        = "${var.gh_role_name}-packer"
132  path        = "/"
133  description = "Grants permissions for GitHub OIDC Role (tf-aws repo) required permissions for packer role"
134  policy      = data.aws_iam_policy_document.gh_role_permissions_packer.json
135}

#Configuring the GitHub Actions workflow

#Secrets

Depending on which cloud provider you're using, the secrets you need to create are quite different.

For AWS, you only need to create a secret called AWS_ROLE_ARN in your GitHub repository settings. The value of this should be the ARN of the role that you created.

You can get the value of this by running the following AWS CLI command, it should output something similar to below:

1aws iam get-role --role-name YOUR_ROLE_NAME_HERE --query 'Role.Arn'
2# in this case, mine is called 'tf-aws-github-actions'.
3"arn:aws:iam::12345678901:role/github-actions/tf-aws-gh-actions"

If you're running on Azure, you'll need to create the following secrets:

• AZURE_CLIENT_ID
• AZURE_SUBSCRIPTION_ID
• AZURE_TENANT_ID

1# get AZURE_CLIENT_ID
2
3# first, get the ID of the service principal - REPLACE tf-aws WITH YOUR REPAL NAME HERE
4REPO_NAME="tf-aws"
5SP_DISPLAY_NAME="${REPO_NAME}-gh-actions"
6
7az ad app list --display-name $SP_DISPLAY_NAME --query "[*].appId" -o tsv
8
9# get AZURE_SUBSCRIPTION_ID
10az account show | jq  -r '.id'
11# get AZURE_TENANT_ID
12az account tenant list

You can use the GitHub CLI, GitHub's web app or GitHub's terraform provider to create the secrets described above:

See the corresponding documentation below:

• https://docs.github.com/en/actions/security-guides/encrypted-secrets
• https://cli.github.com/manual/gh_secret_set
• https://registry.terraform.io/providers/integrations/github/latest/docs

#Workflow Configuration

As I mentioned earlier, the configuration in the GitHub Actions workflow isn't that different to if you were using Access Keys.

However, there is one significant difference - the permissions directive. If you don't add this, it won't work as the workflow won't have permissions to request the OIDC JWT token from GitHub's own OIDC server.

1permissions:
2  id-token: write
3  contents: read

Here's a breakdown of the permission fields above:

• The id-token: write field is required for requesting the JWT/OIDC token itself
• And the contents: read is required for passing it between multiple jobs in the workflow - if you've only got one job that'd use it, then you probably could omit this field.

Authentication / Example Workflows

There's not a whole lot of differences in configuration regarding the cloud provider login Action configurations (az/login, aws-actions/configure-aws-credentials)

For AWS, It's pretty simple - you just don't pass any credentials in:

1
2---
3name: Build VM Images
4on:
5  push:
6    branches:
7      - main
8  pull_request:
9    branches:
10      - main
11permissions:
12  id-token: write
13  contents: read
14
15jobs:
16  build-aws-ami:
17    runs-on: ubuntu-latest
18    container:
19      image: joelfreeman/ansible-packer-boto3:latest
20    steps:
21      - name: Check out repository code
22        uses: actions/checkout@v2
23
24      - name: Configure AWS Credentials
25        uses: aws-actions/configure-aws-credentials@v2
26        with:
27          role-to-assume: arn:aws:iam::012345678910:role/github-actions/tf-aws-gh-actions
28          role-duration-seconds: 2100
29          aws-region: us-east-1
30      # and now you've got credentials!

And for azure, you need to pass in the GitHub actions secrets we created above:

1name: Build VM Images
2on:
3  push:
4    branches:
5      - main
6  pull_request:
7    branches:
8      - main
9permissions:
10  id-token: write
11  contents: read
12
13jobs:
14  build-azure-image:
15    runs-on: ubuntu-latest
16    container:
17      image: joelfreeman/ansible-packer-boto3:latest
18    steps:
19      - name: Check out repository code
20        uses: actions/checkout@v2
21
22      - name: 'Az CLI login'
23        uses: azure/login@v1
24        with:
25          client-id: ${{ secrets.AZURE_CLIENT_ID }}
26          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
27          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Then you're good to go!

If you're curious to see what my full workflow for building VM images is like, take a look below:

1---
2name: Build VM Images
3on:
4  push:
5    branches:
6      - main
7  pull_request:
8    branches:
9      - main
10permissions:
11  id-token: write
12  contents: read
13
14jobs:
15  #TODO: ADD MOLECULE TESTING FOR ANSIBLE
16  ansible-lint:
17    runs-on: ubuntu-latest
18    container:
19      image: joelfreeman/ansible-packer-boto3:latest
20    steps:
21      - name: Check out repository code
22        uses: actions/checkout@v2
23
24      - name: Fix git permissions
25        run: git config --global --add safe.directory /__w/tf-aws/tf-aws
26      - name: Force ansible to use dummy vault
27        run: |
28          cp ./ansible/golden-image/vault.yaml ./ansible/golden-image/vault.old.yaml
29          mv ./ansible/golden-image/dummy_vault.yaml ./ansible/golden-image/vault.yaml
30      - name: Check Ansible Playbook Syntax
31        env:
32          ANSIBLE_VAULT_PASSWORD_FILE: ./ansible/golden-image/dummy-vault-pass
33        run: |
34          ansible-playbook \
35            --syntax-check \
36            --vault-password-file ./ansible/golden-image/dummy-vault-pass \
37            ./ansible/golden-image/base.yml
38          ansible-playbook \
39            --syntax-check \
40            --vault-password-file ./ansible/golden-image/dummy-vault-pass \
41            ./ansible/golden-image/base_azure.yml
42
43      - name: Run ansible-lint
44        env:
45          ANSIBLE_VAULT_PASSWORD_FILE: ./ansible/golden-image/dummy-vault-pass
46        run: ansible-lint ./ansible/
47
48  build-aws-ami:
49    runs-on: ubuntu-latest
50    container:
51      image: joelfreeman/ansible-packer-boto3:latest
52    steps:
53      - name: Check out repository code
54        uses: actions/checkout@v2
55
56      - name: Configure AWS Credentials
57        uses: aws-actions/configure-aws-credentials@v2
58        with:
59          role-to-assume: arn:aws:iam::012345678910:role/github-actions/tf-aws-gh-actions
60          role-duration-seconds: 2100
61          aws-region: us-east-1
62
63      - name: Setup `packer`
64        uses: hashicorp/setup-packer@main
65        id: setup
66        with:
67          version: "1.8.3" # or `latest`
68
69      - name: Fix git error
70        run: git config --global --add safe.directory /__w/tf-aws/tf-aws
71
72      - name: Validate Packerfile
73        env:
74          VAULT_AWS_SECRET_NAME: ${{ secrets.VAULT_AWS_SECRET_NAME }}
75        run: |
76          packer validate \
77            -var "subnet_id=subnet-0b72ba8262022bc88" \
78            -var "vault_pw_file_path=./bin/get_vault_pw.py" \
79            -var "vault_path=./ansible/golden-image/vault.yaml" \
80            ./components/golden-img/alma.pkr.hcl
81
82      - name: Build base AWS AMI
83        env:
84          VAULT_AWS_SECRET_NAME: ${{ secrets.VAULT_AWS_SECRET_NAME }}
85        run: |
86          packer build \
87            -var "subnet_id=subnet-0b72ba8262022bc88" \
88            -var "vault_pw_file_path=./bin/get_vault_pw.py" \
89            -var "vault_path=./ansible/golden-image/vault.yaml" \
90            -timestamp-ui \
91            ./components/golden-img/alma.pkr.hcl
92
93          echo "AMI_ID=$(jq -r '.builds[-1].artifact_id' packer-manifest.json | cut -d ":" -f2)" >> $GITHUB_ENV
94        shell: bash
95      - name: Set SSM Parameter
96        run: |
97          echo "Setting SSM Parameter golden-image-id to $AMI_ID"
98          aws ssm put-parameter --name "golden-image-id" --value $AMI_ID --overwrite
99
100  build-azure-image:
101    runs-on: ubuntu-latest
102    container:
103      image: joelfreeman/ansible-packer-boto3:latest
104    steps:
105      - name: Check out repository code
106        uses: actions/checkout@v2
107
108      - name: 'Az CLI login'
109        uses: azure/login@v1
110        with:
111          client-id: ${{ secrets.AZURE_CLIENT_ID }}
112          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
113          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
114
115      - name: Setup `packer`
116        uses: hashicorp/setup-packer@main
117        id: setup
118        with:
119          version: "1.8.3"  # or `latest`
120
121      - name: Fix git error
122        run: git config --global --add safe.directory /__w/tf-aws/tf-aws
123
124      - name: Validate Packerfile
125        env:
126          VAULT_AZURE_SECRET_NAME: ${{ secrets.VAULT_AZURE_SECRET_NAME }}
127          VAULT_AZURE_VAULT_NAME: ${{ secrets.VAULT_AZURE_VAULT_NAME }}
128        run: |
129          packer validate \
130            -var "vault_pw_file_path=./bin/get_vault_pw_azure.sh" \
131            -var "vault_path=./ansible/golden-image/vault.yaml" \
132            ./components/golden-img/alma-azure.pkr.hcl
133
134      - name: Build base image on azure
135        env:
136          VAULT_AZURE_SECRET_NAME: ${{ secrets.VAULT_AZURE_SECRET_NAME }}
137          VAULT_AZURE_VAULT_NAME: ${{ secrets.VAULT_AZURE_VAULT_NAME }}
138        run: |
139          packer build \
140            -var "vault_pw_file_path=./bin/get_vault_pw_azure.sh" \
141            -var "vault_path=./ansible/golden-image/vault.yaml" \
142            -timestamp-ui \
143            ./components/golden-img/alma-azure.pkr.hcl

#Next Steps

This concept can be applied to most CI/CD providers, and most large services that access cloud resources - such as terraform cloud.

If you're reading this post then you're most likely interested in security - here's some great open source resources/tooling to look at including in your pipelines:

• tfsec - Terraform Static Analysis tool with a focus on security
• chekov - Static Analyis (Security scanning), Policy as code tool
• docker scan - scan docker images for application and package vulnerabilities
• trivy - find vulnerabilities, misconfigurations, secrets in your code, container images, kubernetes manifests and more

Lastly, thank you for reading - If you liked this post, feel free to connect with me on LinkedIn

If you have any questions or inquiries, Message me on LinkedIn above - or send me an email, contact@jxel.dev