Don't Repeat Yourself: Environment Variables in GitHub Actions and locally

As part of some Azure work I did recently (and which will inform some upcoming blog posts) I had to work with environment variables both in a local dev environment as well as in GitHub Actions across workflow steps. This post is about how I did this without duplicating variable management logic.

GitHub Actions and CI/CD

GitHub Actions (GHA) is GitHub's Continuous Integration and Deployment (CI/CD) platform. Like other CI/CD platforms, workflows execute as a sequence of jobs and steps. Your jobs execute in a runner - a Linux or Windows environment, specified by you in the workflow file. So far, so good; this will all be very familiar to any DevOps engineer or developer.

Environment Variables

Workflow steps can use environment variables to set resource names or properties, or for various other operations. GHA runners support environment variables, and there are several ways to set environment variables for use in a step.

Recently I have been doing some work which includes several shell scripts. As usual, I develop and debug these locally. I want to also run these scripts from GHA steps so I can re-use the same code and logic and avoid duplication (Don't Repeat Yourself).

Each script uses the same environment variables. During a work session, I initially run one script to set all my environment variables, used next by other scripts. I wrote a separate script to set the environment variables as some values come from Azure CLI calls, so I don't want to repeat this work (retrieve values from Azure and set variable value) repeatedly in many work scripts.

In bash, it's trivial to set an environment variable: export MY_VAR="myValue". This works in my local dev environment, and it works within the context of the GHA step. That is, in GHA an environment variable I set in a shell script is available later within that same script (and in the context of the GHA job step that called the script).

So far, I can use the same approach to set an environment variable in my local environment as on GHA. However: when I want to use the environment variable in later scripts (i.e. job steps on GHA), this doesn't work.

in GitHub Actions, later steps cannot access environment variables set inside scripts run by earlier steps. Reference: "you can use the value in subsequent steps of the same job by assigning the value to an existing or new environment variable and then writing this to the GITHUB_ENV environment file".

What does this mean? The script I wrote to set environment variables, which worked fine locally, will not work as is in a GitHub Actions workflow step.

In GHA, I have to write my variables to GITHUB_ENV so that later steps can retrieve the variable values. Outside GHA, this is not relevant and I'd like to avoid script errors.

So: I needed a way to do my "set environment variable" logic once, and be able to share this functionality across both local/wherever shell and GitHub environments.

How to do this? I need to detect whether I'm in a GitHub Actions workflow, and only if so, write to GITHUB_ENV so that later workflow steps can use the environment variables I set.

Where am I?

GitHub Actions sets several default environment variables in the runner environments. These GitHub Actions environment variables will not be present in my local shell. I can check for the existence of one of these and, if it's present, my script can assume it's running in a GitHub Actions workflow.

I decided to check for the GITHUB_ACTIONS environment variable. If it's present, my script is running in GitHub Actions. If the variable is not present... I'm not in GitHub Actions. This is how I only write to $GITHUB_ENV if I know I'm in GHA.

# Always write environment variable the shell way - works locally and in GHA for same script/step
export MY_VAR="myValue"

if [ ! -z $GITHUB_ACTIONS ]
then
     echo "MY_VAR='myValue'" >> $GITHUB_ENV
    (... other variables)
fi

Put it in a function and forget about it

Since I have a lot of environment variables to set, I'd like to make that as easy as possible. So I wrote a function that abstracts the above logic away.

setEnvVar() {
    # Set an env var's value at runtime with dynamic variable name
    # If in GitHub Actions runner, will export env var both to Actions and local shell
  # Usage:
  # setEnvVar "variableName" "variableValue"

  varName=$1
  varValue=$2

    if [ ! -z $GITHUB_ACTIONS ]
    then
        # We are in GitHub CI environment - export to GitHub Actions workflow context for availability in later steps in this workflow
        cmd=$(echo -e "echo \x22""$varName""=""$varValue""\x22 \x3E\x3E \x24GITHUB_ENV")
        eval $cmd
    fi

    # Export for local/immediate use, whether on GHA runner or shell/wherever
    cmd="export ""$varName""=\"""$varValue""\""
    eval $cmd
}

This script takes two arguments, like this:

setEnvVar "MY_VAR" "myValue"

Note: I have to pass my variable name in as a string! If I wrote something like setEnvVar $MY_VAR "myValue", then my script wouldn't get the name of the environment variable ($MY_VAR)... it would get the value held in that variable, which so far would likely be blank (after all, I'm trying to initialize the variable here... it doesn't exist or contain a value yet).

I have to pass the name of the environment variable, along with the value to set on that variable.

So my setEnvVar function has to take in a string representing the variable name, and use that to set an environment variable using that name.

How do we do that in bash? A little echo and eval magic. In the setEnvVar script above, note the line that reads

cmd=$(echo -e "echo \x22""$varName""=""$varValue""\x22 \x3E\x3E \x24GITHUB_ENV")

This is where I use the name of the variable ($varName) to prepare (not run, yet) a command that echoes the value ($varValue) into the variable whose name was passed in $varName. I also escape the various symbols. In plain(er) bash, the inside command being prepared is this:

echo "$varName"="$varValue" >> $GITHUB_ENV

We use echo -e to in turn echo that command outside, un-escaping the ASCII codes (e.g. \x24 becomes $), to set the $cmd variable. We then eval the $cmd, which actually runs it, which is how the $varName variable is actually set. So this is where the command is actually run, and the variable is exported to the GHA environment for use by later steps/scripts.

Finally, I can use this in my script that sets all the environment variables, including some I retrieve from Azure with CLI:

# (setEnvVar function defined above but omitted here for brevity)

setEnvVar "MY_VAR_1" "myValue1"
setEnvVar "MY_VAR_2" "my Value2"

# Azure Subscription ID
subscriptionId=$(echo "$(az account show -s $subscriptionName -o tsv --query 'id')" | sed "s/\r//")
setEnvVar "SUBSCRIPTION_ID" "$subscriptionId"

# Azure Tenant ID for Subscription. Need this to create User-Assigned Managed Identity and Key Vault.
tenantId=$(echo "$(az account show -s $subscriptionName -o tsv --query 'tenantId')" | sed "s/\r//")
setEnvVar "TENANT_ID" "$tenantId"

And indeed, these values will be available to my scripts locally, as well as in GHA to later workflow steps and scripts.

Let's say I have a simple script env-var-set.sh to set an environment variable. It runs in a GHA step.

# (setEnvVar function definition omitted for brevity)

# Set an environment variable normally
setEnvVar "MY_VAR" "Hello world"

# Verify it's a local environment variable now
echo $MY_VAR

Now I have a different script env-var-get.sh which runs in a separate GHA step, and gets the environment variable. Works locally and works in GHA.

echo $MY_VAR

Here's the GHA workflow YAML which first calls a script that sets the environment variable on one step, then calls the script the gets the environment variable value on a later step.

- name: Set Environment Variable
if: success()
run: |
  chmod +x ./scripts/env-var-set.sh
  ./scripts/env-var-set.sh

- name: Get Environment Variable
if: success()
run: |
  chmod +x ./scripts/env-var-get.sh
  ./scripts/env-var-get.sh

And the GHA run output:

-3s
Run chmod +x ./scripts/env-var-set.sh
  chmod +x ./scripts/env-var-set.sh
  ./scripts/env-var-set.sh
  shell: /usr/bin/bash -e {0}
Hello world

-3s
Run chmod +x ./scripts/env-var-get.sh
  chmod +x ./scripts/env-var-get.sh
  ./scripts/env-var-get.sh
  shell: /usr/bin/bash -e {0}
  env:
    MY_VAR: Hello world
Hello world

Wrapping Up

In summary: my script detects whether it's running in GHA or not. If it is, it takes an extra step to make sure the custom environment variables I set are available in the same step/script, as well as in later steps/scripts in the job.

I use one call to set environment variables: setEnvVar "MY_VAR" "myValue". This way I abstract away the environment detection logic and avoid duplication. Now back to the actual work this enabled! Happy bashing.