CI/CD for Embedded Firmware: Automated Build, Test, and OTA Release Pipeline
Embedded firmware teams have a well-worn habit of building firmware manually on a developer's laptop, copying the .bin to a shared drive, and telling field technicians to flash devices with a USB cable. This is not shipping software — it is artisanal firmware production, and it does not scale past a dozen devices.
A proper CI/CD pipeline for firmware changes everything: reproducible builds, automated testing, signed binaries, and controlled OTA rollouts that you can pause if a bad release is detected. This guide covers our exact implementation for ESP32 firmware using GitHub Actions and AWS IoT Jobs.
Why Firmware CI/CD Is Different
Software CI/CD is relatively mature. Firmware CI/CD has unique challenges:
The build environment problem: ESP-IDF, the official Espressif framework for ESP32, requires a specific toolchain version pinned to your SDK version. One engineer's local toolchain produces a different binary hash than another's, making reproducibility impossible without containerisation.
No easy unit testing: You cannot run firmware unit tests on x86 without an abstraction layer (HAL mocking). Most teams skip tests entirely — which means bugs reach production hardware.
Deployment is destructive: A bad software deploy is rolled back in 30 seconds. A bad firmware update can brick a device, requiring physical intervention to recover. Staged rollouts are not optional.
Pipeline Architecture
Git Push / PR
↓
GitHub Actions trigger
↓
ESP-IDF Docker build (reproducible binary)
↓
Unity unit tests (mocked HAL layer)
↓
Firmware signing (private key in GitHub Secrets)
↓
S3 upload (versioned binary + manifest)
↓
AWS IoT Jobs: staged rollout (5% → 25% → 100%)
↓
CloudWatch metrics: success/failure rate
↓
Slack notification: release summary
Step 1: Reproducible Build with ESP-IDF Docker
Espressif publishes official Docker images for each ESP-IDF version. Pinning to a specific image tag guarantees byte-identical builds across all environments.
# .github/workflows/firmware-build.yml
name: Firmware CI/CDon:
push:
branches: [main, 'release/**']
pull_request:
branches: [main]
env:
FIRMWARE_VERSION: ${{ github.run_number }}
S3_BUCKET: your-company-firmware-releases
AWS_REGION: us-east-1
jobs:
build-and-test:
runs-on: ubuntu-latest
container:
image: espressif/idf:v5.1.2 # Pin exact version
options: --user root
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Configure project
run: |
. /opt/esp/idf/export.sh
idf.py set-target esp32s3
echo "CONFIG_FIRMWARE_VERSION="${FIRMWARE_VERSION}"" >> sdkconfig.defaults
- name: Build firmware
run: |
. /opt/esp/idf/export.sh
idf.py build
working-directory: firmware/
- name: Run unit tests
run: |
cd firmware/test
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build
./build/firmware_tests --output-on-failure
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: firmware-binaries
path: |
firmware/build/*.bin
firmware/build/*.elf
firmware/build/bootloader/bootloader.bin
firmware/build/partition_table/partition-table.bin
Step 2: Firmware Signing
Never publish unsigned firmware. A device that accepts any binary over OTA is a device that an attacker can own with a crafted update.
sign-and-upload:
needs: build-and-test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: firmware-binaries
path: ./build
- name: Sign firmware
run: |
# Sign using the ESP-IDF secure boot signing tool
python3 -m espsecure sign_data --version 2 --keyfile <(echo "${{ secrets.FIRMWARE_SIGNING_KEY }}") --output ./build/firmware-signed.bin ./build/firmware.bin
# Generate SHA-256 manifest
SHA256=$(sha256sum ./build/firmware-signed.bin | awk '{print $1}')
SIZE=$(stat -c%s ./build/firmware-signed.bin)
cat > manifest.json << EOF
{
"version": "${FIRMWARE_VERSION}",
"sha256": "${SHA256}",
"size": ${SIZE},
"url": "https://${S3_BUCKET}.s3.${AWS_REGION}.amazonaws.com/releases/${FIRMWARE_VERSION}/firmware-signed.bin",
"releaseNotes": "${{ github.event.head_commit.message }}"
}
EOF
- name: Upload to S3
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}
- name: S3 sync
run: |
aws s3 cp ./build/firmware-signed.bin s3://${S3_BUCKET}/releases/${FIRMWARE_VERSION}/firmware-signed.bin --sse AES256
aws s3 cp manifest.json s3://${S3_BUCKET}/releases/${FIRMWARE_VERSION}/manifest.json --sse AES256
# Update the "latest" pointer
aws s3 cp manifest.json s3://${S3_BUCKET}/latest/manifest.json --sse AES256
Step 3: Staged Rollout via AWS IoT Jobs
AWS IoT Jobs provides built-in staged rollout with abort criteria. Configure it to push to 5% of your fleet, wait for success confirmation, then proceed.
- name: Create IoT OTA Job
run: |
aws iot create-job --job-id "firmware-release-${FIRMWARE_VERSION}" --targets "arn:aws:iot:${AWS_REGION}:ACCOUNT_ID:thinggroup/production-devices" --document-source "s3://${S3_BUCKET}/releases/${FIRMWARE_VERSION}/manifest.json" --job-executions-rollout-config '{
"exponentialRate": {
"baseRatePerMinute": 5,
"incrementFactor": 2,
"rateIncreaseCriteria": {
"numberOfSucceededThings": 10
}
},
"maximumPerMinute": 50
}' --abort-config '{
"criteriaList": [{
"failureType": "FAILED",
"action": "CANCEL",
"thresholdPercentage": 10,
"minNumberOfExecutedThings": 20
}]
}' --timeout-config '{"inProgressTimeoutInMinutes": 30}' - name: Notify Slack
uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
text: |
Firmware v${{ env.FIRMWARE_VERSION }} release started.
Staged rollout: 5% → 25% → 100% over 2 hours.
Monitor: https://console.aws.amazon.com/iot/home#/jobs
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
Unit Testing Without Hardware
Mocking the HAL (Hardware Abstraction Layer) lets you test business logic on x86. Structure your firmware to separate hardware drivers from application logic:
// firmware/src/telemetry_processor.c — pure logic, no hardware calls
#include "telemetry_processor.h"
#include "hal_interface.h" // Abstract interface, mocked in testsTelemetryResult process_reading(SensorReading *raw) {
TelemetryResult result = {0};
if (raw->temperature < -40.0f || raw->temperature > 125.0f) {
result.valid = false;
result.error_code = ERR_TEMP_OUT_OF_RANGE;
return result;
}
result.temperature_celsius = raw->temperature;
result.humidity_percent = raw->humidity;
result.valid = true;
return result;
}
The pipeline we described has caught 14 regressions before they reached hardware across our last 8 ESP32 projects. The 2-hour investment to set it up has saved an estimated 40+ hours of field debugging.
---
Want a production-grade firmware CI/CD pipeline for your ESP32 or STM32 project? [Contact Code Caracal](/contact) — we set up the full build and OTA infrastructure as part of every firmware engagement.