CI/CD Secrets, Pipeline Security and Making Pipelines Fast and Reliable

A CI server is one of the most privileged machines in a company. It can read all your source, it holds credentials that can deploy to production, and it executes commands written by whoever can open a pull request. Attackers know this. Slow pipelines are a quieter problem with a human cost: developers stop trusting the signal. This lesson covers both halves, trust and speed.

Secrets: where they live and how they leak

Secrets (API keys, deploy tokens, signing keys) belong in your CI platform's encrypted secret store, referenced by name, never in the repository, never in a Dockerfile ARG, and never in a shared config file. In GitHub Actions you reference one as secrets.NAME. The platform masks exact matches of secret values in logs, but masking is a courtesy, not protection: a value that is transformed (base64-encoded, split, URL-encoded) is no longer matched and may print in clear. So do not echo secrets, do not enable shell tracing (set -x) around steps that use them, and be cautious with tools that print their environment on failure.

• Scope them. Use environment-level secrets (for example a production environment) rather than repository-wide ones, so only jobs targeting that environment can read them, optionally after a required reviewer approves.
• Minimise lifetime. Short-lived credentials limit the damage of a leak. Rotate long-lived ones on a schedule and immediately after any suspicion.
• Do not give secrets to untrusted code. Workflows triggered by pull requests from forks do not receive repository secrets by default, which is a deliberate protection. Be very careful before changing that behaviour, for example with triggers that run with elevated privileges on untrusted pull request content.
• If one leaks, rotate it. Deleting the commit or log is not enough, because copies and caches exist. Revoke and replace the credential first.

Replace stored cloud keys with short-lived federation

Storing a permanent cloud access key as a CI secret means a leak gives lasting access. OpenID Connect (OIDC) lets the pipeline prove its identity (this repository, this branch, this environment) to your cloud provider, which hands back credentials that expire in minutes. Nothing long-lived is stored in the CI platform at all. The cloud side is configured to trust only specific repositories and branches. The workflow below sketches the shape for AWS; the account number and role name are placeholders.

name: deploy

on:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    permissions:
      contents: read
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/example-deploy-role
          aws-region: ap-south-1
      - name: Deploy
        run: ./scripts/deploy.sh "$GITHUB_SHA"

Key lines: environment: production ties the job to an environment where you can require a human approval and restrict which branches may deploy. id-token: write lets the job request an OIDC token, and that permission is granted only to this job. The default GITHUB_TOKEN stays read-only everywhere else. The example avoids any stored access key. If your platform is not AWS, the same pattern exists for other major clouds.

Script injection: when a pull request title becomes a command

Expressions like ${{ ... }} are substituted into the workflow text before the shell runs it. If the substituted value is attacker-controlled (a pull request title, a branch name, an issue body), it becomes part of the script and the author can run commands on your runner. The vulnerable pattern and its fix are tiny:

      - name: Announce
        run: echo "Building PR titled ${{ github.event.pull_request.title }}"

      - name: Announce
        env:
          PR_TITLE: ${{ github.event.pull_request.title }}
        run: echo "Building PR titled $PR_TITLE"

In the fixed version the value is passed as an environment variable, so the shell treats it as data and never as code. The rule: never interpolate untrusted event fields directly into run: scripts. Pass them through env: and quote the variable.

Supply-chain hygiene for third-party actions

Every uses: line runs someone else's code with access to your job's token and secrets. Tags such as @v4 can be moved by the action's maintainer. For high-value pipelines, pin to a full commit SHA (a 40-character hash that you looked up and reviewed yourself, with the version in a comment) and let a dependency-update bot propose bumps. Prefer actions from maintainers you trust, keep the list short, and keep permissions minimal so a compromised action has little to steal. Also scan the dependencies and images you build: see the vulnerability scanning section of the Docker course.

Fast and reliable: the engineering side of pipelines

on:
  pull_request:
    paths-ignore:
      - "docs/**"
      - "**.md"

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    strategy:
      fail-fast: false
      matrix:
        node: [20, 22]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: npm
      - run: npm ci
      - run: npm test

The matrix runs the suite on two Node versions in parallel. fail-fast: false lets both finish so you see all failures, rather than cancelling the second run when the first fails. Reliability matters as much as speed. A flaky test (one that fails without a code change) teaches people to click re-run and ignore red. Track which tests flake, quarantine them with a ticket and an owner, and fix the cause: shared state, time-dependence, unseeded randomness, network calls to real services. Retrying a whole job is a bandage; retry only genuinely transient operations such as downloading a package.

Common mistakes and the symptoms they cause

• Committing a secret. Symptom: a scanner or the platform's secret-scanning alert fires, or a cloud provider emails about exposed credentials. Fix: revoke and rotate first, then clean history.
• Over-broad permissions. Symptom: a compromised or buggy step can push code or modify releases. Fix: default to contents: read and grant per job.
• Interpolating event fields into scripts. Symptom: odd behaviour or errors when a pull request title contains quotes or shell symbols. Fix: route values through env:.
• Secrets unavailable on forked pull requests. Symptom: steps fail with an empty value or an authentication error only on pull requests from forks. Fix: keep those jobs secret-free; do not loosen protection casually.
• Cache never hits. Symptom: install time is the same every run. Fix: check the key, the path and whether the lockfile is where the action looks.
• Retries instead of fixes. Symptom: the pipeline's pass rate depends on how many times you click re-run. Fix: find the flaky test.

How this shows up in real projects and interviews

Security-minded interviewers ask "how do you handle secrets in CI?" and want more than "environment variables": scoping, rotation, short-lived credentials via OIDC, and masking limits. Delivery-minded ones ask "the pipeline takes thirty minutes, what do you do?" and expect measurement first, then caching, parallelism, test splitting and flaky-test discipline. On a real team, the hours you spend making the pipeline fast and trustworthy are repaid every day by everyone who waits on it. For wider context on how the pieces fit, revisit Git and GitHub for the pull request workflow that triggers all of this.