Manual Distributed Task Execution on Github Actions
Using Nx Agents is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.
Run Custom Agents on GitHub
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
.github/workflows/ci.yml
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8# Needed for nx-set-shas when run on the main branch
9permissions:
10 actions: read
11 contents: read
12
13env:
14 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
15 NX_BRANCH: ${{ github.event.number || github.ref_name }}
16 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
17 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
18
19jobs:
20 main:
21 name: Nx Cloud - Main Job
22 runs-on: ubuntu-latest
23 steps:
24 - uses: actions/checkout@v4
25 name: Checkout [Pull Request]
26 if: ${{ github.event_name == 'pull_request' }}
27 with:
28 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
29 ref: ${{ github.event.pull_request.head.sha }}
30 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
31 fetch-depth: 0
32 filter: tree:0
33
34 - uses: actions/checkout@v4
35 name: Checkout [Default Branch]
36 if: ${{ github.event_name != 'pull_request' }}
37 with:
38 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
39 fetch-depth: 0
40 filter: tree:0
41
42 # Set node/npm/yarn versions using volta
43 - uses: volta-cli/action@v4
44 with:
45 package-json-path: '${{ github.workspace }}/package.json'
46
47 - name: Use the package manager cache if available
48 uses: actions/setup-node@v3
49 with:
50 node-version: 20
51 cache: 'npm'
52
53 - name: Install dependencies
54 run: npm ci
55
56 - name: Check out the default branch
57 run: git branch --track main origin/main
58
59 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done
60 run: npx nx-cloud start-ci-run --distribute-on="manual" --stop-agents-after=e2e-ci
61
62 - name: Check the formatting
63 run: npx nx-cloud record -- nx format:check
64
65 - name: Lint, test, build, and run e2e
66 run: npx nx affected -t lint,test,build,e2e-ci --configuration=ci
67
68 agents:
69 name: Agent ${{ matrix.agent }}
70 runs-on: ubuntu-latest
71 strategy:
72 matrix:
73 # Add more agents here as your repository expands
74 agent: [1, 2, 3]
75 steps:
76 - name: Checkout
77 uses: actions/checkout@v4
78
79 # Set node/npm/yarn versions using volta
80 - uses: volta-cli/action@v4
81 with:
82 package-json-path: '${{ github.workspace }}/package.json'
83
84 - name: Use the package manager cache if available
85 uses: actions/setup-node@v3
86 with:
87 node-version: 20
88 cache: 'npm'
89
90 - name: Install dependencies
91 run: npm ci
92
93 - name: Start Nx Agent ${{ matrix.agent }}
94 run: npx nx-cloud start-agent
95 env:
96 NX_AGENT_NAME: ${{ matrix.agent }}
97There are comments throughout the workflow to help you understand what is happening in each section.