---
title: Run Queue
description: Learn how Terragrunt orchestrates multiple concurrent OpenTofu/Terraform runs.
slug: features/stacks/run-queue
sidebar:
  order: 5
---

import FileTree from '@components/vendored/starlight/FileTree.astro';
import { Aside, Tabs, TabItem } from '@astrojs/starlight/components';

Terragrunt's "Run Queue" is the mechanism it uses to manage the run order and concurrency when running OpenTofu/Terraform commands across multiple Terragrunt [units](/features/units). This is particularly relevant when using the [`run --all`](/reference/cli/commands/run#all) or [`run --graph`](/reference/cli/commands/run#graph) commands.

## How it Works: The Dependency Graph (DAG)

At its core, the Run Queue relies on a [Directed Acyclic Graph (DAG)](/getting-started/terminology#directed-acyclic-graph-dag) built from the dependencies defined between your Terragrunt units. These dependencies are typically established using [`dependency`](/reference/hcl/blocks#dependency) or [`dependencies`](/reference/hcl/blocks#dependencies) blocks in your `terragrunt.hcl` files.

Terragrunt analyzes these dependencies to determine the correct order of operations:

1.  **Discovery:** Terragrunt discovers configurations that might be relevant to a run based on the current working directory.
2.  **Constructing the Queue:** Based on the command being run, Terragrunt creates an ordered queue.
    *   For commands like `plan` or `apply`, dependencies are run *before* the units that depend on them.
    *   For commands like `destroy`, dependent units are run *before* their dependencies.
3.  **Runs:** Terragrunt dequeues the units in the queue and runs them, respecting the queue order. By default, it runs units concurrently up to a certain limit (controlled by the [`--parallelism`](/reference/cli/commands/run#parallelism) flag), but it will always wait for a unit's dependencies (or dependents for destroys) to complete successfully before running that unit.

### Example DAG

Consider a setup where:

- Unit "dependent" depends on unit "dependency".
- Unit "dependency" depends on unit "ancestor-dependency".
- Unit "independent" has no dependencies nor dependents.

<FileTree>

- root
  - subtree
    - dependent
      - terragrunt.hcl
    - dependency
      - terragrunt.hcl
  - ancestor-dependency
    - terragrunt.hcl
  - independent
    - terragrunt.hcl

</FileTree>


```d2
direction: right

# Define the nodes
dependent: dependent {
  shape: rectangle
}

dependency: dependency {
  shape: rectangle
}

ancestor-dependency: ancestor-dependency {
  shape: rectangle
}

independent: independent {
  shape: rectangle
}

# Define the connections
dependent -> dependency: depends on
dependency -> ancestor-dependency: depends on
```

Assuming a current working directory of the `root` directory, Terragrunt would run units in the following order:

-   **`run --all plan` Order:** Terragrunt would run `independent` and `ancestor-dependency` concurrently. Once `ancestor-dependency` finishes, `dependency` would run. Once `dependency` finishes, `dependent` would run.
-   **`run --all destroy` Order:** Terragrunt would run `dependent` and `independent` concurrently. Once `dependent` finishes, `dependency` would run. Once `dependency` finishes, `ancestor-dependency` would run.

### Visualizing the DAG

To visualize the dependency graph you can use the `dag graph` command (similar to the `tofu/terraform graph` command), or its equivalent `list --format=dot --dependencies --external` command.

The graph is output in DOT format. The typical program used to render this file format is GraphViz, but many web services are available that can do this as well.

```bash
terragrunt dag graph | dot -Tsvg > graph.svg
# Or equivalently:
terragrunt list --format=dot --dependencies --external | dot -Tsvg > graph.svg
```

For a stack with `frontend-app`, `backend-app`, `mysql`, `valkey`, and `vpc` units (where `frontend-app` depends on `backend-app` and `vpc`, `backend-app` depends on `mysql`/`valkey`/`vpc`, and `mysql`/`valkey` depend on `vpc`), the rendered graph looks like:

![terragrunt dag graph](../../../../assets/img/collections/documentation/graph.png)

The arrow points from a unit to its dependency (e.g. `frontend-app` depends on `backend-app`).

For `plan` and `apply`, Terragrunt runs units in the opposite direction of the arrow (e.g. `backend-app` is applied before `frontend-app`). For `destroy` (and `plan/apply -destroy`), Terragrunt runs in the direction of the arrow (e.g. `frontend-app` is destroyed before `backend-app`).

## Controlling the Queue

Several flags allow you to customize how Terragrunt builds and executes the run queue. By default, Terragrunt will include all units that are in the current working directory.

### Include by default

By default, when using the `--all` flag, Terragrunt will include all units that are in the current working directory.

Using any positive filter triggers "Exclude by default" behavior, meaning that Terragrunt will no longer automatically include all units in the current working directory, and will instead selectively include units only if they match a positive filter (and don't match any negative filters).

More on this in the [Filtering Units](#filtering-units) section.

### Filtering Units

You can control which units are included or excluded from the queue using the [`--filter` flag](/features/filter).

#### Positive Filters

Any filter that doesn't start with a `!` prefix is considered a positive filter. When Terragrunt sees that any positive filter is present, it will evaluate every path it encounters while walking the directory tree, and only include units that match a positive filter.

```bash
terragrunt run --all --filter './subtree/**' -- plan
```

This will tell Terragrunt only to run the units that match the glob pattern `./subtree/**` (any directory found in the current working directory starting with `subtree`).

There are many more types of filters you can use, and those details are covered in the [Filter feature documentation](/features/filter).

#### Negative Filters

Any filter that starts with a `!` prefix is considered a negative filter. Negative filters are always evaluated after positive filters (if present).

```bash
terragrunt run --all --filter '!./subtree/**' -- plan
```

This will tell Terragrunt to run all units in the current working directory except those that match the glob pattern `./subtree/**` (any directory found in the current working directory starting with `subtree`).

#### Combining Positive and Negative Filters

You can combine positive and negative filters to ensure that only the units you want are run.

```bash
terragrunt run --all --filter './subtree/**' --filter '!./subtree/dependency/**' -- plan
```

This will tell Terragrunt to run all units in the directory `subtree` except those that match the glob pattern `./subtree/dependency/**` (any directory found in the current working directory starting with `subtree/dependency`).

### Modifying Order and Error Handling

- [`--queue-construct-as`](/reference/cli/commands/list#queue-construct-as) (`--as`): Build the run queue *as if* a particular command was run. Useful for performing dry-runs of [`run`](/reference/cli/commands/run) using discovery commands, like [`find`](/reference/cli/commands/find) and [`list`](/reference/cli/commands/list).

  e.g. `terragrunt list --queue-construct-as destroy`

  This lists the units in the order they'd be processed for `run --all destroy`:

  ```bash
  $ terragrunt list --as destroy -l
  Type  Path
  unit  independent
  unit  subtree/dependent
  unit  subtree/dependency
  unit  ancestor-dependency
  ```

  ```bash
  $ terragrunt list --as plan -l
  Type  Path
  unit  ancestor-dependency
  unit  independent
  unit  subtree/dependency
  unit  subtree/dependent
  ```

- [`--queue-ignore-dag-order`](/reference/cli/commands/run#queue-ignore-dag-order): Run units in the queue concurrently without respecting the dependency order.

  e.g. `terragrunt run --all plan --queue-ignore-dag-order`

  Run `plan` on `ancestor-dependency`, `subtree/dependency`, `subtree/dependent`, and `independent` all concurrently, without waiting for their defined dependencies. For instance, `subtree/dependent`'s plan would not wait for `subtree/dependency`'s plan to complete.

  <Aside type="caution">
  This flag is useful for faster runs in stateless commands like `validate` or `plan`, but is **dangerous** for commands that modify state like `apply` or `destroy`.

  You might encounter failed applies if unit dependencies are not applied before dependents, and conversely, failed destroys if unit dependents are not destroyed before dependencies.
  </Aside>

- [`--queue-ignore-errors`](/reference/cli/commands/run#queue-ignore-errors): Continue processing the queue even if some units fail.

  e.g. `terragrunt run --all plan --queue-ignore-errors`

  If `ancestor-dependency`'s plan fails, Terragrunt will still attempt to run `plan` for `subtree/dependency`, then `subtree/dependent`, and also for `independent`.

  <Aside type="caution">
  This flag is useful for identifying all errors at once, but can lead to inconsistent state if used with `apply` or `destroy`.

  You might encounter failed applies if unit dependencies are not applied successfully before dependents, and conversely, failed destroys if unit dependents are not destroyed successfully before dependencies.
  </Aside>

- [`--fail-fast`](/reference/cli/commands/run#fail-fast): Fail the run if any unit fails, stopping all remaining units immediately.

  e.g. `terragrunt run --all plan --fail-fast`

  If `independent`'s plan fails, Terragrunt will stop running any more units and fail the run, even if `ancestor-dependency`'s plan succeeds.

  <Aside type="tip">
  This flag is useful when you want to fail the run as soon as any unit fails, and stop running any more units.
  </Aside>

### Limiting run parallelism

By default, Terragrunt does not impose a limit on the number of units it executes when it traverses the dependency graph. If it finds 5 units without dependencies, it runs OpenTofu/Terraform 5 times in parallel, once in each unit.

This can create problems when a graph has many units, like hitting a cloud provider's rate limit.

Use the `--parallelism [number]` flag to cap the maximum number of unit executions at any given time:

```sh
terragrunt run --all --parallelism 4 -- apply
```

### Limiting stack output parallelism

`terragrunt stack output` reads unit outputs in parallel. Use the same `--parallelism` flag to lower concurrency if a large stack hits resource limits (for example, file descriptors or cloud-provider rate limits):

```sh
terragrunt stack output --parallelism 4
```

## Saving OpenTofu/Terraform plan output

A powerful feature of OpenTofu/Terraform is the ability to save the result of a plan as a binary file using the [-out flag](https://opentofu.org/docs/cli/commands/plan/).

Terragrunt provides special tooling in `run --all` execution to ensure that the saved plan for a `run --all` against a stack has a corresponding entry for each unit in the stack in a directory structure that mirrors the stack structure.

To save every plan generated when running a stack, use the `--out-dir` flag (or `TG_OUT_DIR` environment variable) as demonstrated below:

```bash
$ terragrunt run --all --out-dir /tmp/tfplan -- plan
```

<FileTree>

- app1
  - tfplan.tfplan
- app2
  - tfplan.tfplan
- app3
  - tfplan.tfplan
- project-2
  - project-2-app1
    - tfplan.tfplan

</FileTree>

This integration also exists for the `apply` command, where the generated plan will be used when performing the apply.

```bash
$ terragrunt run --all --out-dir /tmp/tfplan -- apply
```

<Aside type="note">

Performing a `run --all --out-dir <dir> -- apply` requires that a plan already exists for each unit in the stack.
If a plan is missing for any unit, the command will fail.

If a plan job was scoped with `--filter`, you must pass the **same** `--filter` to the apply so the apply discovery set matches the artifact contents. For example:

```bash
# Plan `<unit>` and its dependencies:
terragrunt run --all --filter '<unit>...' --out-dir /tmp/tfplan -- plan

# Apply must reuse the same filter:
terragrunt run --all --filter '<unit>...' --out-dir /tmp/tfplan -- apply
```

</Aside>

For planning a destroy operation, use the following commands:

```bash
terragrunt run --all --out-dir /tmp/tfplan -- plan -destroy
terragrunt run --all --out-dir /tmp/tfplan -- apply
```

<Aside type="caution">

If you are leveraging [mock outputs](/features/stacks/stack-operations#unapplied-dependency-and-mock-outputs) in your stack, you may get unexpected results when applying saved plans, as the plans will have mock outputs in them.

There's a workaround documented [here](https://github.com/gruntwork-io/terragrunt/issues/2178#issuecomment-2615842856), but no first-class feature in Terragrunt to address this issue, currently.

If you would like Terragrunt to have first-class support for a solution to this, please [create an RFC](https://github.com/gruntwork-io/terragrunt/issues/new?template=03-rfc.yml) to propose it.

</Aside>

To save plans in json format use the `--json-out-dir` flag:

```bash
terragrunt run --all --json-out-dir /tmp/json -- plan
```

<FileTree>

- app1
  - tfplan.json
- app2
  - tfplan.json
- app3
  - tfplan.json
- project-2
  - project-2-app1
    - tfplan.json

</FileTree>


```bash
terragrunt run --all --out-dir /tmp/all --json-out-dir /tmp/all -- plan
```

<FileTree>

- app1
  - tfplan.json
  - tfplan.tfplan
- app2
  - tfplan.json
  - tfplan.tfplan
- app3
  - tfplan.json
  - tfplan.tfplan
- project-2
  - project-2-app1
    - tfplan.json
    - tfplan.tfplan

</FileTree>

## Important Considerations

<Aside type="caution">
When using `run --all plan` with units that have dependencies (e.g. via `dependency` or `dependencies` blocks), the command will fail if those dependencies have never been deployed. This is because Terragrunt cannot resolve dependency outputs without existing state.

To work around this issue, use [mock outputs in dependency blocks](/reference/hcl/blocks/#dependency).
</Aside>

<Aside type="caution">
Do not set `TF_PLUGIN_CACHE_DIR` when using `run --all` (unless using OpenTofu >= 1.10).

This can cause concurrent access issues with the provider cache. Instead, use Terragrunt's built-in [Provider Cache Server](/features/caching/provider-cache-server/).
</Aside>

<Aside type="caution">
When using `run --all` with `apply` or `destroy`, Terragrunt automatically adds the `-auto-approve` flag due to limitations with shared stdin making individual approvals impossible. Use [`--no-auto-approve`](/reference/cli/commands/run#no-auto-approve) to override this, but be aware you might need alternative approval workflows.
</Aside>
