Concurrent
Run multiple wrangles concurrently rather than sequentially. Wrangles must specify output columns to be used concurrently. When using concurrent, Wrangles may not complete in a predictable order and it is not recommended to update overlapping columns with different wrangles.
Run multiple wrangles concurrently instead of sequentially. Concurrent wrangles must declare output columns, may finish in any order, and should not update overlapping columns.
See the Concurrent connector for the connector equivalent.
Parameters
| Name | Description | Accepted Values | Default | Required |
|---|---|---|---|---|
| Conditions | ||||
if | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with ${variable}. | string | — | No |
where | Filter rows before applying the wrangle using SQL-like criteria, such as column1 = 123 OR column2 = 'abc'. | string | — | No |
where_params | Values used with where for parameterized criteria. Uses SQLite placeholder syntax such as ? or :name. | array, object | — | No |
| Execution | ||||
wrangles | The wrangles section of a recipe to execute for each combination of variables. | array | — | Yes |
max_concurrency | The maximum number of wrangles to execute in parallel. | integer | 10 | No |
use_multiprocessing | Use process-based workers instead of threads. This is an advanced runtime option. | boolean | false | No |
Examples
wrangles:
- concurrent:
wrangles:
- extract.codes:
input: Products
output: Part Codes
- extract.attributes:
input: Products
output: Attributes
| Products |
|---|
| SKF ball brg 2" od 6202 |
| brg seal 1" id 5493 |
| 3lb hammer 87102 |
| Part Codes | Attributes |
|---|---|
| 6202 | {"length":["2in"]} |
| 5493 | {"length":["1in"]} |
| 87102 | {"weight":["3lb"]} |
Access
| Requirement | Value |
|---|---|
| AI-powered | No |
| Requires WrangleWorks account | No |
| Requires subscription | No |
| Requires external API key | No |
Technical details
| Field | Value |
|---|---|
| Catalog ID | 73 |
| Catalog key | concurrent |
| Recipe key | concurrent |
| Catalog status | active |
| Lifecycle status | active |
| Recipe Writer eligible | No |
| Recipe Writer exclusion | Concurrent orchestration is not supported in the baseline. |
| Namespace | Root-level |
| Documentation group | utility |
| Aliases | None |
| Runtime symbol | wrangles.recipe_wrangles.main.concurrent |
| Legacy UUID | None |
Sources