MuleSoft For Each, Parallel For Each and Batch Processing Comparison
Note: While copy/paste the code snippet from blog please quotes and double quotes.
As we know MuleSoft provides For Each, Parallel For Each and Batch Processing to process a list of records. In this blog we will compare all of them and see which use cases are suitable to use these.
What is For Each Scope?
- The Foreach scope splits a collection into elements and processes them iteratively through the processors embedded in the scope, then returns the original message to the flow.
What is Parallel For Each Scope?
- The Parallel For Each scope enables you to process a collection of messages by splitting the collection into parts that are simultaneously processed in separate routes within the scope of any limitation configured for concurrent-processing. After all messages are processed, the results are aggregated following the same order they were in before the split, and then the flow continues.
What is Batch Job?
- Mule allows you to process messages in batches. you can initiate a Batch Job scope, which splits messages into individual records, performs actions upon each record, and then reports on the results and potentially pushes the processed output to other systems or queues.
Comparison Between For Each, Parallel For Each and Batch Processing:
For Each Use Cases:
- Sequential Processing Required
- Synchronous Processing Required
- Small Data Set
- Processing of records in Batch required
- Process records only if previous records are processed successfully
Parallel For Each Use Cases:
- Synchronous Processing Required with parallelism
- Medium Data Set
- Accumulated Output Required
- Process records irrespective of previous records status
Parallel For Batch Job Cases:
- Asynchronous Processing Required
- Ordering of process records not needed
- Large Data set
- Processing logic is complex and filtering is optional
- Process records irrespective of previous records status
Conclusion:
In general the # of records and behavior (Sync or Async) determines the which option to choose. However for Medium # of records choosing between Parallel For Each and Batch Job mostly govern whether we want accumulated output or not. But if choosing Parallel For Each just because your use case requires accumulated output just keep in mind if Accumulated output is large it can cause JVM Out of Memory issue.
