How to limit number of threads/sub-processes used in a function in bash

Interesting question. I tried to use xargs for this and I found a way.

Try this:

seq 10 | xargs -i --max-procs=4 bash -c "echo start {}; sleep 3; echo done {}"

--max-procs=4 will ensure that no more than four subprocesses are running at a time.

The output will look like this:

start 2
start 3
start 1
start 4
done 2
done 3
done 1
done 4
start 6
start 5
start 7
start 8
done 6
done 5
start 9
done 8
done 7
start 10
done 9
done 10

Note that the order of execution might not follow the commands in the order you submit them. As you can see 2 started before 1.

Leave a Comment