I might not have been thinking all that clearly, but I was thinking that a short list should take less time to iterate over. Of necessity, this means keeping a separate copy of that list, so that it can be shortened.
To avoid revisiting a completed task, either drop it from the list, or don’t add it in the first place. (Depending on when and where you build the list.) The next time you iterate over the list, that task just won’t be there, and shouldn’t waste any more time.
You bring up a good point. Where to keep the list depends on the circumstances and the lifetime needed. If it has to be dragged out of a database, instead of a session cache, or a memory-resident list, then that might negate any speed advantage. This suggests using another background task as a monitor, to minimize latency.
A lot depends on when and how these background tasks are created, too. If the monitoring task is the only one that creates them, then perhaps the list can be maintained entirely in memory. With no outside interference, the list ought to be accurate and fast.
Whether any of the above is practical, will depend very strongly on the circumstances.
If task-creation is distributed, for example, then a transaction-guarded “task table” may be the only other reliable way to keep track. At least then you can use query conditions to exclude completed tasks – or remove them to a different table entirely, thus keeping the “active” list short.
Will this be faster in practice? I don’t know, but 30 vs. 700 certainly gives this approach a numerical advantage. And it gives you a place to hang additional per-task data (and metadata), which might be helpful in some circumstances.