spomega/adjustco

code base for Adjust command line tool

adjustco's Issues

A small improvement requested

The solution you have provided looks good. But it has the problem that it performs requests in batches. This approach has the downside that each batch has to wait for the slowest request to finish before the next batch can begin.

Consider the case where you have 100 URLs, out of which every 10th takes 10 seconds to finish, while all the others take 1 second each. With a parallelism of 10, you take a batch of 10 URLs. In the first second all ten goroutines are doing work. After that, for the next 9 seconds only one of them is still busy with the long running request, while the other 9 have already finished.

Are you able to make it so that whenever a request completes another request is able to be started immediately, while still only allowing -parallel many requests to run concurrently.