Corb 2.3.1 is throwing IllegalStateException when ArrayQueue capacity is full.

[bbandlamudi]

Corb 2.3.1 is throwing IllegalStateException when ArrayQueue capacity is full.

[hansenmc]

We used to throw an exception if the number of URIs exceeded the URIs count, but rather than IllegalStateException it was an ArrayIndexOutOfBounds:

https://github.com/marklogic/corb2/blob/a161d974ef32ec132cb0b1293e8ad11a958e78f8/src/main/java/com/marklogic/developer/corb/Manager.java#L728

if (count >= urisArray.length) {
    throw new ArrayIndexOutOfBoundsException("received more than " + total + " results: " + uri);
}

We could override the ArrayQueue.add() and change the behavior. If the capacity is exceeded, should we catch the IllegalStateException and then:

• log a WARNING or ERROR
• keep processing (without throwing) ignore that URI and subsequent URIs? Only processing up to the count that was specified?

Or, log the warning and instantiate a new larger Array? That could get expensive/slow depending on whether we grow incrementally, or by some percentage, and if we were to go down that route, we should consider just using an ArrayList instead of Array, or maybe create an ArrayListQueue?

[hansenmc]

Actually, it looks as if we expected to be able to catch when the number of URIs exceeded the count in the QueryUrisLoader, but are using queue.add()
https://github.com/marklogic/corb2/blob/master/src/main/java/com/marklogic/developer/corb/QueryUrisLoader.java#L169
and should instead be using queue.offer()

I'll take a look and do some testing.

[bbandlamudi]

Thanks Mads! Regarding your first question, I am a little unsure about whether to increase (expensive) the array size with a warning or fail with a proper error (and a non-zero exit code). Though it offers flexibility, we may not want to cover up the mistake from the developers. However, if we do allow it, should we opt for having little bit of internal buffer (few extra spaces for few additional elements) rather than unlimited array size adjustments?

We still want to adjust to lesser value as we can get less number of uris when there are empty strings in between.