index-sort-as elements beginning with newline get sorted as "Specials"

[simonbate]

index-sort-as elements beginning with newline get sorted as "Specials"

[raducoravu]

@simonbate if you are using for publishing the DITA OT PDF5 plugin developed by Antenna House maybe they could help you further:
https://www.antennahouse.com/dita-pdf5-plugin

[simonbate]

Unfortunately, we're using the OT pdf2 plugin as our base.
Our temporary solution is to ensure the sources do not include newlines, but that shouldn't be necessary.

[raducoravu]

@simonbate right you can probably define a Schematron rule to catch such problems.
I added a similar problem but for CHM and indexterm elements a couple of days ago: #4336

[chrispy-snps]

Testcase here: 4337.zip

<index-see> and <index-see-also> handle surrounding whitespace correctly; only <index-sort-as> has the issue.

[chrispy-snps]

I think the fix would be inside src/main/java/org/dita/dost/reader/IndexTermReader.java by applying the trimSpaceAtStart() function somewhere, but I can't figure out where it should be. @raducoravu - if you see a potential fix for this, I would be happy to test it.

[raducoravu]

@chrispy-snps I'm afraid I do not have time to look into this, here "org.dita.dost.reader.IndexTermReader.characters(char[], int, int)" there seems to be an initial normalization using normalizeAndCollapseWhitespace which replaces all consecutive spaces with one.
And then there is this code for the index sort as:

  else if (insideSortingAs && temp.length() > 0) {
    final IndexTerm indexTerm = termStack.peek();
    temp = trimSpaceAtStart(temp, indexTerm.getTermKey());
    indexTerm.setTermKey(setOrAppend(indexTerm.getTermKey(), temp, false));
  } 

It's unclear to me what the trimSpaceAtStart does, it seems to not always remove the first space, but removes it when the second parameter to the method also starts with a space. But the "indexTerm.getTermKey()" is probably null at that moment. So maybe replace:

temp = trimSpaceAtStart(temp, indexTerm.getTermKey());

with:

temp = trimSpaceAtStart(temp, indexTerm.getTermKey() != null ? indexTerm.getTermKey() : temp);

so that the first time when the term key is not yet computed to always remove the first space?

[chrispy-snps]

So I think the IndexTermReader.java code is used only by the htmlhelp transformation. For pdf2 transformations, index term processing appears to be provided by the org.dita.index plugin. It looks like there is a unit test for <index-sort-as> in there that could be modified to test and resolve the whitespace issue, but I haven't been able to build and test that plugin yet.

[chrispy-snps]

In org.dita.index, I filed the following issue:

#2: cannot build or test plugin

Once that is figured out, I hope to implement a fix there.