StringByteIterator toArray does not return the entire byte representation of the value

[brianfrankcooper]

StringByteIterator toArray does not return the entire byte representation of the value

Hi Shay,

ByteIterator.toArray() and ByteIterator.toString().getBytes() should return the same byte array. Could you try something like this and see what ByteIterator.toArray() is returning?

  private Map<String, byte[]> convertToBytearrayMap(Map<String,ByteIterator> values) {
    Map<String, byte[]> retVal = new HashMap<String, byte[]>();
    for (String key : values.keySet()) {
      System.out.println("key: " + key + ", bytesLeft: " + values.get(key).bytesLeft());
      byte[] value = values.get(key).toArray();
      System.out.println("value size: " + value.length);
      System.out.println("key: " + key + ", bytesLeft: " + values.get(key).bytesLeft());
      retVal.put(key, value);
    }
    return retVal;             
  }     

Thanks!
--Michi

[westonplatter]

@ShayHassidim - also something that may impact gemfire is that we recently merged a gemfire change that fixed a bug, #123. It might be worth pulling a fresh copy of the codebase.

[busbey]

I think this issue comes down to incorrect handling of String <-> byte[]. In addition to #279, StringByteIterator incorrectly uses the String's character length to determine how many bytes should be made and then incorrectly pulls each character and truncates it to a byte.

That means for Strings made up of non-ascii, toArray() will return a smaller array than if you immediately call toString().getBytes() (if you don't call toString immediately then you get the wrong answer still).

[busbey]

Fixed by #288