greenrobot/essentials

General purpose utilities and hash functions for Android and Java (aka java-common)

essentials's Issues

Add performance comparison project for Android

Like #8, but to run on Android

LinkedListMultimap

Why not use LinkedHashMap for the key to preserve order when iterating through the map ?

Or provide a LinkedListMultimap class like

public class LinkedListMultimap<K, V> extends AbstractMultimap<K, V, List<V>> {
	public static <K, V> LinkedListMultimap<K, V> create() {
		return new LinkedListMultimap<>(new LinkedHashMap<K, List<V>>());
	}

	protected LinkedListMultimap(LinkedHashMap<K, List<V>> map) {
		super(map);
	}

	@Override protected List<V> createNewCollection() {
		return new ArrayList<>();
	}
}

I'd be happy to make a PR with this as I'm using this in my own project with great success

Make fields in CircularByteBuffer protected

I want to extend its functionality (like, direct access to Buffers without copying byte arrays twice), but its fields are private. This forces me to make a copy of the class just to add a few new methods instead of being able to extend it. Can you please change this?

Murmur3A.updateUtf8 not exists

Trying to use Murmur implementation from this library, and got the error for:

        Murmur3A murmur = new Murmur3A();
        murmur.updateUtf8("My test string");

error: cannot find symbol method updateUtf8(String)

Tried with both versions: org.greenrobot:essentials:3.0.0-RC1 and de.greenrobot:java-common:2.3.1

reference links are broken. documentation link missing.

Most of the link in README.md section is broken. Like: stream-based IO

Your website says:

For more details, please have a look at the documentation.

But I can't find a visible documentation link anywhere.

An illegal reflective access operation has occurred

JDK 11

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl (file:/<REDACTED>/essentials-3.0.0-RC1.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Murmur 3F collision

I might not be using the api correctly, but we're getting collisons for the following case:

var m1 = new Murmur3F();
m1.update(0);
m1.update(-17664);

var m2 = new Murmur3F();
m2.update(0);
m2.update(-16640);

assert !Objects.equals(new UUID(m1.getValueHigh(), m1.getValue()), new UUID(m2.getValueHigh(), m2.getValue()));

Do we need a finalize or something?

Add Android project(s) with tests

1. Run the available JUnit tests as android instrumentation tests
2. Run performance benchmarks on Android

Not sure if those Android projects should be Gradle or Maven. On the plus side of Maven would be that base project is Maven. Also, Android Studio has trouble mixing Android with Java projects (see EventBus and greenDAO).

Add performance comparison project

Create another sub project to check performance of...

• LongHashSet vs. HashSet (call add&contains&remove lots of times)
• LongHashMap vs. HashMap (call put&get&remove lots of times)
• PipelineOutputStream vs. PipedOutputStream (pass around some MB with each)
• StringUtils.split vs. String.split
• StringUtils.hex vs. DatatypeConverter.printHexBinary

Reference platform: Java 8

Issues when building with proguard (3.0.0-RC1)

Building with proguard enabled would result in this error:
PrimitiveArrayUtils$UnsafeImpl: can't find referenced class sun.misc.Unsafe

adding the follow to my proguard configs seemed to help:
-dontwarn sun.misc.Unsafe

Negative Seed Values not handled correctly

What steps will reproduce the problem?

1. Supply a negative seed value when constructing an instance of MurmurHash3
2. Compute a hash
3. Get a different hash value than the reference implementation provides for the same input and seed

Please provide any additional information below:

Below is the current constructor for when a seed is supplied.

public Murmur3F(int seed)
{
this.seed = seed & (0xffffffffL); // unsigned
h1 = seed;
h2 = seed;
}

Here is the code from the reference implementation:

void MurmurHash3_x64_128 ( const void * key, const int len,
const uint32_t seed, void * out )
{
const uint8_t * data = (const uint8_t*)key;
const int nblocks = len / 16;

uint64_t h1 = seed;
uint64_t h2 = seed;

Note that h1 and h2 are unsigned 64-bit integers. Because they are unsigned, they will not inherit the sign of the supplied seed. However, in the corresponding java code, h1 and h2 will become negative if the seed is negative (e.g. a value greater than Integer.MAX_VALUE).

A minor modification to the constructor resolves the issue, and makes the output congruent with the reference implementation for negative seed values:

public Murmur3F(int seed)
{
this.seed = seed & (0xffffffffL); // unsigned
h1 = this.seed;
h2 = this.seed;
}

Question: Required java version

Question: What is the required minimum java version of the project?
Officially, unofficially? It's easy enough to test, but is there a decision regarding this? Looking into using it in an embedded environment where some hardware still only has java 1.5. Looks like the collections, dates, files etc can manage with only 1.5. At least not any 1.7. sun.misc.Unsafe might be an issue but isolated use of that.

ObjectCache should use phone uptime instead of time

Currently ObjectCache depends on the system time never changing, which is not optimal in a setup where the app runs for a long time, the user or the system might change the time.

a solution would be not to use the time but the uptime of the phone.

https://developer.android.com/reference/android/os/SystemClock#uptimeMillis()

No subscribers for service

I post EventBus.getDefault().post(new SendPlayer(player)); from a services, which is running in non main thread

<service
        android:name=".player.PlayerService"
        android:process=":player"
        android:enabled="true"
        android:exported="true">
    </service>

I expect to recieve even in my fragment

  @Subscribe(threadMode = ThreadMode.MAIN) public void onEvent(SendPlayer event) {
    Log.w("mcheck", "onEvent");
  }

However, i get message D/EventBus: No subscribers registered for event class yarh.com.tryexo.player.SendPlayer. Events are delivered only if i remove android:process=":player".
Is it a bug or I misunderstud flow of posting events between bachground thread and main thread?

Large File MurMurHash is much more slower than SHA1 (Android)

Test file size: 1.09GB

Here is the code:

    try {
        Log.d(TAG, "SHA1");
        Log.d(TAG, FileUtils.getSha1(file));
    } catch (IOException e) {
        e.printStackTrace();
    }
    try {
        Murmur3F checksum = new Murmur3F(123);
        Log.d(TAG, "murmur");
        FileUtils.updateChecksum(file, checksum);
        Log.d(TAG, "" + checksum.getValueHexString());
    } catch (IOException e) {
        e.printStackTrace();
    }

Here is the result:
09-20 18:18:53.021 D/HashTest: SHA1
09-20 18:19:12.721 D/HashTest: a21321f03c0463927f51f2febaf2ea1e66acff21
09-20 18:19:12.731 D/HashTest: murmur
09-20 18:19:47.951 D/HashTest: 6e838ff35a6d624310df1607cc5892f6

SHA1: ~20s
MurmurHash: ~40s

Is this result reasonable?

Pull Random outside of benchmark measurements

I experienced Random to have significant impact on performance. Please pull the usage of Random out into some set up step, which is not measured. Affects some tests.