Make JSON decoder buffer size dynamic

[bblfsh]

Make JSON decoder buffer size dynamic

[juanjux]

Oooooh yes, I totally forgot about that, I also lost many hours with that the other day with that cache...

[juanjux]

Fixed by PR #135 when merged.

[bzz]

Wow! So awesome, did not realize #135 is merged - going to test now.

[bzz]

I have tested the the latest bblfsh server \w #135 merged and I still get buffer size exceeded sometimes :/ not sure if that is related to this particular buffer size..

Here is the 12k LOC file it happened on
https://github.com/bit-man/bit-tools/blob/master/GenUnitTests-ECI2012/src/test/java/ej2/conInvariante/RandoopTestConInvariante1.java

I would be happy to just skip such files, but after this error - server hangs forever 😞
Might be related to bblfsh/bblfshd#42

[juanjux]

Very odd since that message is not in the SDK code anymore, I'll try with the docker image (but that SDK code is really used in the drivers which also have new versions compiled against the latest SDK).

[juanjux]

I can't reproduce it. Steps I took (using the workaround posted on Slack for the problem with the current server and env variables):

# Remove old images
docker rm (docker ps -a -q)  
docker rmi --force (docker images -q)

# workaround
export BBLFSH_DRIVER_IMAGES="a=a"
# Run the server
docker run -e BBLFSH_DRIVER_IMAGES --privileged -p 9432:9432 --name bblfsh bblfsh/server

# Send the file
docker run  -v $(pwd):/work --link bblfsh bblfsh/server bblfsh client --address=bblfsh:9432 /work/shit.java

time="2017-07-05T19:27:06Z" level=debug msg="reading file: /work/shit.java" 
time="2017-07-05T19:27:06Z" level=debug msg="dialing server at bblfsh:9432" 
time="2017-07-05T19:27:06Z" level=debug msg="instantiating service client" 
time="2017-07-05T19:27:06Z" level=debug msg="sending request" 

(super huge UAST is printed)

I tried again with several consecutive requests just in case it was hanging at the second request like in server bug #36 but it works perfectly.

[bzz]

Ok, I was missing --transport=docker-daemon when running server, so I guess drivers were fetched from DockerHub instead of using latest build/taggeg :\

But this does not explain why buffer size exceeded show up, does it?

[juanjux]

Actually, I should be using the version from Dockerhub to test it, let me test again without that.

[juanjux]

I didn't use --transport=docker-daemon. I just tried without the workaround (since @abeaumont fixed that and published the new image) and it worked again without problems:

docker rm (docker ps -a -q)  
docker rmi --force (docker images -q)
docker run --privileged -p 9432:9432 --name bblfsh bblfsh/server
docker run  -v (pwd):/work --link bblfsh bblfsh/server bblfsh client --address=bblfsh:9432 /work/shit.java

[juanjux]

Closing since the fix was merged.

[bzz]

Ok, after more digging - the reason I was getting this is that if you do not use transport - by default all the runtime, that includes driver containers, is cached under ./tmp and is never updated.

AFAIK there are plans to add update command eventually to bblfsh that would help to avoid it, but that's in future.

I'm removing it manually and it should work now.