BlueData cluster setup for testing

[magsol]

BlueData cluster setup for testing

[milad181]

Perfect! Please use T 176 and P 2687340 for the 4tasks03.txt.
The 4task01.txt was the largest dataset that worked on the local mode with 16 GB RAM. (-T 176 -P 895780)

[magsol]

What about r and e? I believe we generally set m to 100.

On Thu, Mar 10, 2016 at 4:27 PM milad181 [email protected] wrote:

Perfect! Please use T 176 and P 2687340 for the 4tasks03.txt.
The 4task01.txt http://bd.hafni.cs.uga.edu/test/4tasks01.txt was the
largest dataset that worked on the local mode with 16 GB RAM. (-T 176 -P
895780)

—
Reply to this email directly or view it on GitHub
#67 (comment)
.

iPhone'd

[milad181]

We generally used r 0.07 -m 5 -e 0.01 to obtain results faster.

[magsol]

@quinngroup/bigneuron

I seem to have a reliable BlueData image working. It's currently crunching the 4tasks03.txt dataset; so far it's working. I also implemented a few optimizations--broadcasting the random seed at the start of each iteration, and representing v with a SparseVector object--to see how they work. They're not fully tested yet so the job may crash at some point.

In the meantime, feel free to use the image and stress test it against either the cluster I've spun up or your own custom cluster. Let me know if there are any problems.

[MOJTABAFA]

@magsol
Dear Dr.quinn, would you please set some credentials for me to work with your cluster ?
Thanks