Comments (7)
velvia
commented on May 5, 2024
Hmm.... might be a timing issue. You can disable the tests during
assembly; I believe it's "test in assembly := {}"
On Sat, Feb 14, 2015 at 5:06 AM, Prayag Upd [email protected]
wrote:
With spark 1.1.1 and spark-job-server 0.4.1, I'm facing similar issue
here for config/production.sh as below,copied from local.sh.template
Environment and deploy file
For use with bin/server_deploy, bin/server_package etc.
DEPLOY_HOSTS="localhost"
APP_USER=spark
APP_GROUP=sparkoptional SSH Key to login to deploy server
#SSH_KEY=/path/to/keyfile.pem
INSTALL_DIR=/home/spark/job-server
LOG_DIR=/var/log/job-server
PIDFILE=spark-jobserver.pid
SPARK_HOME=/usr/local/spark-1.1.1
SPARK_CONF_HOME=$SPARK_HOME/confOnly needed for Mesos deploys
SPARK_EXECUTOR_URI=/packup/repo.softwares/JVM/BigData/spark-1.1.1.tgz
$ bin/server_deploy.sh production
Deploying job server to localhost...
[info] Loading project definition from /packup/workspace.programming/workspace.scala/spark-jobserver/project
Missing bintray credentials /home/prayagupd/.bintray/.credentials. Some bintray features depend on this.
Missing bintray credentials /home/prayagupd/.bintray/.credentials. Some bintray features depend on this.
Missing bintray credentials /home/prayagupd/.bintray/.credentials. Some bintray features depend on this.
[info] Set current project to root (in build file:/packup/workspace.programming/workspace.scala/spark-jobserver/)
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobFileDAO.scala message=File line length exceeds 110 characters line=138
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=23 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=24 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=25 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=26 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=28 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=36 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=37 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=38 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=39 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=40 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=41 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=42 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=43 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=48 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=49 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/io/JobSqlDAO.scala message=Public method must have explicit type line=50 column=8
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/JobManagerActor.scala message=File line length exceeds 110 characters line=102
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server/src/spark.jobserver/JobManagerActor.scala message=Avoid using return line=186 column=6
Processed 22 file(s)
Found 0 errors
Found 19 warnings
Finished in 11 ms
[success] created: /packup/workspace.programming/workspace.scala/spark-jobserver/job-server/target/scalastyle-result.xml
Processed 5 file(s)
Found 0 errors
Found 0 warnings
Finished in 1 ms
[success] created: /packup/workspace.programming/workspace.scala/spark-jobserver/job-server-tests/target/scalastyle-result.xml
[warn] Credentials file /home/prayagupd/.bintray/.credentials does not exist
[warn] Credentials file /home/prayagupd/.bintray/.credentials does not exist
[info] Updating {file:/packup/workspace.programming/workspace.scala/spark-jobserver/}job-server-api...
[info] Updating {file:/packup/workspace.programming/workspace.scala/spark-jobserver/}akka-app...
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
[warn] Credentials file /home/prayagupd/.bintray/.credentials does not exist
[info] Updating {file:/packup/workspace.programming/workspace.scala/spark-jobserver/}job-server-tests...
[info] Resolving com.typesafe#config;1.0.0 ...
[info] Updating {file:/packup/workspace.programming/workspace.scala/spark-jobserver/}job-server...
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
[info] Resolving io.spray#spray-util;1.2.1 ...
warning file=/packup/workspace.programming/workspace.scala/spark-jobserver/job-server-api/src/spark.jobserver/SparkJob.scala message=Method name does not match the regular expression '^([+-/_]|[a-z][A-Za-z0-9]_)$' line=10 column=6
Processed 2 file(s)
Found 0 errors
Found 1 warnings
Finished in 1 ms
[success] created: /packup/workspace.programming/workspace.scala/spark-jobserver/job-server-api/target/scalastyle-result.xml
[info] Resolving com.h2database#h2;1.3.170 ...
[info] Compiling 5 Scala sources to /packup/workspace.programming/workspace.scala/spark-jobserver/job-server-tests/target/classes...
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
Processed 11 file(s)
Found 0 errors
Found 0 warnings
Finished in 0 ms
[success] created: /packup/workspace.programming/workspace.scala/spark-jobserver/akka-app/target/scalastyle-result.xml
[info] Compiling 6 Scala sources to /packup/workspace.programming/workspace.scala/spark-jobserver/job-server/target/classes...
[info] Packaging /packup/workspace.programming/workspace.scala/spark-jobserver/job-server-tests/target/job-server-tests-0.4.1.jar ...
[info] Done packaging.
[info] Compiling 4 Scala sources to /packup/workspace.programming/workspace.scala/spark-jobserver/job-server/target/classes...
[info] Compiling 4 Scala sources to /packup/workspace.programming/workspace.scala/spark-jobserver/job-server/target/test-classes...
[info] Including from cache: slf4j-api-1.7.2.jar
[info] Including from cache: spray-util-1.2.1.jar
[info] Including from cache: spray-httpx-1.2.1.jar
[info] Including from cache: netty-3.6.6.Final.jar
[info] Including from cache: joda-convert-1.2.jar
[info] Including from cache: spray-http-1.2.1.jar
[info] Including from cache: parboiled-scala_2.10-1.1.6.jar
[info] Including from cache: config-1.0.0.jar
[info] Including from cache: mimepull-1.9.4.jar
[info] Including from cache: spray-json_2.10-1.2.5.jar
[info] Including from cache: shapeless_2.10-1.2.4.jar
[info] Including from cache: joda-time-2.1.jar
[info] Including from cache: spray-client-1.2.1.jar
[info] Including from cache: parboiled-core-1.1.6.jar
[info] Including from cache: metrics-core-2.2.0.jar
[info] Including from cache: spray-can-1.2.1.jar
[info] Including from cache: spray-routing-1.2.1.jar
[info] Including from cache: spray-io-1.2.1.jar
[info] Including from cache: slick_2.10-2.0.2-RC1.jar
[info] Including from cache: h2-1.3.170.jar
[info] JobStatusActorSpec:
[info] JobStatusActor
[info] - should return empty sequence if there is no job infos
[info] - should return error if non-existing job is unsubscribed
[info] - should not initialize a job more than two times
[info] - should be informed JobStarted until it is unsubscribed
[info] - should be ok to subscribe beofore job init
[info] - should be informed JobValidationFailed once
[info] - should be informed JobFinished until it is unsubscribed
[info] - should be informed JobErroredOut until it is unsubscribed
[info] - should update status correctly
[info] - should update JobValidationFailed status correctly
[info] - should update JobErroredOut status correctly
[info] NamedRddsSpec:
[info] NamedRdds
[info] - get() should return None when RDD does not exist
[info] - get() should return Some(RDD) when it exists
[info] - destroy() should do nothing when RDD with given name doesn't exist
[info] - destroy() should destroy an RDD that exists
[info] - getNames() should return names of all managed RDDs
[info] - getOrElseCreate() should call generator function if RDD does not exist
[info] - getOrElseCreate() should not call generator function, should return existing RDD if one exists
[info] - update() should replace existing RDD
[info] - should include underlying exception when error occurs
[info] SparkJobSpec:
[info] Sample tests for default validation && method
[info] - should return valid
[info] - should return invalid if one of them is invalid
[info] - should return invalid if both of them are invalid with the first message
[ERROR] [02/14/2015 06:56:15.772] [test-akka.actor.default-dispatcher-6] [akka.dispatch.Dispatcher] null
java.lang.NullPointerException
at spark.jobserver.JobManagerActor.spark$jobserver$JobManagerActor$$postEachJob(JobManagerActor.scala:257)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:239)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:232)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:434)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:433)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:385)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[ERROR] [02/14/2015 06:56:15.772] [test-akka.actor.default-dispatcher-2] [akka.dispatch.Dispatcher] null
java.lang.NullPointerException
at spark.jobserver.JobManagerActor.spark$jobserver$JobManagerActor$$postEachJob(JobManagerActor.scala:257)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:239)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:232)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:434)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:433)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:385)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[info] JobManagerActorAdHocSpec:
[info] error conditions
[info] - should return errors if appName does not match
[info] - should return error message if classPath does not match *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.JobManagerActor$Initialized
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply$mcV$sp(JobManagerSpec.scala:76)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:73)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:73)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should error out if loading garbage jar *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for NoSuchClass
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:327)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply$mcV$sp(JobManagerSpec.scala:86)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:81)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:81)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should error out if job validation fails *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected class spark.jobserver.CommonMessages$JobValidationFailed, found class spark.jobserver.CommonMessages$NoSuchClass$
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:413)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply$mcV$sp(JobManagerSpec.scala:95)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:89)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:89)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] starting jobs
[info] - should start job and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply$mcV$sp(JobManagerSpec.scala:113)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:107)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:107)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should start job more than one time and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply$mcV$sp(JobManagerSpec.scala:124)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:118)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:118)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should start job and return results (sync route) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply$mcV$sp(JobManagerSpec.scala:141)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:135)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:135)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should start job and return JobStarted (async) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply$mcV$sp(JobManagerSpec.scala:153)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:147)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:147)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should return error if job throws an error *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobErroredOut
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply$mcV$sp(JobManagerSpec.scala:163)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:157)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:157)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - job should get jobConfig passed in to StartJob message *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply$mcV$sp(JobManagerSpec.scala:175)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:167)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:167)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should properly serialize case classes and other job jar classes *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected: Did not get JobResult but got unexpected message JobResult(8f3ca434-11d2-45f7-8702-38046f7739b5,ArrayBuffer(foo))
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:346)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply$mcV$sp(JobManagerSpec.scala:188)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:181)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:181)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should refuse to start a job when too many jobs in the context are running *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Expected a message but didn't get one!
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply$mcV$sp(JobManagerSpec.scala:212)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:196)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:196)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should start a job that's an object rather than class *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected: Did not get JobResult but got unexpected message JobStarted(2c9d6a4a-7aa0-4226-a8b1-15dcd60e5444,test,2015-02-14T06:56:13.741-06:00)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:346)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply$mcV$sp(JobManagerSpec.scala:238)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:231)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:231)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] WebApiSpec:
[info] jars routes
[info] - should list all jars
[info] - should respond with OK if jar uploaded successfully
[info] - should respond with bad request if jar formatted incorrectly
[info] list jobs
[info] - should list jobs correctly
[info] /jobs routes
[info] - should respond with bad request if jobConfig cannot be parsed
[info] - should merge user passed jobConfig with default jobConfig
[info] - async route should return 202 if job starts successfully
[info] - adhoc job of sync route should return 200 and result
[info] - should be able to take a timeout param
[info] - adhoc job started successfully of async route should return 202
[info] - should be able to query job result from /jobs/ route
[info] - should be able to query job config from /jobs//config route
[info] - should respond with 404 Not Found from /jobs//config route if jobId does not exist
[info] - should respond with 404 Not Found if context does not exist
[info] - should respond with 404 Not Found if app or class not found
[info] - sync route should return Ok with ERROR in JSON response if job failed
[info] serializing complex data types
[info] - should be able to serialize nested Seq's and Map's within Map's to JSON
[info] - should be able to serialize Seq's with different types to JSON
[info] - should be able to serialize base types (eg float, numbers) to JSON
[info] - should convert non-understood types to string
[info] context routes
[info] - should list all contexts
[info] - should respond with 404 Not Found if stopping unknown context
[info] - should return OK if stopping known context
[info] - should respond with bad request if starting an already started context
[info] - should return OK if starting a new context
[info] spark alive workers
[info] - should return OK
[info] SparkWebUiActorSpec:
[info] SparkWebUiActor
[info] - should get worker info
[info] JobResultActorSpec:
[info] JobResultActor
[info] - should return error if non-existing jobs are asked
[info] - should get back existing result
[info] - should be informed only once by subscribed result
[info] - should not be informed unsubscribed result
[info] - should not publish if do not subscribe to JobResult events
[info] - should return error if non-existing subscription is unsubscribed
[info] JobSqlDAOSpec:
[info] save and get the jars
[info] - should be able to save one jar and get it back
[info] - should be able to retrieve the jar file
[info] saveJobConfig() and getJobConfigs() tests
[info] - should provide an empty map on getJobConfigs() for an empty CONFIGS table
[info] - should save and get the same config
[info] - should be able to get previously saved config
[info] - Save a new config, bring down DB, bring up DB, should get configs from DB
[info] Basic saveJobInfo() and getJobInfos() tests
[info] - should provide an empty map on getJobInfos() for an empty JOBS table
[info] - should save a new JobInfo and get the same JobInfo
[info] - should be able to get previously saved JobInfo
[info] - Save another new jobInfo, bring down DB, bring up DB, should JobInfos from DB
[info] - saving a JobInfo with the same jobId should update the JOBS table
[info] JobManagerActorSpec:
[info] error conditions
[info] - should return errors if appName does not match
[info] - should return error message if classPath does not match *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.JobManagerActor$Initialized
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply$mcV$sp(JobManagerSpec.scala:76)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:73)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:73)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should error out if loading garbage jar *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for NoSuchClass
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:327)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply$mcV$sp(JobManagerSpec.scala:86)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:81)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:81)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should error out if job validation fails *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected class spark.jobserver.CommonMessages$JobValidationFailed, found class spark.jobserver.CommonMessages$NoSuchClass$
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:413)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply$mcV$sp(JobManagerSpec.scala:95)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:89)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:89)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] starting jobs
[info] - should start job and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply$mcV$sp(JobManagerSpec.scala:113)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:107)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:107)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should start job more than one time and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply$mcV$sp(JobManagerSpec.scala:124)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:118)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:118)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should start job and return results (sync route) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply$mcV$sp(JobManagerSpec.scala:141)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:135)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:135)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should start job and return JobStarted (async) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply$mcV$sp(JobManagerSpec.scala:153)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:147)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:147)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - should return error if job throws an error *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobErroredOut
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:412)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply$mcV$sp(JobManagerSpec.scala:163)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:157)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:157)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - job should get jobConfig passed in to StartJob message *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply$mcV$sp(JobManagerSpec.scala:175)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:167)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:167)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should properly serialize case classes and other job jar classes *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (4 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply$mcV$sp(JobManagerSpec.scala:188)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:181)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:181)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should refuse to start a job when too many jobs in the context are running *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Expected a message but didn't get one!
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:345)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply$mcV$sp(JobManagerSpec.scala:212)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:196)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:196)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] - should start a job that's an object rather than class *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected: Did not get JobResult but got unexpected message JobStarted(a962d12c-2ef4-4c39-92dc-5807c7eed5b1,test,2015-02-14T06:57:50.865-06:00)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:346)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:707)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply$mcV$sp(JobManagerSpec.scala:238)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:231)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:231)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] at org.scalatest.FunSpec$class.invokeWithFixture$1(FunSpec.scala:1597)
[info] ...
[info] starting jobs
[info] - jobs should be able to cache RDDs and retrieve them through getPersistentRDDs *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected class spark.jobserver.JobManagerActor$Initialized, found class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:413)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$2.apply$mcV$sp(JobManagerActorSpec.scala:18)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$2.apply(JobManagerActorSpec.scala:16)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$2.apply(JobManagerActorSpec.scala:16)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] - jobs should be able to cache and retrieve RDDs by name *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected class spark.jobserver.JobManagerActor$Initialized, found class spark.jobserver.CommonMessages$NoJobSlotsAvailable
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:413)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:399)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:707)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$3.apply$mcV$sp(JobManagerActorSpec.scala:34)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$3.apply(JobManagerActorSpec.scala:32)
[info] at spark.jobserver.JobManagerActorSpec$$anonfun$2$$anonfun$apply$mcV$sp$3.apply(JobManagerActorSpec.scala:32)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.JobManagerSpec.withFixture(JobManagerSpec.scala:36)
[info] ...
[info] LocalContextSupervisorSpec:
[info] context management
[info] - should list empty contexts at startup
[info] - can add contexts from jobConfig *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected List(olap-demo), found ArrayBuffer()
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:328)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$2.apply$mcV$sp(LocalContextSupervisorSpec.scala:77)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$2.apply(LocalContextSupervisorSpec.scala:73)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$2.apply(LocalContextSupervisorSpec.scala:73)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.LocalContextSupervisorSpec.withFixture(LocalContextSupervisorSpec.scala:41)
[info] ...
[info] - should be able to add multiple new contexts *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for ContextInitialized
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:327)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$3.apply$mcV$sp(LocalContextSupervisorSpec.scala:83)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$3.apply(LocalContextSupervisorSpec.scala:80)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$3.apply(LocalContextSupervisorSpec.scala:80)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.LocalContextSupervisorSpec.withFixture(LocalContextSupervisorSpec.scala:41)
[info] ...
[info] - should be able to stop contexts already running *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected ContextInitialized, found ContextInitError(akka.pattern.AskTimeoutException: Timed out)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:328)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$4.apply$mcV$sp(LocalContextSupervisorSpec.scala:96)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$4.apply(LocalContextSupervisorSpec.scala:93)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$4.apply(LocalContextSupervisorSpec.scala:93)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.LocalContextSupervisorSpec.withFixture(LocalContextSupervisorSpec.scala:41)
[info] ...
[info] - should return NoSuchContext if attempt to stop nonexisting context *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected NoSuchContext, found ContextInitError(akka.pattern.AskTimeoutException: Timed out)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:328)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$5.apply$mcV$sp(LocalContextSupervisorSpec.scala:110)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$5.apply(LocalContextSupervisorSpec.scala:108)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$5.apply(LocalContextSupervisorSpec.scala:108)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.LocalContextSupervisorSpec.withFixture(LocalContextSupervisorSpec.scala:41)
[info] ...
[info] - should not allow creation of an already existing context *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected ContextInitialized, found ContextInitError(akka.pattern.AskTimeoutException: Timed out)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:328)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:314)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:707)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$6.apply$mcV$sp(LocalContextSupervisorSpec.scala:115)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$6.apply(LocalContextSupervisorSpec.scala:113)
[info] at spark.jobserver.LocalContextSupervisorSpec$$anonfun$3$$anonfun$apply$mcV$sp$6.apply(LocalContextSupervisorSpec.scala:113)
[info] at org.scalatest.FunSpec$$anon$1.apply(FunSpec.scala:1600)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at spark.jobserver.LocalContextSupervisorSpec.withFixture(LocalContextSupervisorSpec.scala:41)
[info] ...
[info] SparkJobUtilsSpec:
[info] SparkJobUtils.configToSparkConf
[info] - should translate num-cpu-cores and memory-per-node properly
[info] - should add other arbitrary settings
[info] JobInfoActorSpec:
[info] JobInfoActor
[info] - should store a job configuration
[info] - should return a job configuration when the jobId exists
[info] - should return error if jobId does not exist
[error] Failed: Total 106, Failed 31, Errors 0, Passed 75
[error] Failed tests:
[error] spark.jobserver.JobManagerActorAdHocSpec
[error] spark.jobserver.JobManagerActorSpec
[error] spark.jobserver.LocalContextSupervisorSpec
error sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 374 s, completed Feb 14, 2015 6:58:31 AM
Assembly failedSimilar issue at ooyala/spark-jobserver#59
ooyala/spark-jobserver#59—
Reply to this email directly or view it on GitHub
#72.
The fruit of silence is prayer;
the fruit of prayer is faith;
the fruit of faith is love;
the fruit of love is service;
the fruit of service is peace. -- Mother Teresa
from spark-jobserver.
MikhailGolubtsov
commented on May 5, 2024
I also tried run tests on v0.4.0 and on latest master, there were failures. test in assembly := {} in Assembly.scala helped to skip those during build.
On master:
[success] created: /root/spark-jobserver/job-server/target/scalastyle-result.xml
[info] Compiling 22 Scala sources to /root/spark-jobserver/job-server/target/classes...
[info] Compiling 16 Scala sources to /root/spark-jobserver/job-server/target/test-classes...
[warn] /root/spark-jobserver/job-server/test/spark.jobserver/JobManagerSpec.scala:141: non-variable type argument String in type pattern Map[String,Int] is unchecked since it is eliminated by erasure
[warn] case JobResult(_, result: Map[String, Int]) => println("I got results! " + result)
[warn] ^
[warn] /root/spark-jobserver/job-server/test/spark.jobserver/JobManagerSpec.scala:175: non-variable type argument String in type pattern Seq[String] is unchecked since it is eliminated by erasure
[warn] case JobResult(_, keys: Seq[String]) =>
[warn] ^
[warn] /root/spark-jobserver/job-server/test/spark.jobserver/JobStatusActorSpec.scala:15: trait ShouldMatchers in package matchers is deprecated: Please use org.scalatest.Matchers instead.
[warn] with FunSpecLike with ShouldMatchers with BeforeAndAfter with BeforeAndAfterAll {
[warn] ^
[warn] /root/spark-jobserver/job-server/test/spark.jobserver/NamedRddsSpec.scala:11: trait ShouldMatchers in package matchers is deprecated: Please use org.scalatest.Matchers instead.
[warn] with ImplicitSender with ShouldMatchers with BeforeAndAfter with BeforeAndAfterAll {
[warn] ^
[warn] four warnings found
[info] JobStatusActorSpec:
[info] JobStatusActor
[info] - should return empty sequence if there is no job infos
[info] - should return error if non-existing job is unsubscribed
[info] - should not initialize a job more than two times
[info] - should be informed JobStarted until it is unsubscribed
[info] - should be ok to subscribe beofore job init
[info] - should be informed JobValidationFailed once
[info] - should be informed JobFinished until it is unsubscribed
[info] - should be informed JobErroredOut until it is unsubscribed
[info] - should update status correctly
[info] - should update JobValidationFailed status correctly
[info] - should update JobErroredOut status correctly
[info] JobInfoActorSpec:
[info] JobInfoActor
[info] - should store a job configuration
[info] - should return a job configuration when the jobId exists
[info] - should return error if jobId does not exist
[info] JobResultActorSpec:
[info] JobResultActor
[info] - should return error if non-existing jobs are asked
[info] - should get back existing result
[info] - should be informed only once by subscribed result
[info] - should not be informed unsubscribed result
[info] - should not publish if do not subscribe to JobResult events
[info] - should return error if non-existing subscription is unsubscribed
[info] JobSqlDAOSpec:
[info] save and get the jars
[info] - should be able to save one jar and get it back
[info] - should be able to retrieve the jar file
[info] saveJobConfig() and getJobConfigs() tests
[info] - should provide an empty map on getJobConfigs() for an empty CONFIGS table
[info] - should save and get the same config
[info] - should be able to get previously saved config
[info] - Save a new config, bring down DB, bring up DB, should get configs from DB
[info] Basic saveJobInfo() and getJobInfos() tests
[info] - should provide an empty map on getJobInfos() for an empty JOBS table
[info] - should save a new JobInfo and get the same JobInfo
[info] - should be able to get previously saved JobInfo
[info] - Save another new jobInfo, bring down DB, bring up DB, should JobInfos from DB
[info] - saving a JobInfo with the same jobId should update the JOBS table
[info] SparkJobUtilsSpec:
[info] SparkJobUtils.configToSparkConf
[info] - should translate num-cpu-cores and memory-per-node properly
[info] - should add other arbitrary settings
[info] SparkWebUiActorSpec:
[info] SparkWebUiActor
[info] - should get worker info *** FAILED ***
[info] 0 did not equal 1 (SparkWebUiActorSpec.scala:126)
[info] JobManagerActorSpec:
[info] error conditions
[info] - should return errors if appName does not match
[info] - should return error message if classPath does not match *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.JobManagerActor$Initialized
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply$mcV$sp(JobManagerSpec.scala:75)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:72)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:72)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] Exception encountered when attempting to run a suite with class name: spark.jobserver.JobManagerActorSpec *** ABORTED ***
[info] akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/$b#-266591500]] after [10000 ms]
[info] at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:333)
[info] at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117)
[info] at scala.concurrent.Future$InternalCallbackExecutor$.scala$concurrent$Future$InternalCallbackExecutor$$unbatchedExecute(Future.scala:694)
[info] at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:691)
[info] at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467)
[info] at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419)
[info] at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423)
[info] at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375)
[info] at java.lang.Thread.run(Thread.java:745)
[info] ...
[info] LocalContextSupervisorSpec:
[info] context management
[info] - should list empty contexts at startup
[info] - can add contexts from jobConfig
[info] - should be able to add multiple new contexts
[info] - should be able to stop contexts already running
[info] - should return NoSuchContext if attempt to stop nonexisting context
[info] - should not allow creation of an already existing context
[info] WebApiSpec:
[info] jars routes
[info] - should list all jars
[info] - should respond with OK if jar uploaded successfully
[info] - should respond with bad request if jar formatted incorrectly
[info] list jobs
[info] - should list jobs correctly
[info] /jobs routes
[info] - should respond with bad request if jobConfig cannot be parsed
[info] - should merge user passed jobConfig with default jobConfig
[info] - async route should return 202 if job starts successfully
[info] - adhoc job of sync route should return 200 and result
[info] - should be able to take a timeout param
[info] - adhoc job started successfully of async route should return 202
[info] - should be able to query job result from /jobs/<id> route
[info] - should be able to query job config from /jobs/<id>/config route
[info] - should respond with 404 Not Found from /jobs/<id>/config route if jobId does not exist
[info] - should respond with 404 Not Found if context does not exist
[info] - should respond with 404 Not Found if app or class not found
[info] - sync route should return Ok with ERROR in JSON response if job failed
[info] serializing complex data types
[info] - should be able to serialize nested Seq's and Map's within Map's to JSON
[info] - should be able to serialize Seq's with different types to JSON
[info] - should be able to serialize base types (eg float, numbers) to JSON
[info] - should convert non-understood types to string
[info] context routes
[info] - should list all contexts
[info] - should respond with 404 Not Found if stopping unknown context
[info] - should return OK if stopping known context
[info] - should respond with bad request if starting an already started context
[info] - should return OK if starting a new context
[info] spark alive workers
[info] - should return OK
[ERROR] [02/26/2015 11:59:35.495] [test-akka.actor.default-dispatcher-3] [akka.dispatch.Dispatcher] null
java.lang.NullPointerException
at spark.jobserver.JobManagerActor.spark$jobserver$JobManagerActor$$postEachJob(JobManagerActor.scala:258)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:240)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:233)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:434)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:433)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[ERROR] [02/26/2015 11:59:35.495] [test-akka.actor.default-dispatcher-5] [akka.dispatch.Dispatcher] null
java.lang.NullPointerException
at spark.jobserver.JobManagerActor.spark$jobserver$JobManagerActor$$postEachJob(JobManagerActor.scala:258)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:240)
at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$2.applyOrElse(JobManagerActor.scala:233)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:434)
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:433)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[info] JobManagerActorAdHocSpec:
[info] error conditions
[info] - should return errors if appName does not match
[info] - should return error message if classPath does not match *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.JobManagerActor$Initialized
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply$mcV$sp(JobManagerSpec.scala:75)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:72)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$7.apply(JobManagerSpec.scala:72)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - should error out if loading garbage jar *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for NoSuchClass
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:338)
[info] at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:315)
[info] at akka.testkit.TestKit.expectMsg(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply$mcV$sp(JobManagerSpec.scala:85)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:80)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$8.apply(JobManagerSpec.scala:80)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - should error out if job validation fails *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected class spark.jobserver.CommonMessages$JobValidationFailed, found class spark.jobserver.CommonMessages$NoSuchClass$
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:424)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply$mcV$sp(JobManagerSpec.scala:94)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:88)
[info] at spark.jobserver.JobManagerSpec$$anonfun$2$$anonfun$apply$mcV$sp$9.apply(JobManagerSpec.scala:88)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] starting jobs
[info] - should start job and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply$mcV$sp(JobManagerSpec.scala:112)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:106)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$10.apply(JobManagerSpec.scala:106)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - should start job more than one time and return result successfully (all events) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply$mcV$sp(JobManagerSpec.scala:123)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:117)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$11.apply(JobManagerSpec.scala:117)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - should start job and return results (sync route) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:356)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply$mcV$sp(JobManagerSpec.scala:140)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:134)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$12.apply(JobManagerSpec.scala:134)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] ...
[info] - should start job and return JobStarted (async) *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobStarted
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply$mcV$sp(JobManagerSpec.scala:152)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:146)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$13.apply(JobManagerSpec.scala:146)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - should return error if job throws an error *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass waiting for class spark.jobserver.CommonMessages$JobErroredOut
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgClass_internal(TestKit.scala:423)
[info] at akka.testkit.TestKitBase$class.expectMsgClass(TestKit.scala:410)
[info] at akka.testkit.TestKit.expectMsgClass(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply$mcV$sp(JobManagerSpec.scala:162)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:156)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$14.apply(JobManagerSpec.scala:156)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] ...
[info] - job should get jobConfig passed in to StartJob message *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:356)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply$mcV$sp(JobManagerSpec.scala:174)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:166)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$15.apply(JobManagerSpec.scala:166)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] ...
[info] - should properly serialize case classes and other job jar classes *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (4 seconds) during expectMsg: Did not get JobResult
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:356)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply$mcV$sp(JobManagerSpec.scala:187)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:180)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$16.apply(JobManagerSpec.scala:180)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] ...
[info] - should refuse to start a job when too many jobs in the context are running *** FAILED ***
[info] java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg: Expected a message but didn't get one!
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:356)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply$mcV$sp(JobManagerSpec.scala:211)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:195)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$17.apply(JobManagerSpec.scala:195)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] ...
[info] - should start a job that's an object rather than class *** FAILED ***
[info] java.lang.AssertionError: assertion failed: expected: Did not get JobResult but got unexpected message NoJobSlotsAvailable(2)
[info] at scala.Predef$.assert(Predef.scala:179)
[info] at akka.testkit.TestKitBase$class.expectMsgPF(TestKit.scala:357)
[info] at akka.testkit.TestKit.expectMsgPF(TestKit.scala:718)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply$mcV$sp(JobManagerSpec.scala:237)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:230)
[info] at spark.jobserver.JobManagerSpec$$anonfun$3$$anonfun$apply$mcV$sp$19.apply(JobManagerSpec.scala:230)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] ...
[info] SparkJobSpec:
[info] Sample tests for default validation && method
[info] - should return valid
[info] - should return invalid if one of them is invalid
[info] - should return invalid if both of them are invalid with the first message
[info] NamedRddsSpec:
[info] NamedRdds
[info] - get() should return None when RDD does not exist
[info] - get() should return Some(RDD) when it exists
[info] - destroy() should do nothing when RDD with given name doesn't exist
[info] - destroy() should destroy an RDD that exists
[info] - getNames() should return names of all managed RDDs
[info] - getOrElseCreate() should call generator function if RDD does not exist
[info] - getOrElseCreate() should not call generator function, should return existing RDD if one exists
[info] - update() should replace existing RDD
[info] - update() should update an RDD even if it was persisted before
[info] - should include underlying exception when error occurs
[info] ScalaTest
[info] Run completed in 2 minutes, 38 seconds.
[info] Total number of tests run: 94
[info] Suites: completed 11, aborted 1
[info] Tests: succeeded 80, failed 14, canceled 0, ignored 0, pending 0
[info] *** 1 SUITE ABORTED ***
[info] *** 14 TESTS FAILED ***
[error] Error: Total 95, Failed 14, Errors 1, Passed 80
[error] Failed tests:
[error] spark.jobserver.JobManagerActorAdHocSpec
[error] spark.jobserver.SparkWebUiActorSpec
[error] Error during tests:
[error] spark.jobserver.JobManagerActorSpec
[error] (job-server/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 275 s, completed Feb 26, 2015 11:59:53 AM
from spark-jobserver.
velvia
commented on May 5, 2024
Do you want to try latest master?
from spark-jobserver.
prayagupa
commented on May 5, 2024
Hi @velvia , You seem super busy. I don't think master is working yet, as I get some other Whitespace error in file YammerMetrics.scala
log,
$ bin/server_deploy.sh production
.................
[info] Resolving org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0 ...
[error] /home/prayagupd/spark-jobserver/akka-app/src/ooyala.common.akka/metrics/YammerMetrics.scala:20:0: Whitespace at end of line
[error] /home/prayagupd/spark-jobserver/akka-app/src/ooyala.common.akka/metrics/YammerMetrics.scala:21: File line length exceeds 110 characters
[info] Processed 11 file(s)
[info] Found 2 errors
[info] Found 0 warnings
[info] Found 0 infos
[info] Finished in 7 ms
[success] created output: /home/prayagupd/spark-jobserver/akka-app/target
[info] Compiling 3 Scala sources to /home/prayagupd/spark-jobserver/job-server-api/target/classes...
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
[info] Compiling 6 Scala sources to /home/prayagupd/spark-jobserver/job-server-tests/target/classes...
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list
[info] Packaging /home/prayagupd/spark-jobserver/job-server-tests/target/job-server-tests-0.4.2-SNAPSHOT.jar ...
[info] Done packaging.
java.lang.RuntimeException: errors exist
at scala.sys.package$.error(package.scala:27)
at org.scalastyle.sbt.Tasks$.onHasErrors$1(Plugin.scala:123)
at org.scalastyle.sbt.Tasks$.doScalastyleWithConfig$1(Plugin.scala:167)
at org.scalastyle.sbt.Tasks$.doScalastyle(Plugin.scala:175)
at org.scalastyle.sbt.ScalastylePlugin$$anonfun$rawScalastyleSettings$1$$anonfun$apply$1$$anonfun$apply$2.apply(Plugin.scala:88)
at org.scalastyle.sbt.ScalastylePlugin$$anonfun$rawScalastyleSettings$1$$anonfun$apply$1$$anonfun$apply$2.apply(Plugin.scala:76)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[error] (akka-app/compile:scalastyle) errors exist
[error] Total time: 93 s, completed Feb 27, 2015 6:16:40 PM
Assembly failed
YammerMetrics
1 package ooyala.common.akka.metrics
2
3 import com.yammer.metrics.Metrics
4 import com.yammer.metrics.core.{Histogram, Meter, Gauge, Timer}
5 import java.util.concurrent.TimeUnit
6
7 /**
8 * Utility trait to make metrics creation slightly less verbose
9 */
10 trait YammerMetrics {
11 def meter(name: String, eventType: String): Meter =
12 Metrics.newMeter(getClass, name, eventType, TimeUnit.SECONDS)
13
14 def gauge[T](name: String, metric: => T, scope: String = null): Gauge[T] =
15 Metrics.newGauge(getClass, name, scope, new Gauge[T] {
16 override def value(): T = metric
17 })
18
19 def histogram(name: String): Histogram = Metrics.newHistogram(getClass, name, true)
20
21 def timer(name: String, durationUnit:TimeUnit=TimeUnit.NANOSECONDS, rateUnit:TimeUnit=TimeUnit.SECONDS): Timer =
22 Metrics.newTimer(getClass, name, durationUnit, rateUnit)
23 } from spark-jobserver.
velvia
commented on May 5, 2024
Ah, ok. Let me look into this and fix it. I merged in some style enforcements. Should be easy to fix.
-Evan
"Never doubt that a small group of thoughtful, committed citizens can change the world" - M. Mead
On Feb 27, 2015, at 4:22 PM, Prayag Upd [email protected] wrote:
Hi @velvia , You seem super busy. I don't think master is working yet, as I get some other Whitespace error in file YammerMetrics.scala
log,
[info] Resolving org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0 ...
[error] /home/prayagupd/spark-jobserver/akka-app/src/ooyala.common.akka/metrics/YammerMetrics.scala:20:0: Whitespace at end of line
[error] /home/prayagupd/spark-jobserver/akka-app/src/ooyala.common.akka/metrics/YammerMetrics.scala:21: File line length exceeds 110 characters
[info] Processed 11 file(s)
[info] Found 2 errors
[info] Found 0 warnings
[info] Found 0 infos
[info] Finished in 7 ms
[success] created output: /home/prayagupd/spark-jobserver/akka-app/target
[info] Compiling 3 Scala sources to /home/prayagupd/spark-jobserver/job-server-api/target/classes...
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Done updating.
[info] Compiling 6 Scala sources to /home/prayagupd/spark-jobserver/job-server-tests/target/classes...
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list
[info] Packaging /home/prayagupd/spark-jobserver/job-server-tests/target/job-server-tests-0.4.2-SNAPSHOT.jar ...
[info] Done packaging.
java.lang.RuntimeException: errors exist
at scala.sys.package$.error(package.scala:27)
at org.scalastyle.sbt.Tasks$.onHasErrors$1(Plugin.scala:123)
at org.scalastyle.sbt.Tasks$.doScalastyleWithConfig$1(Plugin.scala:167)
at org.scalastyle.sbt.Tasks$.doScalastyle(Plugin.scala:175)
at org.scalastyle.sbt.ScalastylePlugin$$anonfun$rawScalastyleSettings$1$$anonfun$apply$1$$anonfun$apply$2.apply(Plugin.scala:88)
at org.scalastyle.sbt.ScalastylePlugin$$anonfun$rawScalastyleSettings$1$$anonfun$apply$1$$anonfun$apply$2.apply(Plugin.scala:76)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
error errors exist
[error] Total time: 93 s, completed Feb 27, 2015 6:16:40 PM
Assembly failed
21 def timer(name: String, durationUnit:TimeUnit=TimeUnit.NANOSECONDS, rateUnit:TimeUnit=TimeUnit.SECONDS): Timer =
22 Metrics.newTimer(getClass, name, durationUnit, rateUnit)23 }
—
Reply to this email directly or view it on GitHub.
from spark-jobserver.
velvia
commented on May 5, 2024
Latest master is now fixed and running. Tests pass for me, as well as Travis CI.
from spark-jobserver.
noorul
commented on May 5, 2024
Closing this as this has been open for long enough and current master is working. Re-open if required.
from spark-jobserver.
Related Issues (20)
- CVE-2017-1000034: Upgrading Akka from 2.2.4 to 2.5.32 in Spark-jobserver HOT 1
- User did not initialize spark context! HOT 19
- Please update documentation for EMR deployment for latest version HOT 1
- Can't get config, when I post a job with 'sync=true' HOT 1
- Not able to create context HOT 3
- Not able to start spark-jobserver on emr cluster. Permission denied on creating logs in /mnt/var/log/spark-jobserver HOT 1
- Job Server going Out of Memory HOT 7
- While submitting Java jobs to Spark Jobserver, it shows JOB Loading Error HOT 1
- Spark-jobserver api dependency not resolving in scala and python jobs HOT 1
- resolvers += "Artifactory" at "https://sparkjobserver.jfrog.io/artifactory/jobserver/" not found HOT 1
- Spark 3 support HOT 1
- Spark Job Server CVEs
- jobserver disconnect with yarn application HOT 1
- Spark-jobserver 0.11.1 has vulnerabilities
- Spark-jobserver 0.11.1 has vulnerabilities
- Another Spark jobserver project HOT 1
- Update documentation HOT 1
- JFrog account will be suspended HOT 2
- Updating ReadMe.md Cadenz.ai has been changed to Lentra.AI HOT 1
- JavaWordCountJob failed: No configuration setting found for key 'input
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-jobserver.