Average interarrival time

[larissayukimiz]

Average interarrival time

[hongzimao]

So for Poisson process (https://www.probabilitycourse.com/chapter11/11_1_2_basic_concepts_of_the_poisson_process.php), the interarrival time of a job is drawn from an exponential distribution. In our example, the mean of this distribution is 45 seconds.

If you asked about the code, the interarrival time is passed through a global parameter: https://github.com/hongzimao/decima-sim/blob/master/param.py#L34-L35. It's used in the job generation process here:

decima-sim/spark_env/job_generator.py

Lines 128 to 129 in c010dd7

	# poisson process
	t += int(np_random.exponential(args.stream_interval))

Hope this helps!

[larissayukimiz]

I see. So it doesn't generate automatically when I execute the code?

[hongzimao]

Sorry I don't get your question --- what doesn't generate automatically? The job generator is called in the beginning to generate a set of jobs (initial jobs and streaming jobs) in a batch. It just generates the job size and assigns an interarrival time for each of them. The job generator is called here:

decima-sim/spark_env/env.py

Lines 371 to 373 in c010dd7

	# generate a set of new jobs
	self.job_dags = generate_jobs(
	self.np_random, self.timeline, self.wall_time)

[larissayukimiz]

Sorry, Mao. I was talking about how you calculated the average interarrival time (where you got the values).

[hongzimao]

It's just a hyperparameter. You can try different values to generate different loads. We just use an interarrival average so that the system load is ~80%. You can calculate the average load using the job distribution (so that you know the average job size) and the average interarrival time (so that you know the total work in a window). The work arrived in a window / server capacity will be the load in percentage.

[larissayukimiz]

Ohhh I get it now. So it was on default down here. Sorry for the misunderstanding! And thank you for the explanation!