❓ Confused about `batch_size`

[astariul]

❓ Confused about `batch_size`

[nlpyang]

For extractive, batch_size is maximum number of sentences in the source document
For abstractive, batch_size is maximum number of tokens in the target summary

It is designed to use the memory more effectively.

[astariul]

As I understood so far :

Each step, the Trainer class run accum_count mini-batch.
For each mini-batch, there is X samples, where X vary following :

X * max(len(x) for x in X) <= batch_size

Please let me know if I understood right !

---

I think some comments on this function :

PreSumm/src/models/data_loader.py

Lines 112 to 124 in fa69433

	def ext_batch_size_fn(new, count):
	if (len(new) == 4):
	pass
	src, labels = new[0], new[4]
	global max_n_sents, max_n_tokens, max_size
	if count == 1:
	max_size = 0
	max_n_sents = 0
	max_n_tokens = 0
	max_n_sents = max(max_n_sents, len(src))
	max_size = max(max_size, max_n_sents)
	src_elements = count * max_size
	return src_elements

might be helpful !

[jdh3577]

Then, what is the exactly number of real(original mean) batch size?

[astariul]

Batchsize (by its traditional definition) is not a fixed number here. This is designed to use the memory much more efficiently than using a fixed number.

[nlpyang]

@jdh3577 As I said, this is dynamic during training.

[Shashi456]

@nlpyang would decreasing the batch size to 512 in Extractive summarization affect performance?

[gaozhiguang]

Hi, Does the batchsize here have something to do with the number of GPU, it uses the distributed training, how does the model update its parameters? is it all gpus gradient merge and then update?

[gaozhiguang]

if self.grad_accum_count > 1:
if self.n_gpu > 1:
grads = [p.grad.data for p in self.model.parameters()
if p.requires_grad
and p.grad is not None]
distributed.all_reduce_and_rescale_tensors(
grads, float(1))
for o in self.optims:
o.step()
maybe the code here

[pretidav]

@nlpyang could you please shed some light on the meaning of this parameter, it clearly isn't the number of documents in the batch, but something related to the number of word-pieces multiplied by a funny factor 300. Is the latter a typo or a magic number inserted on purpose ?
Thanks