Confusion about the code of train

[ise-uiuc]

Confusion about the code of train

[UniverseFly]

Hi, such options are given in the shell command, which we have not documented yet. Roughly here is how the training is invoked:

accelerate launch -m src/magicoder/train.py \
	--model_key $MODEL_KEY \
	--model_name_or_path $MODEL_KEY \
	--use_flash_attention True \
	--datafile_paths $DATASET_PATH \
	--output_dir $OUTPUT_DIR \
	--bf16 True \
	--num_train_epochs 2 \
	--per_device_train_batch_size 2 \
	--gradient_accumulation_steps 128 \
	--group_by_length False \
	--ddp_find_unused_parameters False \
	--optim adafactor \
	--max_grad_norm -1 \
	--warmup_steps $WARMUP_STEP \
	--learning_rate 5e-5 \
	--lr_scheduler_type linear

We will give a more clear documentation later.

[jaywongs]

Thank you for your reply, it worked! Looking forward to the clear documentation~

[VoiceBeer]

Hey thx for the answer, looking forward to the whole scripts

[shatealaboxiaowang]

Hi, such options are given in the shell command, which we have not documented yet. Roughly here is how the training is invoked:

accelerate launch -m src/magicoder/train.py \
	--model_key $MODEL_KEY \
	--model_name_or_path $MODEL_KEY \
	--use_flash_attention True \
	--datafile_paths $DATASET_PATH \
	--output_dir $OUTPUT_DIR \
	--bf16 True \
	--num_train_epochs 2 \
	--per_device_train_batch_size 2 \
	--gradient_accumulation_steps 128 \
	--group_by_length False \
	--ddp_find_unused_parameters False \
	--optim adafactor \
	--max_grad_norm -1 \
	--warmup_steps $WARMUP_STEP \
	--learning_rate 5e-5 \
	--lr_scheduler_type linear

We will give a more clear documentation later.

Hey thx for the answer, Is clear documentation done?