If I want to train one epoch only, how to set lr?

[nissansz]

Optimizer:
name: AdamW
beta1: 0.9
beta2: 0.999
epsilon: 1.e-8
weight_decay: 0.05
no_weight_decay_name: norm
one_dim_param_no_weight_decay: True
lr:
name: Cosine
learning_rate: 0.001 # 8gpus 192bs
warmup_epoch: 5

[ir2718]

Hi,

can't you just achieve this by setting the lr in the config to your preferred value and removing the warmup_epoch?

Optimizer:
  name: AdamW
  beta1: 0.9
  beta2: 0.999
  epsilon: 1.e-8
  weight_decay: 0.05
  no_weight_decay_name: norm
  one_dim_param_no_weight_decay: True
  name: Cosine
  lr: 0.001 # 8gpus 192bs
  warmup_epoch: 0

[nissansz]

if batch size = 4, what is suitable lr?

[ir2718]

@nissansz

There is no definitive answer to that question, as this depends on the model and dataset. A good start would be the learning rate the authors use in the original paper of the model. They're probably using a much bigger batch size than 4, so I would advise you to lower it.