AdamW_adamw weight decay 大小

$$AdamW$$

一 基于Torch实现

optimizer = torch.optim.AdamW(model.parameters(),
                  lr=1e-4, weight_decay=1e-3)
1
2

optimizer.zero_grad()
# loss.backward()  # 反向传播
optimizer.step()
1
2
3

optimizer = torch.optim.AdamW(model.parameters(),
                  lr=1e-4, weight_decay=1e-3)
1
2

lr_step = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, factor=0.5, patience=5)
lr_step.step(val_dice)
1
2