Skip to main content
 首页 » 编程设计

python之如何在我的损失函数中添加 L2 正则化项

2024年12月31日9Renyi-Fan

我要比较有无正则化的区别,所以我想自定义两个损失函数。

我的 L2 范数损失函数:

###NET   
class CNN(nn.Module): 
def __init__(self): 
    super(CNN,self).__init__() 
    self.layer1 = nn.Sequential( 
        nn.Conv2d(3, 16, kernel_size = 5, padding=2), 
        nn.ReLU(), 
        nn.MaxPool2d(2)) 
    self.layer2 = nn.Sequential( 
        nn.Conv2d(16, 32, kernel_size = 5, padding=2), 
        nn.ReLU(), 
        nn.MaxPool2d(2)) 
    self.layer3 = nn.Sequential( 
        nn.Conv2d(32, 32, kernel_size = 5, padding=2), 
        nn.ReLU(), 
        nn.MaxPool2d(4)) 
    self.fc = nn.Linear(32*32*32,11) 
def forward(self, x): 
    out = self.layer1(x) 
    out = self.layer2(out) 
    out = self.layer3(out) 
    out = out.view(out.size(0), -1) 
    out = self.fc(out) 
    return out 
 
net = CNN() 
 
###OPTIMIZER 
criterion = nn.CrossEntropyLoss() 
optimizer = optim.SGD(net.parameters(), lr = LR, momentum = MOMENTUM) 

1.How can I add a L2 norm in my loss function?

2.If I want to write the loss function by myself (without using optim.SGD) and do the grad-decent by autograd, how can I do?

感谢您的帮助!

请您参考如下方法:

您可以自己明确计算权重的范数,并将其添加到损失中。

reg = 0 
for param in CNN.parameters(): 
  reg += 0.5 * (param ** 2).sum()  # you can replace it with abs().sum() to get L1 regularization 
loss = criterion(CNN(x), y) + reg_lambda * reg  # make the regularization part of the loss 
loss.backward()  # continue as usuall 

参见 this thread了解更多信息。