PyTorch实现ResNet50v2：从结构解析到代码重构
引言
残差网络（ResNet）通过引入跳跃连接解决了深层网络中的梯度消失问题，而ResNetV2在此基础上进一步优化了内部模块的结构顺序。本文将深入探讨ResNetV2相较于原始版本的关键改进，并使用PyTorch从零构建一个完整的ResNet50v2模型，用于图像分类任务。
ResNetV2 核心思想解析
与原始 ResNet 相比，ResNetV2 最显著的变化在于激活函数和批归一化（Batch Normalization, BN）的位置调整。在标准 ResNet 中，每个残差块通常遵循"卷积 → BN → 激活"的流程；而在 ResNetV2 中，这一顺序被修改为"BN → 激活 → 卷积"，即所谓的 pre-activation 设计。
这种改变使得信息流在进入卷积层之前就已经完成了非线性变换，有助于缓解训练初期的信号衰减问题，提升深层网络的收敛速度与最终性能。实验表明，在 CIFAR-10 上使用 1001 层的 ResNet 模型时，ResNetV2 的测试错误率可降至 4.92%，优于原始结构的 7.61%。
不同 shortcut 结构对比分析
研究者尝试了多种 shortcut 连接方式，包括加入 BN、ReLU 或卷积操作等变体。结果显示，保持恒等映射（identity mapping）的原始 shortcut 表现最佳。一旦在 shortcut 路径中引入额外变换，反而可能导致训练不稳定或性能下降。这说明简洁有效的恒等连接仍是当前最优选择。
激活函数位置的探索
关于激活函数的位置，作者测试了六种不同的排列组合。其中效果最好的是"full pre-activation"模式——即在每一个卷积层前都进行 BN 和 ReLU 处理。该设计不仅提升了准确率，还增强了梯度传播能力，成为 ResNetV2 的核心特征之一。
基于 PyTorch 的 ResNet50v2 实现
1. 环境配置与依赖导入
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, random_split
from torchvision import datasets, transforms
import matplotlib.pyplot as plt
import numpy as np
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
2. 数据预处理与加载
加载鸟类图像数据集并进行标准化处理：
data_dir = "bird_photos/"
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
dataset = datasets.ImageFolder(root=data_dir, transform=transform)
num_classes = len(dataset.classes)
# 划分训练集与测试集
train_size = int(0.8 * len(dataset))
test_size = len(dataset) - train_size
train_data, test_data = random_split(dataset, [train_size, test_size])
train_loader = DataLoader(train_data, batch_size=8, shuffle=True)
test_loader = DataLoader(test_data, batch_size=8, shuffle=False)
3. 构建 Pre-Activation 残差块
定义 ResNetV2 使用的基本模块，包含两种类型：基础残差块与投影快捷连接块。
class PreActBlock(nn.Module):
expansion = 4
def __init__(self, in_channels, out_channels, stride=1, downsample=None):
super(PreActBlock, self).__init__()
self.bn1 = nn.BatchNorm2d(in_channels)
self.relu1 = nn.ReLU(inplace=True)
self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=1, bias=False)
self.bn2 = nn.BatchNorm2d(out_channels)
self.relu2 = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3,
stride=stride, padding=1, bias=False)
self.bn3 = nn.BatchNorm2d(out_channels)
self.relu3 = nn.ReLU(inplace=True)
self.conv3 = nn.Conv2d(out_channels, out_channels * self.expansion,
kernel_size=1, bias=False)
self.downsample = downsample
self.stride = stride
def forward(self, x):
identity = x
# Pre-activation: BN → ReLU → Conv
x = self.bn1(x)
x = self.relu1(x)
if self.downsample is not None:
identity = self.downsample(x)
x = self.conv1(x)
x = self.bn2(x)
x = self.relu2(x)
x = self.conv2(x)
x = self.bn3(x)
x = self.relu3(x)
x = self.conv3(x)
x += identity
return x
4. 构建完整 ResNet50v2 网络
按照 ResNet50 的层级结构堆叠残差块：
class ResNet50v2(nn.Module):
def __init__(self, block, layers, num_classes=1000):
super(ResNet50v2, self).__init__()
self.in_channels = 64
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
self.fc = nn.Linear(512 * block.expansion, num_classes)
# 初始化权重
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
elif isinstance(m, (nn.BatchNorm2d, nn.GroupNorm)):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
def _make_layer(self, block, out_channels, blocks, stride=1):
downsample = None
if stride != 1 or self.in_channels != out_channels * block.expansion:
downsample = nn.Sequential(
nn.AvgPool2d(kernel_size=stride, stride=stride),
nn.Conv2d(self.in_channels, out_channels * block.expansion,
kernel_size=1, stride=1, bias=False),
)
layers = []
layers.append(block(self.in_channels, out_channels, stride, downsample))
self.in_channels = out_channels * block.expansion
for _ in range(1, blocks):
layers.append(block(self.in_channels, out_channels))
return nn.Sequential(*layers)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
x = self.avgpool(x)
x = torch.flatten(x, 1)
x = self.fc(x)
return x
# 创建 ResNet50v2 实例
model = ResNet50v2(PreActBlock, [3, 4, 6, 3], num_classes=num_classes).to(device)
5. 模型概览
可通过以下代码查看模型结构：
print(model)
输出将显示详细的网络层次结构，验证 pre-activation 块是否正确嵌入。
