前言

随着人工智能的不断发展,机器学习这门技术也越来越重要,很多人都开启了学习机器学习,本文就介绍了机器学习的基础内容。

一、搭建MobileNet网络

用MobileNetv2学习,轮次训练5轮次(代码),五个epoch2分钟,设备太重要了(服务器显卡P4000,网上说相当于1070)。

MobileSAM训练coco_深度学习


MobileSAM训练coco_MobileSAM训练coco_02

二、代码部分

1.module.py----定义MobileNet的网络结构

代码如下(示例):

from torch import nn
import torch


def _make_divisible(ch, divisor=8, min_ch=None):
    """
    This function is taken from the original tf repo.
    It ensures that all layers have a channel number that is divisible by 8
    It can be seen here:
    https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet.py
    """
    if min_ch is None:
        min_ch = divisor
    new_ch = max(min_ch, int(ch + divisor / 2) // divisor * divisor)
    # Make sure that round down does not go down by more than 10%.
    if new_ch < 0.9 * ch:
        new_ch += divisor
    return new_ch


class ConvBNReLU(nn.Sequential):
    def __init__(self, in_channel, out_channel, kernel_size=3, stride=1, groups=1):
        padding = (kernel_size - 1) // 2
        super(ConvBNReLU, self).__init__(
            nn.Conv2d(in_channel, out_channel, kernel_size, stride, padding, groups=groups, bias=False),
            nn.BatchNorm2d(out_channel),
            nn.ReLU6(inplace=True)
        )


class InvertedResidual(nn.Module):
    def __init__(self, in_channel, out_channel, stride, expand_ratio):
        super(InvertedResidual, self).__init__()
        hidden_channel = in_channel * expand_ratio
        self.use_shortcut = stride == 1 and in_channel == out_channel

        layers = []
        if expand_ratio != 1:
            # 1x1 pointwise conv
            layers.append(ConvBNReLU(in_channel, hidden_channel, kernel_size=1))
        layers.extend([
            # 3x3 depthwise conv
            ConvBNReLU(hidden_channel, hidden_channel, stride=stride, groups=hidden_channel),
            # 1x1 pointwise conv(linear)
            nn.Conv2d(hidden_channel, out_channel, kernel_size=1, bias=False),
            nn.BatchNorm2d(out_channel),
        ])

        self.conv = nn.Sequential(*layers)

    def forward(self, x):
        if self.use_shortcut:
            return x + self.conv(x)
        else:
            return self.conv(x)


class MobileNetV2(nn.Module):
    def __init__(self, num_classes=1000, alpha=1.0, round_nearest=8):
        super(MobileNetV2, self).__init__()
        block = InvertedResidual
        input_channel = _make_divisible(32 * alpha, round_nearest)
        last_channel = _make_divisible(1280 * alpha, round_nearest)

        inverted_residual_setting = [
            # t, c, n, s
            [1, 16, 1, 1],
            [6, 24, 2, 2],
            [6, 32, 3, 2],
            [6, 64, 4, 2],
            [6, 96, 3, 1],
            [6, 160, 3, 2],
            [6, 320, 1, 1],
        ]

        features = []
        # conv1 layer
        features.append(ConvBNReLU(3, input_channel, stride=2))
        # building inverted residual residual blockes
        for t, c, n, s in inverted_residual_setting:
            output_channel = _make_divisible(c * alpha, round_nearest)
            for i in range(n):
                stride = s if i == 0 else 1
                features.append(block(input_channel, output_channel, stride, expand_ratio=t))
                input_channel = output_channel
        # building last several layers
        features.append(ConvBNReLU(input_channel, last_channel, 1))
        # combine feature layers
        self.features = nn.Sequential(*features)

        # building classifier
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.classifier = nn.Sequential(
            nn.Dropout(0.2),
            nn.Linear(last_channel, num_classes)
        )

        # weight initialization
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out')
                if m.bias is not None:
                    nn.init.zeros_(m.bias)
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.ones_(m.weight)
                nn.init.zeros_(m.bias)
            elif isinstance(m, nn.Linear):
                nn.init.normal_(m.weight, 0, 0.01)
                nn.init.zeros_(m.bias)

    def forward(self, x):
        x = self.features(x)
        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        x = self.classifier(x)
        return x

2.train.py----加载数据集并进行训练,训练集计算loss,测试集计算accuracy,保存训练好的网络参数

代码如下(示例):

import os
import sys
import json

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import transforms, datasets
from tqdm import tqdm

from model_v2 import MobileNetV2


def main():
    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
    print("using {} device.".format(device))

    batch_size = 16
    epochs = 5

    data_transform = {
        "train": transforms.Compose([transforms.RandomResizedCrop(224),
                                     transforms.RandomHorizontalFlip(),
                                     transforms.ToTensor(),
                                     transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]),
        "val": transforms.Compose([transforms.Resize(256),
                                   transforms.CenterCrop(224),
                                   transforms.ToTensor(),
                                   transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])}

    data_root = os.path.abspath(os.path.join(os.getcwd(), "../.."))  # get data root path
    image_path = os.path.join(data_root, "data_set", "flower_data")  # flower data set path
    assert os.path.exists(image_path), "{} path does not exist.".format(image_path)
    train_dataset = datasets.ImageFolder(root=os.path.join(image_path, "train"),
                                         transform=data_transform["train"])
    train_num = len(train_dataset)

    # {'daisy':0, 'dandelion':1, 'roses':2, 'sunflower':3, 'tulips':4}
    flower_list = train_dataset.class_to_idx
    cla_dict = dict((val, key) for key, val in flower_list.items())
    # write dict into json file
    json_str = json.dumps(cla_dict, indent=4)
    with open('class_indices.json', 'w') as json_file:
        json_file.write(json_str)

    nw = min([os.cpu_count(), batch_size if batch_size > 1 else 0, 8])  # number of workers
    print('Using {} dataloader workers every process'.format(nw))

    train_loader = torch.utils.data.DataLoader(train_dataset,
                                               batch_size=batch_size, shuffle=True,
                                               num_workers=nw)

    validate_dataset = datasets.ImageFolder(root=os.path.join(image_path, "val"),
                                            transform=data_transform["val"])
    val_num = len(validate_dataset)
    validate_loader = torch.utils.data.DataLoader(validate_dataset,
                                                  batch_size=batch_size, shuffle=False,
                                                  num_workers=nw)

    print("using {} images for training, {} images for validation.".format(train_num,
                                                                           val_num))

    # create model
    net = MobileNetV2(num_classes=5)

    # load pretrain weights
    # download url: https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
    model_weight_path = "./mobilenet_v2.pth"
    assert os.path.exists(model_weight_path), "file {} dose not exist.".format(model_weight_path)
    pre_weights = torch.load(model_weight_path, map_location='cpu')

    # delete classifier weights
    pre_dict = {k: v for k, v in pre_weights.items() if net.state_dict()[k].numel() == v.numel()}
    missing_keys, unexpected_keys = net.load_state_dict(pre_dict, strict=False)

    # freeze features weights
    for param in net.features.parameters():
        param.requires_grad = False

    net.to(device)

    # define loss function
    loss_function = nn.CrossEntropyLoss()

    # construct an optimizer
    params = [p for p in net.parameters() if p.requires_grad]
    optimizer = optim.Adam(params, lr=0.0001)

    best_acc = 0.0
    save_path = './MobileNetV2.pth'
    train_steps = len(train_loader)
    for epoch in range(epochs):
        # train
        net.train()
        running_loss = 0.0
        train_bar = tqdm(train_loader, file=sys.stdout)
        for step, data in enumerate(train_bar):
            images, labels = data
            optimizer.zero_grad()
            logits = net(images.to(device))
            loss = loss_function(logits, labels.to(device))
            loss.backward()
            optimizer.step()

            # print statistics
            running_loss += loss.item()

            train_bar.desc = "train epoch[{}/{}] loss:{:.3f}".format(epoch + 1,
                                                                     epochs,
                                                                     loss)

        # validate
        net.eval()
        acc = 0.0  # accumulate accurate number / epoch
        with torch.no_grad():
            val_bar = tqdm(validate_loader, file=sys.stdout)
            for val_data in val_bar:
                val_images, val_labels = val_data
                outputs = net(val_images.to(device))
                # loss = loss_function(outputs, test_labels)
                predict_y = torch.max(outputs, dim=1)[1]
                acc += torch.eq(predict_y, val_labels.to(device)).sum().item()

                val_bar.desc = "valid epoch[{}/{}]".format(epoch + 1,
                                                           epochs)
        val_accurate = acc / val_num
        print('[epoch %d] train_loss: %.3f  val_accuracy: %.3f' %
              (epoch + 1, running_loss / train_steps, val_accurate))

        if val_accurate > best_acc:
            best_acc = val_accurate
            torch.save(net.state_dict(), save_path)

    print('Finished Training')


if __name__ == '__main__':
    main()
和之前train.py代码有不一样:预训练模型在IMAGENET训练得到的权重最后全连接展开是1000个类别,我们数据集是用的5个类别,所以要将其delete classifier weights并使训练效果更好,进行freeze features weights。
预训练权重下载方式:在train.py代码中输入:import torchvision.models.mobilenet进行查看。官方预训练权重链接

3.predict.py——得到训练好的网络参数后,用自己找的图像进行分类测试

import os
import json

import torch
from PIL import Image
from torchvision import transforms
import matplotlib.pyplot as plt

from model_v2 import MobileNetV2


def main():
    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

    data_transform = transforms.Compose(
        [transforms.Resize(256),
         transforms.CenterCrop(224),
         transforms.ToTensor(),
         transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

    # load image
    img_path = "../tulip.jpg"
    assert os.path.exists(img_path), "file: '{}' dose not exist.".format(img_path)
    img = Image.open(img_path)
    plt.imshow(img)
    # [N, C, H, W]
    img = data_transform(img)
    # expand batch dimension
    img = torch.unsqueeze(img, dim=0)

    # read class_indict
    json_path = './class_indices.json'
    assert os.path.exists(json_path), "file: '{}' dose not exist.".format(json_path)

    with open(json_path, "r") as f:
        class_indict = json.load(f)

    # create model
    model = MobileNetV2(num_classes=5).to(device)
    # load model weights
    model_weight_path = "./MobileNetV2.pth"
    model.load_state_dict(torch.load(model_weight_path, map_location=device))
    model.eval()
    with torch.no_grad():
        # predict class
        output = torch.squeeze(model(img.to(device))).cpu()
        predict = torch.softmax(output, dim=0)
        predict_cla = torch.argmax(predict).numpy()

    print_res = "class: {}   prob: {:.3}".format(class_indict[str(predict_cla)],
                                                 predict[predict_cla].numpy())
    plt.title(print_res)
    for i in range(len(predict)):
        print("class: {:10}   prob: {:.3}".format(class_indict[str(i)],
                                                  predict[i].numpy()))
    plt.show()


if __name__ == '__main__':
    main()

三、MobileNetv1

MobileSAM训练coco_深度学习_03

VGG16权重文件大概有490MB,Resnet152权重文件大概有644MB,对于移动设备和嵌入式不大现实。

MobileSAM训练coco_人工智能_04

MobileNet就是一个轻量级网络:精确度降低非常少,可以忽略的前提,模型的参数可以减少原来的32倍。也是牛逼,自我感觉,就是深度学习一直都是在刷榜单,我们要top最好的精确度,看谁是老大,但是MobileNet感觉是在首先在参数上进行创新,而且不影响精确度,就是走了一个极少数当时不热的方向,结果证明是可行的。(通过DW卷积实现减少模型参数)

总结:MoileNet就是轻量级网络 (通过DW卷积实现减少模型参数)

MobileSAM训练coco_人工智能_05

传统卷积:卷积核的深度=输入特征矩阵的深度;输出矩阵的通道数=卷积核的个数

DW卷积:卷积核的深度=1;输入矩阵的通道数=卷积核的个数=输出特征矩阵的通道数(就是不改变输入和输出的通道数)

MobileSAM训练coco_深度学习_06

总结:DW卷积是卷积核的深度=1(1个卷积核只负责一个输入图像的深度);输入矩阵的通道数=卷积核的个数=输出特征矩阵的通道数(就是不改变输入和输出的通道数)

深度可分离卷积就是由DW卷积和PW卷积组成,其中PW卷积就是普通卷积,只不过卷积核大小是=1.(所以也满足普通卷积的特点:卷积核的深度与输入特征的深度一样,卷积核的个数与输出特征矩阵的通道数一样)

通常DW卷积是和PW卷积一起使用的,使用这种组合可以对比于普通的卷积,大大减少参数的参与。

MobileSAM训练coco_人工智能_07

普通卷积:卷积核大小(DK DK)x输入特征矩阵深度(M)x输出特征矩阵深度(N)x输入特征矩阵大小(DF DF)

DW+PW卷积:卷积核大小(DK)x输入特征矩阵深度(1)x输出特征矩阵深度(M)x输入特征矩阵大小(DF DF)+卷积核大小(1X1)x输入特征矩阵深度(M)x输出特征矩阵深度(N)x输入特征矩阵大小(DF DF)

总结:理论上普通卷积计算量是DW+PW的8-9倍。

MobileSAM训练coco_学习_08

普通卷积Conv 步距 3x3x3x32 前两个3是卷积核的尺寸,第三个3是输入图像的深度,第四个3是卷积核的个数。DW卷积深度是1.

a是卷积核个数的倍率,β是分辨率参数。

四、MobileNetv2

MobileSAM训练coco_ide_09

优点:倒残差结构inverted residuals+linear bottlenecks,这个两个优点就是MobileNetV2的论文题目。

MobileSAM训练coco_人工智能_10

ResNet残差结构是先是1x1降维,然后是3x3卷积处理,最后1x1卷积升维。(“V”,两头大,中间小)

Inverted Residual Block是先1x1升维,然后是3x3卷积DW,最后是1x1卷积降维。(^,两头小,中间大)

激活函数也不一样,一个是Relu,一个是Relu6

MobileSAM训练coco_学习_11

Relu6L:当输入小于0时候,置为0;当输入大于0小于6时候,相当于y=x,大于6时候,置为0。

MobileSAM训练coco_深度学习_12

Relu激活函数对低维度特征信息造成大量损失。

MobileSAM训练coco_ide_13

MobileNetv2先1x1升维卷积(Relu6),然后是3x3卷积DW(Relu6),最后是1x1卷积降维(Linear线性激活)。(^,两头小,中间大)

t是扩展因子,前一个卷积得到的输出矩阵的深度是后一个卷积的输入矩阵的深度,DW卷积不改变输入和输出的通道数。

当步距=1时候且输入特征矩阵与输出矩阵的shape相同时候才有shortcut连接。

MobileSAM训练coco_ide_14

t扩展因子,c是输出特征矩阵的深度channel,n是bottleneck(倒残差结构)重复次数。s是步距(一个block由一系列bottleneck组成):比如bottleneck,n=2时候,bottleneck重复两遍,对他而言,第一层bottleneck的步距为2,第二层为1.

当步距=1时候且输入特征矩阵与输出矩阵的shape相同时候才有shortcut连接。上图的block,他采用了三个bottleneck结构,stride=1,按照论文应该有捷径分支,但实际上是没有的。因为输入特征矩阵深度是=64,而输出特征矩阵深度是=96,深度不一样,不可以直接相加,即没有捷径分支。

MobileSAM训练coco_MobileSAM训练coco_15

当步距=1时候且输入特征矩阵与输出矩阵的shape相同时候才有shortcut连接。上图的block,他采用了三个bottleneck结构,stride=1,按照论文应该有捷径分支,但实际上是没有的。因为输入特征矩阵深度是=64,而输出特征矩阵深度是=96,深度不一样,不可以直接相加,即没有捷径分支。

对于第二层bottleneck而言,步距=1,输入特征矩阵深度=上一层输出特征矩阵深度96,输出特征矩阵=96,宽高也不发生变化,可以有捷径分支。

MobileSAM训练coco_ide_16

性能对比在CPU上,移动设备实时处理。感觉就是也有点神奇。

五、MobileNetv3

MobileSAM训练coco_学习_17

MobileNet网络,更新Block(bneck)

MobileSAM训练coco_深度学习_18

MobileNetV3在Image Net上比MobileNetV2性能提高20%,在延时上比MoblieNetV2延时少6.6%。

总结:MobileNetV3在精确度和延时上比较好。

MobileSAM训练coco_深度学习_19

MobileNetV2就是先是1x1卷积升维(BN+Relu6),然后再3x3卷积DW卷积(BN+Relu6),最后1x1卷积降维(Relu6)。

1.增加了SE模块(注意力机制):针对特征矩阵的每一个channel进行池化处理,channel是多少,一维向量就有多少个元素。再通过两个全连接层得到输出向量。第一个全连接层节点数目是通道数的1/4,第二个全连接层节点数目是和通道数保持一致。最后输出的是和每一个通道数有关系,赋予不同的权重,即可实现SE模块的功能。

MobileSAM训练coco_深度学习_20


MobileSAM训练coco_学习_21

1.SE模块(注意力机制)

MobileSAM训练coco_人工智能_22

2.NL非线性激活函数,最后的1x1的卷积层没有经过激活函数处理

MobileSAM训练coco_MobileSAM训练coco_23

重新设计耗时层结构:

1.减少第一个卷积层的个数,由32变为16准确度差不多。但是参数减少和时间减少。

2.Efficient Last Stage减少运算参数。

MobileSAM训练coco_学习_24

SWISH函数计算求导不方便,h-swish函数比较方便。可以用h-sigmoid函数替代sigmoid函数,两个曲线相似。

MobileSAM训练coco_人工智能_25

输入的shape out=输出特征矩阵Chanel ;bneck3x3对应的是Dwise卷积的大小 ;exp size代表第一个升维的1x1卷积将维度升到多少维度。