deepspeed训练一个pytorch模型流程

转载

mob64ca140651e5 2024-07-18 23:28:28

文章标签 神经网络机器学习人工智能深度学习计算机视觉 文章分类 PyTorch 人工智能

Are you interested in learning about Deep Learning? We are hosting a free 6-week live course on our YouTube channel, starting Saturday, May 23rd at 8:30 AM PST.

您对学习深度学习感兴趣吗？我们将于5月23日(星期六)太平洋标准时间上午8:30开始在我们的YouTube频道上举办为期6周的免费实时课程。

Passively watching a video is often not enough to learn a software concept. You need to be able to ask questions and build real projects. That is exactly what you will be able to do in the course “Deep Learning with PyTorch: Zero to GANs”.

被动地观看视频通常不足以学习软件概念。您需要能够提出问题并建立真实的项目。这就是您在“使用PyTorch进行深度学习：从GAN归零”课程中可以做到的。

This is an online course intended to provide a coding-first introduction to deep learning using the PyTorch framework. The course takes a hands-on coding-focused approach and will be taught using live interactive Jupyter notebooks, allowing students to follow along and experiment.

这是一门在线课程，旨在为使用PyTorch框架的深度学习提供编码优先的入门知识。该课程采用动手编程为重点的方法，并将使用实时交互式Jupyter笔记本电脑进行授课，使学生可以继续学习并进行实验。

This course is taught by Aakash N S. He is the co-founder and CEO of Jovian.ml, a project management and collaboration platform for machine learning.

该课程由Aakash NS教授。他是Jovian.ml(机器学习的项目管理和协作平台)的联合创始人兼首席执行官。

Theoretical concepts will be explained in simple terms using code. Students will receive weekly assignments, work on a project with real-world datasets and participate in a private data science competition to test their skills. Upon successful completion of the course, students will receive a certificate of completion.

理论概念将使用代码以简单的术语进行解释。学生将接受每周的作业，使用真实数据集进行项目研究，并参加私人数据科学竞赛以测试其技能。成功完成课程后，学生将获得结业证书。

This is a beginner-friendly course, and no prior knowledge of data science, machine learning or deep learning is assumed. It is preferable to have some background in the following areas:

这是一门适合初学者的课程，不假定您具备数据科学，机器学习或深度学习的先验知识。最好具有以下方面的背景：

Programming knowledge, preferably in Python

Basics of linear algebra (vectors, matrices, dot products)

Basics of calculus (differentiation, geometric interpretation of derivative)

(Syllabus)

The course is divided into 6 modules, and will be taught over 6 weeks via video lectures and interactive Jupyter notebooks. Each lecture will be around 2 hours long.

该课程分为6个模块，将通过视频讲座和交互式Jupyter笔记本电脑进行为期6周的教学。每个讲座将持续2个小时左右。

(Module 1: PyTorch Basics - Tensors & Gradients)

Introduction to Jupyter notebooks & Data Science in Python
Creating vectors, matrices & Tensors in PyTorch
Tensor operations and gradient computations
Interoperability of PyTorch with Numpy

(Module 2: Linear Regression & Gradient Descent)

Linear Regression from scratch using Tensor operations
Weights, biases and the mean squared error loss function
Gradient descent and model training with PyTorch Autograd
Linear Regression using PyTorch built-ins (nn.Linear, nn.functional etc.)

(Module 3: Logistic Regression for Image Classification)

Working with images from the MNIST dataset
Training and validation dataset creation
Softmax function and categorical cross entropy loss
Model training, evaluation and sample predictions

(Module 4: Feedforward Neural Networks & GPUs)

Working with cloud GPU platforms like Kaggle & Colab
Creating a multilayer neural network using nn.Module
Activation function, non-linearity and universal approximation theorem
Moving with datasets and models to the GPU for faster training

(Module 5a: Image Classification using Convolutional Neural Networks)

Working with the 3-channel RGB images from the CIFAR10 dataset
Introduction to Convolutions, kernels & features maps
Underfitting, overfitting and techniques to improve model performance

(Module 5b: Data Augmentation, Regularization and Residual Networks)

Improving the dataset using data normalization and data augmentation
Improving the model using residual connections and batch normalization
Improving the training loop using learning rate annealing, weight decay and gradient clip
Training a state of the art image classifier from scratch in 10 minutes

(Module 6: Image Generation using Generative Adversarial Networks (GANs))

Introduction to generative modeling and application of GANs
Creating generator and discriminator neural networks
Generating and evaluating fake images of handwritten digits
Training the generator and discriminator in tandem and visualizing results

(Exercises & Assignments)

(Weekly Assignments)

Week 1: Linear Regression
Week 2: Image Classification
Week 3: Feedforward neural networks

(Course Project )

For the course project, students will create an image classification model using Convolutional neural networks, on a real-world dataset of their choice. The project will allow students to experiment with different types of models and regularization techniques. Students will also present their work at the end of the course and publish a blog post describing their approach and results.

对于本课程项目，学生将使用卷积神经网络在他们选择的真实数据集上创建图像分类模型。该项目将允许学生尝试不同类型的模型和正则化技术。学生还将在课程结束时介绍他们的工作，并发布一篇博客文章，介绍他们的方法和结果。

(Kaggle In-Class Competition)

Students will participate in a private data science competition hosted on the Kaggle platform. The competition will run for 3 weeks, allowing students to apply & improve their skills in a competitive environment. Students will gain exposure to working with cloud GPU platforms.

学生将参加在Kaggle平台上举办的私人数据科学竞赛。比赛将持续3周，让学生在竞争激烈的环境中申请并提高自己的技能。学生将接触到使用云GPU平台的知识。

(Certificate of Completion)

Students who attend at least 5 out of 6 video lectures and make valid submissions for all assignments will be eligible to receive a Certificate of Completion by Jovian.ml. Selected projects will also be receive a Best Project Award based on evaluation criteria determined by the instructors.

参加至少6个视频讲座中的5个并为所有作业提交有效作品的学生将有资格获得Jovian.ml的结业证书。选定的项目还将根据讲师确定的评估标准获得“最佳项目奖”。