Dev66

i like old-fashioned

Nvidia GPU Cloud

Posted on | 1758 words | ~4 mins
Nvidia GPU

NGC全称为Nvidia GPU Cloud。提供了一组容器镜像，包装了基于GPU的各种软件框架作为运行环境，方便研发和生产使用；同时也提供各种pre-trained的深度学习模型，方便大家学习。

Nvidia GPU Cloud（简称NGC）名字里有个cloud， 听上去是个IaaS或者PaaS类的产品，类似Azure或者AWS。甚至NGC官网介绍中还包含了cloud platform描述。第一次听说这个产品的人，肯定会以为这是Nvidia搭的一个IaaS平台，提供各种GPU服务器的租赁服务。

NVIDIA GPU Cloud (NGC) is a GPU-accelerated cloud platform optimized for deep learning and scientific computing. In this release, NGC includes NGC containers, the NGC container registry, the NGC website, and platform software for running the deep learning containers. This document provides an overview of the NVIDIA GPU Cloud and how to use it.

然而，通读介绍后就会知道NGC是一个“资源库”，包含：

  • NGC Container Registry，容器镜像仓库
  • NGC Container，各种深度学习框架镜像，作为开发和生产环境使用
  • NGC Websites，浏览/搜索镜像，并管理使用这些镜像的api key
  • Model，各种流行的深度学习模型
  • NGC Registry CLI，在命令行中浏览/搜索/使用镜像和模型

NGC并不会有什么服务器给你用。你能用的是各种环境镜像以及pre-trained的深度学习模型。

NGC Container Registry

NGC最重要的功能就是从NGC Container Registry上pull镜像使用。大部分镜像都是无需登录即可pull。有些预览版或者特别要求的镜像需要登录使用。具体方法是要注册一个NGC Website的账号，登录后在Setup页面选择Generate API Key

ngc_setup

在Generate API Key页面点击右上角按钮生成一个key。在NGC网站上貌似没有查询当前key的页面，请注意将key保存在其他地方。否则就只能再次生成新的key，所有使用旧key的机器都要重新登录一遍。

ngc_generate_api_key

拿到key之后，登录到需要pull镜像的服务器完成登录即可。并pull需要的镜像

1$ docker login nvcr.io
2Username: $oauthtoken
3Password: k7cqFTUvKKdiwGsPnWnyQFYGnlAlsCIRmlP67Qxa
4
5$ docker pull nvcr.io/nvidia/caffe:17.04

NGC Registry CLI

NGC Registry CLI是一个CLI工具用于浏览NGC containers，models，model scripts。但是这个cli无法pull containers。pull依旧需要用docker pull。登录NGC Website之后，可以按Setup CLI Install完成安装。安装完成后，执行ngc config set输入API key等信息完成初始化工作。执行ngc registry image list可以看到所有可用的镜像

 1$ ngc registry image list | jq '.[].displayName'
 2"BigDFT"
 3"CANDLE"
 4"CHROMA"
 5"GAMESS"
 6"GROMACS"
 7"HOOMD-blue"
 8"kipoi"
 9"LAMMPS"
10"Lattice Microbes"
11"Microvolution"
12"MILC"
13"NAMD"
14"Parabricks"
15"PGI Compilers"
16"PIConGPU"
17"QMCPACK"
18"RELION"
19"vmd"
20"DeepStream"
21"IndeX"
22"ParaView"
23"ParaView Holodeck"
24"ParaView IndeX"
25"ParaView OptiX"
26"NVCaffe"
27"Caffe2"
28"CT Organ Segmentation AI"
29"Clara Render Server"
30"Clara-Train-SDK"
31"Clara Deploy SDK"
32"Microsoft Cognitive Toolkit"
33"conundrum-aircraftengine-cuda10-pytorch11"
34"CUDA"
35"CUDA GL"
36"DeepStream"
37"DeepStream-l4t"
38"Smart Parking Detection"
39"DIGITS"
40"Inference Server"
41"CUDA Sample"
42"DCGM Exporter"
43"Device Plugin"
44"Kaldi"
45"l4t-base"
46"MXNet"
47"PyTorch"
48"RAPIDS"
49"TensorFlow"
50"TensorRT"
51"TensorRT Inference Server"
52"Theano"
53"Transfer Learning Toolkit for Video Streaming Analytics"
54"Torch"
55"Chainer"
56"Deep Cognition Studio"
57"Dotscience Runner"
58"H2O Driverless AI"
59"Kinetica"
60"OmniSci (MapD)"
61"MATLAB"
62"OmniSci EE"
63"OmniSci Open Source"
64"MapR PACC"
65"PaddlePaddle"

或者ngc registry model list输出所有的模型列表

  1$ ngc registry model list
  2"BERT-Large (pre-training) for TensorFlow"
  3"BERT-Large(pre-training using LAMB optimizer) for TensorFlow"
  4"BERT-Base(fine-tuning) - SQuAD 1.1, seqLen=128"
  5"BERT-Base(fine-tuning) - SQuAD 1.1, seqLen=384"
  6"BERT-Base(fine-tuning) - SQuAD 1.1, seqLen=128"
  7"BERT-Base(fine-tuning) - SQuAD 1.1, seqLen=384"
  8"BERT-Large(fine-tuning) - SQuAD 1.1, seqLen=128"
  9"BERT-Large(fine-tuning) - SQuAD 1.1, seqLen=384"
 10"BERT-Large(fine-tuning) - SQuAD 1.1, seqLen=128"
 11"BERT-Large(fine-tuning) - SQuAD 1.1, seqLen=384"
 12"BERT-Base(fine-tuning) - SQuAD 2.0, seqLen=128"
 13"BERT-Base(fine-tuning) - SQuAD 2.0, seqLen=384"
 14"BERT-Base(fine-tuning) - SQuAD 2.0, seqLen=128"
 15"BERT-Base(fine-tuning) - SQuAD 2.0, seqLen=384"
 16"BERT-Large(fine-tuning) - SQuAD  2.0, seqLen=128"
 17"BERT-Large(fine-tuning) - SQuAD  2.0, seqLen=384"
 18"BERT-Large(fine-tuning) - SQuAD 2.0, seqLen=128"
 19"BERT-Large(fine-tuning) - SQuAD 2.0, seqLen=384"
 20"BERT-Large(pre-training) for TensorFlow"
 21"BERT-Large(pre-training using LAMB optimizer) for Pytorch"
 22"GNMT for PyTorch"
 23"GNMT for PyTorch"
 24"GNMT for TensorFlow"
 25"GNMT for TensorFlow"
 26"TLT AlexNet Classification"
 27"TLT GoogLeNet Classification"
 28"TLT GoogLeNet DetectNet_v2"
 29"TLT GoogLeNet FasterRCNN"
 30"TLT Classification with Resnet18."
 31"TLT MobileNet v1 Classification"
 32"TLT MobileNet v1 DetectNet_v2"
 33"TLT MobileNet v1 FasterRCNN"
 34"TLT MobileNet v2 Classification"
 35"TLT MobileNet v2 DetectNet_v2"
 36"TLT MobileNet v2 FasterRCNN"
 37"TLT ResNet10 Classification"
 38"TLT Resnet10 DetectNet_v2"
 39"TLT ResNet10 FasterRCNN"
 40"TLT ResNet10 SSD"
 41"TLT ResNet18 Classification"
 42"TLT ResNet18 DetectNet_v2"
 43"TLT ResNet18 FasterRCNN"
 44"TLT ResNet18 SSD"
 45"TLT ResNet50 Classification"
 46"TLT ResNet50 DetectNet_v2"
 47"TLT ResNet50 FasterRCNN"
 48"TLT SqueezeNet Classification"
 49"TLT VGG16 Classification"
 50"TLT VGG16 DetectNet_v2"
 51"TLT VGG16 FasterRCNN"
 52"TLT VGG19 Classification"
 53"TLT VGG19 DetectNet_v2"
 54"TLT VGG19 FasterRCNN"
 55"JasperNetDr 10x5 for NeMo"
 56"Jasper for PyTorch"
 57"annotation_ct_colon_tumor"
 58"annotation_ct_hepatic_vessel"
 59"annotation_ct_liver"
 60"annotation_ct_liver_tumor"
 61"annotation_ct_lung_tumor"
 62"annotation_ct_pancreas"
 63"annotation_ct_spleen"
 64"annotation_mri_brain_tumors_t1ce_tc"
 65"annotation_mri_brain_tumors_t2_wt"
 66"annotation_mri_hippocampus"
 67"annotation_mri_left_atrium"
 68"annotation_mri_prostate_cg_and_pz"
 69"classification_chest_xray"
 70"segmentation_ct_colon_tumor"
 71"segmentation_ct_hepatic_vessel_and_tumor"
 72"segmentation_ct_liver_and_tumor"
 73"segmentation_ct_lung_tumor"
 74"segmentation_ct_pancreas_and_tumor"
 75"segmentation_ct_spleen"
 76"segmentation_mri_brain_tumors_br16_full"
 77"segmentation_mri_brain_tumors_br16_t1c2tc"
 78"segmentation_mri_hippocampus"
 79"segmentation_mri_left_atrium"
 80"segmentation_mri_prostate_cg_and_pz"
 81"MaskRCNN for PyTorch"
 82"MaskRCNN for PyTorch"
 83"pix2pixHD"
 84"QuartzNet 15x5 for NeMo"
 85"ResNet v1.5 for MXNet"
 86"ResNet v1.5 for MXNet"
 87"ResNet v1.5 for TensorFlow"
 88"ResNet v1.5 for TensorFlow"
 89"segmentation_ct_lung_tumor"
 90"SSD for PyTorch"
 91"SSD for PyTorch"
 92"Tacotron2 for PyTorch"
 93"Tacotron2 for PyTorch"
 94"TLT Classification with AlexNet"
 95"TLT Classification with Googlenet."
 96"TLT Classification with Resnet18."
 97"TLT Classification with Resnet50."
 98"TLT Classification with VGG16."
 99"TLT Classification with VGG19."
100"TLT Object Detection with Googlenet."
101"TLT Object Detection with Resnet10."
102"TLT Object Detection with Resnet18."
103"TLT Object Detection with VGG16."
104"Transformer-BIG-en-de-8K for NeMo"
105"ONNX InceptionV1 TensorRT 5.0.2 T4 FP16"
106"ONNX InceptionV1 TensorRT 5.0.2 T4 FP32"
107"ONNX InceptionV1 TensorRT 5.0.2 T4 INT8"
108"ONNX InceptionV1 TensorRT 5.0.2 V100-16G FP16"
109"ONNX InceptionV1 TensorRT 5.0.2 V100-16G FP32"
110"ONNX InceptionV1 TensorRT 5.0.2 V100-16G INT8"
111"ONNX MobileNetv2_1.0 TensorRT 5.0.2 T4 FP16"
112"ONNX MobileNetv2_1.0 TensorRT 5.0.2 T4 FP32"
113"ONNX MobileNetv2_1.0 TensorRT 5.0.2 T4 INT8"
114"ONNX MobileNetv2_1.0 TensorRT 5.0.2 V100-16G FP16"
115"ONNX MobileNetv2_1.0 TensorRT 5.0.2 V100-16G FP32"
116"ONNX MobileNetv2_1.0 TensorRT 5.0.2 V100-16G INT8"
117"ONNX ResNet50 TensorRT 5.0.2 T4 FP16"
118"ONNX ResNet50 TensorRT 5.0.2 T4 FP32"
119"ONNX ResNet50 TensorRT 5.0.2 T4 INT8"
120"ONNX ResNet50 TensorRT 5.0.2 V100-16G FP16"
121"ONNX ResNet50 TensorRT 5.0.2 V100-16G FP32"
122"ONNX ResNet50 TensorRT 5.0.2 V100-16G INT8"
123"ONNX VGG16 TensorRT 5.0.2 T4 FP16"
124"ONNX VGG16 TensorRT 5.0.2 T4 FP32"
125"ONNX VGG16 TensorRT 5.0.2 T4 INT8"
126"ONNX VGG16 TensorRT 5.0.2 V100-16G FP16"
127"ONNX VGG16 TensorRT 5.0.2 V100-16G FP32"
128"ONNX VGG16 TensorRT 5.0.2 V100-16G INT8"
129"WaveGlow LJS 256 Channels"
130"Waveglow for PyTorch"
131"Waveglow for PyTorch"

也可以下载一个模型到本地

1$ ngc registry model download-version nvidia/model/trt_onnx_vgg16_v100_16g_int8:1 -d ./models

不过下载镜像依旧要使用docker pull或者nvidia-docker pull来完成

 1$ nvidia-docker run nvidia/cuda:10.0-base-centos7 nvidia-smi
 2Unable to find image 'nvidia/cuda:10.0-base-centos7' locally
 310.0-base-centos7: Pulling from nvidia/cuda
 4d8d02d457314: Pull complete 
 56a0ea995a5ea: Pull complete 
 6dd7361080d9b: Pull complete 
 78fa7f021536d: Pull complete 
 8fb17db70444a: Pull complete 
 9Digest: sha256:52ac5f3ad6873b5b1276509e0bdfbe107565bf731ab22a960c3c7bf42415d020
10Status: Downloaded newer image for nvidia/cuda:10.0-base-centos7
11Sun Oct 20 07:24:15 2019       
12+-----------------------------------------------------------------------------+
13| NVIDIA-SMI 418.87.00    Driver Version: 418.87.00    CUDA Version: 10.1     |
14|-------------------------------+----------------------+----------------------+
15| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
16| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
17|===============================+======================+======================|
18|   0  Tesla P4            On   | 00000000:00:07.0 Off |                    0 |
19| N/A   27C    P8     6W /  75W |      0MiB /  7611MiB |      0%      Default |
20+-------------------------------+----------------------+----------------------+
21                                                                            
22+-----------------------------------------------------------------------------+
23| Processes:                                                       GPU Memory |
24|  GPU       PID   Type   Process name                             Usage      |
25|=============================================================================|
26|  No running processes found                                                 |
27+-----------------------------------------------------------------------------+