NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA
driver. Make sure that the latest NVIDIA driver is installed and
running.

英伟达驱动掉了感觉很常见
显卡刚训练完就关机,大概率会出现这个问题

1.重装驱动就可以了

or

2.降低内核版本