python memray的使用

1.下载发行版本21

Unity-Technologies/ml-agents at release_21 (github.com)

2.安装python和pytorch

Conda python 设置

在系统中安装 conda 后，打开终端并执行以下命令以设置 python 3.10.12 虚拟环境 并激活虚拟环境。

conda create -n mlagents python=3.10.12 && conda activate mlagents

安装 PyTorch

在 Windows 上，必须先单独安装 PyTorch 包，然后才能安装 安装 ML-Agents 以确保使用启用了 cuda 的版本， 而不是仅 CPU 版本。激活您的虚拟环境并从 命令行输入以下安装代码：

pip3 install torch~=2.2.1 --index-url https://download.pytorch.org/whl/cu121

 检查下载的版本

(mlagents) D:\intelligent_grasp\ml-agents-release_21>python
Python 3.10.12 | packaged by Anaconda, Inc. | (main, Jul  5 2023, 19:01:18) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.__version__)
2.2.2+cu121
>>>

3.安装mlagents

需要在python环境里面去安装ml-agents-envs   ml-agents

(所以要激活创建的虚拟环境）

要安装 Python 包，请激活虚拟环境并 从命令行运行：mlagents：

cd /path/to/ml-agents
python -m pip install ./ml-agents-envs
python -m pip install ./ml-agents

请注意，这将从克隆的存储库安装 ，而不是从PyPi安装 存储 库。

注意：切换盘

如果是要换到另一个盘就是 比如从C盘换到D盘 [ 如果你下载的的源码  （源码就是你下载 ml-agents-release_21）    的不在你当前打开命令行的盘里面 需要先换盘]

 输入 D:   回车就是换盘

然后 cd D:\intelligent_grasp\ml-agents-release_21>

到达对应下载的文件下就是cd /path/to/ml-agents 这个的意思

完整的步骤

（1）打开终端/命令行窗口（这与前面相同）

Copy code：

activate malgents

（2）输入命令并执行:

Copy code：

cd D:\intelligent_grasp\ml-agents-release_21

这将把您当前的工作目录切换到D:\intelligent_grasp\ml-agents-release_21

（1）然后在该目录下执行pip install命令安装ml-agents及其依赖:

Copy code：

python -m pip install ./ml-agents-envs

python -m pip install ./ml-agents

确保您已经正确设置并激活了Python虚拟环境。这两条pip install命令会从本地的ml-agents-release_21目录下安装所需的Python包到您的虚拟环境中。

切换到正确的目录是安装本地包的关键一步,因为pip install命令中的./表示在当前目录下查找需要安装的包。所以先切换目录,再执行后续的安装命令即可。

注意：依赖项下载

对应的下载时遇到的错误 这里提示有一些依赖项没有下载

Installing collected packages: numpy

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.

mlagents-envs 0.28.0 requires grpcio>=1.11.0, which is not installed.

mlagents-envs 0.28.0 requires Pillow>=4.2.1, which is not installed.

mlagents-envs 0.28.0 requires protobuf>=3.6, which is not installed.

tensorboard 2.16.2 requires grpcio>=1.48.2, which is not installed.

tensorboard 2.16.2 requires protobuf!=4.24.0,>=3.19.6, which is not installed.

tensorboard 2.16.2 requires six>1.9, which is not installed.

Successfully installed numpy-1.23.3

(mlagents) D:\intelligent_grasp\ml-agents-release_21>

根据错误信息,虽然numpy已经成功安装,但是在安装mlagents-envs和tensorboard等其他依赖包时,出现了一些依赖项缺失或版本冲突的问题。

具体来说:

1. mlagents-envs需要grpcio>=1.11.0、Pillow>=4.2.1和protobuf>=3.6,但是它们似乎没有被安装。
2. tensorboard需要grpcio>=1.48.2、protobuf!=4.24.0且>=3.19.6、six>1.9,但是它们也没有被安装或存在版本冲突。

为了解决这些依赖问题,您可以尝试以下步骤:

1. 先卸载已安装的numpy:

Copy code：

pip uninstall numpy

1. 然后使用pip一次性安装所有需要的依赖包:

Copy code：

pip install grpcio pillow protobuf six mlagents-envs tensorboard

这将安装所有缺失的依赖包,并处理好版本冲突的问题。

1. 最后再重新安装numpy:

Copy code：

pip install numpy

通过这种安装顺序,应该可以解决当前的依赖问题,使所有需要的包都正确安装并且版本一致。

安装依赖项

由于 ML-Agents 开发速度放缓，因此 PyPi 的发布频率将降低。但是，您可以通过执行 PyPi 进行安装 以下命令：

python -m pip install mlagents==1.0.0

它将安装最新版本的 ML-Agents 和 PyPi 上可用的相关依赖项。请注意，您需要 具有匹配的版本 带有特定版本的 python 包的 Unity 包。您可以在此处找到发布历史记录

通过安装包，还会安装 setup.py 文件中列出的依赖项。其中包括 PyTorch

numpy版本与python3.10 不兼容

还有就是我安装的是ml-agents-release_21  然会这里会遇到numpy版本与python3.10 不兼容

安装python -m pip install ./ml-agents-envs遇到的问题 在论坛已找到答案

Bumped numpy version. (#5997) · Unity-Technologies/ml-agents@f3dc8f6 (github.com)

就是需要将ml-agents-release_21 里面 相应的 set.up的文件里面的numpy的版本该成

"numpy>=1.21.2,<1.24.0",  等等

并且这个问题已在develop版本里面修改了

 3.判断是否下载成功：

mlagents-learn --help

(mlagents) D:\intelligent_grasp\ml-agents-release_21>mlagents-learn --help
C:\Annoconda\annoconda3\envs\mlagents\lib\site-packages\torch\__init__.py:696: UserWarning: torch.set_default_tensor_type() is deprecated as of PyTorch 2.1, please use torch.set_default_dtype() and torch.set_default_device() as alternatives. (Triggered internally at ..\torch\csrc\tensor\python_tensor.cpp:453.)
  _C._set_default_tensor_type(t)
usage: mlagents-learn.exe [-h] [--env ENV_PATH] [--resume] [--deterministic] [--force] [--run-id RUN_ID]
                          [--initialize-from RUN_ID] [--seed SEED] [--inference] [--base-port BASE_PORT]
                          [--num-envs NUM_ENVS] [--num-areas NUM_AREAS] [--debug] [--env-args ...]
                          [--max-lifetime-restarts MAX_LIFETIME_RESTARTS]
                          [--restarts-rate-limit-n RESTARTS_RATE_LIMIT_N]
                          [--restarts-rate-limit-period-s RESTARTS_RATE_LIMIT_PERIOD_S] [--torch] [--tensorflow]
                          [--results-dir RESULTS_DIR] [--timeout-wait TIMEOUT_WAIT] [--width WIDTH] [--height HEIGHT]
                          [--quality-level QUALITY_LEVEL] [--time-scale TIME_SCALE]
                          [--target-frame-rate TARGET_FRAME_RATE] [--capture-frame-rate CAPTURE_FRAME_RATE]
                          [--no-graphics] [--torch-device DEVICE]
                          [trainer_config_path]

positional arguments:
  trainer_config_path

options:
  -h, --help            show this help message and exit
  --env ENV_PATH        Path to the Unity executable to train (default: None)
  --resume              Whether to resume training from a checkpoint. Specify a --run-id to use this option. If set,
                        the training code loads an already trained model to initialize the neural network before
                        resuming training. This option is only valid when the models exist, and have the same behavior
                        names as the current agents in your scene. (default: False)
  --deterministic       Whether to select actions deterministically in policy. `dist.mean` for continuous action
                        space, and `dist.argmax` for deterministic action space (default: False)
  --force               Whether to force-overwrite this run-id's existing summary and model data. (Without this flag,
                        attempting to train a model with a run-id that has been used before will throw an error.
                        (default: False)
  --run-id RUN_ID       The identifier for the training run. This identifier is used to name the subdirectories in
                        which the trained model and summary statistics are saved as well as the saved model itself. If
                        you use TensorBoard to view the training statistics, always set a unique run-id for each
                        training run. (The statistics for all runs with the same id are combined as if they were
                        produced by a the same session.) (default: ppo)
  --initialize-from RUN_ID
                        Specify a previously saved run ID from which to initialize the model from. This can be used,
                        for instance, to fine-tune an existing model on a new environment. Note that the previously
                        saved models must have the same behavior parameters as your current environment. (default:
                        None)
  --seed SEED           A number to use as a seed for the random number generator used by the training code (default:
                        -1)
  --inference           Whether to run in Python inference mode (i.e. no training). Use with --resume to load a model
                        trained with an existing run ID. (default: False)
  --base-port BASE_PORT
                        The starting port for environment communication. Each concurrent Unity environment instance
                        will get assigned a port sequentially, starting from the base-port. Each instance will use the
                        port (base_port + worker_id), where the worker_id is sequential IDs given to each instance
                        from 0 to (num_envs - 1). Note that when training using the Editor rather than an executable,
                        the base port will be ignored. (default: 5005)
  --num-envs NUM_ENVS   The number of concurrent Unity environment instances to collect experiences from when training
                        (default: 1)
  --num-areas NUM_AREAS
                        The number of parallel training areas in each Unity environment instance. (default: 1)
  --debug               Whether to enable debug-level logging for some parts of the code (default: False)
  --env-args ...        Arguments passed to the Unity executable. Be aware that the standalone build will also process
                        these as Unity Command Line Arguments. You should choose different argument names if you want
                        to create environment-specific arguments. All arguments after this flag will be passed to the
                        executable. (default: None)
  --max-lifetime-restarts MAX_LIFETIME_RESTARTS
                        The max number of times a single Unity executable can crash over its lifetime before ml-agents
                        exits. Can be set to -1 if no limit is desired. (default: 10)
  --restarts-rate-limit-n RESTARTS_RATE_LIMIT_N
                        The maximum number of times a single Unity executable can crash over a period of time (period
                        set in restarts-rate-limit-period-s). Can be set to -1 to not use rate limiting with restarts.
                        (default: 1)
  --restarts-rate-limit-period-s RESTARTS_RATE_LIMIT_PERIOD_S
                        The period of time --restarts-rate-limit-n applies to. (default: 60)
  --torch               (Removed) Use the PyTorch framework. (default: False)
  --tensorflow          (Removed) Use the TensorFlow framework. (default: False)
  --results-dir RESULTS_DIR
                        Results base directory (default: results)
  --timeout-wait TIMEOUT_WAIT
                        The period of time to wait on a Unity environment to startup for training. (default: 60)

Engine Configuration:
  --width WIDTH         The width of the executable window of the environment(s) in pixels (ignored for editor
                        training). (default: 84)
  --height HEIGHT       The height of the executable window of the environment(s) in pixels (ignored for editor
                        training) (default: 84)
  --quality-level QUALITY_LEVEL
                        The quality level of the environment(s). Equivalent to calling QualitySettings.SetQualityLevel
                        in Unity. (default: 5)
  --time-scale TIME_SCALE
                        The time scale of the Unity environment(s). Equivalent to setting Time.timeScale in Unity.
                        (default: 20)
  --target-frame-rate TARGET_FRAME_RATE
                        The target frame rate of the Unity environment(s). Equivalent to setting
                        Application.targetFrameRate in Unity. (default: -1)
  --capture-frame-rate CAPTURE_FRAME_RATE
                        The capture frame rate of the Unity environment(s). Equivalent to setting
                        Time.captureFramerate in Unity. (default: 60)
  --no-graphics         Whether to run the Unity executable in no-graphics mode (i.e. without initializing the
                        graphics driver. Use this only if your agents don't use visual observations. (default: False)

Torch Configuration:
  --torch-device DEVICE
                        Settings for the default torch.device used in training, for example, "cpu", "cuda", or
                        "cuda:0" (default: None)

(mlagents) D:\intelligent_grasp\ml-agents-release_21>
(mlagents) D:\intelligent_grasp\ml-agents-release_21>