PPO algorithm clarification

[unity_Z5c3tcfoTCizWA]

Hi All,

I have tried to find out what type of PPO algorithm is used in ML-Agents.
According to OpenAI there are two types of PPO algorithms, namely PPO-Penalty and PPO-Clip
(Proximal Policy Optimization — Spinning Up documentation)
Going through the trainer.py and optimizer_torch.py files in this location: ml-agents\mlagents\trainers\ppo
hinted towards PPO-Clip, but I wanted to confirm this.

So is the PPO used in ML-Agents PPO-Clip?

Thanks a lot,
Kindest,
Philipp

[aakarshanc01]

It is PPO-Clip I believe. I went through the utils and found this in the trust_region_policy_loss which is used to calculate loss for PPO and POCA.