DSAC (Distributional Soft Actor-Critic)
I proposed an RL algorithm, named DSAC, to tackle the overestimation problem encountered by most model-free off-policy RL algorithms. It achieves state-of-the-art (SOTA) performance in most MuJoCo benchmarks, surpassing other model-free RL baselines such as SAC, TD3, PPO.
GOPS (General Optimal control Problems Solver)
I am currently serving as the co-leader in developing GOPS. Solving optimal control problems serves as basic demands of industrial control tasks. Existing methods like model predictive control often suffer from heavy online computational burdens. Reinforcement learning (RL) has shown great promise in computer and board games but has yet to be widely adopted in industrial applications due to lacking accessible and high-accuracy solvers. Therefore, “Intelligent Driving Lab (iDLab)” at Tsinghua University has developed GOPS, an easy-to-use RL solver package that aims to build real-time and high-performance controllers in industrial fields. GOPS is built with a highly modular structure that retains a flexible framework for secondary development. Considering the diversity of industrial control tasks, GOPS also includes a conversion tool that allows for the use of Matlab/Simulink to support environment construction, controller design, and performance validation. To handle large-scale control problems, GOPS can automatically create various serial and parallel trainers by flexibly combining embedded buffers and samplers. It offers a variety of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, convolutional neural network, etc. Additionally, constrained and robust training algorithms for special industrial control systems with state constraints and model uncertainties are also integrated into GOPS.