At long last, I present my Scholars project, where I engineered a (somewhat primitive) framework that disentangles data containing many behaviors from different experts to learn to steer a model towards one mode of behavior or another.

As I round the corner of the final days of my project, I am striving to succeed in provably demonstrating its practical utility, simplicity and contributions. The neural architecture I have designed so far, however, is anything but simple.

Over the last two weeks I have been delving into new depths, turning a problem over in my mind for days at a time, without certainty of success. Continuing to probe at an idea in the face of possible failure can be daunting, and I wanted to share some of the tips that helped me overcome the challenges of designing novel solutions.

Reinforcement learning literature involves learning to pursue actions that provide sufficient enough rewards (or minimize the agent’s cost). As we have previously seen, encouraging continuous actions defined as an “environment step” can be tricky because of the credit assignment problem, wherein the learning function must attribute credit for rewards or costs to some actions taken along a trajectory**.

The Expert We trained multiple experts at different thresholds and constraints, but in this report we will discuss a configuration set (alias Marigold), for which we ran multiple smaller cloning experiments.