[Solved]: What’s the difference between Adaptive Control and Hierarchical Reinforcement Learning?

The difference between the two tasks really comes down to the level of continuity assumed in the models of the problem. In adaptive control, continuity is assumed at all levels; the problem space and the actions to be executed are all continuous. In hierarchical reinforcement learning, although the problem space is continuous, the actions to execute upon the space are discrete. A good analogy for this is the robot arm trying to move to a certain goal. Adaptive control would work with Jacobian to find a continuous solution to this problem. Reinforcement learning would select actions (up, down, left, right) to learn the problem space and find a solution. Hierarchical reinforcement learning and adaptive control are really complements of each other. When the problem gets too complex to compute with Adaptive Control, hierarchical learning steps in to make it discrete and more scalable. For another perspective, especially since my explanation breaks down when you consider continuous HRL (which does exist), consider this explanation from my personal correspondence with Daniel Rasumussan:

Practically, there are many differences between how they [Adaptive Control and HRL in a continuous space] are implemented; they use different algorithms, different learning rules, different representations, etc. For example, adaptive control is not typically hierarchical. You could arguably call it a hierarchy with a depth of 1 (with e.g. the “high level” being control in the x,y space and “low level” being control in the motor space). But there’s no temporal abstraction in that hierarchy; the high level isn’t trying to encapsulate low level decisions, the problem is how to find a nice reliable one-to-one mapping between those two spaces. Of course you could make adaptive control hierarchical, as you can make pretty much anything hierarchical, but that isn’t typically what they’re interested in. Perhaps a more critical difference is that in motor control typically it is assumed that the transition and reward functions are known to the agent. There may be some unknown variability in the case of adaptive control, but by and large the agent has a pretty good idea what the effects of its actions will be, and what the goal is that it is trying to achieve. It’s one of the key principle of reinforcement learning that that information is not known, and must be figured out by the agent by exploring the environment. Now again here, you could come up with a motor control problem where the agent doesn’t assume that information is known, but that just isn’t the kind of problem those techniques are designed for, in general. So, to sum up, there certainly is a gradient between HRL and adaptive control, and by tweaking various assumptions you can bring them closer together or farther apart. But just implementing continuous actions in HRL would still leave you quite a ways from adaptive control.

As for an example combination this, one might imagine using hierarchical reinforcement learning to play a video game like Pong and then changing the physics of the game (Pong on slippery ice!) so that Adaptive Control might step in.