Combination of actor/critic algorithm with the goal-directed reasoning

We combine an actor/critic algorithm with goal-directed reasoning. There is a claim that the actor/critic algorithm can be a model of the basal ganglia. The basal ganglia seems to contribute to the higher functions such as goal-directed reasoning. Therefore, an important problem is understanding goal-directed reasoning in the framework of the actor/critic algorithm. As the goal-directed reasoning is realized by changing the current goal, we consider changing the goal as an action and incorporate it into the actor/critic algorithm. One fundamental algorithm and its extensions are proposed with simulation results.