Malliavin calculus used to derive a stochastic maximum principle for system driven by fractional Brownian and standard Wiener motions with application

Abstract We consider a stochastic control problem in the case where the set of the control domain is convex, and the system is governed by fractional Brownian motion with Hurst parameter H ∈ ( 1 2 , 1 ) {H\in(\frac{1}{2},1)} and standard Wiener motion. The criterion to be minimized is in the general form, with initial cost. We derive a stochastic maximum principle of optimality by using two famous approaches. The first one is the Doss–Sussmann transformation and the second one is the Malliavin derivative.