Photochromic azobenzenes undergo light-mediated trans–cis isomerization. The cis isomer reverts to the trans isomer thermally. To investigate the effect of different monomer sequences on the thermal stability of cis-azobenzene, a series of copolymers, namely, block, and random structures containing stearic acid and azobenzene moieties as their side chains have been synthesized through reversible addition–fragmentation chain transfer (RAFT) polymerization. In this study, we investigate the photoisomerization of the trans and cis forms of the polymers and also the thermal reversal of the cis-azobenzene photochromic systems. The isomerization data revealed significant differences in the isomerization timescales between the polymers and the corresponding monomer. It was observed that the local polarity around the azobenzene units within a polymer was significantly influenced by the chain segment depending on whether it was in the vicinity of the hydrophobic alkyl chains or other azobenzene units. This local environment of the azobenzene units regulates the stabilization of the transition states during the cis–trans thermal isomerization, consequently affecting the half-life of the cis isomer.