Fish-like robots have become underwater vehicles with broad application prospects, owing to their flexibility and adaptability. Inspired by the swimming motion of fish, an undulating fin with composite motion is proposed in this study. The composite motion is a combination of oscillation and pitch perpendicular to and parallel to the horizontal direction, respectively. First, the kinematic and dynamic modes of the motion mechanism in the free propulsive mode are established with variable locomotive parameters, such as the oscillating frequency and deflection angle. The hydrodynamic performance of the undulating fin is then investigated through analyses of the force mechanism, propulsive velocity, non-dimensional number, and vortex flow using the dynamic mesh method. The energy efficiency of the composite motion of undulating fin is compared with that of a simple oscillating motion. Finally, experimental measurements of an undulating-fin robot prototype are conducted. The results show that the underwater robot accelerates to a maximum velocity of 0.54 m/s with a frequency of 2.5 Hz. Owing to the composite motion, alternating high and low-velocity regions occur in the surrounding fluid of the undulating fin. The highest energy efficiency is 0.88 with a frequency of 2.0 Hz, and the maximum improvement is 10.1% compared with the simple oscillating motion. A streamwise central jet is formed with vortices shedding into the wake flow, which generates a thrust force with two peaks and troughs in one cycle. The negative and positive vorticities correspond to the alternation of the pressure distribution. The experimental measurements demonstrate the propulsive velocity and force are consistent with the numerical simulation. The composite motion with a higher Strouhal number reflects the considerable instability of the flow.