A two-phase flow model accelerated by graphical processing unit (GPU) is developed to solve fluid-solid interaction (FSI) using the sharp-interface immersed boundary method (IBM). This model solves the incompressible Navier-Stokes equations using the projection-based fractional step method in a fixed staggered Cartesian grid system. A volume of fluid (VOF) method with second-order accuracy is employed to trace the free surface. To represent the intricate surface geometry, the structure is discretized using the unstructured triangle mesh. Additionally, a ray tracing method is employed to classify fluid and solid points. A high-order stable scheme has been introduced to reconstruct the local velocity at interface points. Three FSI problems, including wave evolution around a breakwater, interaction between a periodic wave train and a moving float, and a 3-D moving object interacting with the free surface, were investigated to validate the accuracy and stability of the proposed model. The numerical results are in good agreement with the experimental data. Additionally, we evaluated the computational performance of the proposed GPU-based model. The GPU-based model achieved a 42.29 times speedup compared with the single-core CPU-based model in the three-dimension test. Additionally, the results regarding the time cost of each code section indicate that achieving more significant acceleration is associated with solving the turbulence, advection, and diffusion terms, while solving the pressure Poisson equation (PPE) saves the most time. Furthermore, the impact of grid number on computational efficiency indicates that as the number of grids increases, the GPU-based model outperforms the multi-core CPU-based model.