Vanishing point detection is an important component of vision based autonomous navigation for unmanned vehicles. Due to the lack of clear road lines and complex background interference in unstructured scenes, existing detection methods generally have the drawbacks of low accuracy and long calculation time. Therefore, a vanishing point detection method combining stereo vision is proposed based on the characteristics of unstructured roads. Using binocular stereo vision technology to obtain disparity maps of road images, and using breadth first algorithm to quickly estimate the background area of the road image; Design a four sided five scale Gabor filter bank to estimate the pixel response amplitude, and reduce detection errors through amplitude correction; Design a series of voting point selection strategies based on the background area to eliminate interference in the background area and improve algorithm accuracy; Adopting a strategy of dynamically adjusting the range of candidate points to reduce the search range of vanishing points, thereby improving algorithm efficiency; We have designed an angle first voting function that treats candidate points that obtain the maximum number of votes in the voting space as vanishing points. The results show that the improved method has good robustness in complex background interference scenarios, and has significant improvements in detection speed and accuracy.