Structural instability has been one of the central research questions in economics and finance over many decades. This paper systematically investigates structural instabilities in high dimensional factor models, which portray both structural breaks and threshold effects simultaneously. The observed high dimensional time series are concatenated at an unknown number of break points, while they are described by multiple threshold factor models that are heterogeneous between any two consecutive subsamples. Both joint and sequential procedures for estimating the break points are developed based on the second moment of the pseudo factor estimates that fully ignore the structural instabilities. In each separated subsample, the group Lasso approach recently proposed by Ma and Tu (2023b) is adopted to efficiently identify the threshold factor structure. An information criterion is further proposed to determine the number of break points, which also serves the purpose to distinguish the two types of instabilities. Theoretical properties of the proposed estimators are established, and their finite sample performance is evaluated in Monte Carlo simulations. An empirical application to the U.S. financial market dataset demonstrates the consequences when structural break meets threshold effect in factor analysis.