COVID-19 has ravaged Brazil, and its spread showed spatial heterogeneity. Changes in the environment have been implicated as potential factors involved in COVID-19 transmission. However, considerable research efforts have not elucidated the risk of environmental factors on COVID-19 transmission from the perspective of infectious disease dynamics. The aim of this study is to model the influence of the environment on COVID-19 transmission and to analyze how the socio-ecological factors affecting the probability of virus transmission in 10 states dramatically shifted during the early stages of the epidemic in Brazil. First, this study used a Pearson correlation to analyze the interconnection between COVID-19 morbidity and socio-ecological factors and identified factors with significant correlations as the dominant factors affecting COVID-19 transmission. Then, the time-lag effect of dominant factors on the morbidity of COVID-19 was investigated by constructing a distributed lag nonlinear model and standard two-stage meta-analytic model, and the results were considered in the improved SEIR model. Lastly, a machine learning method was introduced to explore the nonlinear relationship between the environmental propagation probability and socio-ecological factors. By analyzing the impact of environmental factors on virus transmission, it can be found that population mobility directly caused by human activities had a greater impact on virus transmission than temperature and humidity. The heterogeneity of meteorological factors can be accounted for by the diverse climate patterns in Brazil. The improved SEIR model was adopted to explore the interconnection of COVID-19 transmission and the environment, which revealed a new strategy to probe the causal links between them.