In this paper, we investigate variable past inflection in four New Englishes. Our data are drawn from the conversational parts of the Hong Kong, India, Jamaica, and Philippine subcomponents of the International Corpus of English. We investigate the entire range of language-internal factors that have been found to influence non-obligatory past marking in varieties of English. This includes morpho-phonological verb class, lexical aspect, grammatical aspect, marker persistence, the presence or absence of a temporal adverbial, and, for consonant-final regular verbs, preceding and following phonological environment. We also consider verb frequency, which has received only scant attention in past inflection research so far. Employing both mixed-effects regression and random forests, we argue that, despite inter-variety differences, there is a core grammar of past inflection, which is constrained by general structural and cognitive phenomena such as grammatical aspect and marker persistence, with frequency also exerting an important and consistent effect. This has implications for debates about universals vs. substrate influence or creole effects in morphosyntactic variation in English.