The link between E. coli strains contaminating foods and human disease is unclear, with some reports supporting a direct transmission of pathogenic strains via food and others highlighting their role as reservoirs for resistance and virulence genes. Here we take a genomics approach, analyzing a large set of fully-assembled genomic sequences from E. coli available in GenBank. Most of the strains isolated in food are more closely related to each other than to clinical strains, arguing against a frequent direct transmission of pathogenic strains from food to the clinic. We also provide strong evidence of genetic exchanges between food and clinical strains that are facilitated by plasmids. This is based on an overlapped representation of virulence and resistance genes in plasmids isolated from these two sources. We identify clusters of phylogenetically-related plasmids that are largely responsible for the observed overlap and see evidence of specialization, with some food plasmid clusters preferentially transferring virulence factors over resistance genes. Consistent with these observations, food plasmids have a high mobilization potential based on their plasmid taxonomic unit classification and on an analysis of mobilization gene content. We report antibiotic resistance genes of high clinical relevance and their specific incompatibility group associations. Finally, we also report a striking enrichment for adhesins in food plasmids and their association with specific IncF replicon subtypes. The identification of food plasmids with specific markers (Inc and PTU combinations) as mediators of horizontal transfer between food and clinical strains opens new research avenues and should assist with the design of surveillance strategies.