With increasing interest to replace dairy proteins in food products, questions of performance and stability of these alternatives rise. Plant-based proteins and carbohydrates that are used for dried emulsion systems, like plant-based milk or creamer powder, are expected to influence micro structure of final powders with further impact on storage stability. The aim of this work was to investigate the plant-based matrix, with focus on pea protein content and carbohydrate molecular weight, on spray-dried powder structure, fat distribution and oxidation stability of the bioactive component beta carotene. Spray-dried powders were produced from plant-based emulsions with varying protein content (2.4, 20% wt) and maltodextrin dextrose equivalent (DE 6, 21, 40). Both factors significantly impacted particle structure and free fat content of spray-dried powders. While the effect on structure and morphology did not follow clear trends, free fat differed greatly between powders (4.5–88.8%) and showed to increase with protein content and decreased maltodextrin DE. Beta carotene stability during storage followed similar trends, with highest retention after storage measured for coarse, low protein and maltodextrin DE 40 powder (37.9%), while poorest performance was found for powders with maltodextrin DE 6 (9.5–12.3%), independent of the protein content. The study showcases the high impact of carbohydrate molecular weight and plant proteins on structure and thus stability indicators of plant-based powders. For a sustainable substitution of dairy protein in food powders, these differences need to be considered during processing and production.