Can art improve health and wellbeing? The claim that there is strong evidence that engaging with art ameliorates symptoms of mental and physical disorders and increases wellbeing is gaining acceptance among researchers and clinicians. This claim deserves thorough scrutiny, as it is used to justify a broad range of arts-based clinical interventions and health policies. Here we show that the evidence cited in favor of the efficacy of arts-based interventions is far weaker than it is claimed to be. First, we examined the methodological and statistical quality of studies that have been cited as proof for the efficacy of arts-based interventions. This analysis found that many of these studies lack key clinical trial features, such as defining the therapeutic agent, randomizing group assignment, controlling for patient or researcher allegiance, controlling for the effects of other concurrent interventions and medications, comparing art-based interventions to other kinds of interventions, or conducting and reporting statistical analyses appropriately. Second, in a broader examination of experiments on arts-based interventions, we looked for the experimental designs that would actually allow demonstrating that the putative health benefits owe to the effect of art. This analysis revealed that (i) most studies fail to define what art is, making it impossible to compare the effects of “art” and “non-art” stimuli and activities on health and wellbeing; (ii) fail to demonstrate that art stimuli and activities elicit a distinct class of art-induced physiological processes capable of modulating the cause of targeted disorders; (iii) and fail to manipulate neural processes believed to be mechanisms of action that could prove that arts-based interventions directly affect the etiology of disorders. These methodological weaknesses and inappropriate experimental designs cast serious doubt on claims that engaging with art can induce physiological changes that improve health and wellbeing. We discuss why arts-based interventions have neglected these problems and the ethical implications for patients who are treated with them.