Implicit elasticity presents the general response of materials without imposing assumptions at the fundamental level. A popular implausible assumption of continuum mechanics is that the reference configuration is stress-free, since residual stress is ubiquitous in Nature. This paper develops large and small deformation implicit elasticity frameworks using residually stressed reference configurations. The general forms of constitutive relations, in finite deformations, are obtained by pull-back or push-forward of all the associated tensors to the same (Eulerian or Lagrangian) configuration. These general forms are used to study the relationship between “residual stress and material symmetry” for implicit elasticity. Further, we use a virtual stress-free body, which is implicit elastic, to exactly determine the response of an initially stressed reference configuration. A number of such exact implicit relations are presented for residually stressed reference configurations, which are further simplified through interesting tensor analysis. The simplified implicit relations directly evaluates strain from a given Cauchy stress and residual stress tensor. One of these constitutive relations are employed for investigating the finite inflation of a residually-stressed, thick sphere. Finally, a small deformation implicit theory is attained by linearizing the developed relations for small strain and small rotation. To represent the small strain from a stressed reference, we need to invert a fourth order tensor. The closed-form inverse is determined in a new approach presented in the paper.