Though the Moon does not possess a global magnetic field like the Earth, there are localized crustal magnetic fields on the lunar surface. Because of the plasma interaction with the crustal magnetic fields, electrostatic and electromagnetic environments near magnetized regions can differ from those near non-magnetized regions on the Moon. Previous studies observationally revealed the difference in the electrostatic potential on the lunar surface between magnetized and non-magnetized regions of the Moon in the solar wind, which was attributed to upward electric fields formed by electron-ion decoupling above the magnetic anomaly regions. However, these inhomogeneous distributions of surface potentials associated with lunar crustal magnetic fields remain uncharacterized in plasma regimes different from the solar wind. In this study, we use a large number of observations by Kaguya and a numerical model of photoelectrons emitted from the sunlit lunar surface to investigate the horizontal distributions of the lunar surface potential in the terrestrial magnetotail lobes. We estimate the relative surface potential variations from the measured energy shift of lunar surface photoelectrons. The results indicate that photoelectrons emitted from relatively strong crustal magnetic field regions tend to be more decelerated, suggesting more positive potentials on the magnetized surface. This implies that upward electric fields are formed by the interaction of terrestrial magnetotail plasma with the lunar crustal magnetic fields in a similar manner to the solar wind interaction with lunar crustal magnetic fields.