Do large language models “understand” their knowledge?

Abstract

Large language models (LLMs) are often criticized for lacking true “understanding” and the ability to “reason” with their knowledge, being seen merely as autocomplete engines. I suggest that this assessment might be missing a nuanced insight. LLMs do develop a kind of empirical “understanding” that is “geometry”-like, which is adequate for many applications. However, this “geometric” understanding, built from incomplete and noisy data, makes them unreliable, difficult to generalize, and lacking in inference capabilities and explanations. To overcome these limitations, LLMs should be integrated with an “algebraic” representation of knowledge that includes symbolic AI elements used in expert systems. This integration aims to create large knowledge models (LKMs) grounded in first principles that can reason and explain, mimicking human expert capabilities. Furthermore, we need a conceptual breakthrough, such as the transformation from Newtonian mechanics to statistical mechanics, to create a new science of LLMs.