LLM-Oracle Machines

Abstract

Contemporary AI applications leverage large language models (LLMs) for their knowledge and inference capabilities in natural language processing tasks. This approach aligns with the concept of oracle Turing machines (OTMs). To capture the essence of these computations, including those desired but not yet in practice, we extend the notion of OTMs by employing a cluster of LLMs as the oracle. We present four variants: basic, augmented, fault-avoidance, and $\epsilon$-fault. The first two variants are commonly observed, whereas the latter two are specifically designed to ensure reliable outcomes by addressing LLM hallucinations, biases, and inconsistencies.