Enhanced and Efficient Reasoning in Large Language Models

In current Large Language Models the production of smoothly flowing prose is understandable in terms of the principles of machine learning. However, there is no comparable principled basis to justify trust in the content of the text produced. The widely recognized phenomenon of hallucinations is just one manifestation of this situation. While instances may be detected and corrected where alternative methods of verification exist, the pervasiveness of hallucinations where those precautions are not taken leads one to wonder about the trustworthiness of outputs in general. It appears to be conventional wisdom that addressing this issue by adding more principled reasoning to large language models is not computationally feasible and hence not an appropriate goal for technology.

Here we describe some new results that indicate, to the contrary, that one can enhance current language models with more principled reasoning, even while retaining much of the current software and hardware base. The method can be interpreted as providing more of a world model, where the individual objects and relations of the world are represented as first-class entities, and not treated indirectly and impressionistically as is current practice. On this model learned or programmed rules can be chained with certain soundness guarantees.