John, I agree with you about the importance of combining syntax with semantics and pragmatics in order to create adequate models of language, and I agree with your comparison of LLMs with Searle's Chinese Room. But I doubt that even RRG is enough to model human language behavior adequately. Specifically, I don't think it can account for deictic language in conversation. I have written about that elsewhere, as summarized below.
There are several kinds of human language brhavior pragmatically that differ in the nature of the environment(s) in which the speaker and hearer, or writer and reader reside and interact.
First, there is one-way language interaction through language only. That is where the speaker or writer produces utterances or texts without otherwise interacting with the hearer or reader. Stories, other narratives, and poems are examples. Those are the easiest kinds of language production for AI programs to fake, by patching together saved phrases, and possibly the hardest for such programs to really understand, because of the cognitive effort that would require.
Second, there is two-way language interaction in the absence of a shared perceptual environment. Actually, I must immediately modify that description, because the sequence of language acts in a two-way language interaction is always, itself, a shared perceptual environment, or at least a shared experiential environment. The participants can refer to each other's and their own language acts. That is the kind of language interaction that you and I are having over your article. Its depth can range from very superficial, as demonstrated by Weizenbaum's original ELIZA program, to deep, as in my response to your article, John, and your possible response to this comment of mine.
In some previous articles I have described how a computer can be programmed to carry on real conversations with people about food recipes:
https://medium.datadriveninvestor.com/components-of-a-recipe-expert-system-1f038fd0c79b
https://jmason37-80878.medium.com/recipeexpert-as-a-philosophical-thought-experiment-f9d567ca6a4f
Similar systems might be designed to deal with aspects of human law, for example. They require the system to model the pragmatics and semantics of a situation in a fair amount of detail.
Third, there is two-way language interaction in a shared perceptual and (possibly also) action environment. I have also described in some previous articles an example of such an environment in which a computer program could understand human language and use it to interact with people in a significant way:
https://medium.datadriveninvestor.com/a-computer-program-that-exhibits-consciousness-964ab03f61e5
In addition to accurately modeling the pragmatics and semantics of the situation, the computer program must be able to track the short-term memory and attention of its human conversational partner.
Those examples demonstrate that it is not impossible for AI systems to master human language, but it is difficult to create ones that do so, and it will be most feasible if they can be restricted to well-defined pragmatic domains and given sensory organs and effector organs that are suitable to those domains.
