Exposure is Optional: Learning Unlike Coordination in Language Models

This paper investigates a long-standing debate in theoretical linguistics: the Law of Coordination of Likes (LCL), which posits that only syntactically identical constituents can be conjoined. While natural language contains many counterexamples—known as "unlike coordination" (e.g., coordinating a noun phrase with a clause)—it remains unclear whether this ability is innate or learned. The authors use language models (LMs) as a computational testbed to determine if direct exposure to unlike coordination is necessary for acquisition or if it emerges organically from general compositional abilities.

To test this, the researchers employed Filtered-Corpus Training (FiCT). They trained GPT-2 models from scratch on three distinct corpora: a baseline (unfiltered) corpus, and two filtered corpora where all instances of unlike coordination (or specifically those using the conjunction "and") were systematically removed. By comparing the performance of these models on targeted grammaticality tests and analyzing their internal representations, the authors isolated the conditions required for learning these structures.

The results demonstrate that direct exposure to unlike coordination is not necessary for a model to master it. Models trained on filtered data achieved perplexity and grammaticality judgment scores comparable to those trained on unfiltered text. The authors found that LMs process unlike coordination using two primary mechanisms: treating conjoined elements as belonging to a shared "supercategory" or employing a process akin to deletion. Both mechanisms appear to be learnable from exposure to alike coordination alone, suggesting that the LCL is not a hard constraint that models must be explicitly taught to violate.

This work provides empirical evidence that complex linguistic structures can emerge from simpler, more general patterns. By showing that models can generalize to unlike coordination without seeing it during training, the authors argue that specialized grammatical rules or innate constraints—often proposed in theoretical linguistics to explain these structures—may be unnecessary. The study also highlights the utility of FiCT as a methodology for testing hypotheses about the origins of linguistic competence in neural networks.