Appearing Legitimate is Not Enough: Interrogating Synthetic Agents in Representational Processes through a Participatory Design Lens

As large language models and synthetic agents increasingly substitute for human participants in research, they are also being extended into high-stakes institutional settings such as policy consultation, enterprise jury deliberation, and global humanitarian diplomacy. In these domains, participation is not merely an informational input or a tool for consensus generation; rather, it is a necessary, legitimizing condition of democratic processes whose authority derives from procedural inclusion rather than outcome correctness. Treating synthetic agents as human substitutes raises profound political, representational, and ethical concerns. This paper investigates how synthetic agents manufacture the appearance of legitimate participation across different scales, and asks what boundaries should guide their design and deployment.

The authors conduct a comparative case study of three synthetic agents deployed at varying representational scales: local policy consultation (Ana, a community-voice chatbot in Pittsburgh), enterprise legal trial preparation (Synthetic Juror), and global humanitarian diplomacy (Ask Amina and Ask Abdalla, developed by the United Nations University). Using a close reading of publicly available project documentation, technical reports, and secondary coverage, the authors trace how personhood is produced. They complement their findings by drawing on Sanders et al.’s Participatory Design framework—specifically the modes of probing, priming, understanding, and generating—to build a process-oriented critique that contrasts with traditional outcome-centric evaluations like believability, faithfulness, and algorithmic fidelity.

Across the three cases, the authors identify a four-step process through which synthetic agents manufacture personhood: problem framing, institutionalized curation of data, design of user encounters with the persona, and technical evaluation of validity. This sequence produces what the authors call manufactured personhood—an artificial appearance of legitimate representation that bypasses the actual lived experiences of marginalized groups who would otherwise participate. Standard evaluation frameworks fail to catch this bypass because they measure surface-level output similarity rather than the integrity of the underlying processes. Furthermore, synthetic agents lack consciousness, intentionality, and the capacity for reciprocal moral relations, rendering them fundamentally unsuited to serve as moral or legal actors in deliberative institutions.