Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling

As large language models (LLMs) are increasingly integrated into decision-making and advisory roles, understanding their behavioral regularities is critical. This study investigates whether standardized psychological instruments—typically designed for humans—can elicit reproducible, interpretable, and model-specific behavioral signatures from LLMs. The researchers evaluated nine contemporary LLMs from both international and China-based providers using seven psychological instruments covering personality, moral foundations, cognitive styles, and social orientations. Each model was tested in both English and Chinese across five independent administrations, with the study design specifically retaining both valid scale responses and non-answer (NA) responses to capture the boundaries of model self-reporting.

The study reveals that LLMs possess distinct, model-specific psychometric profiles. While all models share a common alignment-shaped pattern—characterized by high prosocial and self-regulatory responses and low endorsement of dominance, moral disengagement, and harmful intent—they remain clearly distinguishable. These differences are not confined to a single domain but are distributed across personality, moral, and cognitive dimensions.

Crucially, the researchers found that NA responses are not random; they are systematic and provide a valuable signal regarding the boundaries of synthetic self-report. Models frequently abstained from answering items requiring deep affective mapping or social self-identification. Furthermore, language and provider origin were found to influence both profile configuration and answerability. Despite these shifts, the models demonstrated high reproducibility, allowing researchers to recover model identity from both scored and NA-response patterns with high accuracy. Comparisons with human data indicate that LLMs occupy restricted, model-specific regions of the human response space rather than mimicking human personality distributions.

This framework provides a quantitative method for auditing and comparing LLM behavior at the deployment level. By treating psychometric profiling as a tool for characterizing response tendencies rather than inferring internal psychological states, this approach offers a robust way to monitor model alignment, track longitudinal changes across versions, and evaluate how different models navigate value-laden or socially sensitive requests. It highlights that behavioral signatures are context-dependent, emphasizing the need for transparency in administration protocols when evaluating AI systems.