Measuring Message Credibility: Construction and Validation of an Exclusive Scale

Credibility research has historically struggled with conceptual ambiguity, often conflating the credibility of the source, the medium, and the message itself. While researchers have developed robust tools for measuring source and medium credibility, there has been a notable lack of an exclusive, validated scale for measuring the credibility of a message. This gap is particularly problematic in the modern digital landscape, where content is frequently consumed in isolation from its original source or delivery platform. This study aims to fill this void by constructing and validating a parsimonious scale specifically for message credibility.

The authors conducted a two-stage process to develop the scale. First, they performed a comprehensive literature review and focus groups to identify potential indicators of message credibility. This resulted in a list of 31 items, categorized into formative indicators (factors that contribute to credibility, such as writing quality and fairness) and reflective indicators (factors that directly reflect the perception of credibility).

Using a within-subjects experiment with 322 participants recruited via Amazon Mechanical Turk, the researchers tested these items against six news articles (three high-credibility, three low-credibility). Through confirmatory factor analysis (CFA), they refined the model to identify the most robust indicators, ultimately distilling the measure down to three reflective items: accurate, authentic, and believable.

The final three-item scale demonstrated high reliability (Cronbach’s alpha = .87) and strong validity. The study confirmed that message credibility is statistically distinct from related constructs like liking and newsworthiness, as well as from source-related measures. The results suggest that while factors like professional writing quality and fairness contribute to the perception of credibility, the core of the construct is captured by the three-item scale. This tool provides media scholars and industry professionals with a standardized, theory-driven metric to isolate message-level perceptions, enabling more precise research into how content characteristics influence audience trust.