FRANK PASQUALE
5 min
Frank Pasquale argues that we are entering an era where critical social and economic decisions—ranging from credit scoring and employment rankings to search engine results and financial interventions—are increasingly delegated to automated, proprietary algorithms. These systems function as 'black boxes': we can observe the inputs and outputs, but the internal processes remain hidden from the public, regulators, and even the individuals affected by them. This opacity is not an inherent feature of technology but is often a deliberate strategy protected by trade secrecy laws, nondisclosure agreements, and intentional obfuscation.
While the digital age promised transparency, the reality has become a 'one-way mirror.' Corporations and government agencies collect vast amounts of granular data on individuals, yet they simultaneously fight to keep their own decision-making procedures, risk models, and influence campaigns secret. This asymmetry undermines the fairness of markets and the democratic process. When authority is expressed algorithmically, the lack of transparency prevents the correction of errors, biases, or conflicts of interest, effectively insulating powerful actors from the consequences of their decisions.
This shift toward algorithmic governance threatens individual autonomy and economic stability. By treating complex social judgments as purely technical problems, companies and governments can mask contestable value judgments as objective, data-driven outcomes. Pasquale contends that transparency alone is insufficient because powerful actors often respond to disclosure requirements by increasing the complexity of their systems to the point of incomprehensibility. Therefore, he advocates for a more robust regulatory framework that prioritizes the 'intelligibility' of these systems, ensuring that when algorithms exert power over people, they are subject to oversight, accountability, and ethical standards.
Alex: Which raises the accountability question directly. If the algorithm functions as a judge, but the judge's reasoning is permanently sealed, what does due process actually mean in that context?
Sam: Pasquale pushes on this hard. His argument is that secrecy is reaching a kind of critical mass where the asymmetry becomes self-reinforcing. Institutions that can see everything about you while remaining invisible themselves accumulate structural advantages that are very difficult to dislodge through normal market mechanisms. The promise of competitive markets — that better information leads to better outcomes — breaks down when one side of the transaction controls what can be known.
Alex: It reframes the whole conversation. This isn't primarily about data privacy in the individual sense. It's about the distribution of epistemic power — who gets to know what about whom, and who gets to enforce that arrangement.
Sam: And his policy instinct follows directly from that framing. The goal isn't to eliminate institutional secrecy entirely — there are legitimate reasons for some confidentiality. The argument is about recalibration. One formulation he offers is pointed: if a process is too complex to be explained to a regulator, it arguably shouldn't be deployed. That's a strong claim. It would have significant implications for large parts of contemporary finance and algorithmic decision-making.
Alex: That's a position that would face serious pushback — from industry obviously, but also from people who'd argue that complexity is sometimes genuinely unavoidable rather than manufactured.
Sam: It would, and that's probably the sharpest tension in the book. Pasquale is making a normative argument about where the burden of proof should sit. Right now, the burden falls on the individual or regulator to demonstrate that an opaque system is harmful. His inversion would place the burden on the institution to demonstrate that its opacity is justified. Whether that's practically achievable is a real question — but as a framing for what accountability should require, it's a substantive one.
Alex: It's the kind of argument that's easy to dismiss as idealistic until you think carefully about what the current arrangement actually permits — and who bears the cost of it.
Sam: Which is precisely why the agnotology framing matters. It asks you to treat the current distribution of knowledge not as a neutral baseline, but as a choice — one that was made, that benefits specific actors, and that could in principle be made differently. That's a harder case to wave away.
Alex: Thanks for listening to ResearchPod.