Practical Mechanism Design for Information Elicitation and ...

to combine the information implicit in each probability, treating each opinion as data in a larger inference
procedure or mimicking a process of Aumann agreement through discussion. Prediction markets
based on market scoring rules are one means of simultaneously eliciting and aggregating probabilities.
Chapter 2: Truthful Revelation without External Verication
I next survey work on peer-prediction mechanisms, including Drazen Prelec’s Bayesian truth serum.
As “peer-prediction” suggests, participants predict the answers of other agents in addition to stating
their own opinion. Comparing predictions against each other rather than against an exogenously
revealed outcome generates truthfulness internally, enabling the elicitation of judgments on vague,
hypothetical, or very long-term questions. As long as one is willing to assume agents share a common
prior, I show large classes of these mechanisms exist. Mechanisms can be easily tailored to the question
at hand, taking advantage of natural orderings or hierarchies in possible answers.
Chapter 3: Accounting for Preferences
Scoring rules, prediction markets, and peer-prediction mechanisms assume participants care only
about maximizing their score from the mechanism, oen given to the expert as a payment. However,
agents are also likely to care about how their responses are used. e principal-agent and informationtransmission
literature in economics describes how conicts in preferences can limit communication,
but yields little practical advice. Although preferences constrain what can be implemented, this comes
with a silver lining. If agents want the nal outcome to reect their opinion (though are still willing
to misrepresent their evidence in order to persuade), honest revelation can be achieved without relying
on payments. Since agents care about the conclusions drawn from reports, inference and decision
procedures must also satisfy incentive-compatibility constraints. I investigate maximally accurate inference
procedures in this setting.
Chapter 4: Robust Mechanism Design for Elicitation
A mechanism is robust if it doesn’t depend on a precise specication of agent preferences or beliefs.
In particular, robust mechanism design typically attempts to relax assumptions about the common
knowledge of participants. In common economic settings, higher-order beliefs of participants have
no relevance for how a good should be allocated, so a robust mechanism should not be sensitive to beliefs
about the beliefs of other agents. In an information elicitation setting, higher-order beliefs could
contain useful information and can be exploited with peer-prediction mechanisms. I argue for appropriate
notions of strategic robustness in elicitation, centering around the notion of rationalizability.
Chapter 5: Assessing Qualitative Information
Although tools for collecting and aggregating probabilistic beliefs are vital, participants might be unwilling
or unable to translate qualitative knowledge into quantitative judgments. In situations when
probabilities can’t be naturally interpreted as frequencies, mechanisms should be capable of handling
qualitative input. is design challenge is tied to developing practical mechanisms for innite hypothesis
or type spaces. One initial solution, proposed by Vincent Conitzer, separates agents into two
groups: one providing information and the other translating statements into probabilitiesis. I investigated
this proposal further.
2