\section{Setting and problem statement}

\begin{figure}
    \begin{tikzpicture}[->,>=stealth',node distance=1.5cm, semithick]

  \tikzstyle{every state}=[fill=none,draw=black,text=black]

  \node[state] (R)                    {$R$};
  \node[state] (X) [right of=R] {$X$};
  \node[state] (T) [below of=X] {$T$};
  \node[state] (Z) [rectangle, right of=X] {$Z$};
  \node[state] (Y) [below of=Z] {$Y$};

  \path (R) edge (T)
        (X) edge (T)
	     edge (Y)
        (Z) edge (T)
	     edge (Y)
        (T) edge (Y);
\end{tikzpicture}
\caption{The causal model of the decision makers \human and \machine. 
$R$ is the leniency of the decision maker, $T$ is a binary decision, $Y$ is the outcome that is selectively labeled. Background features  $X$ for a subject affect the decision and the outcome. Additional background features  $Z$ are visible only to decision maker \human. }\label{fig:causalmodel}
\end{figure}

The setting we consider is described in terms of two decision processes.
%
In the first one, a decision maker \human considers a case described by a set of features $F$ and makes a binary decision $T = T_{_H} \in\{0, 1\}$, nominally referred to as {\it positive} ($T = 1$) or non-positive ($T = 0$).
%
Intuitively, in our bail-or-jail example of Section~\ref{sec:introduction}, \human corresponds to the human judge deciding whether to grant bail ($T = 1$) or not ($T = 0$).
%
The decision is followed with a binary outcome $Y = Y_{_H}$, which is nominally referred to as {\it successful} ($Y = 1$) or {\it unsuccessful} ($Y = 0$).
%
An outcome can be {\it unsuccessful} ($Y = 0$) only if the decision that preceded it was positive ($T = 1$).
%
If the decision was not positive ($T = 0$), then the outcome is considered by default successful ($Y = 1$).
%
Back in our example, the decision of the judge is unsuccessful only if the judge grants bail ($T = 1$) but the defendant violates its terms ($Y = 0$).
%
Otherwise, if the decision of the judge was to keep the defendant in jail ($T = 0$), the outcome is by default successful ($Y = 1$) since there can be no bail violation.
%
Moreover, we assume that decision maker \human is associated with a leniency level $R$, which determines the fraction of cases for which they produce a positive decision, in expectation. 
%
Formally, for leniency level $R = r\in [0, 1]$, we have
\begin{equation}
	P(T = 1 | R = r) = \sum_{X, Z} P(T = 1, X, Z | R = r) = r .
\end{equation}

The product of this process is a record $(H, X, T, Y)$ that contains only a subset $X\subseteq F$ of the features of the case, the decision $T$ of the judge and the outcome $Y$ -- but leaves no trace for a subset $Z = F - X$ of the features.
%
Intuitively, in our example, $X$ corresponds to publicly recorded information about the bail-or-jail case decided by the judge (e.g., the gender and age of the defendant) and $Z$ corresponds to features that are observed by the judge but do not appear on record (e.g., whether the defendant appeared anxious in court).
%
The set of records $\{(H, X, T, Y)\}$ produced by decision maker \human becomes part of what we refer to as the {\bf dataset} -- and the dataset may include records from more than one decision makers.
%
Figure~\ref{fig:causalmodel} shows the causal diagram that describes the operation of a single decision-maker \human.



In the second decision process, a decision maker \machine considers a case from the dataset, described by the set of recorded features $X$ and makes its own binary decision $T = T_{_M}$ based on those features, followed by a binary outcome $Y = Y_{_M}$.
%
In our example, \machine corresponds to a machine-based automated-decision system that is considered for replacing the human judge in bail-or-jail decisions.
%
% Notice that we assume \machine has access only to some of the features that were available to \human, to model cases where the system would use only the recorded features and not other ones that would be available to a human judge.
%
The definitions and semantics of decision $T$ and outcome $Y$ follow those of the first process and are not repeated.
%
Moreover, decision maker \machine is also associated with a leniency level $R$, defined as before for \human.
%
The causal diagram for decision-maker \machine is the same as that for \human (Figure~\ref{fig:causalmodel}), except that \machine does not observe variables $Z$.

\subsection{Model definition}

The causal diagram of Figure~\ref{fig:causalmodel} provides the structure of causal relationships for quantities of interest.
%
In addition, we consider $J$ instances $\{H_j, j = 1..J\}$ of decision makers \human --
and, for the purposes of applied data analysis, we explicate probabilistically our assumptions about them below.
%
Note that index $j$ refers to decision maker $H_j$ and \invlogit is the logistic function.

\noindent
\hrulefill
\begin{align} 
\prob{\outcome=1~|~\decision, \features=\featuresValue, \unobservable=\unobservableValue}
	& =
	\begin{cases}
		1,~\text{if}~\decision = 0\\
		\invlogit(\alpha+\beta_{x}x+\beta_{z}z),~\text{o/w}
	\end{cases}\label{eq:defendantmodel}\\
\prob{\decision = 1~|~\leniency_j = \leniencyValue, \features = \featuresValue, \unobservable = \unobservableValue}
	& = \invlogit(\alpha_{j} + \gamma_{x} x + \gamma_{z}z) \label{eq:judgemodel}\\ 
	\text{for}\ \alpha_{j} & = \logit(\leniencyValue) \nonumber\\
	& \ldots . 
\end{align}
\hrulefill

As stated in the equations above, we consider normalized features \features and \unobservable.
%
Moreover, the probability that the decision maker makes a positive decision takes the form of a logistic function (Equation~\ref{eq:judgemodel}).
% 
Note that we are making the simplifying assumption that coefficients $\gamma$ are the same for all defendants, but decision makers are allowed to differ in coefficient $\alpha_j$, so as to allow for varying leniency levels among the decision makers.
%
The probability that the outcome is successful conditional on a positive decision (Equation~\ref{eq:defendantmodel}) is also provided by a logistic function, applied on the same features as the logistic of (Equation~\ref{eq:judgemodel}).
%
In general, these two logistic functions may differ.
%
However, in certain scenarios, a decision maker would be considered good if the two functions were the same -- i.e., if the probability to make a positive decision was the same as the probability to be followed with a successful outcome.

\todo{Michael to Riku}{Define the full model above.}

\subsection{Evaluating decision makers}

The quality of a decision maker is measured in terms of its failure rate {\bf FR} -- i.e., the fraction of undesired outcomes ($Y=0$) out of all the cases for which a decision is made. 
%
A good decision maker achieves as low failure rate FR  as possible.
%
Note, however, that a decision maker that always makes a negative decision $T=0$, has failure rate $FR = 0$, by definition.
%
For comparisons to be meaningful, we compare decision makers at the same leniency level $R$.


The main challenge in estimating FR, however, is that in general the dataset does not directly provide a way to evaluate FR. 
%
In particular, let us consider the case where we wish to evaluate decision maker \machine\ -- and suppose that \machine is making a decision $T_{_M}$ for the case corresponding to record $(H, X, T_{_H}, Y_{_H})$.
%
Suppose also that the decision by \human was $T_{_H} = 0$, in which case the outcome is always positive, $Y_{_H} = 1$.
%
If the decision by \machine is $T_{_M} = 1$, then it is not possible to tell directly from the dataset what the outcome $Y_{_M}$ would have been in the hypothetical case where decision maker's \machine's decision had been followed in the first place.
%
The approach we take to deal with this challenge is to use counterfactual reasoning to infer $Y_{_M}$.

Ultimately, our goal is to obtain an estimate of the failure rate FR for a decision maker \machine.
\begin{problem}[Evaluation]
Given a dataset $\{(H, X, T, Y)\}$, and a decision maker \machine, provide an estimate of the failure rate FR.
\end{problem}
\noindent
\mcomment{I think that leniency does not need to be part of the problem formulation, since imputation allows us to evaluate a decision maker even if we do not know its leniency level.}
Sometimes, we may have control over the leniency level of the decision maker we evaluate.
%
In such cases, we would like to evaluate decision maker $M = M(R = r)$ at various leniency levels $R$.
%
Ideally, the estimate returned by the evaluation should also be accurate for all levels of leniency.

\todo{Michael}{Create and use macros for all main terms and mathematical quantities, so that they stay consistent throughout the paper.}
