...

paper/experiments.tex

@@ -22,11 +22,11 @@ Each experiment is described in terms of
 
 \subsection{Synthetic setting}
 
-Decision makers can be of the following types:
+We employed the following decision makers in our experiments:
 \begin{itemize}
-\item \textbf{Random}: given leniency $r$ they release random subjects. (with probability or portion?)
-\item \textbf{Batch}: Following Lakkaraju, these sort their subjects according to a logistic regression model $y \sim x$ or $y\sim x+z$ and release $r$ portion of the subjects.
-\item \textbf{Independent}: Each subject is released with respect to a cumulative distribution function based on the logistic regression model. EXPLAIN BETTER.
+\item \textbf{Random}: given leniency $r$ they release each subject with probability $r$.
+\item \textbf{Batch}: Following Lakkaraju, this decision maker sorts its subjects according to a logistic regression model $y \sim x$ or $y\sim x+z$ and releases $r$ portion of the subjects.
+\item \textbf{Independent}: Each subject is released with respect to a cumulative distribution function based on the logistic regression model. \acomment{EXPLAIN BETTER.}
 \item \textbf{Probabilistic}: Each subject is released with probability based on the logistic regression model.
 \end{itemize}
 Decision makers in the data (\human) have access to \unobservable. Evaluated decision makers do not have access to \unobservable. All parameters of the models are for evaluated decision makers are learned from the training data set.
@@ -170,14 +170,14 @@ This model was used as decision-maker \machine and these same features were used
 \begin{figure}
 %\centering
 \includegraphics[width=\linewidth]{./img/sl_rmax05}
-\caption{Results when $\max(\leniencyValue)=0.5$. Here we observe how our proposed method is able to estimate the true failure rate accurately despite the maximum leniency in the data. Contraction however is only able to estimate the true failure rate only up to $\max(\leniencyValue)$ and does it with lower accuracy.}
+\caption{Results when $\max(\leniencyValue)=0.5$. Here we observe how our proposed method is able to estimate the true failure rate accurately despite the maximum leniency in the data. Contraction however is only able to estimate the true failure rate only up to $\max(\leniencyValue)$ and does it with loer accuracy.}
 \label{fig:results_compas}
 \end{figure}
 
 \begin{figure}
 %\centering
 \includegraphics[width=\linewidth]{./img/sl_absolute_errors}
-\caption{Results using different decision-makers and settings. Here $\max(\leniencyValue)=0.9$.}
+\caption{Results using different decision-makers and settings with leniences from $0.1$ to $0.9$. The most lenient decision maker in the data set had  $\max(\leniencyValue)=0.9$.}
 \label{fig:results_compas}
 \end{figure}
 
@@ -204,7 +204,7 @@ Decision-maker \human random 					& 0.01522     & 0.00137         \\
 Decision-maker \machine random 					& 0.03005     & 0.00327              \\
 Lakkaraju's decision-maker \human \cite{lakkaraju2017selective} & 0.01187          & 0.00288            \\ \bottomrule
 \end{tabular}
-\caption{Comparison of mean absolute error w.r.t true evaluation between contraction and the counterfactual-based method we have  presented. The table shows that our method can perform welll despite violations of the assumptions (eg. having decision-maker \human giving random and non-informative decisions). Here $\max(\leniencyValue)=0.9$.}
+\caption{Comparison of mean absolute error w.r.t true evaluation between contraction and the counterfactual-based imputation with leniences from $0.1$ to $0.9$. The table shows that our method can perform well despite violations of the assumptions (eg. having decision-maker \human giving random and non-informative decisions). Here data had $\max(\leniencyValue)=0.9$.}
 \label{tab:}
 \end{table}