...

paper/sl.tex

@@ -38,7 +38,9 @@
 \usepackage{footnote} % show footnotes in tables
 \makesavenoteenv{table}
 
-\newcommand{\antti}[1]{{{\color{orange} [AH: #1]}}}
+\newcommand{\acomment}[1]{{{\color{orange} [A: #1]}}}
+\newcommand{\rcomment}[1]{{{\color{red} [R: #1]}}}
+\newcommand{\mcomment}[1]{{{\color{blue} [M: #1]}}}
 
 \newcommand{\ourtitle}{A Causal Approach for Selective Labels}
 
@@ -62,7 +64,7 @@
 Increasing number of important decision affecting people's lives are being made by machine learning and AI systems. 
 We study evaluating the quality of such decision makers.
 The major difficulty in such evaluation is that existing decision makers in use, whether AI or human, influence the data the evaluation is based on. For example, when
-deciding whether of defendant should be given bail or kept in jail, we are not able to directly observe the possible offences by defendants that the decision making system in use decides to keep in jail. To evaluate decision makers in these difficult settings, we derive a flexible Bayesian approach, that utilizes counterfactual-based imputation. Compared to previous state-of-the-art, the approach gives more accurate predictions on the decision quality with lower variance. The approach is also shown to be robust to different variations in the mechanisms producing the data.
+deciding whether of defendant should be given bail or kept in jail, we are not able to directly observe the possible offences by defendants that the decision making system in use decides to keep in jail. To evaluate decision makers in these difficult settings, we derive a flexible Bayesian approach, that utilizes counterfactual-based imputation. Compared to previous state-of-the-art, the approach gives more accurate predictions on the decision quality with lower variance. The approach is also shown to be robust to different variations in the decision mechanisms in the data.
 \end{abstract}
 
 
@@ -76,6 +78,8 @@ deciding whether of defendant should be given bail or kept in jail, we are not a
 
 
 \section{Introduction} 
+\acomment{'Decision maker' sounds and looks much better than 'decider'! Can we use that?}
+
 
 \begin{itemize}
 \item What we study
@@ -121,7 +125,7 @@ deciding whether of defendant should be given bail or kept in jail, we are not a
 \end{itemize}
 
 
-\section{Framework}
+\section{The Selective Labels Framework}
 
 
 \begin{figure}
@@ -147,7 +151,8 @@ deciding whether of defendant should be given bail or kept in jail, we are not a
 
 
 
-In this section, we define the key terms used in this paper, present the modular framework for selective labels problems and state our problem.
+%In this section, we define the key terms used in this paper, present the modular framework for selective labels problems and state our problem.
+%Antti: In conference papers we do not waste space for such in this paper stuff!! In journals one can do that.
 
 \begin{itemize}
 \item Definitions \\