conclusions.tex

%!TEX root = sl.tex
% The above command helps compiling in TexShop on a MAc. Hitting typeset complies sl.tex directly instead of producing an error here.

\section{Conclusions and Future work}

In this paper we considered evaluation of (automatic) decision makers, which is vitally needed for the current aims of replacing human decision making with different kinds of automatic decision making procedures. The challenge in this is that the evaluation often needs to be based on data where present decisions imply selective labeling and missing data, thus biasing any standard statistical data analysis results.  We showed that with proper causal modeling, evaluation of decision makers is possible even on this selectively labeled data. Contrary to the previous methods, our proposed approach allows for more accurate evaluations, with less variation, also in settings that evaluation was not possible before.

In the future we will examine further generalizing the setting and modeling assumption: more intricate differences in decision maker's behaviour could be modeled e.g. by hiearchical Bayesian modeling. 
%
Since our approach predicts outcomes based on decision made by educated decision makers, it is an open question, whether this information can be used also when learning the statistical models the automatic decision makers are ultimately based on.
%
We believe such approaches will allow for better evaluations in new application fields, ensuring the accuracy and fairness of automatic decision making procedures that can be then adopted in the society.



%\begin{itemize}
%\item Conclusions 
%\item Future work / Impact
%\end{itemize}