@@ -323,14 +323,14 @@ This observation is vitally important in the sense that decision makers based on
\spara{The effect of unobservables.} So far in our synthetic experiments, we have assumed that observed and unobserved features are of equal importance in determining possible outcomes, an assumption encoded in the value of parameters $\beta_\obsFeatures=\beta_\unobservable=1$ (see Section~\ref{sec:syntheticsetting}).
\spara{The effect of unobservables.} So far in our synthetic experiments, we have assumed that observed and unobserved features are of equal importance in determining possible outcomes, an assumption encoded in the value of parameters $\beta_\obsFeatures,~\gamma_\obsFeatures,~\beta_\unobservable$ and $\gamma_\unobservable$ which all were equal to $1$ (see Section~\ref{sec:syntheticsetting}).
%
To explore situations where the importance of unobservables is higher, we now also consider settings with
The results are shown in Figure~\ref{fig:highz}, which is produced just like Figure~\ref{fig:results_errors}, the only difference being the parameters $\beta_\obsFeatures$, $\beta_\unobservable$.
The results are shown in Figure~\ref{fig:highz}, which is produced just like Figure~\ref{fig:results_errors}, the only difference being the values of parameters $\beta_\unobservable$ and $\gamma_\unobservable$.
%
In these settings, the decisions in the data are made mostly based on background factors not observed by the decision maker $\machine$ being evaluated, thus the performance $\machine$ is worse than in Fig.~\ref{fig:results_errors}.
In these settings, the decisions in the data are made mostly based on background factors not observed by the decision maker $\machine$ being evaluated, thus the performance $\machine$ is worse than in Fig.~\ref{fig:results_errors}.
%In these settings, the decisions in the data are made mostly based on background factors not observed by the decision maker $\machine$ being evaluated, thus the performance $\machine$ is expected to be as good as in Fig.~\ref{fig:results_errors}.
% WHAT??? NOT AS GOOD
Nevertheless, the proposed method (\cfbi) is able to evaluate different decision makers $\machine$ accurately.