Off-policy evaluation methods are important in recommendation systems and
search engines, whereby data collected under an old logging policy is used to
predict the performance of a new target policy. However, in practice most systems
are not observed to recommend most of the possible actions, which is an issue since
existing methods require that the probability of the target policy recommending an
item can only be non-zero when the probability of the logging policy is non-zero
(known as absolute continuity). To circumvent this issue, we explore the use of
action embeddings. By representing contexts and actions in an embedding space,
we are able to share information to extrapolate behaviors for actions and contexts
previously unseen.