Effective optimization is essential for interactive systems to provide a satisfactory user experience. However, it is often challenging to find an objective to optimize for. Generally, such objectives are manually crafted and rarely capture complex user needs in an accurate manner. We propose to infer the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. It is promising if we model the objectives directly from the user interactions which we use to optimize interactive systems, which will improve user experience and dynamically reacts to user actions.