QUESTO: Interactive Construction of Objective Functions for Classification Tasks

Building effective classifiers requires providing the modeling algorithms with information about the training data and modeling goals in order to create a model that makes proper tradeoffs. Machine learning algorithms allow for flexible specification of such meta-information through the design of the objective functions that they solve. However, such objective functions are hard for users to specify as they are a specific mathematical formulation of their intents. We present an approach that allows users to generate objective functions for classification problems through an interactive visual interface. Our approach adopts a semantic interaction design in that user interactions over data elements in the visualization are translated into objective function terms. The generated objective functions are solved by a machine learning solver that provides candidate models, which can be inspected by the user, and used to suggest refinements to the specifications. Our visual analytic (VA) system called QUESTO translates user interactions with data into objective functions for classification. QUESTO facilitates interactive construction of a classifier by allowing users to formulate their preferences as an objective function while they explore and interact with a tabular dataset. QUESTO visualizes the underlying objective function to help users understand their specified goals and constraints. Furthermore, through iterative interactions with the data, the user can refine the specified constraints and the objective function to find a model that correctly characterises the data. The resulting user-defined multi-objective objective function can be used with Auto-ML optimizers (such as Hyperopt) to guide and facilitate the construction of classification models, with the advantage of users being able to specify domain-specific constraints and objectives.