Abstract: A computer-implemented Tree Alternating Optimization (TAO) algorithm for learning decision trees to find an approximate minimizer of an objective function over the parameters of the bee. Generally, the method comprises inputting an initial decision tree and a training set of instances, processing the initial decision tree by partitioning nodes into sets of non-descendant nodes, processing the nodes in each set by updating the nodes' parameters at each iteration so that the objective function decreases monotonically, and pruning the free, which produces a final free of a size no larger than that of the initial free. TAO applies to many different types of loss functions, regularization terms and constraints, and types of models at both the decision nodes and the leaves, and makes it possible to learn better decision trees than with traditional algorithms, and to learn bees for problems where traditional algorithms do not apply.

Abstract: Computer-implemented methods for learning decision trees to optimize classification accuracy, comprising inputting an initial decision tree and an initial data training set and, for nodes not descendants of each other, if the node is a leaf, assigning a label based on a majority label of training points that reach the leaf, and if the node is a decision node, updating the parameters of the node's decision function based on solution of a reduced problem, iterating over the all nodes of the tree until parameters change less than a set threshold, or a number of iterations reaches a set limit, pruning the resulting tree to remove dead branches and pure subtrees, and using the resulting tree to make predictions from target data. In some embodiments, the TAO algorithm employs a sparsity penalty to learn sparse oblique trees where each decision function is a hyperplane involving only a small subset of features.