Abstract: A method for identifying derived compounds exhibiting activity for a target macromolecule generates experiences. Each experience uses an initial compound in plurality of initial compounds to construct a derived compound through a hierarchical proximal policy. The policy has a parent molecular reaction model and a child reactant model that uses an environment of the target macromolecule. The parent model evaluates a plurality of molecular reactions. The child model evaluates a corresponding plurality of reactants for a selected molecular reaction. Using the plurality of experiences, the parameters of the parent model are updated in accordance with a first surrogate objective while the parameters of the child model are updated in accordance with a second surrogate objective. The generation of derived compounds and hierarchical proximal policy updating continues until convergence. Then, a subset of the derived compounds from the experiences is tested for activity against the target macromolecule.