Abstract: The technology disclosed enables a user to optimize a sampling logic to increase the future sampling likelihood of those instances that are similar to the instances that the user believes are informative, and decrease the future sampling likelihood of those instances that are similar to the instances that the user believes are non-informative.

Abstract: The technology disclosed implements Human-in-the-loop (HITL) active learning with a feedback look via a user interface that is expressly designed for the suggested images to admit multiple fast feedbacks, including selection, dismissal, and annotation. Then, the downstream selection policy for subsequent sampling iterations is based on the available data interpreted in the context of the previous selections, dismissals, and annotations.

Abstract: The technology disclosed presents a system that comprises a memory, a data partitioning logic, and an annotation logic. The memory stores a sequence of frames of a video. The data partitioning logic is configured to partition the sequence of frames into an oracle set and an unannotated set. Frames in the oracle set are annotated by a user. Frames in the unannotated set are candidates for user annotation conditional upon being members of a core set, and for machine annotation conditional upon being non-members of the core set. The annotation logic is configured to generate annotations for the frames in the unannotated set. The annotations include user annotations based on membership in the core set, and machine annotations based on non-membership in the core set.

Abstract: The technology disclosed extends Human-in-the-loop (HITL) active learning to incorporate real-time human feedback to influence future sampling priority for choosing the best instances to annotate for accelerated convergence to model optima. The technology disclosed enables the user to communicate with the model that generates machine annotations for unannotated instances. The technology disclosed also enables the user to communicate with the sampling logic that selects instances to be annotated next. The technology disclosed enables the user to generate ground truth annotations, from scratch or by correcting erroneous model annotations, which guide future model predictions to more accurate results.