Abstract: A simplified handwriting recognition approach includes a first network comprising convolutional neural network comprising one or more convolutional layers and one or more max-pooling layers. The first network receives an input image of handwriting and outputs an embedding based thereon. A second network comprises a network of cascaded convolutional layers including one or more subnetworks configured to receive an embedding of a handwriting image and output one or more character predictions. The subnetworks are configured to downsample and flatten the embedding to a feature map and then a vector before passing the vector to a dense neural network for character prediction. Certain subnetworks are configured to concatenate an input embedding with an upsampled version of the feature map.