Alicia Fornés
Computer Vision Center
Universitat Autònoma de Barcelona

Recognition of Handwritten Textual and Graphical Documents in the Deep Learning Era.

In the last decades, Document Image Analysis and Recognition has become a fundamental technology for recognizing, searching and extracting information from handwritten document collections, thus helping in the preservation, access and indexing of our cultural heritage. However, and even with the recent advances in deep learning, historical handwritten documents are still challenging mainly because of the high variability among handwriting styles and the few available labelled data. Lately, several techniques have been proposed to alleviate such problems. Firstly, synthetic data generation and data augmentation have been used to train deep learning systems. Nevertheless, those approaches are still insufficient when dealing with collections from different domains. For this reason, unsupervised writer adaptation approaches have shown a promising ability to automatically adjust the handwriting recognizer towards the new incoming handwriting style. Secondly, few-shot learning and semi-supervised approaches have emerged as an alternative to the low available labelled data, showing a good trade-off between performance and the required human effort in manual annotation or validation. Finally, the combination of those techniques with the inclusion of the human in the loop, crowdsourcing and gamification have shown to be a promising solution for speeding up the transcription of historical manuscripts while offering an engaging experience.
This talk will overview some of these techniques, showing examples of their application to both textual and graphical handwritten documents (e.g. demographic documents, music scores or enciphered manuscripts), and discussing some of the future challenges and research directions.

Dr. Alicia Fornés received the Ph.D. degree in 2009 from the Universitat Autònoma de Barcelona (UAB). Her Ph.D. work on writer identification of old music scores received the best thesis award 2009-2010 by the AERFAI (Spanish Branch of the IAPR – International Association for Pattern Recognition). She has done several research stays abroad, including the University of Bern (Switzerland), University of La Rochelle (France), Osaka Prefecture University (Japan) and Uppsala University (Sweden). She is currently a Senior Research Fellow at the Computer Vision Center (CVC) and the UAB. She has published more than 100 papers in international conferences and journals, and she has participated in many research and technology transfer projects related to the recognition of handwritten documents. She received the IAPR/ICDAR Young Investigator Award in 2017 for outstanding contributions in the recognition of handwriting, text and graphics, with high impact to the field of Digital Humanities. Since 2019, she serves as chair of the IAPR TC-10 on Graphics Recognition ( She also serves as associate editor of the Pattern Recognition journal. Her research interests include historical document image analysis, graphics recognition, handwriting recognition and optical music recognition. More info: