A Brief Exploration of OCR Technology: 1. Overview
A note up front: as mentioned in an earlier post, last month I took part in the 4th Taidi Cup Data Mining Competition, working on Problem A, which had some connection to OCR systems, and I promised to open-source the final results. I've been busy with graduation and moving lately, so I haven't had time to organize this material — now I finally have a chance to write it up.
I'm publishing these results not because they're particularly impressive or state-of-the-art (quite the opposite, actually — after comparing our work with Baidu's paper Advances in Deep-Learning-Based Image Recognition: Some Practices at Baidu, I realized that the content of that paper is, in essence, still built on the traditional approach, far behind the current trends), but because, although OCR technology is fairly mature by now, there is essentially no article on the internet that explains OCR systems in reasonable detail. So consider this post my attempt to fill that gap. I've always believed that technology can only advance if it's open-sourced (though admittedly, in China this is a debatable point, since open-sourcing can easily lead to knock-offs). Whether it's research in math and physics or data mining, I try to publish most of what I do on this blog, so as to exchange ideas with everyone.more
To get back on track: although our results aren't outstanding, overall we did manage to build a reasonably complete, usable OCR system. In other words, we went through basically all the steps one should take to build an OCR system — whether we did them well is another matter, but let's just say it was passable. There may be some exaggerated rhetorical flourishes in the write-up below; readers are encouraged to take them with a grain of salt.
Below is the abstract of our paper.
We designed a series of algorithms to accomplish tasks such as text feature extraction and text localization, and built a character recognition model based on convolutional neural networks (CNNs). Finally, we incorporated a statistical language model to further improve performance, successfully constructing a complete OCR (optical character recognition) system.
For feature extraction, we abandoned the traditional "edge detection + erosion/dilation" approach. Based on a few basic assumptions, we obtained good text features through steps such as grayscale clustering, layer decomposition, and denoising. These text features can be used both for text localization in the second step and fed directly into the model in the third step for recognition, without requiring additional feature extraction work.
For text localization, we first merged fragmented features via a nearest-neighbor search method to obtain single-line text features, and then segmented each line into individual characters using a forward-backward statistical method. Testing showed that this segmentation approach handles mixed Chinese-English text segmentation quite well.
For optical recognition, we built a single-character recognition model based on a CNN deep learning model, generating 1.4 million samples ourselves for training. This yielded a good single-character recognition model, achieving a training accuracy of 99.7% and a test accuracy of 92.1%; even when image noise was increased to 15%, the model still achieved around 90% accuracy.
Finally, to further improve performance on top of the previous work, we incorporated a language model: using hundreds of thousands of WeChat text samples, we computed a transition probability matrix for common Chinese characters, and used dynamic programming via the Viterbi algorithm to obtain the optimal recognition combination.
Combining the above four components yields a complete OCR system. Testing shows that our system performs well on printed text recognition and can serve as an image text recognition tool for platforms such as e-commerce sites and WeChat.
References
[1] Li Meng; Research on Text Detection Algorithms Based on Multi-Scale Gabor Filters and BP Neural Networks; Computer Software and Theory; 2007
[2] Kernel Density Estimation; https://zh.wikipedia.org/zh-cn/核密度估计; Wikipedia
[3] Xavier Glorot, Antoine Bordes, Yoshua Bengio; Deep Sparse Rectifier Neural Networks
[4] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton; ImageNet Classification with Deep Convolutional Neural Networks
[5] Dropout: A Simple Way to Prevent Neural Networks from Overfitting
[6] Wu Jun; The Beauty of Mathematics (2nd edition); Chapter 3
[7] Wu Jun; The Beauty of Mathematics (2nd edition); Chapter 26
Translated automatically with claude-sonnet-5; all equations are reproduced verbatim from the source. Copyright remains with the original author.