The aim of this project is to quickly send text in a digital medium. I will utilize a machine learning algorithm to learn to recognize characters in digital writing from the new shorthand I developed last year. The second part of the algorithm will then convert the Digital Shorthand Key (DSK) into English text. I will then train the algorithm to recognize the characters via training and testing data and measure loss. I will also write a series of rules to turn the shorthand into English from the DSK. This program would prove to be useful for those in time sensitive situations where they do not have time to type up a message to send on a traditional keyboard. Machine learning well adapted to these tasks such as Natural Language Processing and image recognition and transcription. Furthermore, numerous qualities of shorthands have been examined in order to help make the Digital Shorthand Key that this project will be using.
Society demands efficiency yet typing on the standard keyboard can prove detrimental in fast-paced environments. This project aims to digitally transcribe information more efficiently, by interpreting and translating a new, custom, and efficient shorthand, a way of quickly writing English text using loops, curls, and squiggles, via Machine Learning image recognition software.
The first step in building the system is to transcribe common phonetic texts into the shorthand, take pictures, and annotate characters using the program LabelImg built specifically for this task. About 80% of these pictures and their xml files denoting character locations are the training set used to train the Tensorflow Recursive Neural Network. After training for 160,000 steps, the model is run with some code and a UI to add context to the words written. The total loss (inaccuracy of the Machine Learning model in identifying characters) was 0.11 to 0.12. Although the loss was not zero, the system provides a quick and easy way to communicate information through this model majority of the time. It provides a quantitative metric of the effectiveness of the trained model and a new medium for faster digital transcription for everyone in all environments. The digital shorthand system can help people save time writing on touchscreen devices such as iPads, write without the traditional keyboard (easier for the visually impaired), and standardize or facilitate shorthands in the medical industry.
Shorthands are the art of transcribing words and information (in English) quickly by hand, since traditional words take too long to write. For the past few hundred years, Gregg shorthand has been widely considered one of the most popular and effective shorthands (4). Pitman shorthand is also a very popular shorthand used by many professionals to record information, but relies on the thickness of stroke (6). However, my project last year focused on the number of strokes and how many pixels they took up (all being the same length). For these reasons, Gregg seemed to be the better option. The new shorthand that could beat Gregg thanks to a smaller phonetic inventory represented by only 12 distinct characters. There are numerous benefits to a phonetic system, including future application in different languages too (3).
Image recognition is an important part of this project because it is the first step of getting the shorthand from an image into a format where functions can be performed to get it into English. The output of the first Machine Learning program will output information written in the DSK, however this is not English. In order to get it back into English, lexers and parsers will have to be used, or another Machine Learning algorithm that strictly deals with text to put the information into context (7). Furthermore, although the information about English words in IPA is phonetic like the shorthand, the IPA will have to go through a dichotomy in order to resemble the DSK.
Download here