This project is being developed alongside Dr. Kevin Scannell.
Throughout the course of the Fall 2010 semester I will investigate the expansion of an existing Mozilla Firefox extension (hereafter “add-on,” “extension,” “plug-in”). This extension, licensed under GNU General Public License version three (GPLv3), interfaces Charlifter [1], also GPLv3, a server which provides access to language processing resources for over one-hundred languages. The purpose of Charlifter is to promote the proper localization of text, which is presently represented either with incorrect diacritic placement or none at all. Charlifter uses ever-growing corpora to add diacritics, or unicodify, text which is passed to it either in ASCII form or in an incompletely or incorrectly unicodified state, using context and the aforementioned corpora for statistical analysis and unicodification of the text. Communication between the extension and the server are made via HTTP POSTs where data is represented in JavaScript Object Notation(JSON). Any request from the plug-in to unicodify the text is also referenced as “lifting” or “accentuating,” meant to represent the act of adding diacritics.
The goal in extending the add-on is to focus on an intuitive interface for seamless integration into the browsing experience. The target expertise level of the end-user is to range anywhere from novice to expert, meaning that the extension will avoid technical and linguistics jargon throughout.
Presently the plug-in supports only synchronous lifting, thereby requiring a conscious act from the user to lift the text. To meet the goal of making this plug-in intuitive, asynchronous lifting will be made the focal point of the add-on. Charlifter uses word-level trigrams with its corpora for unicodification, so a primary concern of the asynchronous lifting aspect will be to accentuate with as much context as possible. The ideal situation is that the user types three or more words and the plug-in will then request a lift, but in situations in which the user types fewer than three words, the extension will employ techniques to make an accurate guess as to when the user is finished typing. In cases like this where the trigram is not present, the server surrounds the submitted text with placeholder words to make a less accurate unicodification guess.
Should the user then revisit the text input and begin typing more, the extension must remain intuitive and include words which may have already been lifted. If the server responds with a different unicodification based on this more accurate context, the plug-in is to replace the old, likely less accurate, version with the new response.
The user’s manipulations of the text are unpredictable as a whole. Therefore, it is imperative that the add-on also support mutations where the user deletes just characters, a single word, or groups thereof, re-evaluating nearby words for changes in the context. The client may also add large blocks of text via common methods such as copying and pasting, either in an empty box or near un-lifted or already lifted text. In all such cases, the plug-in is still to resend the text and that which is nearby for lifting based on new context.
The extension must also account for common network issues such as slow communication (lag) or connectivity being lost altogether. Such a situation is fine under the plug-in’s policy of failing gracefully, especially during synchronous accentuations, but the lag issue deserves special attention when considering asynchronous lifting. There may be situations in which the add-on submits a request to have text lifted and network lag pauses the response, during the interim of which the user adds more text, changing the context. As another request is submitted, the original returns and the add-on modifies the text accordingly. The problem arises when the second response returns and the extension must find, through common text matching, the old block of text that was lifted. As the context used for this has since received diacritics, this would be an impossible task. The proposed solution for this is to use other clues to locate which blocks of text were to be lifted, such as cursor position at the time of the call and difference between original and lifted text. Weighing such records of the text as it was at lift-time, the extension can more accurately discern which text was sent for lifting, even with subsequent text mutations. In order to relate these weighted values to the current text, it is proposed that a simple numerical ID be added to the Application Programming Interface (API) request, which the server is to simply echo back in the response, thereby not violating the server’s statelessness.
With asynchronous functionality comes the ability for the user to add text to multiple page elements in multiple tabs within multiple browser windows at the same time. The plug-in must coordinate which lift response belongs to a particular browser window, tab, page, and page element. The proposed method of such tracking is to assign a pseudo-random unique identifier (ID) to each element (of the page, the window, and so forth), such that an associative array in combination with this ID can be used to locate the element and mutate the text accordingly.
With the addition of asynchronous calls, the rate at which calls are made becomes an issue of the extension rather than one of the user. In order to determine when a user is done editing, at least momentarily, the plug-in will use two methods: timeouts and blur events.
The timeout is triggered after a delay in user activity. For the delay to be effective in terms of promoting a sense of live accentuating while still giving the user sufficient time to begin adding the next characters for new words, the time will typically be under one second, although further experimentation needs to be conducted to determine an adequate idle time.
The other method is responding to an “onblur” event. As soon as a user begins typing in a page element, with asynchronous lifting enabled, the element will be tracked as mentioned prior, and an event listener will be added for the “onblur” event. When the input field loses focus, whether a trigram or more has been met or not, the text will be lifted. This follows that the user is at least temporarily done making modifications to that elements’ text.
It is important in the consideration of asynchronous calling and efficiency both to use such methods in order to minimize the request rate, saving bandwidth, while still maintaining a sense of the accentuating being live.
Currently the user is provided with a list of approximately 112 languages to choose from, along with a context menu entry for their most recently used language. In line with the goal of being intuitive, the extension will originally provide a list where the ISO-639 codes are sorted alphabetically. The extension will then weigh clues to order the list in a way that provides languages that the user is most likely to target at the top. Incorporating top-level domains of the currently visited website, the Mozilla application’s language setting, and use frequency of past languages, the plug-in will resort the list. Any unweighted items will remain at the end of the list, still ordered alphabetically.
I like learning new things. Previously: Kenchi founder, eng & ops teams at Stripe from 2012-2019. Say hi! 🏳️🌈