HTML Conditional Comments with blaze-html

Why

A couple of days ago I ran across a really neat boilerplate for mobile-friendly development called Skeleton. This seemed great, and because I do my web development in Haskell and use Jasper’s excellent blaze-html, I wanted the index.html coded in Haskell.

No problem, right? Wrong.

As soon as you look at the index.html file, these top lines raise concern:

Can blaze-html do comments like that, or more complex still, are its combinators capable of closing the final if statement but leaving the HTML tag unclosed?

How

The answer to this is yes, although I admit my solution is kind of hackish. As I got farther into this, I began to realize that perhaps I should have just hard-coded these if statements with preEscapedText and moved on with my life, but it was too late at this point. I was already deep in Text.Blaze.Internal’s source.

The first realization is that, due to the nature of the combinators, I had to define my own html and body tags that would allow me to:

• Open the html tag without closing it
• Open the body tag but close both it and html

This resulted in:

Here, Parent is used to define the opening and closing tags. Note that the missing > is intentional in the opening string: blaze adds it automatically after appending any attributes.

Next came actually allowing for the creation of the comments. This was actually quite simple after StaticString/OverloadedStrings were understood.

comment is very similar to html' and htmlBody from above, but with one key difference: the addition of ss. This is needed to repack the StaticString that blaze uses for efficiency, which is typically created automagically via the OverloadedStrings language pragma.

Usage

Usage is actually quite simple once the combinators are setup, which was of course my goal all along. First, I created htmlTag, a function with the specific task of outputting just the conditional HTML tags:

I was then able to use this effortlessly in the skeletonBase code that actually output the template:

And at last, we have HTML conditional comments in blaze!

Notes

As mentioned before, I know this seems a bit hackish, but it feels less hackish (and more fun!) than the alternative of just shoving a hard-coded string into the template via preEscapedText. Plus, it was a great reason to get to know blaze’s internals a bit better.

If you can think of a better way to do this, please do let me know.

Announcing hs-vcard

I recently began working on a new venture, QR Card Us to help support my education and fund another venture–it just launched tonight. This isn’t so much a sales pitch about that though as it is an announcement of an open-sourced Haskell module I wrote in the process (though, you’re certainly welcome to go check it out, tell your friends, and order some!).

As I was reading up on vCards, I found it most helpful to read RFC 2426. I wanted to easily play around with vCards in my favorite language, but didn’t care for the existing vCard module, so I decided to write my own instead, thus letting me announce hs-vcard. I think it’s fairly straight forward and well-documented, so I’ll end with an example input and the output.

This is the vCard for Frank Dawson, one of the RFC 2426 authors, constructed in Haskell:

and printed out as:

Please let me know if you find any problems, and even better: fix them and submit patches to the GitHub repository. I also chose to not implement reading them in, primarily because I had absolutely no use for that, but people are again welcome to contribute and do so.

I hope you find it useful! Please feel free to comment with any questions or feedback you might have on how I could do things better.

Indigenous Tweets: True Worldwide Twitter Discovery

tl;dr

My mentor, Prof. Kevin Scannell, made a pretty awesome website called Indigenous Tweets. It finds and ranks tweeters who tweet in one of the over 30 indexed languages. He’ll also be blogging in order to talk with some of the main tweeters in each of these languages and help grow the online presence of these language communities.  

Intro

My mentor and esteemed SLU Professor Kevin Scannell is at it again: he’s providing a way for members of language communities to harness the power of the Internet in order to connect with one another, this time by finding the top users of over 30 languages on Twitter and ranking both them and the languages on Indigenous Tweets.

Why is this needed?

Twitter describes itself as “the best way to discover what’s new in your world,”¹ but there is a fundamental issue with this: “world” is presently limited by the inclusion of only a handful of languages. Although people can tweet in any language on Twitter, finding users who speak the same language is a difficult, or even a seemingly impossible, task. This is especially true for the minority languages on which Prof. Scannell focuses. Further, while Twitter attempts to classify the language of every tweet, it does a poor job.

This isn’t to blame Twitter–in fact, classifying many of these languages can be difficult due to a lack of data, but Prof. Scannell has been working on similar problems for many years and as such has amassed large corpora with which to classify and analyze languages. With his data, Twitter’s API, and the magic of Perl at hand, Prof. Scannell was able to write a bot that crawls Twitter as far as the API allows, seeded by a search of common but distinctive words in each language. For every user that is encountered in a search, the bot then considers not only that tweeter’s timeline for ranking, but also his or her following and follower graphs in an attempt to find other language users.

A bit of social.

Of course, as Twitter continues to grow, Indigenous Tweets aims to do the same. Twitter’s API was very helpful in gathering the data, Prof. Scannell has told me, but he knows that some tweeters were likely missed in the process. To counter this, every page is affixed with a form where the usernames of those thought to have been missed can be suggested, letting the community be directly involved in the website. As he continues to crawl Twitter, those suggested will be added to the queue for consideration.

He’s also created a blog (that he’ll definitely keep updated²). Through the blog, he plans to further engage the community, primarily by interviewing top tweeters in each language. He hopes that this in conjunction with the ranking system on Indigenous Tweets will put the need for increased Internet communication at the forefront of language communities’ minds.

Conclusion

It’s a really great service, so I should just stop talking about it so that you can go check it out!

Footnotes

¹ From Twitter’s about page. ² I visit his office frequently, so I’ll be sure to pester him if I don’t see a new post every once in awhile.

A Voyage Into the Universalization of Localization

This project is being developed alongside Dr. Kevin Scannell.

Problem

The current status of localization leaves much to be desired, to which any localizer who has ever had conflict with a website designer will attest. Presently, localization is an oddly local task; that is, translations are handled per-website. This leads to three issues:

1. each website must implement its own, often decentralized, system for managing translations,
2. translations lag behind that of the primary language (e.g., an English website’s new text may not see translation into Irish for some time), and
3. websites without direct programmer intervention are left without any ability to be translated, even if users would find such functionality useful

These three aspects serve as significantly negative roadblocks to the continued effort to make the Internet more accessible.

Current Solutions

We are aware of only one current solution, Dakwak, which aims to localize websites into any of over 60 languages. Unfortunately, this neither alleviates the need for site developer intervention nor provides a more natural user interface through which to submit the translations.

Solution 

Our proposed solution to this problem is to create a global localization system: localize once, reap the benefits everywhere, which will be most likely interfaced through a JavaScript bookmarklet, presenting cross-browser, cross-platform usage. This service will incorporate a few primary properties:

1. the user interface will allow in-place localization on the website itself, maintaining context and preventing the need to leave a familiar interface,
2. translations will be stored in a centralized location based on a convenient fragment size, most likely sentences, minimizing the needless duplication that the localization world sees today, and
3. users can localize any website without developer intervention, making the process one in which language communities can directly and effortlessly contribute to their web presence

Although we may use machine translation services such as Apertium to fill in the gaps, the current state of machine translation is such that the quality is not always high enough for fully readable text, bringing us to the focus on collaborative human translation.

Ultimately, in addition to making the web a more accessible place, the goal is to use the human-provided translations to improve today’s machine learning systems. Some considerations will need to be taken into account in the implementation of the interface itself depending on the intended system to train. Apertium, for example, uses rule-based translation, so it may be wise to provide an optional third step which would allow the user to identify the meaning of words not yet in the Apertium dictionary.

Solution 

This post is just intended to outline the general idea as it stands. I have more specific implementation details in mind and underway, so will touch back later to give more information. Any input is certainly welcome.

Announcing accentuateus-0.9 (Haskell implementation of the Accentuate.us API)

Introduction

As some of you know, I co-created the Accentuate.us project with Kevin Scannell and have been hacking away at various client implementations ever since (including our flagship [Firefox]( http://accentuate.us/get/firefox) client).

Haskell

So, since I have been working on so many implementations of the API, it only seemed natural that I give some attention to my favourite language.

This is my first release on Hackage and it is available at http://hackage.haskell.org/package/accentuateus. I appreciate any feedback you guys have so that I can improve both it and any future code I write.

Code

The source code is available on Spearhead’s GitHub.

Usage

The API implementation boils down to Accentuate.us’ three calls:

• langs — Accepting an optional locale and mandatory version number (0 if unknown), langs will tell you whether or not you are in sync with the API servers’ current version. If not (supplied version != server version), it will supply a new list of (ISO-639, Language Name) pairs. When possible, we will return the language names localized to the supplied locale.
• accentuate — This is the magic call in the service: given a language, optional locale (to localize any error messages), and input text, it will use statistics to add diacritics.
• feedback — The other part of the Accentuate.us service that we really want to make more use of is feedback. Since we use statistics, the diacritic restorations will not always be correct (especially for the more under-resourced languages we support). By supplying a language name and corrected text, we will add it to our training text so that results will improve by user contributions.

Feedback

Again, this is my first upload to Hackage and I’m really excited about it. I appreciate any feedback you guys have. The latest source can be found at https://github.com/spearhead/hs-accentuateus/blob/master/Text/AccentuateUs.hs.

Video of s/2 Years/10 Minutes/ of Photography: A Strange Passions Talk

Thanks to Alex Miller of Strange Loop 2010 fame for getting the video and slides of my Strange Passions talk online!

http://blip.tv/file/4536453

Mozilla Add-On Manager Version Difficulties

Problem

While developing the Accentuate.us Firefox add-on we ran into a bit of a snag. Our API requires that the user-agent begin with Accentuate.us/version for analytics, random-hit spam prevention, and the ability to handle any future version conflicts. This requires gathering the add-on’s version number, which is specified in install.rdf.

The problem arose when testing Gecko 2 browsers such as Firefox 4, which features a spiffy, new add-on manager.

Solution

The solution was rather simple but a little difficult to track down at first, so I’ve included it here for convenience:

The difference is minor but important. The code in try is for Gecko < 2, which grabs the extension manager from Components.classes. In Gecko > 2, the add-on manager is imported. Once you have the add-on manager object, obtaining the version is trivial.

Relatedly, we had to have the line for initializing the Charlifter (legacy name for Accentuate.us) Lifter object in both places because the version number is needed immediately upon initialization for a language list call, which the callback style of Gecko 2’s add-on manager interferes with.

Creating a Mac OS X Service (Part I)

Introduction

I was back working in OS X the other day and got to thinking about an often underused portion of the operating system: Services. Services provide the user with contextual operations: working with text, images, and so forth. One that Apple provides is to highlight a word and, in the context menu or the application’s services menu (i.e., Safari > Services), you will be provided with an option to look the word up in the dictionary.

Wanting yet another example of how easy it is to implement the Accentuate.us API, I decided to take on the task and implement an Accentuate.us OS X Service myself. Note that at the moment it is still under development, so it only accentuates into Irish for testing purposes. That is being ironed out and will be committed soon, though its absense will not impact the contents of this Part I post.

Background

I was disappointed by the documentation that I found online. Although Apple’s developer documentation did prove useful once the project was underway, I felt that there was too little detail about setting up the project itself within Xcode, which led to much undue frustration. I found much detail online about helper utilities to make services, but little material on coding my own, except for CocoaDev’s making services, which had the same issue of not discussing the setup of the Xcode project itself, and Simone’s very detailed walkthrough. Unfortunately, after following Simone’s step-by-step, I encountered the note at the end of the post admitting that “the bundle doesn’t actually work,” which was slightly important to the project. There was a workaround mentioned that involved compiling two different targets, but I found that unnecessary and document the solution within this post.

It is important to reiterate that the three services mentioned brought me close to what I needed, but all lacked in explaining the setup itself or did so in an unfavorable way, which is what I will correct in this part of the series. This will successfully lay the foundation for creating an OS X service and get it to the point that it can lowercase any selected text. In subsequent posts, we will dig deep with the Accentuate.us API and see just how easy it is to make a system-wide utility to accentuate or process selected text in a way you desire!

Setup

Actually configuring the project in Xcode was the area that I found most difficult, so for your benefit, I will err on the side of caution and be overly detailed.

Create a new Xcode project and under the Mac OS X pane on the left, select Framework & Library. Choose Bundle with the Core Foundation framework and save it as AccentuateUsService.

The first problem is that the default bundle is called AccentuateUsService.bundle, but we need it to be AccentuateUsService.service. Go to Project > Edit Active Target "AccentuateUsService"... and select the Build tab. Under the Packaging category, change the Wrapper Extension settings from bundle to service.

While we’re in the target info, switch over to the properties tab and change the Identifier field to read com.accentuateus.${PRODUCT_NAME:rfc1034Identifier}, which turns out after parsing to be com.accentuateus.AccentuateUsService. This will be useful in a subsequent post.

The next problem to tackle is the need for an executable, otherwise the logic for handling the selected text is nonexistent in the build. You can do this by right clicking on the Executables group and selecting New Custom Executable... under the Add menu, which you should fill out as pictured above (AccentuateUsService for the executable name, AccentuateUsService.service for the path).

Though the executable exists, the binary file at this stage will be useless. To change this, navigate to the Project menu and select Edit Project Settings. Under the Build tab, beneath the Linking category, change Mach-O Type from Bundle to Executable.

Our work in this service depends on the Cocoa framework, so right click the External Frameworks and Libraries group and navigate to Existing Frameworks... under the Add menu. Select the Cocoa.framework entry and click Add.

Now, we’re getting to the fun part! We need to add our first source file, main.m, which will hold the logic to register the service with OS X. Under AccentuateUsService’s Source group, navigate to New File... beneath Add. We’ll want to navigate to the Mac OS X Cocoa Class entry in the left pane and then select the Objective-C class as a subclass of NSObject. The file name is main.m and we do not need a header file, so uncheck that box. All other fields can stay at their default, so click Finish.

We also need a place for the service logic. Although this could be done also in main.m, or main.m’s logic could be in this Service class, I like to separate them out. Follow the same steps as above, but this time use the file name Services.m and do check the Also create "Services.h" box.

main.m

In main.m we initialize the below-implemented Service class and register it with the services provider, making service calls possible.

Services.h

In Services.h, we simply define the accentuate method signature. Note that while the first argument, accentuate, may be whatever you please, both userData and error must remain that. The custom accentuate term, although it may be whatever you desire, must match with Info.plist’s NSMessage, as discussed later.

Services.m

Services.m is where the magic happens—it is the logic for accentuate, the method through which all service calls pass. We first check that types does indeed contain an object matching one of NSSendTypes’ entries (discussed later when speaking of Info.plist) and that we can extract the supplied text from it. This is particularly useful for handling the processing of multiple types of data, although that is not covered in this particular post. To be safe, we also check that text has been set, otherwise throw an error and abort.

After we are sure that we have been provided well-formed data, we clear pboard and process the text. In this case, we are simply calling lowercaseString which, intuitively, provides a lowercased version of the string. We will in the next post be replacing this with a different call that actually accentuates the text.

After all is said and done, the processed text is written to pboard. That’s all there is to it!

We’re getting close to finish. The next step is to make sure OS X can have an easy time finding your service, so for that we’ll need to modify the Info.plist file found under AccentuateUsService's Resources group. Right click and select Source Code File under Open As to follow along directly as I do here. You can edit it in the other modes as well, but I prefer this.

Info.plist

The property values are fairly self-explanatory or well documented in the Mac OS X Reference Library, but I’d like to highlight just a few:

• LSBackgroundOnly keeps the application from appearing in the Dock.
• NSMessage is a key item, so I’ll say it again: NSMessage. Earlier we named a method in Services.h called accentuate, or whatever silly name you chose while following, and this must be the same.
• NSPortName is another important entry. I set its value to \${PRODUCT_NAME}, which resolves to AccentuateUsService, because I used that earlier in main.m.

Installation

Phew, we’re done with the setup! In future posts, now that this foundation has been laid, we’ll be able to jump straight into code, but for now it is time to test out our brand new service!

You can navigate easily to the code’s folder by right clicking on the AccentuateUsService group and selecting Reveal in Finder. Navigate to AccentuateUsService/build/Debug/. Open a new finder window (⌘N) and go to the folder (⇧⌘G) ~/Library/Services/.

Copy the AccentuateUsService.service file from the first finder window (in Debug) to the Services folder. You will now need to either log out and back in again or open Terminal (/Applications/Utilities/Terminal.app) and run

$ /System/Library/CoreServices/pbs -dump_pboard |grep AccentuateUsService

This will force OS X to reregister any available services and save you the trouble of login cycling. I added the grep just because -dump_pboard outputs so much contextually irrelevant information.

Please note well that pbs is not guaranteed to stay in OS X and is subject to change functionality, move, or be removed entirely, but it is great for testing.

Next you will need to tell OS X that it’s okay to actually display this new service. Navigate to Services under the Application’s menu (i.e., Finder > Services) and choose Services Preferences.... Under the Keyboard Shortcuts tab, navigate to the Services pane (left) and check Text > Accentuate Text.

Demo

This is my favorite part: demo time! For this, I open Apple’s TextEdit and type a very important letter to my guests, who are unfortunately averse to caps. Thankfully, we wrote this wonderful service that lowers the case of all selected text! Go to the application’s menu and select Accentuate Text under the Services submenu (TextEdit > Services > Accentuate Text).

If all goes well, your letter should be caps free and your guests quite happy indeed!

Wrap-Up

As noted, this particular installation of the Accentuate.us service posts was more detailed than perhaps necessary because I found a few key steps of the setup in Xcode to be undocumented or at least extremely difficult to find and did not want you to suffer as I did. In the next installment, we’ll dive right in with some more code and get a first-hand account of just how easy the Accentuate.us API is.

Photography Roots

I started photography only two years ago, but my real first steps in graphic design holds their roots in eighth grade, about five years ago. My parents owned a local newspaper, so we were in no short supply of the Adobe® products. Prowling about the Internet, I came across some Photoshop® tutorial websites and went to town—doing only what is natural to young boys, of course, turning (photos of) family members into zombies!

I found this to be great fun and also realized in myself a certain knack for design, though I was troubled at first with this. I’m no practiced painter or drawer, so finding my way in the graphics design world seemed at first a dead end, until I met photography. Since it took me some time to find my way, I thought I may speed the process for any budding photo artists and list some of the resources I came across. I’ll also list some more generic Photoshop® resources, just because there is so much overlap.

Outside of the following list, I want to pay special mention to the National Association of Photoshop Professionals (NAPP), Larry Becker in particular. I won’t go into details, but he helped me find my way in the intricate world of design. If you are at all into design, highly consider membership with NAPP—the benefits are well worth it.

The list as promised, in no particular order:

• Photoshop® Insider, the blog of Scott Kelby, president of NAPP
• Photoshop® User TV, a fantastic, informative, and hilarious show. I’ve watched it evolve over the years and the quality and content have only grown richer.
• Layers Magazine, for everything Adobe®
• Photoshop® Lightroom Killer Tips (the world’s best presets?)
• Strobist, for providing advice that makes your lighting otherworldly
• and, This Week in Photo, another great way of staying up-to-date in the photography world

If I had to pick only ten websites to visit again, those would definitely be at the top. For purposes of shameless plug, but also out of all truthfulness, I’d count Accentuate.us and Spearhead Development in that list too :)

Upcoming Talks: Photography and Accentuate.us

Strange Loop: Strange Passions

On October 14, I’ll be giving an extended version of my July 21st Perl Mongers s/2 years/5 minutes/ of Photography lightning talk for Strange Loop’s Strange Passions track. Alex tells me we’ll be getting our hands on video thanks to the lovely InfoQ, so I’ll have that and slides posted as soon as it comes my way.

Be sure to also check out Matt Follett’s Perl 6 Talk, which he’ll also be giving at Strange Loop.

Saint Louis University Math/CS Club: Accentuate Us!

Dr. Kevin Scannell and I will be giving a talk on November 10th to the Math/CS Club, starting around 4pm, to discuss Accentuate.us, a new system allowing you to type quickly and easily in over 100 languages. Video is not yet confirmed, but I’m looking into it.