Singularity on the Plane

The more our information society progresses, the more we realize the value of having a program available in different languages. The effort towards internationalization and localization is an amusing trend for me, because at childhood I once thought that the existence of multiple languages is a bother, and envisioned that at some age the human civilization will settle on one ultimate lingua-gaea.

I couldn’t be more wrong. Now I realize that each language is beautiful and unique in its own right and every effort must be spent to conserve them, or at least document them sufficiently before its last speaker dies.

Anyway, multilingual programs… People might need a certain language simply because she couldn’t understand any other languages. In this case, the availability of a program in a certain language is crucial to reduce the technological gap. People might also want a certain language simply because she enjoys that particular language the most. Another use is for studying foreign languages, like me who tries to set every possible program to Japanese in order to immerse myself in the language.

With that in mind, we will investigate how Windows programs currently handle the user’s desire to choose her preferred language. (Some of the discussion might apply to other OSes such as GNU/Linux and OS X)

One installer to rule them all

The most convenient case is when a program installer includes all available translations. Examples are Inkscape, Pidgin, Battle for Wesnoth, Paint.NET, and iTunes. For example, when I tried to download the newest Inkscape, there’s only one installer for Windows: Inkscape-0.46.win32.exe. It has English, Japanese, and a myriad of other languages included. Even Indonesian!

The next question is, what language will such programs use by default? Some programs, most notably GTK programs, are smart enough to detect the operating system’s language settings. Where is it set?

In Windows XP, the user can set what language she prefers from “Control Panel” → “Regional and Language Options” → “Advanced” → “Language for non-Unicode programs”, like so:

Note that this setting is actually to enable non-Unicode (e.g., ancient) programs to display its text correctly instead of mojibake. However modern Unicode-aware programs use the value we set here to decide what language it should present to the user.

Mine is set to Japanese, so Inkscape appears like this:

Other programs ask for what language the user would like to use, perhaps at install time or when the program is run for the first time. An example is Paint.NET:

The rest just set the default language to English or whatever else the developer prefers. If the user desires, she can change the language through some means because the language data are already installed anyway. An example is an old version of OpenTTD which defaults to English:

If the default language doesn’t suit you, how do you change it? Some programs offer the convenience of setting it within the program itself. An example is iTunes:

And The Battle for Wesnoth:

GTK programs does not visibly offer any such options, because it assumes that you will in most cases want the language you set on the operating system. However, it atually checks for the availability of the LANG environment variable (probably ISO 639-1 codes). You can use it to quickly try out a language. For example, go to the command prompt and type:

set LANG=th

And from the same command prompt, run the program, say inkscape.exe:

Ah, I feel nostalgic :).

Exceptionally Easy to Extend (E3)

Some programs come with only one language, and to choose another we must download the required language files. Though quite inconvenient, it is not that bad because language files shouldn’t be that large. An example is µTorrent:

Predestination, believe it or not

Other programs offer seperate installer for each language. If you go to their web sites, you will find one installer for the English version, another for the French version, and so on. The concept is very simple: what you download is what you get. Needless to say, it’s a pain the arse for the curious or the language learners out there. Two glaring examples are OpenOffice.org and Mozilla Firefox. Here’s a Japanese version of Firefox that I recently installed:

It’s of course better than no multilingual support at all, but still it’s a waste of bandwidth to download another version and troublesome to actually install it (uninstall the other-language version first).

My take

A modern program should at all cost include all available translations in its installer. It should then detect and display the user’s preferred language by default.

About language-changing facility, I can understand GTK’s decision to hide it from the program’s preferences. They are probably following Gnome’s guideline that every GTK app should look the same (think about themes), and if the user wants to change anything, she can apply a system-wide change. To take things into perspective, it’s bizzare to imagine that every GTK app has its own theme settings, right? At least you can set it per program using an environment variable.

For µTorrent that separates its language pack, I think it’s an acceptable special case because µTorrent aims to be a small no-frills downloadable program. Heck, even the language pack is larger than the core English program itself! However, I think for most other programs the size of the language files shouldn’t matter that much.

So big cheers for GTK apps and other programs that include all translations by default. A big, big boo for OpenOffice.org, Firefox, and the gangs that require a different download for each language.

The plan for my final project is a web-based application. Because I don’t usually play with web technologies, there are many things that I have to learn. I chose Javascript as my first subject.

So, I’ve been reading w3school’s tutorial on Javascript. It’s great that the syntax is C-ish, so I could skim most of the syntatic stuffs.

Please don’t misundertand the title of this post. It doesn’t mean that “Learning Javascript is a simple game”. It means that “after learning some Javascript, I made a simple game” :). Here’s the game, please test it. (That’s the beauty of web apps, people can run your programs in any platform without the hassle of installing extra programs)

About the game itself, there were 3 major rewrites. The initial version could actually produce starting points that could not be solved! I rewrote it so that the initial state of the puzzle is generated from the completed state through a series of LEGAL moves. Second, the generator could produce an initial state that could be solved very easily. That sucks, so I added a check to keep shuffling if the initial state is too easy. Third, the “empty” cell is changed from the lower right to the middle. The reason is obvious, so that there is more incentive to solve the puzzle

I found a great IME shortcut key (Windows XP’s IME)…

The default replacement for characters we input is hiragana. For example, after activating the IME and writing "suki", the replacement string that appears on screen is "すき". By pressing spacebar repeatedly, we can cycle through the alternatives like "好き".

The magic shortcut key is F7. It directly transforms the string into katakana! キーボードショートカット大スキ!!!

Windows isn’t Windows if it doesn’t behave crazy from time to time.

OK, so I installed Japanese fonts (for an obvious purpose). I also installed Japanese IME so that I could input Japanese in a sane way. Last, I set “Language for non-Unicode programs” to Japanese so that I could activate IME on console programs (”Regional and Language Options” -> “Advanced”).

See, I did all those Japanese-related tweaking, but never did I order Windows to set its user interface to Japanese (and I couldn’t see any way to do it). But this is what I get:

I don’t know what caused it. When started, the program was fully in English. When the menu went crazy, other programs were also still fully English.

Oh, and previously I’ve observed half of my Notepad’s menu turning into Japanese. A precious moment, but I delayed taking a screenshot until the computer went down (electricity outage or Windows crash).

Or perhaps Windows has an advanced detection technology that could detect my kanji level. I’ve learned all kanji in the above screenshot. Maybe as I improve, more menu items will turn into Japanese.

goproblems.com is a wonderful community site containing thousands of… well… go problems. Problems are presented and solved using a Java applet. For me, the best thing about the site is that it is downloadable.

The downloaded problems can be navigated using the “previous” and “next” button. However, it iterates through problem id, which for me isn’t very useful.

My idea is to traverse the problems from the easiest to the hardest. My first solution was to copy-paste the contents of the “all problems” index into a spreadsheet and sort them there. Because the links in the speadsheet is clickable, I would navigate the problems from the spreadsheet.

My second solution was to use the sorted spreadsheet data to generate a set of htmls which allows me to traverse the problems without leaving the browser. Because OpenOffice.org Calc uses XML, parsing it was quite doable (I didn’t use System.XML, just some ad hoc parsing). The result was a frame based solution. The original goproblems.com page is contained in a frame and there is another frame for navigating backwards/forward. Bad usability because there were two sets of navigational buttons and the difference wasn’t obvious. Not to mention that frames are ugly.

Some days ago I downloaded a the latest goproblems.com snapshot (around 1000 new problems from my last one). I then finally forced myself to make a final solution. It was essentially a complete rewrite. I made the program read the difficulty directly from the html files, removing the need for manual sorting inside OOo Calc. I also made the program modify the URL of the already-existing navigational buttons, removing the need for frames.

So, here’s the result:

The program is here and the source code is here. It is made in C# so Mono or Microsoft .NET needs to be installed. To run it, just type:

gpsorter [input-directory] [output-directory]

or using Mono:

mono gpsorter.exe [input-directory] [output-directory]

Where [input-directory] is the directory of the unpacked and orginal goproblems.com.

Google now searches source code! The URL is http://www.google.com/codesearch. Other than indexing easily accessible source files (.cpp, .cs, .java, etc stored in a web server), it also searches inside a compressed file (like .zip) and code repositories (like cvs).

Anyway, I got 2 hits on Google code search :)…

To run another program from a .NET program, call the System.Diagnostics.Process.Start method. An example C# program is:

using System.Diagnostics;

class RunNewProcessTest
{
	static void Main()
	{
		Process.Start("notepad");
	}
}

In the example, Notepad will be invoked.

A document can be given to the start method (for example “c:\test.txt”). If there is an associated application for the file, the application will launch. Else an exception will be thrown.

If any arguments must be given to the invoked process, put it in the second argument of the Start method. An example is:

Process.Start("notepad", "c:\\test.txt");

(Note: backslash (’\') must be escaped)

In this regard, the Process.Start method is different from C or C++’s system function (defined on stdlib.h (C) or cstdlib (C++)), which would be:

system("notepad c:\\test.txt");

Sometimes our program needs resources in the form of files. An example is icons to show on buttons. Another example is the text of GNU GPL for display on a certain dialog box. Kanji dictionaries might use the text-based kanjidic file.

The obvious approach is to put the files on the file system. The advantage is that the resources can be updated or replaced easily. The disadvantage is file system clutter and external dependency (remove the resources and the program won’t run).

The other solution is to embed the resources in the executable or library itself. In this way, the executable/library will be self-contained.

Here’s how to do it in .NET using C#.

For the purpose of this guide, suppose the file to embed is test.txt which contains:

Line 1 of test.txt
Line 2 of test.txt

To embed the file into an assembly, just add the switch /res:FILENAME when compiling. For example, if the source code is Embed.cs then the command to compile should be:

csc /res:test.txt Embed.cs

(Replace csc with mcs if Mono’s compiler is used)

If the file is disassembled using Mono’s monodis, this will be found:

.mresource public 'test.txt'
{
}

which indicates that the file test.txt is indeed embedded into the assembly.

An example of a program which loads the resource is:

using System;
using System.IO;
using System.Reflection;

class Embed
{
  static void Main()
  {
    Assembly a = Assembly.GetAssembly(typeof(Embed));
    using(Stream s = a.GetManifestResourceStream("test.txt"))
    {
      using(StreamReader sr = new StreamReader(s))
      {
        Console.WriteLine(sr.ReadToEnd());
      }
    }
  }
}

The class from the System namespace that is used is Console. From System.IO is Stream and StreamReader while from System.Reflection is Assembly.

First, load the assembly that contains the resouce. Because the only class in the assembly is Embed, it is used to get the assembly:

Assembly a = Assembly.GetAssembly(typeof(Embed));

Now is the real meat. To open the stream to a resource in the assembly, call GetManifestResourceStream:

using(Stream s = a.GetManifestResourceStream("test.txt"))

The Stream object returned is just like a Stream object when a file is loaded from the file system. After that it’s just normal I/O.

If for some reason the embedded filename must differ from the original filename, use the option:

/res:FILENAME,EMBEDDED_FILENAME

I like to use the McCune-Reischauer form of Korean Go players’ name because that’s what Sensei’s Library use for page titles. Therefore, it is “Yi Se-tol” not “Lee Sedol” and “Yi Ch’ang-ho” not “Lee Changho”. Unlike other romanizations, we can reconstruct the hangul correctly from its McC-R form.

It was a pain in the arm to write articles containing Korean Go player names because I had to repeatedly look at the reference and do lots of “find & replace”. Therefore I made a small tool to do that job:

$ mcrconverter --help
This program will change Korean Go player names in a file into
their McCune-Reischauer form.

Usage:
mcrconverter [options] file:
    change the contents of a file
or
mcrconverter [options] folder:
    change the contents of SGF files in a folder

Options can be:
–change-file-name:
    also changes the file/folder name
–recursive:
    search for files inside subfolders recursively

Other than useful for writing articles, this tool can also be used to mass-rename the contents of an sgf file, and even the filename itself.

The source and executable are here (requires .NET Framework 2.0).

Problems worthy of attack
Prove their worth by fighting back
    - Piet Hein

A Princeton computer science major writes about his/her annoyance of being asked to fix computer problems.

What are problems in the field called “Computer Science” anyway? It’s NOT about how to make a web site. It’s NOT about fixing a machine that won’t boot up. It’s NOT about getting rid of worms investing your computer. It’s NOT about making the best hardware purchase.

Wikipedia has a list of unsolved computer science problems. It includes the famous P=NP. There is also a similar, significantly longer, list for mathematics and physics. This probably reflects the relatively young age of computer science. (those thinking mathematics is finished is dead wrong; the amount of mathematical research activity is in fact getting much bigger from time to time)

Anyway, does the reverse happen? (which means, people enrolling CS expecting to be taught about fixing computer problems)