Translate Toolkit and Firefox – a Primer

Velika planina

Image via Wikipedia

I recently took over the string management for the Slovene (sl) locale of Firefox. To make a long story short, the strings have not been maintained for a while and nobody was doing it for Firefox 3. To miss out on Firefox 3 would be a disaster, especially considering that Slovenia has consistently been in the top 2 for Firefox adoption in Europe of the last couple of years.

THE WHY

I am also involved with the Irish (ga-IE) locale, but more on an advisory rather than technical level. Kevin Scannell does a fantastic job managing the strings backed up by his small team of translators. So I needed a tool that would a) get the job done quickly, and b) utilise the strings already available from Firefox 2, and c) be relatively easy to use for an l10n maintainer novice like myself. It has been years since I tried Mozilla Translator, and I had memories of it being quite clunky (no offense, I’m sure it gets the job done). There are also quite a few new kids on the block, mainly web-based, as sethb talked about recently. But in the end I went with Translate Toolkit (TT) better known perhaps as moz2po, po2moz, and friends for a few reasons. First, it is proven and being used by a number of other locales. It is command line driven and quite elegant at that. And finally, I met Dwayne Bailey at FOSDEM a few years back and was very impressed at the passion he brings to the project and how the Translate Toolkit set of tools has brought a multitude of software to millions of new users in Africa and around the world.

What seemed to be lacking however, was decent up-to-date documentation for use with Firefox. So here I hope to fill in that gap, at least in some modest way. And I promised Dwayne, Friedal, and Axel that I would! So here it goes.

THE HOW

DISCALIMER : In the spirit of “there is more than one way to do it”, this is not intended to be THE WORD, but more of a nudge in the right direction. It works for me, but if you are using TT and know some better way, by all means let me know. I plan to put this on a wiki soon, so contributions are welcome.

Getting Started

I was coming late to the game with some 40 other Firefox 3 locales released during the beta period, so my goal was to catch-up quickly. Herein begins some instructions. For it to work of course with your locale, change sl to whatever your locale code is.

Firstly, cheat. Get Axel to do some CVS voodoo to move the FF2 strings over to the trunk.

Next, set up your base source and sync with CVS.

$ cvs -z3 -d:pserver:anonymous@cvs -mirror.mozilla.org:/cvsroot co mozilla/client.mk
$ cd mozilla
(make your .mozconfig here and put in in mozilla/ -- see below)
$ make -f client.mk l10n-checkout

My .mozconfig looks like this:

mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_CO_MODULE="mozilla/tools/l10n"
mk_add_options MOZ_CO_LOCALES=sl
mk_add_options LOCALES_CVSROOT=brian%mozdev.org@cvs.mozilla.org:/l10n
ac_add_options --disable-compile-environment
ac_add_options --enable-application=browser

Generate your first set of PO files.

$ moz2po -t l10n/en-US l10n/sl po/sl2
$ cd mozilla
$ python tools/l10n/l10n.py en-US
$ moz2po -P --duplicates=msgctxt l10n/en-US l10n/pot
$ pomigrate2 --use-compendium --pot2po po/sl2 po/sl l10n/pot

In short, these commands compare the en-US strings with the current set of sl strings, generates intermediary POT files, and from there generates the PO files. These initial set of PO files are probably the most important set you will generate, because they will contain the fuzzies, i.e. the strings moved over from FF2 that the engine thinks are a match but flags them for special attention. It is also a good idea to run pocount at this stage, to give the translators an idea of the task ahead of them.

Updating

So you have sent out the PO files and got them back, or have done a bit of translation. What’s next. You’ll want to convert them and get them checked in.

$ po2moz -t l10n/en-US/ po/sl l10n/sl

I always do a diff before checking in, to make sure some things are as you expect. Especially with the larger patches, I noticed a few files with bad or unwanted conversions that had to be cleaned up by hand. It is also a good idea to run compare-locales before checking in.

Syncing with en-US

Finally, new en-US base strings come down the pipe from time to time. Granted, it slows to a trickle the closer to release, but it still needs to be managed.

$ cd mozilla
$ make -f client.mk l10n-checkout
$ cd..
$ rm -rf l10n/en-US l10n/pot
$ cd mozilla
$ python tools/l10n/l10n.py en-US
$ cd..
$ moz2po -P --duplicates=msgctxt l10n/en-US l10n/pot

You now have a new set of .pot files in l10n/pot. Then…

pot2po -t po/sl l10n/pot po-updated/sl

Dwayne says could output po-updated/sl to simply po/sl and overwrite your old translations, but advises it only if you have po/ under version control. Each time I did an update, I just create a new foder — po-updated/, po-updated2/, and so on.

Now you can edit the new and updated strings. When you convert the PO files for checkin, remember to use the new PO folder:

po2moz -t l10n/en-US po-updated/sl l10n/sl

Now you have updates in l10n/sl for your translations.

That’s basically it. If you run a tight ship, and keep your strings in sync with en-US and checkin regularly, then everything should go smoothly. I did expereince a few mishaps where I had to go in and hand edit files, but that was more likely my fault though sometime the tools produced bad conversions. However, kudos to the Translate Toolkit team for making and maintaining such a useful and powerful set of tools.

If you have any tips on how to improve this process, or just want to share your TT experiences, I’d love to hear about it.

Tags: , , , , , , ,

3 Responses to “Translate Toolkit and Firefox – a Primer”

  1. Dwayne Bailey 23-May-2008 at 14:07 #

    Thanks Brian for the notes.

    Some things to add. It is important to note that when you generate your first set of PO files that both the l10n/en-US and your locale in l10n/ are in sync. So if your last official release was Fx2.0.0.3 check that out and create your PO files.

    Also for that step you will need to run l10.py before the recovery as we need the en-US directory that that script creates.

  2. Axel Hecht 23-May-2008 at 17:50 #

    I think in the “Generate your first set of po-files” code chunk, you need to generate l10n/en-US with l10n.py before doing the moz2po.

Trackbacks/Pingbacks

  1. seth’s blog » Blog Archive » “Replaneting” some blog posts - 28-May-2008

    [...] A post from May 21 showing a few localization video demos from the Translate.org.za… You might have seen this because Brian King blogged about it here. [...]

Leave a Reply