Translate Toolkit and Firefox – a Primer

Velika planina

Image via Wikipedia

I recently took over the string management for the Slovene (sl) locale of Firefox. To make a long story short, the strings have not been maintained for a while and nobody was doing it for Firefox 3. To miss out on Firefox 3 would be a disaster, especially considering that Slovenia has consistently been in the top 2 for Firefox adoption in Europe of the last couple of years.

THE WHY

I am also involved with the Irish (ga-IE) locale, but more on an advisory rather than technical level. Kevin Scannell does a fantastic job managing the strings backed up by his small team of translators. So I needed a tool that would a) get the job done quickly, and b) utilise the strings already available from Firefox 2, and c) be relatively easy to use for an l10n maintainer novice like myself. It has been years since I tried Mozilla Translator, and I had memories of it being quite clunky (no offense, I’m sure it gets the job done). There are also quite a few new kids on the block, mainly web-based, as sethb talked about recently. But in the end I went with Translate Toolkit (TT) better known perhaps as moz2po, po2moz, and friends for a few reasons. First, it is proven and being used by a number of other locales. It is command line driven and quite elegant at that. And finally, I met Dwayne Bailey at FOSDEM a few years back and was very impressed at the passion he brings to the project and how the Translate Toolkit set of tools has brought a multitude of software to millions of new users in Africa and around the world.

What seemed to be lacking however, was decent up-to-date documentation for use with Firefox. So here I hope to fill in that gap, at least in some modest way. And I promised Dwayne, Friedal, and Axel that I would! So here it goes.

THE HOW

DISCALIMER : In the spirit of “there is more than one way to do it”, this is not intended to be THE WORD, but more of a nudge in the right direction. It works for me, but if you are using TT and know some better way, by all means let me know. I plan to put this on a wiki soon, so contributions are welcome.

Getting Started

I was coming late to the game with some 40 other Firefox 3 locales released during the beta period, so my goal was to catch-up quickly. Herein begins some instructions. For it to work of course with your locale, change sl to whatever your locale code is.

Firstly, cheat. Get Axel to do some CVS voodoo to move the FF2 strings over to the trunk.

Next, set up your base source and sync with CVS.

$ cvs -z3 -d:pserver:anonymous@cvs -mirror.mozilla.org:/cvsroot co mozilla/client.mk
$ cd mozilla
(make your .mozconfig here and put in in mozilla/ -- see below)
$ make -f client.mk l10n-checkout

My .mozconfig looks like this:

mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_CO_MODULE="mozilla/tools/l10n"
mk_add_options MOZ_CO_LOCALES=sl
mk_add_options LOCALES_CVSROOT=brian%mozdev.org@cvs.mozilla.org:/l10n
ac_add_options --disable-compile-environment
ac_add_options --enable-application=browser

Generate your first set of PO files.

$ moz2po -t l10n/en-US l10n/sl po/sl2
$ cd mozilla
$ python tools/l10n/l10n.py en-US
$ moz2po -P --duplicates=msgctxt l10n/en-US l10n/pot
$ pomigrate2 --use-compendium --pot2po po/sl2 po/sl l10n/pot

In short, these commands compare the en-US strings with the current set of sl strings, generates intermediary POT files, and from there generates the PO files. These initial set of PO files are probably the most important set you will generate, because they will contain the fuzzies, i.e. the strings moved over from FF2 that the engine thinks are a match but flags them for special attention. It is also a good idea to run pocount at this stage, to give the translators an idea of the task ahead of them.

Updating

So you have sent out the PO files and got them back, or have done a bit of translation. What’s next. You’ll want to convert them and get them checked in.

$ po2moz -t l10n/en-US/ po/sl l10n/sl

I always do a diff before checking in, to make sure some things are as you expect. Especially with the larger patches, I noticed a few files with bad or unwanted conversions that had to be cleaned up by hand. It is also a good idea to run compare-locales before checking in.

Syncing with en-US

Finally, new en-US base strings come down the pipe from time to time. Granted, it slows to a trickle the closer to release, but it still needs to be managed.

$ cd mozilla
$ make -f client.mk l10n-checkout
$ cd..
$ rm -rf l10n/en-US l10n/pot
$ cd mozilla
$ python tools/l10n/l10n.py en-US
$ cd..
$ moz2po -P --duplicates=msgctxt l10n/en-US l10n/pot

You now have a new set of .pot files in l10n/pot. Then…

pot2po -t po/sl l10n/pot po-updated/sl

Dwayne says could output po-updated/sl to simply po/sl and overwrite your old translations, but advises it only if you have po/ under version control. Each time I did an update, I just create a new foder — po-updated/, po-updated2/, and so on.

Now you can edit the new and updated strings. When you convert the PO files for checkin, remember to use the new PO folder:

po2moz -t l10n/en-US po-updated/sl l10n/sl

Now you have updates in l10n/sl for your translations.

That’s basically it. If you run a tight ship, and keep your strings in sync with en-US and checkin regularly, then everything should go smoothly. I did expereince a few mishaps where I had to go in and hand edit files, but that was more likely my fault though sometime the tools produced bad conversions. However, kudos to the Translate Toolkit team for making and maintaining such a useful and powerful set of tools.

If you have any tips on how to improve this process, or just want to share your TT experiences, I’d love to hear about it.

76 Replies to “Translate Toolkit and Firefox – a Primer”

  1. I wish to show my appreciation to you for bailing me out of this type of matter. Just after exploring throughout the the web and coming across ideas that were not powerful, I assumed my entire life was well over. Living devoid of the solutions to the problems you have fixed through your good posting is a critical case, and ones which may have adversely damaged my entire career if I hadn’t encountered the website. Your ability and kindness in touching all the things was invaluable. I don’t know what I would’ve done if I had not come upon such a stuff like this. I can also at this moment relish my future. Thanks for your time very much for this professional and effective guide. I will not hesitate to recommend the website to any individual who should get support on this issue.

  2. Thanks so much for giving everyone an extremely nice possiblity to discover important secrets from this website. It is often very cool and as well , jam-packed with amusement for me and my office peers to visit your web site at minimum thrice every week to study the newest guidance you have. Not to mention, I am usually amazed with all the wonderful methods you give. Some 1 facts in this posting are certainly the most impressive we have all ever had.

  3. I truly wanted to write a small note to be able to appreciate you for all of the fantastic tips you are giving on this site. My rather long internet lookup has at the end been compensated with brilliant insight to exchange with my visitors. I ‘d admit that many of us website visitors are unquestionably fortunate to be in a good place with very many perfect people with useful guidelines. I feel rather blessed to have discovered the website page and look forward to some more thrilling moments reading here. Thank you once more for everything.

  4. I would like to convey my love for your kindness in support of those individuals that absolutely need assistance with this particular area. Your very own dedication to passing the solution along became unbelievably helpful and have all the time enabled professionals just like me to arrive at their objectives. This warm and friendly advice can mean this much a person like me and substantially more to my fellow workers. Many thanks; from all of us.

  5. I wish to express thanks to you for rescuing me from such a scenario. Right after researching throughout the world wide web and finding concepts that were not helpful, I thought my life was gone. Being alive without the presence of approaches to the problems you’ve fixed all through your good article content is a critical case, as well as the ones that might have in a negative way damaged my entire career if I hadn’t encountered your blog post. Your main ability and kindness in taking care of all areas was very useful. I am not sure what I would’ve done if I hadn’t discovered such a stuff like this. I can also at this point look forward to my future. Thanks a lot so much for this high quality and sensible help. I will not hesitate to recommend your blog post to anybody who requires guidance about this subject matter.

  6. An impressive share, I just given this onto a colleague who was doing a bit analysis on this. And he actually purchased me breakfast as a result of I found it for him.. smile. So let me reword that: Thnx for the deal with! However yeah Thnkx for spending the time to discuss this, I really feel strongly about it and love reading extra on this topic. If attainable, as you develop into experience, would you mind updating your weblog with extra details? It’s highly helpful for me. Massive thumb up for this weblog submit!

  7. I am just commenting to make you know of the superb discovery my wife’s princess gained viewing your blog. She even learned plenty of issues, including what it’s like to possess a great helping nature to make many people with no trouble know just exactly a variety of specialized subject matter. You undoubtedly surpassed visitors’ expectations. Thanks for rendering such great, trusted, edifying and also easy tips on your topic to Gloria.

  8. I precisely needed to thank you very much once again. I’m not certain the things that I would’ve gone through in the absence of those information provided by you about that subject matter. It truly was a very frightful scenario in my opinion, however , finding out this well-written technique you resolved that forced me to leap with contentment. Now i am thankful for this advice as well as hope you know what a powerful job your are undertaking instructing men and women using your blog. I know that you’ve never got to know all of us.

  9. I just wanted to post a quick note to express gratitude to you for these precious concepts you are writing on this website. My considerable internet lookup has at the end of the day been rewarded with awesome content to talk about with my good friends. I would state that that most of us website visitors actually are unquestionably blessed to dwell in a great community with so many perfect professionals with very beneficial methods. I feel truly lucky to have encountered your entire web site and look forward to some more amazing minutes reading here. Thank you again for everything.

  10. 5 billion deal with DynCorp International to train Iraqi police forces, The other wanted to return to taekwondo,By Miranda Leitsinger Ban should immediately distance the UN from this shameful conference, Lastly,Peters said she had no immediate word on the six survivors’ condition.Jim Hickerson,??nd while added sugar has been reduced,??General Mills spokeswoman told the Tribune. including Flurry.
    kobe 10 elite

  11. I simply needed to thank you so much all over again. I am not sure the things I would have implemented in the absence of the ideas discussed by you directly on my subject. This has been an absolute frustrating dilemma in my view, nevertheless finding out a new professional style you processed that made me to jump over joy. Extremely happy for your service and in addition wish you know what a powerful job that you are providing training people today via your webpage. I’m certain you have never encountered all of us.

  12. I wanted to compose you the tiny note to finally give many thanks the moment again on your pretty tips you’ve shared above. This is unbelievably open-handed of people like you to supply unhampered precisely what a lot of people would have supplied as an ebook to generate some bucks for themselves, primarily seeing that you might well have tried it if you decided. The inspiring ideas as well acted to become a fantastic way to be sure that other individuals have the identical zeal much like my own to understand significantly more in terms of this issue. I think there are thousands of more pleasant periods up front for individuals who browse through your blog post.

  13. I want to express my admiration for your kindness giving support to all those that need help on in this concept. Your special commitment to getting the message all through appeared to be surprisingly helpful and has without exception made workers just like me to get to their pursuits. Your important tutorial means a lot a person like me and especially to my mates. Thanks a lot; from each one of us.

  14. I wanted to post a word to be able to say thanks to you for the unique ideas you are sharing at this website. My extensive internet investigation has at the end of the day been paid with awesome strategies to talk about with my guests. I ‘d suppose that we readers actually are quite endowed to live in a superb website with so many brilliant people with good suggestions. I feel very lucky to have seen the weblog and look forward to tons of more amazing minutes reading here. Thanks a lot again for everything.

  15. I precisely desired to appreciate you once again. I’m not certain the things I would have tried without these methods shared by you on such a topic. This was a real troublesome concern for me, but being able to see this specialised style you solved that made me to leap over delight. Now i am thankful for the guidance as well as pray you realize what a great job that you’re getting into teaching others through the use of your website. Probably you haven’t come across any of us.

  16. I precisely wanted to thank you so much again. I am not sure what I would’ve tried without the pointers shown by you regarding such a theme. It absolutely was a very scary crisis for me, but encountering a new professional technique you resolved it forced me to jump for happiness. I am just grateful for the help and then expect you really know what an amazing job that you’re providing instructing some other people by way of your web blog. Most probably you have never got to know any of us.

  17. I have to express my affection for your generosity giving support to those who absolutely need assistance with this question. Your real dedication to getting the message all through came to be exceedingly important and have constantly empowered guys and women like me to attain their desired goals. The warm and friendly recommendations can mean a lot a person like me and additionally to my mates. Thanks a lot; from everyone of us.

Leave a Reply

Your email address will not be published. Required fields are marked *