Firefox OS Phone Localization and Zanata

Background

Along with couple of other contributors I have been part of Mozilla Gujarati Localization Team from quite a long time now. As part of our community commitments, we do localize various products of Mozilla based on the need and availability of time. The mission is to get everything localized into Gujarati in this world of web. We really need more contributors joining the task force. Please join us, if you want to be part of this mission.

Gujarati got it’s place in one of the first two Indic languages released with early Firefox and Thunderbird versions and we always wanted to make sure we don’t lag behind for any localization project for the sake of community of native language users. I came to know about first release of Firefox OS Phone Localization very late, and I spent couple of nights in the race to complete the translations 100%. But later realized that the OS doesn’t have rendering support for complex scripts including Indic languages. Waste of efforts!!! :-(

Recent announcement Samsung phones to support 9 Indian languages caught attention of Mozilla L10n drivers and they started approaching currently active Indic Mozilla localizers in order to ship Indic languages with Firefox OS Phone in the upcoming releases. Certainly a good news.

Now, I had to equip myself to (re)start the localization!

Mozilla Tools & Processes

Mercurial repository called gaia where your translations are supposed to be submitted.
Localization Dashboard will tell you the missing bits you need to complete.

These are the tools provided by Mozilla Localization for anyone to work on localization. Choice lies with you (contributors) how you make use of it.

  1. Use any offline plain (or rich) text editors to translate .properties files and submit them to the Mercurial repository
  2. Use any online translation management systems to translate .properties files and submit them to the Mercurial repository

First case is pretty simple, straight-forward and convenient but won’t give you rich experience of localization in this era!

In the second case, Translate House has been landing their volunteer support in terms of managing Mozilla Localizations over their Pootle Online Instance. All you need to do is submit your translations using Pootle Online Instance for Mozilla. Someone from Translate House commits your translations to Mozilla Mercurial repository at a regular frequency. The most favourite method of contributing to Mozilla Localization these days.

Open Source is a place where you have a choice and so Localization @ Mozilla!

I have decided to try out Zanata for Firefox OS Phone Localization into Gujarati. I would describe my experience below. Please keep reading…

Zanata Server Configuration (one-time)

Project Creation

  1. Create an account or Log In to Zanata Open Instance
  2. Click “Projects” link at the top
  3. Click “Create Project” button at the right-top from “Actions”
  4. Enter “Project ID”, “Name” and “Description” as you find it appropriate
  5. Select “Project Type” as “UTF8Properties”
  6. Tick-mark “Would you like to add a customized list of locales?” and add only your locale that you want to be localized
  7. Rest of the fields can be filled or left untouched as per your requirement

Version Creation

  1. Click “Create Version” button at the right-top from “Actions”
  2. Enter the “Version ID”
  3. Rest of the fields can be filled or left untouched as per your requirement

Zanata Client

Sync Mozilla upstream repository into Zanata

  1. Download zanata.xml configuration file from “Config file” button at the right-top from “Actions”
  2. Open your terminal to run following commands
  3. Switch to your Home directory

    cd ~

  4. Create a directory from where you would manage translations

    mkdir zanata_firefoxos

  5. Switch to that newly created directory

    cd zanata_firefoxos

  6. Copy the downloaded zanata.xml file to your zanata directory

    cp ~/Downloads/zanata.xml .

  7. Switch to your Home directory

    cd ~

  8. Create a directory from where you would sync source/translations with Mozilla repo

    mkdir mozilla_firefoxos

  9. Switch to that newly created directory

    cd mozilla_firefoxos

  10. Clone source contents from Mozilla upstream repository

    hg clone http://hg.mozilla.org/gaia-l10n/en-US/

  11. Clone translations too that we would need later (replace “gu” with your locale code)

    hg clone http://hg.mozilla.org/gaia-l10n/gu/

  12. Change the names of the translation files as it’s required for Zanata to differentiate between source and translation contents (replace “gu” with your locale code)

    for i in `find gu/ -type f`; do j=`echo $i | sed ‘s#.properties#_gu.properties#’`; mv $i $j; done

    “_gu.properties” files needs to be renamed back to “.properties” when you have to submit translations!

  13. Copy the source and translation contents into Zanata directory (replace “gu” with your locale code)

    cp -r en-US/* ~/zanata_firefoxos/

    cp -r gu/* ~/zanata_firefoxos/

  14. Switch to Zanata directory

    cd ~/zanata_firefoxos/

  15. Push translations to Zanata Server.

    mvn zanata:push -Dzanata.pushType=both

    I am using Maven Client to push/pull translations to/from Zanata. One can use other clients available too to interact with Zanata Server.

Manage translations with Zanata

After you have accomplished all of the above steps, you are ready to start translations using Zanata Online at: https://translate.zanata.org/zanata/iteration/files/gaia/Aurora/gu

You could also download the translations in the .po file format as well as .properties format too. This will also allow you to use your favourite offline translation tools like lokalize or virtaal or so. Uploading of translations to Zanata using these file-formats is available too.

Please check the Zanata documentation for better information about how to use Zanata to translate online.

Sync translations from Zanata into Mozilla upstream repository

  1. Open your terminal to run following commands
  2. Switch to your Home directory

    cd ~

  3. Switch to the Zanata directory

    cd zanata_firefoxos

  4. Pull translations from Zanata Server.

    mvn zanata:pull -Dzanata.pullType=trans

  5. Change the names of the translation files as it’s required for Mozilla (replace “gu” with your locale code)

    for i in `find . -type f`; do j=`echo $i | sed ‘s#_gu.properties#.properties#’`; mv $i $j; done

  6. Copy the translation contents into Mozilla directory (replace “gu” with your locale code)

    cp -r ~/zanata_firefoxos/apps/ ~/mozilla_firefoxos/gu/

    cp -r ~/zanata_firefoxos/shared/ ~/mozilla_firefoxos/gu/

  7. Switch to your mozilla Gujarati directory

    cd ~/mozilla_firefoxos/gu/

  8. Add, commit and push translations to Mozilla hg repository

    hg add .

    hg commit -m “Added and updated translations”

    hg push

  9. That’s it!

What next?

For anyone interested in using Zanata to manage their Firefox OS localization may not necessarily have to go through all these steps I have mentioned above. Instead they could use existing setup to host and manage their translations. All you would have to do is translate using Zanata online just like you do use Pootle these days. Rest will be taken care of. Please reach out to me, if you are interested and let’s talk to Mozilla l10n drivers as well in case there’s a confusion.

Gujarati translations of Firefox OS Phone are completed and now it requires heavy amount of translation quality review and testing. Please come and contribute.

As always happy to hear your feedback, inputs, suggestions, complaints, questions or anything about the stuff that I write.

Thank You,
Ankit Patel

Translation Quality Assessment Matrix

Rajesh Ranjan, the man behind FUEL Project, brought the idea of building some formula that helps to assess the quality of translations. To fulfill this need, we worked together to prepare a Translation Quality Assessment Matrix, TQAM.

TQAM is released at the recent FUEL GILT Conference 2013 under GNU GPL v3 license. To my knowledge, TQAM is the first of it’s kind in the field of Open Source Localization.

TQAM is available here: http://fuelproject.org/tqam/index

What is TQAM? – A Concept

TQAM is a web based tool to help you assess a translation.

Why would you need TQAM? – The Advantage

If an answer to any of the following questions is yes, you may need it.

  • Want to assess translations?
  • Want to identify translators’ strengths or improvement areas?
  • Want to figure out whether a particular translation could be critical or blocker?

Examples:

  • I am a Gujarati translator for the past couple of years for Fedora Project and I have a new member requesting to join the Gujarati Localization efforts. How do I know whether the new member is really capable of translating? How do I know the areas of improvement for the new member? Whether I should approve or deny his/her request?
  • I am Hindi Language Co-ordinator for LibreOffice Localization project and I would like to give “Translator”, “Reviewer” and “Committer” roles to different contributors from my group. How would I identify and differentiate their strengths & areas of improvements and assign roles respectively?
  • I am a Localization Lead for Mozilla Localization project and I have a member requesting to start a new language that’s unknown to me. How would I know whether the person requesting to start a new language is really native to that language?

Of course, you can roughly figure out and make a decision, though there’s no concrete backing that could be common across. TQAM would make it a common tool and process across the evaluators to bring consistency while assessing translations.

How to use TQAM? – The Process

TQAM works on assessment of translations based on the formula created with the help of Error Categories, Assessment Parameters & Sub-Parameters and Error Points.

Error Categories:
Trivial – can ignore
Venial – can forgive
Critical – must be fixed
Blocker – unacceptable

Parameters:
Accuracy
Language
Technical Validation
Locale
Style & Structure
Terminology

and various other sub-parameters for these parameters.

Definitions or descriptions about each of these parameters or sub-parameters could be found from TQAM web.

Steps:

  1. Enter the word count (of the source) in “Total Words” section
  2. Go through the translations manually
  3. Provide number of errors found in translations for the respective parameters/sub-parameters
  4. Hit Enter or click “Calculate” button if you are done
  5. End of the assessment phase TQAM would produce an assessment result in terms of marks out of 100
  6. TQAM would also gives you scoring for each parameter

Inputs/Feedback/Suggestions:

I have already started receiving feedback: https://fedorahosted.org/fuel/wiki/fuel_tqam/feedback You can jump in too otherwise do not hesitate to drop an e-mail on https://fedorahosted.org/mailman/listinfo/fuel-discuss

Thank You!
Ankit Patel

Contextualization for Software Localization

Localization industry has been revamping and coming up with lots of improvements these days. Number of languages getting increased in most of the projects’ portfolios. Many good news around! At the same time there are number of challenges yet to overcome. e.g. Data encoding, file formats, machine translations, online translation tools, translation memory, spell checkers, meta data, insufficient information/reference about translation messages, and you know there are many more to this list…

My focus in this blog post would be to emphasize and overcome the issue of Insufficient information/reference about translation messages or in other words Context for the translation messages. Lack of Context for translation messages is really a critical issue for the localization industry today, not just for translators! e.g. How would you translate the message “Empty Trash” in your native language. Would you translate it like whether the trash is empty or make the trash empty? Unless you see the actual source message in the user interface and use the functionality you wouldn’t be able to translate these kind of ambiguous messages correctly!

Projects like Deckard are of great help for such instances, however I think that’s not the only or real solution to the problem we are facing today!

During the Open Source Language Summit organized at Red Hat office Pune, I got a chance to meet all awesome members from Wikimedia Language Engineering Team. I came to know that Siebrand Mazeland, takes care of reviewing almost every message of the source material that comes to translatewiki.net before it goes to translators. We call it i18n review. Hat’s off Siebrand! Not an easy job! Certainly, this is very much helpful to translators.

For localizers translating various open source projects, where nobody takes care of reviewing translatable messages nor provide context it becomes the most challenging translation job at times!

How can we fix the problem?

Another, not an easy solution, though I would like to give it a try!

Concept: Review and modify the source code of software applications in order to add comments or references or context for the translatable messages in order to make them more meaningful so that it could be easily understood by translators that will help us get the best localization in the industry!

Skill-sets required:

  • Software Engineer – to help us create patches for software source code and communicate with mainstream developers about it
  • Quality Engineer – to help us understand the functionality of the software functionalities and better understand the usage of the translatable messages
  • Technical Writer – to help us write better translatable messages

Strategy:

  1. Form a group consisting of people with each of the skill-sets as mentioned above!
  2. List out various important open source software projects that come from Fedora, Gnome, Kde, Firefox, LibreOffice, etc.
  3. Start approaching them with the concept and idea behind the initiative
  4. Figure out contextual information for each and every translatable message
  5. Accordingly, provide software source code updates or patches whatever is possible
  6. Ensure that the contextual information goes into the translatable file formats provided to translators
  7. Translators do get the contextual information in the form of translation formats
  8. And we receive good quality translations!
  9. That’s pretty much all I guess!

Though I believe there is always a scope of further improvements and changes in the strategy or approach we may come to conclusion as soon as we form a group.

If we are able to convince even a single software application and it’s developers about the idea and get a buy-in, I think we have a ray of hope for the movement that can begun!

Got an interest to do something for the software localization? Got another idea better than this one? Have suggestions to make? Please please do provide your comments and let’s work together to start fixing all software localization issues one by one!

Thank You for reading!
Ankit Patel

“Unicode Text Rendering Reference System” brings six more Indic languages

“Unicode Text Rendering Reference System” also referred as UTRRS.

Few months ago UTRRS developer community announced a development instance of UTRRS, which is hosted on Red Hat’s OpenShift and can be found here: http://utrrs-testing.rhcloud.com/

Initially UTRRS had just five languages (Bengali, Gujarati, Hindi, Punjabi and Tamil). With the addition of six more Indic languages (Assamese, Kannada, Malayalam, Marathi, Oriya, Telugu) to the list UTRRS has now total of 11 Indic languages to be used for developers and localizers or native language users around.

This effort would help reducing the gap of understanding about combinations of scripts between developers and language experts. It would also mark as a standard reference point for anybody looking for accurate native script grammar data.

Please spend some time to go through it and provide your valuable feedback about it on the UTRRS mailing list.

Thanks for reading,
Ankit

Fedora Language Testing Group

I have been engaged with Localization activities for various projects including Fedora, but nowhere I have noticed a dedicated group of volunteers contributing to not just translate or localize software but also test their languages in a professional manner.

In the beginning of March, there’s a new group called “FLTG” (Fedora Language Testing Group) has been formed under Fedora’s SIG (Special Interest Groups).

FLTG folks take care of developing testing strategy, infrastructure, test cases and bug filing, bug triaging, helping testers and many more things associated with Language Testing for Fedora. You may want to join this move and contribute. Initial announcement about the group and information is available here: http://lists.fedoraproject.org/pipermail/trans/2012-March/009703.html

  • If you are a localizer for Fedora, this is the time you want to join this group and help test your language.
  • If you are a native language user of Fedora, this is the time you may want to test your language and take a help from FLTG

Show some love for your language and help testing languages in Fedora!

/Ankit