Translating ckan extensions using the ITranslations

From ckan 2.5 onwards you will be able to translate the strings in ckan extensions in a much more friendly and easy way. Unless there are any major issues in the code review, this pull request should make your life easier.

Previously, there were a few, not ideal solutions (see ckan#959) which involved having a paster command or script that would munge all the po/mo files together using gettext's msgcat command.

This has the downside that the sysadmin of any ckan instance would have to run this script and whatever other series of texts whenever they had a new ckan extension that they wanted to add.

The new pull request allows extension writers to provide a translations that will automatically be included without any additional steps other than the standard step specifying the plugin in the ckan.plugins in the configuration file.

To do this you'll need to copy the example plugin and edit your plugin so it looks a bit like

...
from ckan.lib.plugins import DefaultTranslation
...

class MyPlugin(plugins.SingletonPlugin, DefaultTranslation):
    plugins.implements(plugins.ITranslation)
...

The ITranslation interface and DefaultTranslation will mean that ckan will look for a directory in your extension ckanext/<your extension>/i18n for possible translations, your translation file for each locale should be ckanext-<your extension>. This is actually a gettext domain, in the future we might be able to add some more flexibility it so that only strings in an extension get overwritten even if they are the same as a ckan core string.

If you need help on gettext and how it works for ckan you should check out the ckan translation docs. Or take a look at the example_itranslation_plugin in the pull request

Given the previous attempts to fix this, I was pleasantly surprised with how much fun I had writing this.

Babel has support for an extended translation class that allows you to merge translations with an existing one. The problem was getting ckan to use it, I was expecting some awful hacks or messy code.

After poking around the source code to babel and pylons, it turns out that pylons eventually passes kwargs from set_lang to _get_translator which passes its keyword arguments to gettext , which is not mentioned in the pylons documentation. But that is the price you pay for being stuck on an older framework

 pylons/18n/translation.py

gettext import NullTranslations, translation
...
def _get_translator(lang, **kwargs):
    ....
    try:
        translator = translation(conf['pylons.package'], localedir,
                                 languages=lang, **kwargs)
    except IOError, ioe:
        raise LanguageError('IOError: %s' % ioe)
    translator.pylons_lang = lang
    return translator

def set_lang(lang, **kwargs):
    translator = _get_translator(lang, **kwargs)
    ...

In gettext translation, you can pass a Translation class in as the class parameter, so all it really involves is getting ckan to tell pylons to use the babel Translation class instead.

from babel.support import Translations
...
def _set_lang(lang):
    if config.get('ckan.i18n_directory'):
        fake_config = {'pylons.paths': {'root': config['ckan.i18n_directory']},
                       'pylons.package': config['pylons.package']}
        i18n.set_lang(lang, pylons_config=fake_config, class_=Translations)
    else:
        i18n.set_lang(lang, class_=Translations)

def handle_request(request, tmpl_context):
    lang = request.environ.get('CKAN_LANG') or \
        config.get('ckan.locale_default', 'en')
    if lang != 'en':
        set_lang(lang)

    for plugin in PluginImplementations(ITranslation):
        if lang in plugin.i18n_locales():
            _add_extra_translations(plugin.i18n_directory(), lang, plugin.i18n_domain())

    tmpl_context.language = lang
    return lang

def _add_extra_translations(dirname, locales, domain):
    translator = Translations.load(dirname=dirname, locales=locales,
                                   domain=domain)
    try:
        pylons.translator.merge(translator)
    except AttributeError:
        # this occurs when an extension has 'en' translations that
        # replace the default strings. As set_lang has not been run,
        # pylons.translation is the NullTranslation, so we have to
        # replace the StackedObjectProxy ourselves manually.
        environ = pylons.request.environ
        environ['pylons.pylons'].translator = translator
        if 'paste.registry' in environ:
            environ['paste.registry'].replace(pylons.translator, translator)

So simple! When ckan makes a call to set_lang, we pass in the babel Translation class in _set_lang, then when languages are changed in handle_request, we merge all the relevant translations from plugins implementing ITranslation.

There is an edge case for the 'en' language. The translation class won't exist yet and pylons will set the NullTranslation class, but we want to allow 'en' translations in extensions, this allows users to rename 'Organizations' to whatever they like in their custom instance without changing all the templates!

We do not have worry about permissions of merging files on the server, or making the user provide an extra command. All that is required is for plugin writers to add translations to their extensions and implement ITranslation, making the lives of our users simpler.

In the future, it will be worth exploring allowing extensions to specify strings that they do _not_ want to overwrite in core ckan translations, this will probably be have to be handled in extensions themselves.

Hooray for kwargs!