Feature

dates are currently not localized

Summary

open
Dec 11, 2006
90%
Dec 11, 2006 / pixtur
Jan 5, 2009 / phsouzacruz
ganesh
 

Attached files

Summary
bug_gmstrftime.png
bug_gmstrftime.png

#3795 by pixtur, 20k
Clipboard01.jpg
Clipboard01.jpg

#4028 by cvivanco, 22k
Clipboard02.jpg
Clipboard02.jpg

#4029 by cvivanco, 8k
 
About localization, I have spotted a few places where there's room for improvement:
  • dates are currently not localized, moreover date/gmdate always produces english names for weekdays and months, strftime/setlocale should be used instead,


also see:

29 Comments

madlyr:Please wait a little, I did some work but not finished it yet

8 years ago (2. update 8 years ago)

I localised calendars and as I remember (it was 2 weeks ago) it was not finished yet. In a free time (hehe) I finish it and commit changes.

ganesh:Reply to Please wait a little, I did some work but not finished it yet

8 years ago

How did you localise calendars? I am now finished, but I'm waiting for you to commit. My changes are located mainly in the function renderDate/renderTime, etc. (file render.misc.php), plus locale selection in function setLang (file common.inc.php). Did you use setlocale also?

madlyr:Reply to Reply to Please wait a little, I did some work but not finished it yet

8 years ago

It was not finished yet change made 2-3 weeks ago, so I don't remember ale the things. I have no time at lately, but I try to commit "my way" as is tomorrow.

pixtur:cool.

8 years ago


ganesh:Committed

8 years ago

I've committed the fix. The main differences are:
  1. function setLang() in common.inc.php calls setlocale() to set the proper locale for date formatting.
  2. the config settings FORMAT_DATE, FORMAT_TIME and and FORMAT_TIMESTAMP have been removed. Proper format strings will have to be provided by translators as regular language strings. So each user will see the dates properly formatted according to its preferred language.
  3. a new config setting FORCE_LOCALE has been added: it can be used to override the per-user setting and force the same locale for all languages. The most important use for this setting is that setting value "C" completely disable localized formatting. Useful if the underlying PHP platform does not have proper locale support.
There is a minor issue with the setlocale() call: the names of locales vary among platforms. So, in order to be as portable as possible, I used the feature of setlocale() that allows to specify more than one locale name. Each name is checked in order, until a valid one is found. For example for en-US language, the string is "en_US.utf8,en_US,enu". The first one "en_US.utf8" is the name used by most unices with utf8 support. The second one "en_US" is the same without utf8. The third one "enu" is the name used by Windows-based servers. If you know of any other names for platform we want to support, they must be added to the string. This string needs also be localised in every language tables.

pixtur:Reply to Committed

8 years ago (2. update 8 years ago)

hmm... That are quiet a lot of changes. I wasn't aware that this would leed to so much code. Can you once again explain, why we not just add the FORMAT_TIMESTAMP etc as translation key like:

 confSet('FORMAT_TIMESTAMP', __('D, d.M Y H:i'));

or

if(__('__FORMAT_TIMESTAMP__|check translation guide') != '__FORMAT_TIMESTAMP__') {
 confSet('FORMAT_TIMESTAMP', __('__FORMAT_TIMESTAMP__'));
}

This would keep up us from using php's setlocale() function and still allow a straight forward method of localizing the date format. I had soooo much problems with the php date stuff, I don't want to use it anymore. I have nightmares about someone coming with an early v5.0.0 and telling me setlocale works different or not at all.

setlocale() is one of the reasons I would never use PHP as a programming language again :)

I don't want to be too restrictive, though. Sorry...

pixtur:Sorry. I am not convinced...

8 years ago (2. update 8 years ago)

I just looked at your code and now understand, why using the translation table is not sufficient to translate Dates with Weekdays or Months.

Anyway, since the gmstrftime() function does not work as expected on my version (see attached screenshot. The %e parameter is ignored), I do not trust it at all. I understand that you want a perfect translation, but I am for stability first.

If we cannot fix the missing day number on my php version soon, I vote for reverting the setlocale() and gmstrftime() stuff. Sorry. Saying this is hard.

Please also read I do not like new features

Things like this really makes my day. It makes me hurt to see how crappy PHP is with stability. Although we could probably do not better, there still is the feeling I want to throw something on my office wall. I will build a PHP voodoo puppy which I can torture with needles and stuff.
I honestly thought not just twice how much effort it would be to fuck PHP and rebuild streber from scratch with rubyOnRails. But hey: I only got one life...

Live is a bitch and then you die :)

ganesh:Mmmm

8 years ago

I understand, but really you can't talk of a translation while keeping dates in English. Project management is all about dates! I don't think about this as a "new feature", at least it won't be perceived like that from the end-user.
Anyway, you're the boss.
Apart from ditching the whole stuff, we have two more options:
  1. make FORCE_LOCALE = "C" by default and change the implementation so that in that case it reverts to the old behaviour (with date() instead of gmstrftime()). With this change people using locales will do so at their own risk.
  2. implement our own gmstrftime-like function, using translation tables for weekdays and months. It's actually not that hard. We just need 21 more localized strings (12 months + 7 weekdays + am/pm), not a big deal.
If you agree, I can make the changes in the next few days. As a quick (and dirty) fix for the %e problem, maybe just using %d instead of %e might work.

pixtur:partly fixed...

8 years ago

Sorry for being too restrictive. I don't want to look like a "boss"...

Using %d fixed the missing day problem. It took a while until I figured out, how to get a German date be localized. Since the %d does the fix, let us keep the current implementation.
If some users complain, we can easily revert the changes.

I don't think that implementing an own localized version of gm would be a good idea. Although it would be easy, it would further slow down rendering.

Could you add a small chapter on to be translated locale settings to the Translation Guide. I see that there are four important keys:

 en_US.utf8,en_US,enu|list of locales
 %b %d, %Y|strftime format string
 %I:%M%P|strftime format string
 %a %b %d, %Y %I:%M%P|strftime format string

or are there some more? Maybe some links to the php functions will help translators to figure out what is meant...

ganesh:Reply to partly fixed...

8 years ago

I added a few lines to the guide, as you suggested.

The problem with the %e formatter is rather annoying, I really didn't expect it to happen. Is your PHP-box Windows-based? According to http://php.net/manual/en/function.strftime.php#53340, Windows indeed does not support the %e formatter. I wasn't aware of that. Anyway, as this case can be easily detected in the code, it could be handled separately without too much overhead and without forcing everyone to use the %d formatter. Let me attempt a fix for that.

pixtur:Reply to Reply to partly fixed...

8 years ago

You are perfectly right. I am still working on a windows system :) If this is really a known problem, we could easily replace %e with %s if OS == Win . This would be a sufficient solution for the problem.

btw: Excellent work on the translation guide!

cvivanco:Reply to Reply to Reply to partly fixed...

8 years ago

In the language file you can add instead of %e, %#d to suppress the leading zero for the date. This is a known bug for win32 systems.

ganesh:Reply to Reply to Reply to Reply to partly fixed...

8 years ago (2. update 8 years ago)

Really? I didn't know that. Good to know. I don't have access to a Windows-based server, so could you please help test that? Try changing the following lines:

from render/render_misc.inc.php, lines 585-587

        // fix %e formatter if not supported (e.g. on Windows)
        if(strftime("%e", mktime(12, 0, 0, 1, 1)) != '1')
            $userFormatDate = str_replace("%e", "%d", $userFormatDate);
into

from render/render_misc.inc.php, lines 585-587

        // fix %e formatter if not supported (e.g. on Windows)
        if(strftime("%e", mktime(12, 0, 0, 1, 1)) != '1')
            $userFormatDate = str_replace("%e", "%#d", $userFormatDate);
(third line changed) and let me know if it works. Thanks!

ganesh:

8 years ago

Fixed. I used strftime itself to check if %e is supported. I added a bit of caching so the impact on perfomances should be negligible.

cvivanco:Problem with date encoding with UTF8

8 years ago (2. update 8 years ago)

I work with a language encodede with UTF8, but all the date are wrong encoded
Clipboard01.jpg
Clipboard02.jpg

ganesh:Reply to Problem with date encoding with UTF8

8 years ago (2. update 7 years ago)

I see. First of all, the translation of the string 'en_US.utf8,en_US,enu|list of locales' should keep all possible spellings for the locale, so it should be, in the case of the Spanish language, 'es_ES.utf8,es_ES,esn' (possibly in that order) and not simply 'esn' as it now. Otherwise the translation won't work on any *nix-based machine.
Second, it definetly seems that the string 'esn' is instructing Windows to produce strings in the windows-1252 codepage, rather than utf-8. According to MSDN, the notation language.codepage should be supported, but I don't know what string to use as codepage to get utf-8. Could you please try something like 'esn.utf8' or 'esn.utf-8' or 'esn.utf_8' and tell me if any one of those produces the correct result? Thanks in advance.

ganesh:Reply to Problem with date encoding with UTF8

8 years ago (2. update 8 years ago)

Damn... according to MSDN:

The set of available languages, country/region codes, and code pages includes all those supported by the Win32 NLS API except code pages that require more than two bytes per character, such as UTF-7 and UTF-8. If you provide a code page like UTF-7 or UTF-8, setlocale will fail, returning NULL.

So apparently gmstrftime() can't be used to produce correctly utf-8 encoded strings under Windows. We need to convert them explicitly with utf8_encode(). It's really a shame.

pixtur:Damn

8 years ago (2. update 8 years ago)

I hoped to release v0.08 at the weekend. But still errors pop up here and there. Damn it.

Should we revert to the old date function then, or can the conversion be done soon enough?

madlyr:Reply to Damn

8 years ago

In my opinion we should revert to old functions - we are not prepared to do such a big revolution...

madlyr:I added some info in Translation guide topic

8 years ago (2. update 8 years ago)

I see this topic is more appropriate - see ->

ganesh:

8 years ago

These are really minor problems, that can easily be fixed, IMHO. I just need a hand testing them under Windows...
Anyway, you don't need to actually revert all the code, if you don't want to include it in v0.08. Just make the default value of the configuration setting FORCE_LOCALE to "C" instead of "". People who are unaware of the feature will not even notice that it exists.

pixtur:sounds great, but...

8 years ago (3. update 8 years ago)

I still have a weird feeling about this stuff getting out of control. Although I long gave up perfectly understanding all code in streber, I roughly knew what is going on. But the setlocale-stuff is completely behind my horizon.

If you have any code that needs to be tested under Windows, I will be glad to help out.

If the confChange('FORCE_LOCALE','C') solutions works for madlyr, we should change this. Otherwise I would vote of reverting the code and live with a non perfect translation. Maybe it will get most stable in php6...


ganesh:Reply to sounds great, but...

8 years ago

Well... let's not exaggerate... it's not a bomb! It's not out of control. It works under Linux. We know how to fix it under Windows. There's an easy way to skip the code entirely on all other platforms, in case it doesn't work. We can conceivably check the feature programmatically at install time so that the user is not even aware of a workaround. What more do you want? Frankly I don't understand what are you afraid of.
Anyway, as I replied to madlyr in , we can get away even without setlocale if we want to. The feature can easily be made optional, so performances will be unaffected unless the user wants. Maybe I should just stop talking and provide a working implementation. Maybe we could profile it and find that it doesn't affect performances so much...

madlyr:Reply to sounds great, but... - FORCE_LOCALE

8 years ago (2. update 8 years ago)

confChange('FORCE_LOCALE','C'); disabled that ugly warning.

If it reverts to old functionality, then it's OK, but we have to speak loud in docs and in release info what to do if web server doesn't return result of setlocale or doesn't properly sets locale.

We have to add to other languages proper translations. I updated Polish language at rev. 285.


Personally I don't like names of days and months in lists at all, so for Polish language I changed default strings to strictly numeric dates, these are perfectly what I like ;-)

Maybe we put those blocks to other languages instead of default English one?

from pl.inc.php

### ../render/render_misc.inc.php   ###
'%b %e, %Y|strftime format string'=>'%Y-%m-%d',  # line 579
'%I:%M%P|strftime format string'=>'%H:%M',  # line 592
'%a %b %e, %Y %I:%M%P|strftime format string'=>'%Y-%m-%d %H:%M',  # line 601
'%s min'                      =>'%s min',  # line 696
'%A, %B %e|strftime format string'=>'%Y-%m-%d',  # line 921

pixtur:The is matter of taste...

8 years ago

to ganesh:
You are perfectly right. Discussing for the sake of discussing is of no good. There is always the force of "facts". If you implement a working solution, I am very happy to use it and concentrate to other problems.


to madlyr:
I like names a lot, because it makes a program less technical and more "human". Hard to explain. I think that even in the translations we not go for personal taste but for accurate meaning. In this putting the format into the translation is already a little bit weird.

Anyway. The best option for overwriting the display of names would be an option that could be placed to the customize.inc.php. We could easily overwrite translation keys there, although it needs some documentation.


madlyr:Reply to The is matter of taste...

8 years ago

I agree. Customization for date/time display SHOULD be in configuration.inc.php.

ganesh:Reply to Reply to The is matter of taste...

8 years ago


I agree. Customization for date/time display SHOULD be in configuration.inc.php.

Not everything can go in configuration.inc.php. For example, strings like '%a %b %e, %Y' (or whatever equivalent we will use when we ditch strftime) have to be language dependent. The English "Fri Nov 27th, 2007" becomes in Italian "ven 27 nov 2007": the name of the month has moved in third position and there's no comma. If you put those strings in configuration.inc.php you are forcing the same format for all users regardless of their language settings. For an Italian user, reading "ven nov 27, 2007" is even more confusing than not translating the date at all.
What exactly are you suggesting to put in configuration.inc.php?

pixtur:Customization...

8 years ago

I thought a while about the problem of customized date form versus translation. Maybe we could do it this way:

Currently we get the dateformat by the wrapper function getUserFormatDate() which returns at string like __('%b %e, %Y', 'strftime format string') which is been translated. This string could actually by a configuration variable that could also be overwritten in the customize.inc.php file.

Ganesh: any progress on the locale replacing date function?


pixtur:Hi ganesh...

8 years ago

any progress on this?