[luatex] luatex with unexpected locale

[luigi scarso]

On Thu, Aug 17, 2017 at 7:00 AM, Norbert Preining <preining at logic.at> wrote:
> Hi Luigi,
>
> On Sun, 11 Jun 2017, luigi scarso wrote:
>> >         env_locale = setlocale (LC_ALL, "");
>> >     if (!env_locale) {
>> >       fprintf(stderr,"Unable to read environment locale:exit now.\n");
>
>> > I'm not sure if luatex could be made to work if LANG is set to an
>> > unknown value (eg just act as if it was C)??
>>
>> no, exit is the better way.
>
> What if LANG is not set at all? I disagree that "exit" is the better way.
> There are enough circumstances where LANG is not set.
>
> So what *influence* does LANG have on the luatex format build?

Hm quite old discussion... as is now in experimental,
luatex exits if setlocale (LC_ALL, "") returns NULL,
which means that the locale from the environment  is invalid
(see man setlocale).
If ok, we store lc_type, lc_collate, lc_numeric (so that the user can
read them),
return to the old_locale,
and then we set
LC_CTYPE  LC_COLLATE  LC_NUMERIC to "C"
for luatex (old_locale is "C").

   /* Get the current locale (it should be C )          */
    /* and save LC_CTYPE, LC_COLLATE and LC_NUMERIC.     */
    /* Later luainterpreter() will consciously use them. */
    old_locale = xstrdup(setlocale (LC_ALL, NULL));
    lc_ctype = NULL;
    lc_collate = NULL;
    lc_numeric = NULL;
    if (old_locale) {
        /* If setlocale fails here, then the state   */
        /* could be compromised, and we exit.        */
        env_locale = setlocale (LC_ALL, "");
        if (!env_locale && !lua_only) {
          fprintf(stderr,"Unable to read environment locale: exit now.\n");
          exit(1);
        }
        tmp = setlocale (LC_CTYPE, NULL);
        if (tmp) {
          lc_ctype = xstrdup(tmp);
        }
        tmp = setlocale (LC_COLLATE, NULL);
        if (tmp){
          lc_collate = xstrdup(tmp);
        }
        tmp = setlocale (LC_NUMERIC, NULL);
        if (tmp){
          lc_numeric = xstrdup(tmp);
        }
        /* Back to the previous locale if possible,   */
        /* otherwise it's a serious error and we exit:*/
        /* we can't ensure a 'sane' locale for lua.   */
        env_locale = setlocale (LC_ALL, old_locale);
        if (!env_locale) {
          fprintf(stderr,"Unable to restore original locale %s: exit
now.\n",old_locale);
          exit(1);
        }
        xfree(old_locale);
    } else {
       fprintf(stderr,"Unable to store environment locale.\n");
    }

    /* make sure that the locale is 'sane' (for lua) */
    putenv(LC_CTYPE_C);
    putenv(LC_COLLATE_C);
    putenv(LC_NUMERIC_C);

The rationale behind this choice is that we want that luatex starts
in a sane environment;
of course we could force the locale to C, so that lc_type, lc_collate
and  lc_numeric always store "C" (pretty useless) but   this can also
be done  by the user before to run luatex;
or we can go on if the locale from the environment is not valid, but
in this case we should set lc_type, lc_collate, lc_numeric
to some values (NULL ?) that are not those from the (wrong)
environment, and of course the user will complain saying that luatex
has a bug because doesn't read the locale....

At runtime the user can query/modify the locale with Lua, and setting
a locale different from "C" almost for sure will cause problems to the
tex internals:
it's safe to confine the change inside Lua, ie

\catcode`\%=12
\def\Date{%A %B %Y}
\catcode`\%=14
\directlua{
os.setlocale("it_IT.utf8")
local it_date=os.date("\Date", os.time())
os.setlocale("C")
tex.print(it_date,"")
local C_date=os.date("\Date", os.time())
tex.print(C_date,"")
}
\par
\input knuth
\bye



    giovedì agosto 2017
     Thursday August 2017
     Thus, I came to the conclusion that the designer of a new system
must not only be the implementer
and first large||scale user; the designer should also write the first
user manual.
     The separation of any of these four components would have hurt
TEX significantly. If I had not partici-
pated fully in all these activities, literally hundreds of
improvements would never have been made, because
I would never have thought of them or perceived why they were important.
     But a system cannot be successful if it is too strongly
influenced by a single person. Once the initial
design is complete and fairly robust, the real test begins as people
with many different viewpoints undertake
their own experiments.



but if one forgets to reset to "C", ie

\catcode`\%=12
\def\Date{%A %B %Y}
\catcode`\%=14
\directlua{
os.setlocale("it_IT.utf8")
local it_date=os.date("\Date", os.time())
}
\par
\input knuth
\bye


then
:
:
! error:  (file /opt/luatex/mkvi-experimental/tex/texmf/fonts/opentype/public/l
m/lmroman10-regular.otf) (cff): parsing DICT failed (error=-1)
!  ==> Fatal error occurred, no output PDF file produced!


Note that this is not related to  the format:
the format is made after these lines
/* make sure that the locale is 'sane' (for lua) */
putenv(LC_CTYPE_C);
putenv(LC_COLLATE_C);
putenv(LC_NUMERIC_C);

so in a "C" locale.


-- 
luigi