texlive[53415] Master/texmf-dist: babel (15jan20)
commits+karl at tug.org
commits+karl at tug.org
Wed Jan 15 22:50:07 CET 2020
Revision: 53415
http://tug.org/svn/texlive?view=revision&revision=53415
Author: karl
Date: 2020-01-15 22:50:07 +0100 (Wed, 15 Jan 2020)
Log Message:
-----------
babel (15jan20)
Modified Paths:
--------------
trunk/Master/texmf-dist/doc/latex/babel/README.md
trunk/Master/texmf-dist/doc/latex/babel/babel.pdf
trunk/Master/texmf-dist/source/latex/babel/babel.dtx
trunk/Master/texmf-dist/source/latex/babel/babel.ins
trunk/Master/texmf-dist/source/latex/babel/bbcompat.dtx
trunk/Master/texmf-dist/source/latex/babel/locale.zip
trunk/Master/texmf-dist/tex/generic/babel/babel.def
trunk/Master/texmf-dist/tex/generic/babel/babel.sty
trunk/Master/texmf-dist/tex/generic/babel/hyphen.cfg
trunk/Master/texmf-dist/tex/generic/babel/luababel.def
trunk/Master/texmf-dist/tex/generic/babel/nil.ldf
trunk/Master/texmf-dist/tex/generic/babel/switch.def
Removed Paths:
-------------
trunk/Master/texmf-dist/tex/generic/babel/test-hyphen-post-wiki.tex
trunk/Master/texmf-dist/tex/generic/babel/xe-polygreek.tex
Modified: trunk/Master/texmf-dist/doc/latex/babel/README.md
===================================================================
--- trunk/Master/texmf-dist/doc/latex/babel/README.md 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/doc/latex/babel/README.md 2020-01-15 21:50:07 UTC (rev 53415)
@@ -1,4 +1,4 @@
-## Babel 3.37
+## Babel 3.38
This package manages culturally-determined typographical (and other)
rules, and hyphenation patterns for a wide range of languages. Many
@@ -51,13 +51,22 @@
### Latest changes
```
+3.38 2020-01-15
+ - Automatic switching of ids (\language and \localeid), and fonts
+ based on script blocks (lua).s
+ - New macro - \localeinfo, to access the basic data in the ini
+ file loaded by languages.
+See https://github.com/latex3/babel/wiki/What's-new-in-babel-3.38
+
+
3.37 2019-12-08
- Preliminary code for non-standard hyphenation, like ff ->
ff-f (lua).
- \babelprovide now can be used to add or modify values for the
keys in ini files.
- - Line breaking in South East Asian and CKJ are assimilated to
- hyphenation, and it is activated even without 'import' (lua).
+ - Line breaking in South East Asian and CKJ is assimilated to
+ hyphenation, and it is activated even without 'import' (lua).
+See https://github.com/latex3/babel/wiki/What's-new-in-babel-3.37
3.36 2019-11-14
- New - \babeladjust, with options: bidi.text, bidi.mirroring,
@@ -65,7 +74,8 @@
linebreak.cjk. There are still some limitations (lua).
- New - ini for Polytonic Greek, thanks to Claudio Beccari.
- Fix - Language and script set for Chinese Tradicional and
- Chinese Simplified.
+ Chinese Simplified.
+See https://github.com/latex3/babel/wiki/What's-new-in-babel-3.36
3.35 2019-10-15
- \markboth and \markright made robust with a recent LaTeX.
@@ -116,12 +126,6 @@
very likely the risks are very low, and it is, I think, the
expected behavior.
-3.27 2018-11-13
- - Preliminary support for bidi (by Vafa Khalighi) with xetex.
- - Fix for 3.23 - \ensureascii was redefined even when not
- necessary.
- - Minor improvements in babel-vi.ini.
-
```
Javier Bezos
Modified: trunk/Master/texmf-dist/doc/latex/babel/babel.pdf
===================================================================
(Binary files differ)
Modified: trunk/Master/texmf-dist/source/latex/babel/babel.dtx
===================================================================
--- trunk/Master/texmf-dist/source/latex/babel/babel.dtx 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/source/latex/babel/babel.dtx 2020-01-15 21:50:07 UTC (rev 53415)
@@ -31,7 +31,7 @@
%
% \iffalse
%<*filedriver>
-\ProvidesFile{babel.dtx}[2019/12/08 v3.37 The Babel package]
+\ProvidesFile{babel.dtx}[2020/01/15 v3.38 The Babel package]
\documentclass{ltxdoc}
\GetFileInfo{babel.dtx}
\usepackage{fontspec}
@@ -59,6 +59,7 @@
\newcommand*\xetex{\textsf{xetex}}
\newcommand*\pdftex{\textsf{pdftex}}
\newcommand*\luatex{\textsf{luatex}}
+\newcommand\largetex{T\kern -.1517em\lower .45ex\hbox {E}\kern -.09emX}
\newcommand*\nb[1]{}
\newcommand*\m[1]{\mbox{$\langle$\normalfont\itshape#1\/$\rangle$}}
\newcommand*\langlist{%
@@ -98,7 +99,18 @@
\newtheorem{troubleshooting}{Troubleshooting}
\let\bblxv\verbatim
\let\bblexv\endverbatim
-\def\verbatim{\begin{shaded*}\bblxv\vskip-\baselineskip\vskip2.5\parsep}
+\newcommand\setengine{\def\engine}
+\let\engine\relax
+\def\verbatim{%
+ \begin{shaded*}%
+ \ifx\engine\relax\else
+ \vskip-1.08\baselineskip
+ \leavevmode\llap{\fbox{\footnotesize\textsc{\engine}}\hskip2.8em}%
+ \vskip-1.5\baselineskip
+ \vskip0pt
+ \global\let\engine\relax
+ \fi
+ \bblxv\vskip-\baselineskip\vskip2.5\parsep}
\def\endverbatim{\bblexv\vskip-2\baselineskip\end{shaded*}}
\catcode`\_=\active
\def_{\bgroup\let_\egroup\leavevmode\color{thered}}
@@ -205,11 +217,11 @@
\fontsize{35}{45}\selectfont
\setlength\parskip{3mm}\raggedright
Localization and internationalization\\[1cm]
-\TeX\\
-pdf\TeX\\
-Lua\TeX\\
-LuaHB\TeX\\
-Xe\TeX
+\largetex\\
+pdf\largetex\\
+Lua\largetex\\
+LuaHB\largetex\\
+Xe\largetex
\vspace{20cm}
\end{minipage}
\end{tabular}
@@ -271,6 +283,7 @@
example because typically you will need them (however, the package
\textsf{inputenc} may be omitted with \LaTeX{} $\ge$ 2018-04-01 if
the encoding is UTF-8):
+\setengine{pdftex}
\begin{verbatim}
\documentclass{article}
@@ -294,6 +307,7 @@
in UTF-8 and a so-called Unicode font must be loaded (in this example
|\babelfont| is used, described below).
+\setengine{luatex/xetex}
\begin{verbatim}
\documentclass{article}
@@ -433,6 +447,7 @@
A full bilingual document follows. The main language is |french|, which
is activated when the document begins. The package \textsf{inputenc}
may be omitted with \LaTeX{} $\ge$ 2018-04-01 if the encoding is UTF-8.
+\setengine{pdftex}
\begin{verbatim}
\documentclass{article}
@@ -459,6 +474,7 @@
document in UTF-8 encoding just prints a couple of ‘captions’ and
|\today| in Danish and Vietnamese. No additional packages are
required.
+\setengine{luatex/xetex}
\begin{verbatim}
\documentclass{article}
@@ -892,12 +908,14 @@
(You may also need to activate them as user shorthands in the preamble
with, for example, |\useshorthands| or |\useshorthands*|.)
-Very often, this is a more convenient way to deactivate shorthands
-than |\shorthandoff|, for example if you want to define a macro
-to easy typing phonetic characters with \textsf{tipa}:
+\begin{example}
+ Very often, this is a more convenient way to deactivate shorthands
+ than |\shorthandoff|, for example if you want to define a macro
+ to easy typing phonetic characters with \textsf{tipa}:
\begin{verbatim}
\newcommand{\myipa}[1]{{\languageshorthands{none}\tipaencoding#1}}
\end{verbatim}
+\end{example}
\Describe{\babelshorthand}{\marg{shorthand}}
With this command you can use a shorthand even if (1) not activated in
@@ -908,6 +926,15 @@
\verb|\babelshorthand{:}|. (You can conveniently define your own
macros, or even your own user shorthands provided they do not overlap.)
+\begin{example}
+ Since by default shorthands are not activated until
+ |\begin{document}|, you may use this macro when defining the |\title|
+ in the preamble:
+\begin{verbatim}
+\title{Documento científico\babelshorthand{"-}técnico}
+\end{verbatim}
+\end{example}
+
\bigskip
For your records, here is a list of shorthands, but you must double
@@ -1178,6 +1205,7 @@
declare this language with an |ini| file in Unicode engines.
\begingroup
\setmonofont[Scale=.87,Script=Georgian]{DejaVu Sans Mono}
+\setengine{luatex/xetex}
\begin{verbatim}
\documentclass{book}
@@ -1910,6 +1938,7 @@
\def@#1{\ifcase#1\relax \egroup \or \bgroup\textdir TLT \else
\bgroup\textdir TRT \fontspec[Scale=.87,Script=Hebrew]{Liberation
Mono} \fi}
+\setengine{luatex/xetex}
\begin{verbatim}
\documentclass{article}
@@ -1929,6 +1958,7 @@
If on the other hand you have to resort to different fonts, you could
replace the red line above with, say:
+\setengine{luatex/xetex}
\begin{verbatim}
\babelfont{rm}{Iwona}
\babelfont[hebrew]{rm}{FreeSerif}
@@ -1941,6 +1971,7 @@
\begin{example}
Here is how to do it:
+\setengine{luatex/xetex}
\begin{verbatim}
\babelfont{kai}{FandolKai}
\end{verbatim}
@@ -1950,6 +1981,7 @@
\begin{note}
You may load \textsf{fontspec} explicitly. For example:
+\setengine{luatex/xetex}
\begin{verbatim}
\usepackage{fontspec}
\newfontscript{Devanagari}{deva}
@@ -2241,10 +2273,18 @@
a character has the same direction as the script for the “provided”
language, then change its font to that set for this language’. There
are 3 directions, following the bidi Unicode algorithm, namely,
-Arabic-like, Hebrew-like and left to right.\footnote{In future releases
-a couple of values (\texttt{language} and \texttt{script}) will be
-added.} So, there should be at most 3 directives of this kind.
+Arabic-like, Hebrew-like and left to right. So, there should be at most
+3 directives of this kind.
+\Describe{onchar=}{\texttt{ids} $\string|$ \texttt{fonts}}
+\New{3.38} This options is much like an ‘event’ called with a character
+belonging to the script of the current locale is found. There are two
+action, which can be used at the same time (separated by a space): with
+|ids| the |\language| and the |\localeid| are set to the values of this
+locale; with |fonts|, the fonts are changed to those of the current
+locale (as set with |\babelfont|). This option is not compatible with
+|mapfont|.
+
\Describe{intraspace=}{\meta{base} \meta{shrink} \meta{stretch}}
Sets the interword space for the writing system of the language, in em
units (so, |0 .1 0| is |0em plus .1em|). Like |\spaceskip|, the em unit
@@ -2301,7 +2341,7 @@
bidirectional behavior (unlike |Numbers=Arabic| in \textsf{fontspec},
which is not recommended).
-\subsection{Getting the current language name}
+\subsection{Accessing language info}
\Describe{\languagename}{}
The control sequence |\languagename| contains the name of the
@@ -2317,7 +2357,7 @@
If more than one language is used, it might be necessary to know which
language is active at a specific time. This can be checked by a call
-to |\iflanguage|, but note here ``language'' is used in the \TeX\
+to |\iflanguage|, but note here ``language'' is used in the \TeX
sense, as a set of hyphenation patterns, and \textit{not} as its
\textsf{babel} name. This macro takes three arguments. The first
argument is the name of a language; the second and third arguments are
@@ -2324,11 +2364,33 @@
the actions to take if the result of the test is true or false
respectively.
-\begin{warning}
- The advice about |\languagename| also applies here -- use
- \textsf{iflang} instead of |\iflanguage| if possible.
-\end{warning}
+\Describe{\localeinfo}{\marg{field}}
+\New{3.38} If an |ini| file has been loaded for the current language,
+you may access the information stored in it. This macros is fully
+expandable and the available fields are:
+\begin{description}
+\itemsep=-\parskip
+\item[|name.english|] as provided by the Unicode CLDR.
+%%% \item[|name.locale|] is the equivalent of |\languagename|. Not yet
+%%% activated because the bug in \languagename is far from trivial.
+\item[|tag.ini|] is the tag of the |ini| file (the way this
+ file is identified in its name).
+\item[|tag.bcp47|] is the BCP 47 language tag.
+\item[|tag.opentype|] is the tag used by OpenType (usually, but not
+ always, the same as BCP 47).
+\item[|script.name|] as provided by the Unicode CLDR.
+\item[|script.tag.bcp47|] is the BCP 47 language tag of the script
+ used by this locale.
+\item[|script.tag.opentype|] is the tag used by OpenType (usually,
+ but not always, the same as BCP 47).
+\end{description}
+
+|ini| files are loaded with |\babelprovide| and also when languages are
+selected if there is a |\babelfont|. To ensure the |ini| files are
+loaded (and therefore the corresponding data) even if these two
+conditions are not met, write |\BabelEnsureInfo|.
+
\subsection{Hyphenation and line breaking}
\Describe{\babelhyphen}{%
@@ -4160,8 +4222,8 @@
% \section{Tools}
%
% \begin{macrocode}
-%<<version=3.37>>
-%<<date=2019/12/08>>
+%<<version=3.38>>
+%<<date=2020/01/15>>
% \end{macrocode}
%
% \textbf{Do not use the following macros in \texttt{ldf} files. They
@@ -5270,7 +5332,7 @@
% The file |babel.def| expects some definitions made in the \LaTeXe{}
% style file. So, In \LaTeX2.09 and Plain{} we must provide at least
% some predefined values as well some tools to set them (even if not
-% all options are available). There in no package options, and
+% all options are available). There are no package options, and
% therefore and alternative mechanism is provided. For the moment,
% only |\babeloptionstrings| and |\babeloptionmath| are provided,
% which can be defined before loading \babel. |\BabelModifiers| can be
@@ -8143,6 +8205,7 @@
% be restored.}
% \changes{babel~3.37}{2019/12/07}{SEA and CJK linebreaking activated
% by default.}
+% \changes{babel~3.38}{2020/01/15}{Code for the onchar option.}
%
% \begin{macrocode}
\bbl at trace{Creating languages and reading ini files}
@@ -8151,6 +8214,7 @@
\edef\bbl at savelocaleid{\the\localeid}%
% Set name and locale id
\def\languagename{#2}%
+ % \global\@namedef{bbl at lcname@#2}{#2}%
\bbl at id@assign
\let\bbl at KVP@captions\@nil
\let\bbl at KVP@import\@nil
@@ -8163,6 +8227,8 @@
\let\bbl at KVP@mapdigits\@nil
\let\bbl at KVP@intraspace\@nil
\let\bbl at KVP@intrapenalty\@nil
+ \let\bbl at KVP@onchar\@nil
+ \let\bbl at KVP@chargroups\@nil
\bbl at forkv{#1}{% TODO - error handling
\in@{..}{##1}%
\ifin@
@@ -8227,6 +8293,59 @@
\ifx\bbl at KVP@language\@nil\else
\bbl at csarg\edef{lname@#2}{\bbl at KVP@language}%
\fi
+ % == onchar ==
+ \ifx\bbl at KVP@onchar\@nil\else
+ \bbl at luahyphenate
+ \directlua{
+ if Babel.locale_mapped == nil then
+ Babel.locale_mapped = true
+ Babel.linebreaking.add_before(Babel.locale_map)
+ Babel.loc_to_scr = {}
+ Babel.chr_to_loc = {}
+ end}%
+ \bbl at xin@{ ids }{ \bbl at KVP@onchar\space}%
+ \ifin@
+ % TODO - error/warning if no script
+ \directlua{
+ if Babel.script_blocks['\bbl at cs{sbcp@\languagename}'] then
+ Babel.loc_to_scr[\the\localeid] =
+ Babel.script_blocks['\bbl at cs{sbcp@\languagename}']
+ Babel.locale_props[\the\localeid].lc = \the\localeid\space
+ Babel.locale_props[\the\localeid].lg = \the\@nameuse{l@\languagename}\space
+ end
+ }%
+ \fi
+ \bbl at xin@{ fonts }{ \bbl at KVP@onchar\space}%
+ \ifin@
+ \bbl at ifunset{bbl at lsys@\languagename}{\bbl at provide@lsys{\languagename}}{}%
+ \bbl at ifunset{bbl at wdir@\languagename}{\bbl at provide@dirs{\languagename}}{}%
+ \directlua{
+ if Babel.script_blocks['\bbl at cs{sbcp@\languagename}'] then
+ Babel.loc_to_scr[\the\localeid] =
+ Babel.script_blocks['\bbl at cs{sbcp@\languagename}']
+ end}
+ \ifx\bbl at mapselect\@undefined
+ \AtBeginDocument{%
+ \expandafter\bbl at add\csname selectfont \endcsname{{\bbl at mapselect}}%
+ {\selectfont}}%
+ \def\bbl at mapselect{%
+ \let\bbl at mapselect\relax
+ \edef\bbl at prefontid{\fontid\font}}%
+ \def\bbl at mapdir##1{%
+ {\def\languagename{##1}%
+ \let\bbl at ifrestoring\@firstoftwo % To avoid font warning
+ \bbl at switchfont
+ \directlua{
+ Babel.locale_props[\the\csname bbl at id@@##1\endcsname]%
+ ['/\bbl at prefontid'] = \fontid\font\space}}}%
+ \fi
+ \bbl at exp{\\\bbl at add\\\bbl at mapselect{\\\bbl at mapdir{\languagename}}}%
+ \fi
+ % TODO - catch non-valid values
+ \fi
+% \ifx\bbl at KVP@chargroups\@nil\else
+% \bbl at chargroups
+% \fi
% == mapfont ==
% For bidi texts, to switch the font based on direction
\ifx\bbl at KVP@mapfont\@nil\else
@@ -8477,11 +8596,17 @@
%
% \changes{babel~3.37}{2019/12/07}{Allow to define key/values
% (added \cs{bbl at renewlist}).}
+% \changes{babel~3.38}{2020/01/15}{Read numbers are not hardcoded
+% (passim); use \cs{bbl at readstream}.}
%
% \begin{macrocode}
+\ifx\bbl at readstream\@undefined
+ \csname newread\endcsname\bbl at readstream
+\fi
\def\bbl at read@ini#1#2{%
- \openin1=babel-#1.ini % FIXME - number must not be hardcoded
- \ifeof1
+ \global\@namedef{bbl at lini@\languagename}{#1}%
+ \openin\bbl at readstream=babel-#1.ini
+ \ifeof\bbl at readstream
\bbl at error
{There is no ini file for the requested language\\%
(#1). Perhaps you misspelled it or your installation\\%
@@ -8502,9 +8627,9 @@
\bbl at info{Importing #2 for \languagename\\%
from babel-#1.ini. Reported}%
\loop
- \if T\ifeof1F\fi T\relax % Trick, because inside \loop
+ \if T\ifeof\bbl at readstream F\fi T\relax % Trick, because inside \loop
\endlinechar\m at ne
- \read1 to \bbl at line
+ \read\bbl at readstream to \bbl at line
\endlinechar`\^^M
\ifx\bbl at line\@empty\else
\expandafter\bbl at iniline\bbl at line\bbl at iniline
@@ -8590,14 +8715,14 @@
% \begin{macrocode}
\let\bbl at inikv@identification\bbl at inikv
\def\bbl at secpost@identification{%
- \bbl at ifunset{bbl@@kv at identification.name.opentype}%
- {\bbl at exportkey{lname}{identification.name.english}{}}%
- {\bbl at exportkey{lname}{identification.name.opentype}{}}%
+ \bbl at exportkey{elname}{identification.name.english}{}%
+ \bbl at exp{\\\bbl at exportkey{lname}{identification.name.opentype}%
+ {\csname bbl at elname@\languagename\endcsname}}%
\bbl at exportkey{lbcp}{identification.tag.bcp47}{}%
\bbl at exportkey{lotf}{identification.tag.opentype}{dflt}%
- \bbl at ifunset{bbl@@kv at identification.script.name.opentype}%
- {\bbl at exportkey{sname}{identification.script.name}{}}%
- {\bbl at exportkey{sname}{identification.script.name.opentype}{}}%
+ \bbl at exportkey{esname}{identification.script.name}{}%
+ \bbl at exp{\\\bbl at exportkey{sname}{identification.script.name.opentype}%
+ {\csname bbl at esname@\languagename\endcsname}}%
\bbl at exportkey{sbcp}{identification.script.tag.bcp47}{}%
\bbl at exportkey{sotf}{identification.script.tag.opentype}{DFLT}}
\let\bbl at inikv@typography\bbl at inikv
@@ -8790,12 +8915,45 @@
\def\bbl at ini@basic#1{%
\def\BabelBeforeIni##1##2{%
\begingroup
- \bbl at add\bbl at secpost@identification{\closein1 }%
+ \bbl at add\bbl at secpost@identification{\closein\bbl at readstream }%
\catcode`\[=12 \catcode`\]=12 \catcode`\==12 %
\bbl at read@ini{##1}{font and identification data}%
\endinput % babel- .tex may contain onlypreamble's
\endgroup}% boxed, to avoid extra spaces:
{\setbox\z@\hbox{\InputIfFileExists{babel-#1.tex}{}{}}}}
+% \end{macrocode}
+%
+% The information in the identification section can be useful, so the
+% following macro just exposes it with a user command.
+%
+% \changes{babel~3.38}{2020/01/14}{Added \cs{localeinfo}.}
+%
+% \begin{macrocode}
+\newcommand\localeinfo[1]{%
+ \bbl at ifunset{bbl@\csname bbl at info@#1\endcsname @\languagename}%
+ {\bbl at error{I've found no info for the current locale.\\%
+ The corresponding ini file has not been loaded\\%
+ Perhaps it doesn't exist}%
+ {See the manual for details.}}%
+ {\@nameuse{bbl@\csname bbl at info@#1\endcsname @\languagename}}}
+% \@namedef{bbl at info@name.locale}{lcname}
+\@namedef{bbl at info@tag.ini}{lini}
+\@namedef{bbl at info@name.english}{elname}
+\@namedef{bbl at info@name.opentype}{lname}
+\@namedef{bbl at info@tag.bcp47}{lbcp}
+\@namedef{bbl at info@tag.opentype}{lotf}
+\@namedef{bbl at info@script.name}{esname}
+\@namedef{bbl at info@script.name.opentype}{sname}
+\@namedef{bbl at info@script.tag.bcp47}{sbcp}
+\@namedef{bbl at info@script.tag.opentype}{sotf}
+\let\bbl at ensureinfo\@gobble
+\newcommand\BabelEnsureInfo{%
+ \def\bbl at ensureinfo##1{%
+ \ifx\InputIfFileExists\@undefined\else % not in plain
+ \bbl at ifunset{bbl at lname@##1}{\bbl at ini@basic{##1}}{}%
+ \fi}}
+% \end{macrocode}
+%
% \section{Adjusting the Babel bahavior}
%
% \changes{babel~3.36}{2019/10/30}{New macro \cs{babeladjust}}
@@ -10195,6 +10353,7 @@
Babel = Babel or {}
Babel.locale_props = Babel.locale_props or {}
Babel.locale_props[\bbl at id@last] = {}
+ Babel.locale_props[\bbl at id@last].name = '\languagename'
}%
\fi}%
{}%
@@ -10247,6 +10406,7 @@
\edef\languagename{%
\ifnum\escapechar=\expandafter`\string#1\@empty
\else\string#1\@empty\fi}%
+ % \@namedef{bbl at lcname@#1}{#1}%
\select at language{\languagename}%
% write to auxs
\expandafter\ifx\csname date\languagename\endcsname\relax\else
@@ -10353,6 +10513,8 @@
% \begin{macrocode}
\newif\ifbbl at usedategroup
\def\bbl at switch#1{% from select@, foreign@
+ % make sure there is info for the language if so requested
+ \bbl at ensureinfo{#1}%
% restore
\originalTeX
\expandafter\def\expandafter\originalTeX\expandafter{%
@@ -10563,6 +10725,7 @@
\def\foreign at language#1{%
% set name
\edef\languagename{#1}%
+ % \@namedef{bbl at lcname@#1}{#1}%
\bbl at fixname\languagename
\bbl at iflanguage\languagename{%
\expandafter\ifx\csname date\languagename\endcsname\relax
@@ -11992,6 +12155,9 @@
%<*luatex>
\ifx\AddBabelHook\@undefined
\bbl at trace{Read language.dat}
+\ifx\bbl at readstream\@undefined
+ \csname newread\endcsname\bbl at readstream
+\fi
\begingroup
\toks@{}
\count@\z@ % 0=start, 1=0th, 2=normal
@@ -12055,16 +12221,16 @@
\fi
\def\bbl at elt#1#2#3#4{\@namedef{zth@#1}{}} % Define flags
\bbl at languages
- \openin1=language.dat
- \ifeof1
+ \openin\bbl at readstream=language.dat
+ \ifeof\bbl at readstream
\bbl at warning{I couldn't find language.dat. No additional\\%
patterns loaded. Reported}%
\else
\loop
\endlinechar\m at ne
- \read1 to \bbl at line
+ \read\bbl at readstream to \bbl at line
\endlinechar`\^^M
- \if T\ifeof1F\fi T\relax
+ \if T\ifeof\bbl at readstream F\fi T\relax
\ifx\bbl at line\@empty\else
\edef\bbl at line{\bbl at line\space\space\space}%
\expandafter\bbl at process@line\bbl at line\relax
@@ -12362,7 +12528,7 @@
quad = font.getfont(last_char.font).size
for lg, rg in pairs(sea_ranges) do
if last_char.char > rg[1] and last_char.char < rg[2] then
- lg = lg:sub(1, 4)
+ lg = lg:sub(1, 4) ^^ Remove trailing number of, eg, Cyrl1
local intraspace = Babel.intraspaces[lg]
local intrapenalty = Babel.intrapenalties[lg]
local n
@@ -12407,15 +12573,14 @@
luatexbase.registernumber'bbl at attr@locale')
local props = Babel.locale_props[LOCALE]
- class = Babel.cjk_class[item.char].c
+ local class = Babel.cjk_class[item.char].c
if class == 'cp' then class = 'cl' end % )] as CL
if class == 'id' then class = 'I' end
+ local br = 0
if class and last_class and Babel.cjk_breaks[last_class][class] then
br = Babel.cjk_breaks[last_class][class]
- else
- br = 0
end
if br == 1 and props.linebreak == 'c' and
@@ -12451,14 +12616,14 @@
\directlua{
luatexbase.add_to_callback('hyphenate',
function (head, tail)
- if Babel.cjk_enabled then
- Babel.cjk_linebreak(head)
- end
if Babel.linebreaking.before then
for k, func in ipairs(Babel.linebreaking.before) do
func(head)
end
end
+ if Babel.cjk_enabled then
+ Babel.cjk_linebreak(head)
+ end
lang.hyphenate(head)
if Babel.linebreaking.after then
for k, func in ipairs(Babel.linebreaking.after) do
@@ -12536,6 +12701,116 @@
<@Font selection@>
% \end{macrocode}
%
+% \subsection{Automatic fonts and ids switching}
+%
+% After defining the blocks for a number of scripts (must be extended
+% and very likely fine tuned), we define a short function which just
+% traverse the node list to carry out the replacements. The table
+% |loc_to_scr| gets the locale form a script range (note the locale is
+% the key, and that there is an intermediate table built on the fly for
+% optimization). This locale is then used to get the |\language| and
+% the |\localeid| as stored in |locale_props|, as well as the font (as
+% requested). In the latter table a key starting with |/| maps the font
+% from the global one (the key) to the local one (the value). Maths are
+% skipped and discretionaries are handled in a special way.
+%
+% \changes{babel~3.38}{2020/01/15}{Automatic fonts and ids switching}
+%
+% \begin{macrocode}
+\directlua{
+Babel.script_blocks = {
+ ['Arab'] = {{0x0600, 0x06FF}, {0x08A0, 0x08FF}, {0x0750, 0x077F},
+ {0xFE70, 0xFEFF}, {0xFB50, 0xFDFF}, {0x1EE00, 0x1EEFF}},
+ ['Armn'] = {{0x0530, 0x058F}},
+ ['Beng'] = {{0x0980, 0x09FF}},
+ ['Cher'] = {{0x13A0, 0x13FF}, {0xAB70, 0xABBF}},
+ ['Cyrl'] = {{0x0400, 0x04FF}, {0x0500, 0x052F}, {0x1C80, 0x1C8F},
+ {0x2DE0, 0x2DFF}, {0xA640, 0xA69F}},
+ ['Deva'] = {{0x0900, 0x097F}, {0xA8E0, 0xA8FF}},
+ ['Ethi'] = {{0x1200, 0x137F}, {0x1380, 0x139F}, {0x2D80, 0x2DDF},
+ {0xAB00, 0xAB2F}},
+ ['Geor'] = {{0x10A0, 0x10FF}, {0x2D00, 0x2D2F}},
+ ['Grek'] = {{0x0370, 0x03FF}, {0x1F00, 0x1FFF}},
+ ['Hans'] = {{0x2E80, 0x2EFF}, {0x3000, 0x303F}, {0x31C0, 0x31EF},
+ {0x3300, 0x33FF}, {0x3400, 0x4DBF}, {0x4E00, 0x9FFF},
+ {0xF900, 0xFAFF}, {0xFE30, 0xFE4F}, {0xFF00, 0xFFEF},
+ {0x20000, 0x2A6DF}, {0x2A700, 0x2B73F},
+ {0x2B740, 0x2B81F}, {0x2B820, 0x2CEAF},
+ {0x2CEB0, 0x2EBEF}, {0x2F800, 0x2FA1F}},
+ ['Hebr'] = {{0x0590, 0x05FF}},
+ ['Japa'] = {{0x3000, 0x303F}, {0x3040, 0x309F}, {0x30A0, 0x30FF},
+ {0x4E00, 0x9FAF}, {0xFF00, 0xFFEF}},
+ ['Khmr'] = {{0x1780, 0x17FF}, {0x19E0, 0x19FF}},
+ ['Knda'] = {{0x0C80, 0x0CFF}},
+ ['Kore'] = {{0x1100, 0x11FF}, {0x3000, 0x303F}, {0x3130, 0x318F},
+ {0x4E00, 0x9FAF}, {0xA960, 0xA97F}, {0xAC00, 0xD7AF},
+ {0xD7B0, 0xD7FF}, {0xFF00, 0xFFEF}},
+ ['Laoo'] = {{0x0E80, 0x0EFF}},
+ ['Latn'] = {{0x0000, 0x007F}, {0x0080, 0x00FF}, {0x0100, 0x017F},
+ {0x0180, 0x024F}, {0x1E00, 0x1EFF}, {0x2C60, 0x2C7F},
+ {0xA720, 0xA7FF}, {0xAB30, 0xAB6F}},
+ ['Mahj'] = {{0x11150, 0x1117F}},
+ ['Mlym'] = {{0x0D00, 0x0D7F}},
+ ['Mymr'] = {{0x1000, 0x109F}, {0xAA60, 0xAA7F}, {0xA9E0, 0xA9FF}},
+ ['Orya'] = {{0x0B00, 0x0B7F}},
+ ['Sinh'] = {{0x0D80, 0x0DFF}, {0x111E0, 0x111FF}},
+ ['Taml'] = {{0x0B80, 0x0BFF}},
+ ['Telu'] = {{0x0C00, 0x0C7F}},
+ ['Tfng'] = {{0x2D30, 0x2D7F}},
+ ['Thai'] = {{0x0E00, 0x0E7F}},
+ ['Tibt'] = {{0x0F00, 0x0FFF}},
+ ['Vaii'] = {{0xA500, 0xA63F}},
+ ['Yiii']= {{0xA490, 0xA4CF}, {0xA000, 0xA48F}}
+}
+
+Babel.script_blocks.Hant = Babel.script_blocks.Hans
+
+function Babel.locale_map(head)
+ if not Babel.locale_mapped then return head end
+
+ local LOCALE = luatexbase.registernumber'bbl at attr@locale'
+ local GLYPH = node.id('glyph')
+ local inmath = false
+ for item in node.traverse(head) do
+ local toloc
+ if not inmath and item.id == GLYPH then
+ % Optimization: build a table with the chars found
+ if Babel.chr_to_loc[item.char] then
+ toloc = Babel.chr_to_loc[item.char]
+ else
+ for lc, maps in pairs(Babel.loc_to_scr) do
+ for _, rg in pairs(maps) do
+ if item.char >= rg[1] and item.char <= rg[2] then
+ Babel.chr_to_loc[item.char] = lc
+ toloc = lc
+ break
+ end
+ end
+ end
+ end
+ % Now, take action
+ if toloc then
+ if Babel.locale_props[toloc].lg then
+ item.lang = Babel.locale_props[toloc].lg
+ node.set_attribute(item, LOCALE, toloc)
+ end
+ if Babel.locale_props[toloc]['/'..item.font] then
+ item.font = Babel.locale_props[toloc]['/'..item.font]
+ end
+ end
+ elseif not inmath and item.id == 7 then
+ item.replace = item.replace and Babel.locale_map(item.replace)
+ item.pre = item.pre and Babel.locale_map(item.pre)
+ item.post = item.post and Babel.locale_map(item.post)
+ elseif item.id == node.id'math' then
+ inmath = (item.subtype == 0)
+ end
+ end
+ return head
+end
+}
+% \end{macrocode}
+%
% \changes{babel~3.32}{2019/05/23}{New - \cs{babelcharproperty}.}
%
% The code for |\babelcharproperty| is straightforward. Just note the
@@ -12599,11 +12874,11 @@
% |tex.hyphenate|. This means the automatic hyphenation points are
% known. As empty captures return a byte position (as explained in the
% \luatex{} manual), we must convert it to a utf8 position. With
-% |first|, the last byte can be the leading byte in a utf8 sequence,
-% so we just remove it and add 1 to the resulting length. With |last|
-% we must take into account the capture position points to the next
-% character. Here |word_head| points to the starting node of the text to
-% be matched.
+% |first|, the last byte can be the leading byte in a utf8 sequence, so
+% we just remove it and add 1 to the resulting length. With |last| we
+% must take into account the capture position points to the next
+% character. Here |word_head| points to the starting node of the text
+% to be matched.
%
% \begin{macrocode}
\begingroup
@@ -12775,6 +13050,7 @@
return head
end
+ &% Used below
function Babel.capture_func(key, cap)
local ret = "[[" .. cap:gsub('{([0-9])}', "]]..m[%1]..[[") .. "]]"
ret = ret:gsub("%[%[%]%]%.%.", '')
@@ -12789,11 +13065,11 @@
% These functions handle the |{|\textit{n}|}| syntax. For example,
% |pre={1}{1}-| becomes |function(m) return m[1]..m[1]..'-' end|, where
% |m| are the matches returned after applying the pattern. The way it
-% is done is somewhat tricky, but the effect in not dissimilar to lua
-% |load| – save the code as string in a TeX macro, and expand this
-% macro at the appropriate place. As |\directlua| does not take into
-% account the current catcode of |@|, we just avoid this character in
-% macro names (which explains the internal group, too).
+% is carried out is somewhat tricky, but the effect in not dissimilar
+% to lua |load| – save the code as string in a TeX macro, and expand
+% this macro at the appropriate place. As |\directlua| does not take
+% into account the current catcode of |@|, we just avoid this character
+% in macro names (which explains the internal group, too).
%
% \begin{macrocode}
\catcode`\#=6
Modified: trunk/Master/texmf-dist/source/latex/babel/babel.ins
===================================================================
--- trunk/Master/texmf-dist/source/latex/babel/babel.ins 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/source/latex/babel/babel.ins 2020-01-15 21:50:07 UTC (rev 53415)
@@ -26,7 +26,7 @@
%% and covered by LPPL is defined by the unpacking scripts (with
%% extension .ins) which are part of the distribution.
%%
-\def\filedate{2019/12/08}
+\def\filedate{2020/01/15}
\def\batchfile{babel.ins}
\input docstrip.tex
Modified: trunk/Master/texmf-dist/source/latex/babel/bbcompat.dtx
===================================================================
--- trunk/Master/texmf-dist/source/latex/babel/bbcompat.dtx 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/source/latex/babel/bbcompat.dtx 2020-01-15 21:50:07 UTC (rev 53415)
@@ -30,7 +30,7 @@
%
% \iffalse
%<*dtx>
-\ProvidesFile{bbcompat.dtx}[2019/12/08 v3.37]
+\ProvidesFile{bbcompat.dtx}[2020/01/15 v3.38]
%</dtx>
%
%% File 'bbcompat.dtx'
Modified: trunk/Master/texmf-dist/source/latex/babel/locale.zip
===================================================================
(Binary files differ)
Modified: trunk/Master/texmf-dist/tex/generic/babel/babel.def
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/babel.def 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/babel.def 2020-01-15 21:50:07 UTC (rev 53415)
@@ -41,7 +41,7 @@
\wlog{File: #1 #4 #3 <#2>}%
\let\ProvidesFile\@undefined}
\fi
-\ProvidesFile{babel.def}[2019/12/08 3.37 Babel common definitions]
+\ProvidesFile{babel.def}[2020/01/15 3.38 Babel common definitions]
\ifx\AtBeginDocument\@undefined
\input plain.def\relax
\fi
@@ -1374,6 +1374,7 @@
\edef\bbl at savelocaleid{\the\localeid}%
% Set name and locale id
\def\languagename{#2}%
+ % \global\@namedef{bbl at lcname@#2}{#2}%
\bbl at id@assign
\let\bbl at KVP@captions\@nil
\let\bbl at KVP@import\@nil
@@ -1386,6 +1387,8 @@
\let\bbl at KVP@mapdigits\@nil
\let\bbl at KVP@intraspace\@nil
\let\bbl at KVP@intrapenalty\@nil
+ \let\bbl at KVP@onchar\@nil
+ \let\bbl at KVP@chargroups\@nil
\bbl at forkv{#1}{% TODO - error handling
\in@{..}{##1}%
\ifin@
@@ -1450,6 +1453,56 @@
\ifx\bbl at KVP@language\@nil\else
\bbl at csarg\edef{lname@#2}{\bbl at KVP@language}%
\fi
+ % == onchar ==
+ \ifx\bbl at KVP@onchar\@nil\else
+ \bbl at luahyphenate
+ \directlua{
+ if Babel.locale_mapped == nil then
+ Babel.locale_mapped = true
+ Babel.linebreaking.add_before(Babel.locale_map)
+ Babel.loc_to_scr = {}
+ Babel.chr_to_loc = {}
+ end}%
+ \bbl at xin@{ ids }{ \bbl at KVP@onchar\space}%
+ \ifin@
+ % TODO - error/warning if no script
+ \directlua{
+ if Babel.script_blocks['\bbl at cs{sbcp@\languagename}'] then
+ Babel.loc_to_scr[\the\localeid] =
+ Babel.script_blocks['\bbl at cs{sbcp@\languagename}']
+ Babel.locale_props[\the\localeid].lc = \the\localeid\space
+ Babel.locale_props[\the\localeid].lg = \the\@nameuse{l@\languagename}\space
+ end
+ }%
+ \fi
+ \bbl at xin@{ fonts }{ \bbl at KVP@onchar\space}%
+ \ifin@
+ \bbl at ifunset{bbl at lsys@\languagename}{\bbl at provide@lsys{\languagename}}{}%
+ \bbl at ifunset{bbl at wdir@\languagename}{\bbl at provide@dirs{\languagename}}{}%
+ \directlua{
+ if Babel.script_blocks['\bbl at cs{sbcp@\languagename}'] then
+ Babel.loc_to_scr[\the\localeid] =
+ Babel.script_blocks['\bbl at cs{sbcp@\languagename}']
+ end}
+ \ifx\bbl at mapselect\@undefined
+ \AtBeginDocument{%
+ \expandafter\bbl at add\csname selectfont \endcsname{{\bbl at mapselect}}%
+ {\selectfont}}%
+ \def\bbl at mapselect{%
+ \let\bbl at mapselect\relax
+ \edef\bbl at prefontid{\fontid\font}}%
+ \def\bbl at mapdir##1{%
+ {\def\languagename{##1}%
+ \let\bbl at ifrestoring\@firstoftwo % To avoid font warning
+ \bbl at switchfont
+ \directlua{
+ Babel.locale_props[\the\csname bbl at id@@##1\endcsname]%
+ ['/\bbl at prefontid'] = \fontid\font\space}}}%
+ \fi
+ \bbl at exp{\\\bbl at add\\\bbl at mapselect{\\\bbl at mapdir{\languagename}}}%
+ \fi
+ % TODO - catch non-valid values
+ \fi
% == mapfont ==
% For bidi texts, to switch the font based on direction
\ifx\bbl at KVP@mapfont\@nil\else
@@ -1670,9 +1723,13 @@
{\\\AddBabelHook[\languagename]{babel-prehc-\languagename}{patterns}%
{\prehyphenchar=\@nameuse{bbl at prehc@\languagename}\relax}}}%
\fi}}
+\ifx\bbl at readstream\@undefined
+ \csname newread\endcsname\bbl at readstream
+\fi
\def\bbl at read@ini#1#2{%
- \openin1=babel-#1.ini % FIXME - number must not be hardcoded
- \ifeof1
+ \global\@namedef{bbl at lini@\languagename}{#1}%
+ \openin\bbl at readstream=babel-#1.ini
+ \ifeof\bbl at readstream
\bbl at error
{There is no ini file for the requested language\\%
(#1). Perhaps you misspelled it or your installation\\%
@@ -1693,9 +1750,9 @@
\bbl at info{Importing #2 for \languagename\\%
from babel-#1.ini. Reported}%
\loop
- \if T\ifeof1F\fi T\relax % Trick, because inside \loop
+ \if T\ifeof\bbl at readstream F\fi T\relax % Trick, because inside \loop
\endlinechar\m at ne
- \read1 to \bbl at line
+ \read\bbl at readstream to \bbl at line
\endlinechar`\^^M
\ifx\bbl at line\@empty\else
\expandafter\bbl at iniline\bbl at line\bbl at iniline
@@ -1746,14 +1803,14 @@
\fi}}
\let\bbl at inikv@identification\bbl at inikv
\def\bbl at secpost@identification{%
- \bbl at ifunset{bbl@@kv at identification.name.opentype}%
- {\bbl at exportkey{lname}{identification.name.english}{}}%
- {\bbl at exportkey{lname}{identification.name.opentype}{}}%
+ \bbl at exportkey{elname}{identification.name.english}{}%
+ \bbl at exp{\\\bbl at exportkey{lname}{identification.name.opentype}%
+ {\csname bbl at elname@\languagename\endcsname}}%
\bbl at exportkey{lbcp}{identification.tag.bcp47}{}%
\bbl at exportkey{lotf}{identification.tag.opentype}{dflt}%
- \bbl at ifunset{bbl@@kv at identification.script.name.opentype}%
- {\bbl at exportkey{sname}{identification.script.name}{}}%
- {\bbl at exportkey{sname}{identification.script.name.opentype}{}}%
+ \bbl at exportkey{esname}{identification.script.name}{}%
+ \bbl at exp{\\\bbl at exportkey{sname}{identification.script.name.opentype}%
+ {\csname bbl at esname@\languagename\endcsname}}%
\bbl at exportkey{sbcp}{identification.script.tag.bcp47}{}%
\bbl at exportkey{sotf}{identification.script.tag.opentype}{DFLT}}
\let\bbl at inikv@typography\bbl at inikv
@@ -1889,12 +1946,34 @@
\def\bbl at ini@basic#1{%
\def\BabelBeforeIni##1##2{%
\begingroup
- \bbl at add\bbl at secpost@identification{\closein1 }%
+ \bbl at add\bbl at secpost@identification{\closein\bbl at readstream }%
\catcode`\[=12 \catcode`\]=12 \catcode`\==12 %
\bbl at read@ini{##1}{font and identification data}%
\endinput % babel- .tex may contain onlypreamble's
\endgroup}% boxed, to avoid extra spaces:
{\setbox\z@\hbox{\InputIfFileExists{babel-#1.tex}{}{}}}}
+\newcommand\localeinfo[1]{%
+ \bbl at ifunset{bbl@\csname bbl at info@#1\endcsname @\languagename}%
+ {\bbl at error{I've found no info for the current locale.\\%
+ The corresponding ini file has not been loaded\\%
+ Perhaps it doesn't exist}%
+ {See the manual for details.}}%
+ {\@nameuse{bbl@\csname bbl at info@#1\endcsname @\languagename}}}
+\@namedef{bbl at info@tag.ini}{lini}
+\@namedef{bbl at info@name.english}{elname}
+\@namedef{bbl at info@name.opentype}{lname}
+\@namedef{bbl at info@tag.bcp47}{lbcp}
+\@namedef{bbl at info@tag.opentype}{lotf}
+\@namedef{bbl at info@script.name}{esname}
+\@namedef{bbl at info@script.name.opentype}{sname}
+\@namedef{bbl at info@script.tag.bcp47}{sbcp}
+\@namedef{bbl at info@script.tag.opentype}{sotf}
+\let\bbl at ensureinfo\@gobble
+\newcommand\BabelEnsureInfo{%
+ \def\bbl at ensureinfo##1{%
+ \ifx\InputIfFileExists\@undefined\else % not in plain
+ \bbl at ifunset{bbl at lname@##1}{\bbl at ini@basic{##1}}{}%
+ \fi}}
\newcommand\babeladjust[1]{% TODO. Error handling.
\bbl at forkv{#1}{\@nameuse{bbl at ADJ@##1@##2}}}
\def\bbl at adjust@lua#1#2{%
Modified: trunk/Master/texmf-dist/tex/generic/babel/babel.sty
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/babel.sty 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/babel.sty 2020-01-15 21:50:07 UTC (rev 53415)
@@ -33,7 +33,7 @@
%%
\NeedsTeXFormat{LaTeX2e}[2005/12/01]
-\ProvidesPackage{babel}[2019/12/08 3.37 The Babel package]
+\ProvidesPackage{babel}[2020/01/15 3.38 The Babel package]
\@ifpackagewith{babel}{debug}
{\providecommand\bbl at trace[1]{\message{^^J[ #1 ]}}%
\let\bbl at debug\@firstofone}
Modified: trunk/Master/texmf-dist/tex/generic/babel/hyphen.cfg
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/hyphen.cfg 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/hyphen.cfg 2020-01-15 21:50:07 UTC (rev 53415)
@@ -37,7 +37,7 @@
\wlog{File: #1 #4 #3 <#2>}%
\let\ProvidesFile\@undefined}
\fi
-\ProvidesFile{hyphen.cfg}[2019/12/08 3.37 Babel hyphens]
+\ProvidesFile{hyphen.cfg}[2020/01/15 3.38 Babel hyphens]
\xdef\bbl at format{\jobname}
\ifx\AtBeginDocument\@undefined
\def\@empty{}
Modified: trunk/Master/texmf-dist/tex/generic/babel/luababel.def
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/luababel.def 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/luababel.def 2020-01-15 21:50:07 UTC (rev 53415)
@@ -34,6 +34,9 @@
\ifx\AddBabelHook\@undefined
\bbl at trace{Read language.dat}
+\ifx\bbl at readstream\@undefined
+ \csname newread\endcsname\bbl at readstream
+\fi
\begingroup
\toks@{}
\count@\z@ % 0=start, 1=0th, 2=normal
@@ -97,16 +100,16 @@
\fi
\def\bbl at elt#1#2#3#4{\@namedef{zth@#1}{}} % Define flags
\bbl at languages
- \openin1=language.dat
- \ifeof1
+ \openin\bbl at readstream=language.dat
+ \ifeof\bbl at readstream
\bbl at warning{I couldn't find language.dat. No additional\\%
patterns loaded. Reported}%
\else
\loop
\endlinechar\m at ne
- \read1 to \bbl at line
+ \read\bbl at readstream to \bbl at line
\endlinechar`\^^M
- \if T\ifeof1F\fi T\relax
+ \if T\ifeof\bbl at readstream F\fi T\relax
\ifx\bbl at line\@empty\else
\edef\bbl at line{\bbl at line\space\space\space}%
\expandafter\bbl at process@line\bbl at line\relax
@@ -373,7 +376,7 @@
quad = font.getfont(last_char.font).size
for lg, rg in pairs(sea_ranges) do
if last_char.char > rg[1] and last_char.char < rg[2] then
- lg = lg:sub(1, 4)
+ lg = lg:sub(1, 4) ^^ Remove trailing number of, eg, Cyrl1
local intraspace = Babel.intraspaces[lg]
local intrapenalty = Babel.intrapenalties[lg]
local n
@@ -418,15 +421,14 @@
luatexbase.registernumber'bbl at attr@locale')
local props = Babel.locale_props[LOCALE]
- class = Babel.cjk_class[item.char].c
+ local class = Babel.cjk_class[item.char].c
if class == 'cp' then class = 'cl' end % )] as CL
if class == 'id' then class = 'I' end
+ local br = 0
if class and last_class and Babel.cjk_breaks[last_class][class] then
br = Babel.cjk_breaks[last_class][class]
- else
- br = 0
end
if br == 1 and props.linebreak == 'c' and
@@ -462,14 +464,14 @@
\directlua{
luatexbase.add_to_callback('hyphenate',
function (head, tail)
- if Babel.cjk_enabled then
- Babel.cjk_linebreak(head)
- end
if Babel.linebreaking.before then
for k, func in ipairs(Babel.linebreaking.before) do
func(head)
end
end
+ if Babel.cjk_enabled then
+ Babel.cjk_linebreak(head)
+ end
lang.hyphenate(head)
if Babel.linebreaking.after then
for k, func in ipairs(Babel.linebreaking.after) do
@@ -727,6 +729,98 @@
\expandafter\addto\csname extras#1\endcsname{%
\babel at save\bbl at langfeatures
\edef\bbl at langfeatures{#2,}}}
+\directlua{
+Babel.script_blocks = {
+ ['Arab'] = {{0x0600, 0x06FF}, {0x08A0, 0x08FF}, {0x0750, 0x077F},
+ {0xFE70, 0xFEFF}, {0xFB50, 0xFDFF}, {0x1EE00, 0x1EEFF}},
+ ['Armn'] = {{0x0530, 0x058F}},
+ ['Beng'] = {{0x0980, 0x09FF}},
+ ['Cher'] = {{0x13A0, 0x13FF}, {0xAB70, 0xABBF}},
+ ['Cyrl'] = {{0x0400, 0x04FF}, {0x0500, 0x052F}, {0x1C80, 0x1C8F},
+ {0x2DE0, 0x2DFF}, {0xA640, 0xA69F}},
+ ['Deva'] = {{0x0900, 0x097F}, {0xA8E0, 0xA8FF}},
+ ['Ethi'] = {{0x1200, 0x137F}, {0x1380, 0x139F}, {0x2D80, 0x2DDF},
+ {0xAB00, 0xAB2F}},
+ ['Geor'] = {{0x10A0, 0x10FF}, {0x2D00, 0x2D2F}},
+ ['Grek'] = {{0x0370, 0x03FF}, {0x1F00, 0x1FFF}},
+ ['Hans'] = {{0x2E80, 0x2EFF}, {0x3000, 0x303F}, {0x31C0, 0x31EF},
+ {0x3300, 0x33FF}, {0x3400, 0x4DBF}, {0x4E00, 0x9FFF},
+ {0xF900, 0xFAFF}, {0xFE30, 0xFE4F}, {0xFF00, 0xFFEF},
+ {0x20000, 0x2A6DF}, {0x2A700, 0x2B73F},
+ {0x2B740, 0x2B81F}, {0x2B820, 0x2CEAF},
+ {0x2CEB0, 0x2EBEF}, {0x2F800, 0x2FA1F}},
+ ['Hebr'] = {{0x0590, 0x05FF}},
+ ['Japa'] = {{0x3000, 0x303F}, {0x3040, 0x309F}, {0x30A0, 0x30FF},
+ {0x4E00, 0x9FAF}, {0xFF00, 0xFFEF}},
+ ['Khmr'] = {{0x1780, 0x17FF}, {0x19E0, 0x19FF}},
+ ['Knda'] = {{0x0C80, 0x0CFF}},
+ ['Kore'] = {{0x1100, 0x11FF}, {0x3000, 0x303F}, {0x3130, 0x318F},
+ {0x4E00, 0x9FAF}, {0xA960, 0xA97F}, {0xAC00, 0xD7AF},
+ {0xD7B0, 0xD7FF}, {0xFF00, 0xFFEF}},
+ ['Laoo'] = {{0x0E80, 0x0EFF}},
+ ['Latn'] = {{0x0000, 0x007F}, {0x0080, 0x00FF}, {0x0100, 0x017F},
+ {0x0180, 0x024F}, {0x1E00, 0x1EFF}, {0x2C60, 0x2C7F},
+ {0xA720, 0xA7FF}, {0xAB30, 0xAB6F}},
+ ['Mahj'] = {{0x11150, 0x1117F}},
+ ['Mlym'] = {{0x0D00, 0x0D7F}},
+ ['Mymr'] = {{0x1000, 0x109F}, {0xAA60, 0xAA7F}, {0xA9E0, 0xA9FF}},
+ ['Orya'] = {{0x0B00, 0x0B7F}},
+ ['Sinh'] = {{0x0D80, 0x0DFF}, {0x111E0, 0x111FF}},
+ ['Taml'] = {{0x0B80, 0x0BFF}},
+ ['Telu'] = {{0x0C00, 0x0C7F}},
+ ['Tfng'] = {{0x2D30, 0x2D7F}},
+ ['Thai'] = {{0x0E00, 0x0E7F}},
+ ['Tibt'] = {{0x0F00, 0x0FFF}},
+ ['Vaii'] = {{0xA500, 0xA63F}},
+ ['Yiii']= {{0xA490, 0xA4CF}, {0xA000, 0xA48F}}
+}
+
+Babel.script_blocks.Hant = Babel.script_blocks.Hans
+
+function Babel.locale_map(head)
+ if not Babel.locale_mapped then return head end
+
+ local LOCALE = luatexbase.registernumber'bbl at attr@locale'
+ local GLYPH = node.id('glyph')
+ local inmath = false
+ for item in node.traverse(head) do
+ local toloc
+ if not inmath and item.id == GLYPH then
+ % Optimization: build a table with the chars found
+ if Babel.chr_to_loc[item.char] then
+ toloc = Babel.chr_to_loc[item.char]
+ else
+ for lc, maps in pairs(Babel.loc_to_scr) do
+ for _, rg in pairs(maps) do
+ if item.char >= rg[1] and item.char <= rg[2] then
+ Babel.chr_to_loc[item.char] = lc
+ toloc = lc
+ break
+ end
+ end
+ end
+ end
+ % Now, take action
+ if toloc then
+ if Babel.locale_props[toloc].lg then
+ item.lang = Babel.locale_props[toloc].lg
+ node.set_attribute(item, LOCALE, toloc)
+ end
+ if Babel.locale_props[toloc]['/'..item.font] then
+ item.font = Babel.locale_props[toloc]['/'..item.font]
+ end
+ end
+ elseif not inmath and item.id == 7 then
+ item.replace = item.replace and Babel.locale_map(item.replace)
+ item.pre = item.pre and Babel.locale_map(item.pre)
+ item.post = item.post and Babel.locale_map(item.post)
+ elseif item.id == node.id'math' then
+ inmath = (item.subtype == 0)
+ end
+ end
+ return head
+end
+}
\newcommand\babelcharproperty[1]{%
\count@=#1\relax
\ifvmode
@@ -935,6 +1029,7 @@
return head
end
+ &% Used below
function Babel.capture_func(key, cap)
local ret = "[[" .. cap:gsub('{([0-9])}', "]]..m[%1]..[[") .. "]]"
ret = ret:gsub("%[%[%]%]%.%.", '')
Modified: trunk/Master/texmf-dist/tex/generic/babel/nil.ldf
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/nil.ldf 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/nil.ldf 2020-01-15 21:50:07 UTC (rev 53415)
@@ -32,7 +32,7 @@
%% extension |.ins|) which are part of the distribution.
%%
-\ProvidesLanguage{nil}[2019/12/08 3.37 Nil language]
+\ProvidesLanguage{nil}[2020/01/15 3.38 Nil language]
\LdfInit{nil}{datenil}
\ifx\l at nil\@undefined
\newlanguage\l at nil
Modified: trunk/Master/texmf-dist/tex/generic/babel/switch.def
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/switch.def 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/switch.def 2020-01-15 21:50:07 UTC (rev 53415)
@@ -37,7 +37,7 @@
\wlog{File: #1 #4 #3 <#2>}%
\let\ProvidesFile\@undefined}
\fi
-\ProvidesFile{switch.def}[2019/12/08 3.37 Babel switching mechanism]
+\ProvidesFile{switch.def}[2020/01/15 3.38 Babel switching mechanism]
\ifx\AtBeginDocument\@undefined
\input plain.def\relax
\fi
@@ -58,8 +58,8 @@
\countdef\last at language=19
\def\addlanguage{\alloc at 9\language\chardef\@cclvi}
\fi
-\def\bbl at version{3.37}
-\def\bbl at date{2019/12/08}
+\def\bbl at version{3.38}
+\def\bbl at date{2020/01/15}
\def\adddialect#1#2{%
\global\chardef#1#2\relax
\bbl at usehooks{adddialect}{{#1}{#2}}%
@@ -131,6 +131,7 @@
Babel = Babel or {}
Babel.locale_props = Babel.locale_props or {}
Babel.locale_props[\bbl at id@last] = {}
+ Babel.locale_props[\bbl at id@last].name = '\languagename'
}%
\fi}%
{}%
@@ -145,6 +146,7 @@
\edef\languagename{%
\ifnum\escapechar=\expandafter`\string#1\@empty
\else\string#1\@empty\fi}%
+ % \@namedef{bbl at lcname@#1}{#1}%
\select at language{\languagename}%
% write to auxs
\expandafter\ifx\csname date\languagename\endcsname\relax\else
@@ -195,6 +197,8 @@
\let\select at language@x\select at language
\newif\ifbbl at usedategroup
\def\bbl at switch#1{% from select@, foreign@
+ % make sure there is info for the language if so requested
+ \bbl at ensureinfo{#1}%
% restore
\originalTeX
\expandafter\def\expandafter\originalTeX\expandafter{%
@@ -297,6 +301,7 @@
\def\foreign at language#1{%
% set name
\edef\languagename{#1}%
+ % \@namedef{bbl at lcname@#1}{#1}%
\bbl at fixname\languagename
\bbl at iflanguage\languagename{%
\expandafter\ifx\csname date\languagename\endcsname\relax
Deleted: trunk/Master/texmf-dist/tex/generic/babel/test-hyphen-post-wiki.tex
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/test-hyphen-post-wiki.tex 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/test-hyphen-post-wiki.tex 2020-01-15 21:50:07 UTC (rev 53415)
@@ -1,31 +0,0 @@
-\documentclass{article}
-
-\usepackage[ngerman]{babel}
-
-\directlua{
-
-Babel.linebreaking.add_after(Babel.post_hyphenate_replace)
-
-Babel.linebreaking.add_replacement(
- 'ngerman',
- '([fmtrp])|{1}',
- {
- nil,
- { no = '{1}', pre = '{1}{1}-', post = '', penalty = 150, data = 1 },
- {}
- })
-
-}
-
-\begin{document}
-
-\rightskip5cm
-
-Auffrisierende Auffrisierendem Auffrisierenden Auffrisierender
-Auffrisierendes Auffrisierst Auffrisiert Auffrisierte Auffrisiertem
-Auffrisierten Auffrisierter Auffrisiertes Auffrisiertest Auffrisiertet
-Auffrisst Auffuhr Aufführbar Aufführbare Aufführbarem Aufführbaren
-Aufführbarer Aufführbares Aufführe Auffuhren Aufführen Aufführend
-Aufführende Aufführendem Aufführenden Aufführender Aufführendes
-
-\end{document}
\ No newline at end of file
Deleted: trunk/Master/texmf-dist/tex/generic/babel/xe-polygreek.tex
===================================================================
--- trunk/Master/texmf-dist/tex/generic/babel/xe-polygreek.tex 2020-01-15 21:49:08 UTC (rev 53414)
+++ trunk/Master/texmf-dist/tex/generic/babel/xe-polygreek.tex 2020-01-15 21:50:07 UTC (rev 53415)
@@ -1,79 +0,0 @@
-\makeatletter
-\def\e at alloc#1#2#3#4#5#6{%
- \global\advance#3\@ne
- \e at ch@ck{#3}{#4}{#5}#1%
- \allocationnumber#3\relax
- \global#2#6\allocationnumber}
- % \wlog{\string#6=\string#1\the\allocationnumber}}%
-
-\documentclass{article}
-
-
-
-\usepackage{babel}
-
-\usepackage{multicol}
-
-\babelprovide[import,main]{polytonicgreek}
-\babelfont{rm}{Old Standard}
-
-\babelprovide{danish}
-
-\begin{document}
-
-\section*{Φορτωμένα κόκκαλα}
-
-\textit{Αλέξανδρος Παπαδιαμάντης}\\
-1907\\
-From https://el.wikisource.org/wiki/Φορτωμένα\_κόκκαλα
-
-{\selectlanguage{danish}}
-
-\begin{multicols}{3}
-
-Ἀνεβαίνομεν τὸ βουνὸν, πεζοὶ, μὲ τὸ γαϊδουράκι φορτωμένον, ὁ παπ’
-Ἀνδρέας, ὁ καλός μας εὐχέτης, κι’ ὁ μακαρίτης ὁ Λαμιαῖος, κι’ ἐγὼ, κι’
-ὁ Ἀλέκος τὸ Φωτάκι, ὁ μικρὸς καὶ πρόθυμος φίλος μας. Εἰς ὅλα ἦτον
-πάντοτε ἕτοιμος νὰ τρέχῃ ἀκούραστος, ὅ,τι ἤθελες τὸν διατάξει. Νὰ πάῃ
-στὸ χωριὸ, διὰ θέλημα, δυὸ ὧρες δρόμον, καὶ πάλι, φορτωμένος, ὀπίσω νὰ
-ἔλθῃ· νὰ σκουπίσῃ ὅλον τὸ ἐξωκκλήσι, καὶ τὸν αὐλόγυρον, καὶ τὰ κελλιὰ,
-μὲ πρόχειρον σκούπαν ἀπὸ σπαρτίνες καὶ θάμνους· νὰ τρέξῃ κάτω στὸν
-αἰγιαλόν, διὰ νὰ μᾶς φέρῃ πεταλίδες, καὶ κοχύλια, καὶ πετροκάβουρα, διὰ
-τὸ ὀρεκτικὸν δεῖπνόν μας καὶ νὰ γυρίσῃ μετὰ μίαν ὥραν μὲ μίαν ποδιὰν
-γεμάτην· εἶτα ν’ ἀνάψῃ φωτιάν, νὰ ψήσῃ, νὰ μαγειρεύσῃ ὅλα τὰ ἐδέσματα·
-καὶ νὰ ἔχῃ τὴν ἐπιστασίαν τοῦ παγουριοῦ καὶ τῆς φλάσκας, διὰ νὰ
-εὑρίσκωνται δροσερὰ εἰς τὸ ρεῦμα, ἀκριβῶς ὑπὸ τὴν βρύσην· εἰς ὅλα ἦτον
-μονάκριβος.
-
-Ἕκτος μᾶς εἶχεν ἀκολουθήσει ὁ σκύλος τοῦ Σταμάτη τοῦ Ἀλεξανδράκη,
-σκύλος προωρισμένος ὀψέποτε νὰ μένῃ ἀδέσποτος. Ὁ ἄτυχος καὶ κακοκέφαλος
-φίλος μας, ὁ Σταμάτης, ἀφοῦ ἐμάλωσε μὲ ὅλους τοὺς συγγενεῖς καὶ τοὺς
-φίλους του, καὶ σχεδὸν μὲ ὅλον τὸν κόσμον, ἤρχισε νὰ πάσχῃ ἀπὸ
-περιοδικὰς ἀφανείας, ὁποῦ ἦσαν τὰ προανακρούσματα τῆς ὁριστικῆς
-ἐξαφανίσεώς του ἀπὸ τὸν μάταιον κόσμον. Πότε ἐκρύπτετο, ὡς ἔλεγαν, εἰς
-μίαν ἐρημικὴν σπηλιὰν, πότε ἐπήγαινε νὰ μείνῃ ὀλίγας ἡμέρας εἰς τὸ
-Μοναστήρι, πότε ἐταξίδευεν, ἄγνωστον ποῦ· καὶ ὅλας αὐτὰς τὰς φορὰς, τὸν
-ἄτυχον Σαψώνην, τὸν ἄφηνεν εἰς τὸ ἔλεος τοῦ Θεοῦ καὶ τῶν ἀνθρώπων τῆς
-ἀγορᾶς, ἂν θὰ εὐαρεστοῦντο ποτὲ νὰ τοῦ ρίψουν ἓν ξηροκόμματον. Συχνὰ ὁ
-Γιωργὸς ὁ Λαφκιώτης ὁ ἰδιοκτήτης τοῦ ἀρχαιοπρεποῦς καὶ ἀναλλοιώτου
-καφενείου εἰς τὴν παραθαλασσίαν, ἂν καὶ τοῦ εἶχε φάγῃ ὀλίγας
-ἑκατοντάδας δραχμῶν, καλῇ τῇ πίστει, ὁ ἀφέντης τοῦ Σαψώνη, ᾤκτειρε τὸ
-ἄκακον θρέμμα, καὶ τοῦ ἔρριπτεν ὀλίγα κόκκαλα. Ἐμὲ, ἀφοῦ μὲ
-κατεσκόπευεν ὁ Σαψώνης, ἀπὸ καφενεῖον εἰς ὕπαιθρον, καὶ ἀπὸ κιόσκι εἰς
-τένταν, ἐπὶ τῆς προκυμαίας, τέλος, μὲ ἠκολούθει ὁριστικῶς εἰς τὴν
-οἰκίαν ὅπου ἔπρεπε νὰ τοῦ ρίψω τι ἐκ τοῦ ὑστερήματος.
-
-Ἀλλὰ καὶ εἰς τὰ καλά του ὅταν ἦτο ὁ Σταμάτης, διὰ τὸν περιπαθῶς
-ἀφωσιωμένον σκύλον δὲν εἶχε λάβῃ ἄλλην πρόνοιαν, εἰμὴ νὰ τὸν ρίπτῃ
-αἰφνιδίως εἰς τὸ κῦμα, γαυγίζοντα καὶ μὴ θέλοντα, διὰ νὰ κολυμβᾷ. Τὸν
-εἶχεν ἀφήσει ἀκούρευτον καὶ βαθύτριχον ἀπὸ χρόνων πολλῶν. Ἦτο πολὺ
-μαλλιαρὸς σκύλος. Τὴν φορὰν αὐτὴν, ἀφοῦ μᾶς ἐμυρίσθη πῶς ἡτοιμαζόμεθα
-δι’ ἐκδρομὴν, εἶχε δείξει ἀνήσυχον περιέργειαν, ὅταν ἐφορτώνετο τὸ
-ὀνάριον, καὶ εἶχε πλησιάσει νὰ ὀσφρανθῇ τὶ περιεῖχεν ὁ σάκκος ἐκεῖνος,
-τὸν ὁποῖον ὁ Ἀλέκος εἶχε περιδέσει καὶ φορτώσει περὶ τὸ σάγμα, εἰς τὴν
-ἀριστερὰν πλευρὰν τοῦ ζώου. Δεξιὰ εἶχε φορτωθῇ ὁ μάρσιππος μὲ τὰ ἱερὰ
-τοῦ Παπᾶ, εἷς κάλαθος μὲ τρόφιμα, καὶ μία φλάσκα μὲ οἶνον. Εἶτα, μᾶς
-ἠκολούθησε μὲ βῆμα, αὐτόκλητος.
-\end{multicols}
-
-\end{document}
\ No newline at end of file
More information about the tex-live-commits
mailing list