[latex3-commits] [git/LaTeX3-latex3-latex2e] master: Option bidi=basic for both L and R text. (077215f)
Javier
javier at dante.de
Wed Apr 4 18:19:26 CEST 2018
Repository : https://github.com/latex3/latex2e
On branch : master
Link : https://github.com/latex3/latex2e/commit/077215f312316251fe393bebe233d0d1fd578c09
>---------------------------------------------------------------
commit 077215f312316251fe393bebe233d0d1fd578c09
Author: Javier <javier at localhost>
Date: Wed Apr 4 18:18:33 2018 +0200
Option bidi=basic for both L and R text.
>---------------------------------------------------------------
077215f312316251fe393bebe233d0d1fd578c09
required/babel/README.md | 4 +-
required/babel/babel.dtx | 607 ++++++++++++++++++++++++++----
required/babel/babel.ins | 3 +-
required/babel/babel.pdf | Bin 639170 -> 656726 bytes
required/babel/bbcompat.dtx | 2 +-
required/babel/samples/lua-bidibasic.pdf | Bin 0 -> 43293 bytes
required/babel/samples/lua-bidibasic.tex | 151 ++++++++
7 files changed, 686 insertions(+), 81 deletions(-)
diff --git a/required/babel/README.md b/required/babel/README.md
index 68f077f..593c1de 100644
--- a/required/babel/README.md
+++ b/required/babel/README.md
@@ -1,4 +1,4 @@
-## Babel 3.18
+## Babel 3.18.1236
This package manages culturally-determined typographical (and other)
rules, and hyphenation patterns for a wide range of languages. Many
@@ -118,5 +118,5 @@ respective authors.
```
Javier Bezos
-2018/02/14
+2018/04/04
diff --git a/required/babel/babel.dtx b/required/babel/babel.dtx
index b11d3d5..89bd033 100644
--- a/required/babel/babel.dtx
+++ b/required/babel/babel.dtx
@@ -31,7 +31,7 @@
%
% \iffalse
%<*filedriver>
-\ProvidesFile{babel.dtx}[2018/02/14 v3.18 The Babel package]
+\ProvidesFile{babel.dtx}[2018/04/04 v3.18.1236 The Babel package]
\documentclass{ltxdoc}
\GetFileInfo{babel.dtx}
\usepackage{fontspec}
@@ -85,6 +85,7 @@
\newtheorem{warning}{Warning}
\newtheorem{note}{Note}
\newtheorem{example}{Example}
+\newtheorem{troubleshooting}{Troubleshooting}
\let\bblxv\verbatim
\let\bblexv\endverbatim
\def\verbatim{\begin{shaded*}\bblxv\vskip-\baselineskip\vskip2.5\parsep}
@@ -256,8 +257,8 @@ purpose, namely, passing that language as an optional argument. In
addition, you may want to set the font and input encodings.
\begin{example}
- Here is a simple full example for “traditional” \TeX{} engines (see
- below for \xetex{} and \luatex{}). The packages |fontenc| and
+ Here is a simple full example for “traditional” \TeX{} engines
+ (see below for \xetex{} and \luatex{}). The packages |fontenc| and
|inputenc| do not belong to \babel, but they are included in the
example because typically you will need them:
\begin{verbatim}
@@ -275,10 +276,15 @@ Plus ça change, plus c'est la même chose!
\end{document}
\end{verbatim}
\end{example}
-\begin{warning}
-A common source of trouble is a wrong setting of the input
-encoding. Make sure you set the encoding actually used by your editor.
-\end{warning}
+\begin{troubleshooting}
+\trouble{Paragraph ended before \textbackslash UTFviii at three@octets was complete}
+A common source of trouble is a wrong setting of the input encoding.
+Very often you will get the following somewhat cryptic error:
+\begin{verbatim}
+! Paragraph ended before \UTFviii at three@octets was complete.
+\end{verbatim}
+Make sure you set the encoding actually used by your editor.
+\end{troubleshooting}
Another approach is making the language (\Lopt{french} in the example)
a global option in order to let other packages detect and use it:
@@ -303,6 +309,26 @@ the option and will be able to use it.
specific languages for further info.
\end{note}
+\begin{troubleshooting}
+ The following warning is about hyphenation patterns, which are
+ not under the direct control of \babel:
+\trouble{No hyphenation patterns were preloaded for (babel) the
+ language `LANG' into the format}
+\begin{verbatim}
+Package babel Warning: No hyphenation patterns were preloaded for
+(babel) the language `LANG' into the format.
+(babel) Please, configure your TeX system to add them and
+(babel) rebuild the format. Now I will use the patterns
+(babel) preloaded for \language=0 instead on input line 57.
+\end{verbatim}
+ The document will be typeset, but very likely the text will not be
+ correctly hyphenated. Some languages may be raising this warning
+ wrongly (because they are not hyphenated); it is a bug to be fixed
+ -- just ignore it. See the manual of your distribution (Mac\TeX,
+ Mik\TeX, \TeX Live, etc.) for further info about how to configure
+ it.
+\end{troubleshooting}
+
\subsection{Multilingual documents}
In multilingual documents, just use several options. The last one is
@@ -476,24 +502,6 @@ _\usepackage[russian]{babel}_
all, and therefore you removed it from the option list). In most
cases, the error vanishes when the document is typeset again, but in
more severe ones you will need to remove the |aux| file.
-
-\item The following warning is about hyphenation patterns, which are
- not under the direct control of \babel:
-\trouble{No hyphenation patterns were preloaded for (babel) the
- language `LANG' into the format}
-\begin{verbatim}
-Package babel Warning: No hyphenation patterns were preloaded for
-(babel) the language `LANG' into the format.
-(babel) Please, configure your TeX system to add them and
-(babel) rebuild the format. Now I will use the patterns
-(babel) preloaded for \language=0 instead on input line 57.
-\end{verbatim}
- The document will be typeset, but very likely the text will not be
- correctly hyphenated. Some languages may be raising this warning
- wrongly (because they are not hyphenated); it is a bug to be fixed
- -- just ignore it. See the manual of your distribution (Mac\TeX,
- Mik\TeX, \TeX Live, etc.) for further info about how to configure
- it.
\end{itemize}
\subsection{Plain}
@@ -592,40 +600,39 @@ Same as |\foreignlanguage| but as environment. Spaces after the
environment are \textit{not} ignored.
This environment was originally intended for intermixing left-to-right
-typesetting with right-to-left typesetting in engines not supporting a
-change in the writing direction inside a line. However, by default it never
-complied with the documented behavior and it is just a version as
+ypesetting with right-to-left typesetting in engines not supporting a
+change in the writing direction inside a line. However, by default it
+never complied with the documented behavior and it is just a version as
environment of |\foreignlanguage|.
\Describe{\begin\menv{hyphenrules}}{\marg{language}\Eenv{hyphenrules}}
-The environment \Lenv{hyphenrules} can be used to select \emph{only}
-the hyphenation rules to be used (it can be used as command,
-too). This can for instance be used to select `nohyphenation',
-provided that in \file{language.dat} the `language'
-\textsf{nohyphenation} is defined by loading \file{zerohyph.tex}. It
-deactivates language shorthands, too (but not user shorthands).
+The environment \Lenv{hyphenrules} can be used to select \emph{only} he
+hyphenation rules to be used (it can be used as command, oo). This can
+for instance be used to select `nohyphenation', provided that in
+\file{language.dat} the `language' \textsf{nohyphenation} is defined by
+loading \file{zerohyph.tex}. It deactivates language shorthands, too
+(but not user shorthands).
Except for these simple uses, |hyphenrules| is discouraged and
-|otherlanguage*| (the starred version) is preferred, as the former
-does not take into account possible changes in encodings of characters
-like, say, |'| done by some languages (eg, \textsf{italian},
-\textsf{french}, \textsf{ukraineb}). To set hyphenation exceptions,
-use |\babelhyphenation| (see below).
+|otherlanguage*| (the starred version) is preferred, as the former does
+not take into account possible changes in encodings of characters like,
+say, |'| done by some languages (eg, \textsf{italian}, \textsf{french},
+\textsf{ukraineb}). To set hyphenation exceptions, use
+|\babelhyphenation| (see below).
\subsection{More on selection}
\Describe\babeltags{\char`\{\m{tag1} \texttt{=} \m{language1}, \m{tag2}
\texttt{=} \m{language2}, \dots\char`\}}
-\New{3.9i} In multilingual documents with many language switches
-the commands above can be cumbersome. With this tool shorter names can
-be defined. It adds nothing really new -- it is just syntactical
-sugar.
+\New{3.9i} In multilingual documents with many language switches he
+commands above can be cumbersome. With this tool shorter names can be
+defined. It adds nothing really new -- it is just syntactical sugar.
It defines |\text|\m{tag1}\marg{text} to be
|\foreignlanguage|\marg{language1}\marg{text}, and |\begin|\marg{tag1}
-to be |\begin{otherlanguage*}|\marg{language1}, and so on. Note
+o be |\begin{otherlanguage*}|\marg{language1}, and so on. Note
|\|\m{tag1} is also allowed, but remember to set it locally inside a
group.
\begin{example}
@@ -2468,10 +2475,10 @@ to you to make sure no toc type is duplicated).
\subsection{Languages supported by \babel}
-In the following table most of the languages supported by \babel\ are
-listed, together with the names of the option which you can load
-\babel\ with for each language. Note this list is open and the
-current options may be different. It does not include |ini| files.
+In the following table most of the languages supported by \babel{} with
+and |.ldf| file are listed, together with the names of the option which
+you can load \babel\ with for each language. Note this list is open and
+the current options may be different. It does not include |ini| files.
\begin{description}
\itemsep=-\parskip
@@ -2703,6 +2710,51 @@ chapters), in Hungarian ``from (1)'' is ``(1)-b\H{o}l'', but ``from
\subsection{Tentative and experimental code}
+\textbf{Option \texttt{bidi=basic}}
+
+\New{3.19} With this package option \textit{both} L and R text can be
+mixed without explicit markup (which will be only necessary in some
+special cases where the Unicode algorithm fails). It is used much
+like |bidi=basic-r| but with R text inside L text you may want to
+map the font so that the correct features are in force. This is
+accomplised with an option in |\babelprovide|, as illustrated:
+ \begingroup
+% If you are looking at the code to see how it has been written, you
+% will be disappointed :-). The following example is built ad hoc to
+% emulate the final result to avoid dependencies, and therefore it's
+% not "real" code.
+\setmonofont[Scale=.87,Script=Arabic]{DejaVu Sans Mono} \catcode`@=13
+\def@#1{\ifcase#1\relax \egroup \or \bgroup\textdir TRT \else
+\bgroup\textdir TLT \pardir TLT \fi}
+\begin{verbatim}
+\documentclass{book}
+
+\usepackage[english, _bidi=basic_]{babel}
+
+\babelprovide[_mapfont=direction_]{arabic}
+
+\babelfont{rm}{Crimson}
+\babelfont[*arabic]{rm}{FreeSerif}
+
+\begin{document}
+
+Most Arabic speakers consider the two varieties to be two registers
+of one language, although the two registers can be referred to in
+Arabic as @1فصحى العصر@0 \textit{fuṣḥā l-ʻaṣr} (MSA) and
+ at 1فصحى التراث@0 \textit{fuṣḥā t-turāth} (CA).
+
+\end{document}
+\end{verbatim}
+\endgroup
+What |mapfont=direction| means is, ‘when a character has the same
+direction as the script for the “provided” language (|arabic| in this
+case), then change its font to that set for this language (here defined
+indirectly via |*arabic|)’.
+
+There is a sample on GitHub, under \texttt{/required/babel/samples}.
+
+\textbf{Old stuff}
+
Handling of \textbf{``Unicode'' fonts} is problematic. There is
\textsf{fontspec}, but special macros are required (not only the NFSS
ones) and it doesn't provide ``orthogonal axis'' for features,
@@ -2727,10 +2779,10 @@ So, for example:
\end{verbatim}
\textbf{Bidi writing} is taking its \textit{first steps}.
-\textit{First steps} means exactly that. For example, in \luatex{} any
-Arabic text must be marked up explicitly in L mode. On the other hand,
+\textit{First steps} means exactly that. On the other hand,
\xetex{} poses quite different challenges. Document layout (lists,
-footnotes, etc.) is not touched at all.
+footnotes, etc.) is not touched at all by default, but an option to
+manage them is almost finished (for luatex).
See the code section for |\foreignlanguage*| (a new starred version of
|\foreignlanguage|).
@@ -3635,8 +3687,8 @@ help from Bernd Raichle, for which I am grateful.
% little bit of literate programming.
%
% \begin{macrocode}
-%<<version=3.18>>
-%<<date=2018/02/14>>
+%<<version=3.18.1236>>
+%<<date=2018/04/04>>
% \end{macrocode}
%
% \section{Tools}
@@ -4100,23 +4152,27 @@ help from Bernd Raichle, for which I am grateful.
\bbl at languages
\fi
\ifodd\bbl at engine
- \@ifpackagewith{babel}{bidi=basic-r}{% must go before any \DeclareOption
+ \def\bbl at loadbidi#1{%
\let\bbl at beforeforeign\leavevmode
\AtEndOfPackage{\EnableBabelHook{babel-bidi}}%
\RequirePackage{luatexbase}%
\directlua{
require('babel-bidi.lua')
- require('babel-bidi-basic-r.lua')
+ require('babel-bidi-#1.lua')
luatexbase.add_to_callback('pre_linebreak_filter',
- Babel.pre_otfload,
- 'Babel.pre_otfload',
+ Babel.pre_otfload_v,
+ 'Babel.pre_otfload_v',
luatexbase.priority_in_callback('pre_linebreak_filter',
'luaotfload.node_processor') or nil)
luatexbase.add_to_callback('hpack_filter',
- Babel.pre_otfload,
- 'Babel.pre_otfload',
+ Babel.pre_otfload_h,
+ 'Babel.pre_otfload_h',
luatexbase.priority_in_callback('hpack_filter',
- 'luaotfload.node_processor') or nil)}}{}
+ 'luaotfload.node_processor') or nil)}}
+ \@ifpackagewith{babel}{bidi=basic-r}{% must go before any \DeclareOption
+ \bbl at loadbidi{basic-r}}{}
+ \@ifpackagewith{babel}{bidi=basic}{%
+ \bbl at loadbidi{basic}}{}
\fi
% \end{macrocode}
%
@@ -4584,6 +4640,7 @@ help from Bernd Raichle, for which I am grateful.
You shouldn't try to proceed from here, type x to quit.}
\fi
%</package>
+%<*core>
% \end{macrocode}
%
% \section{The kernel of Babel (\texttt{babel.def}, common)}
@@ -4631,7 +4688,6 @@ help from Bernd Raichle, for which I am grateful.
% \changes{babel~3.9q}{2016/02/12}{Load lua patterns if not lualatex.}
%
% \begin{macrocode}
-%<*core>
\ifx\ldf at quit\@undefined
\else
\expandafter\endinput
@@ -7494,6 +7550,7 @@ help from Bernd Raichle, for which I am grateful.
\let\bbl at KVP@language\@nil
\let\bbl at KVP@dir\@nil
\let\bbl at KVP@hyphenrules\@nil
+ \let\bbl at KVP@mapfont\@nil
\bbl at forkv{#1}{\bbl at csarg\def{KVP@##1}{##2}}% TODO - error handling
\ifx\bbl at KVP@captions\@nil
\let\bbl at KVP@captions\bbl at KVP@import
@@ -7521,6 +7578,28 @@ help from Bernd Raichle, for which I am grateful.
\ifx\bbl at KVP@language\@nil\else
\bbl at csarg\edef{lname@#2}{\bbl at KVP@language}%
\fi
+ \ifx\bbl at KVP@mapfont\@nil\else
+ \bbl at ifsamestring{\bbl at KVP@mapfont}{direction}{}%
+ {\bbl at error{Option `\bbl at KVP@mapfont' unknown for\\%
+ mapfont. Use `direction'.%
+ {See the manual for details.}}}%
+ \bbl at ifunset{bbl at lsys@\languagename}{\bbl at provide@lsys{\languagename}}{}%
+ \bbl at ifunset{bbl at wdir@\languagename}{\bbl at provide@dirs{\languagename}}{}%
+ \ifx\bbl at mapselect\@undefined
+ \AtBeginDocument{%
+ \expandafter\bbl at add\csname selectfont \endcsname{{\bbl at mapselect}}%
+ {\selectfont}}%
+ \def\bbl at mapselect{%
+ \let\bbl at mapselect\relax
+ \edef\bbl at prefontid{\fontid\font}}%
+ \def\bbl at mapdir##1{%
+ {\def\languagename{##1}\bbl at switchfont
+ \directlua{Babel.fontmap
+ [\the\csname bbl at wdir@##1\endcsname]%
+ [\bbl at prefontid]=\fontid\font}}}%
+ \fi
+ \bbl at exp{\\\bbl at add\\\bbl at mapselect{\\\bbl at mapdir{\languagename}}}%
+ \fi
\let\languagename\bbl at savelangname}
% \end{macrocode}
%
@@ -8782,6 +8861,7 @@ help from Bernd Raichle, for which I am grateful.
\ifodd\bbl at engine % luatex=1
\AddBabelHook{babel-bidi}{afterextras}{\bbl at switchdir}
\DisableBabelHook{babel-bidi}
+ \chardef\bbl at thepardir\z@
\def\bbl at getluadir#1{%
\directlua{
if tex.#1dir == 'TLT' then
@@ -8789,7 +8869,7 @@ help from Bernd Raichle, for which I am grateful.
elseif tex.#1dir == 'TRT' then
tex.sprint('1')
end}}
- \def\bbl at setdir#1#2#3{% 1=text/par.. 2=\textdir.. 3=0 lr/1 rl
+ \def\bbl at setluadir#1#2#3{% 1=text/par.. 2=\textdir.. 3=0 lr/1 rl
\ifcase#3\relax
\ifcase\bbl at getluadir{#1}\relax\else
#2 TLT\relax
@@ -8800,11 +8880,12 @@ help from Bernd Raichle, for which I am grateful.
\fi
\fi}
\def\bbl at textdir#1{%
- \bbl at setdir{text}\textdir{#1}% TODO - ?\linedir
- \setattribute\bbl at attr@dir{#1}}
- \def\bbl at pardir{\bbl at setdir{par}\pardir}
- \def\bbl at bodydir{\bbl at setdir{body}\bodydir}
- \def\bbl at pagedir{\bbl at setdir{page}\pagedir}
+ \bbl at setluadir{text}\textdir{#1}% TODO - ?\linedir
+ \setattribute\bbl at attr@dir{\numexpr\bbl at thepardir*3+#1}}
+ \def\bbl at pardir#1{\bbl at setluadir{par}\pardir{#1}%
+ \chardef\bbl at thepardir#1\relax}
+ \def\bbl at bodydir{\bbl at setluadir{body}\bodydir}
+ \def\bbl at pagedir{\bbl at setluadir{page}\pagedir}
\def\bbl at dirparastext{\pardir\the\textdir\relax}% %%%%
\else % pdftex=0, xetex=2
\AddBabelHook{babel-bidi}{afterextras}{\bbl at switchdir}
@@ -8931,6 +9012,7 @@ help from Bernd Raichle, for which I am grateful.
\if at nobreak\ifvmode\nobreak\fi\fi}
\fi
%</core>
+%<*kernel>
% \end{macrocode}%
% \end{macro}
%
@@ -8942,7 +9024,6 @@ help from Bernd Raichle, for which I am grateful.
% counter.
%
% \begin{macrocode}
-%<*kernel>
<@Make sure ProvidesFile is defined@>
\ProvidesFile{switch.def}[<@date@> <@version@> Babel switching mechanism]
<@Load macros for plain if not LaTeX@>
@@ -9801,6 +9882,7 @@ help from Bernd Raichle, for which I am grateful.
preloaded for \bbl at nulllanguage\space instead}}
\let\bbl at usehooks\@gobbletwo
%</kernel>
+%<*patterns>
% \end{macrocode}
%
% \end{macro}
@@ -9864,7 +9946,6 @@ help from Bernd Raichle, for which I am grateful.
% is dumped.
%
% \begin{macrocode}
-%<*patterns>
<@Make sure ProvidesFile is defined@>
\ProvidesFile{hyphen.cfg}[<@date@> <@version@> Babel hyphens]
\xdef\bbl at format{\jobname}
@@ -10258,6 +10339,11 @@ help from Bernd Raichle, for which I am grateful.
\newattribute\bbl at attr@dir
\bbl at exp{\output{\bodydir\pagedir\the\output}}%
\AtEndOfPackage{\EnableBabelHook{babel-bidi}}}
+ \DeclareOption{bidi=basic}% TODO -- copypaste pattern
+ {\let\bbl at beforeforeign\leavevmode
+ \newattribute\bbl at attr@dir
+ \bbl at exp{\output{\bodydir\pagedir\the\output}}%
+ \AtEndOfPackage{\EnableBabelHook{babel-bidi}}}
\else
\DeclareOption{bidi=basic-r}%
{\bbl at error
@@ -10269,6 +10355,16 @@ help from Bernd Raichle, for which I am grateful.
\AtEndOfPackage{%
\EnableBabelHook{babel-bidi}%
\bbl at xebidipar}}
+ \DeclareOption{bidi=basic}% TODO --- copypaste pattern
+ {\bbl at error
+ {The bidi method `basic' is available only in\\%
+ luatex. I'll continue with `bidi=default', so\\%
+ expect wrong results}%
+ {See the manual for further details.}%
+ \let\bbl at beforeforeign\leavevmode
+ \AtEndOfPackage{%
+ \EnableBabelHook{babel-bidi}%
+ \bbl at xebidipar}}
\fi
\DeclareOption{bidi=default}%
{\let\bbl at beforeforeign\leavevmode
@@ -10347,9 +10443,10 @@ help from Bernd Raichle, for which I am grateful.
\<bbl@##1dflt@*\bbl at tempa>}}}%
{}}% 1=T - language, already defined
\def\bbl at tempa{%
- \bbl at warning{The current font is not a standard family.\\%
- Script and Language are not applied. Consider defining\\%
- a new family with \string\babelfont,}}%
+ \bbl at warning{The current font is not a standard family:\\%
+ \fontname\font\\%
+ Script and Language are not applied. Consider defining a\\%
+ new family with \string\babelfont. Reported}}%
\bbl at foreach\bbl at font@fams{% don't gather with prev for
\bbl at ifunset{bbl@##1dflt@\languagename}%
{\bbl at cs{famrst@##1}%
@@ -10370,17 +10467,20 @@ help from Bernd Raichle, for which I am grateful.
% settings will take precedence.
%
% \begin{macrocode}
-\def\bbl at font@set#1#2#3{%
+\def\bbl at font@set#1#2#3{% eg \bbl at rmdflt@lang \rmdefault \rmfamily
\bbl at xin@{<>}{#1}%
\ifin@
\bbl at exp{\\\bbl at fontspec@set\\#1\expandafter\@gobbletwo#1}%
\fi
\bbl at exp{%
- \def\\#2{#1}% eg, \rmdefault{\bbl at rm1dflt@lang}
+ \def\\#2{#1}% eg, \rmdefault{\bbl at rmdflt@lang}
\\\bbl at ifsamestring{#2}{\f at family}{\\#3\let\\\bbl at tempa\relax}{}}}
-\def\bbl at fontspec@set#1#2#3{%
+\def\bbl at fontspec@set#1#2#3{% eg \bbl at rmdflt@lang fnt-opt fnt-nme
+ \let\bbl at tempe\bbl at mapselect
+ \let\bbl at mapselect\relax
\bbl at exp{\<fontspec_set_family:Nnn>\\#1%
{\bbl at cs{lsys@\languagename},#2}}{#3}%
+ \let\bbl at mapselect\bbl at tempe
\bbl at toglobal#1}%
% \end{macrocode}
%
@@ -11201,7 +11301,7 @@ help from Bernd Raichle, for which I am grateful.
\fi
\IfBabelLayout{tabular}
{\def\@tabular{%
- \leavevmode\hbox\bgroup\bbl at nextfake$%
+ \leavevmode\hbox\bgroup\bbl at nextfake$% %$
\let\@acol\@tabacol \let\@classz\@tabclassz
\let\@classiv\@tabclassiv \let\\\@tabularcr\@tabarray}}
{}
@@ -17421,7 +17521,19 @@ local function dir_mark(head, from, to, outer)
node.insert_after(head, to, d)
end
-function Babel.pre_otfload(head)
+function Babel.pre_otfload_v(head)
+ -- head = Babel.numbers(head)
+ head = Babel.bidi(head, true)
+ return head
+end
+
+function Babel.pre_otfload_h(head)
+ -- head = Babel.numbers(head)
+ head = Babel.bidi(head, false)
+ return head
+end
+
+function Babel.bidi(head, ispar)
local first_n, last_n -- first and last char with nums
local last_es -- an auxiliary 'last' used with nums
local first_d, last_d -- first and last char in L/R block
@@ -17649,6 +17761,347 @@ end
%</basic-r>
% \end{macrocode}
%
+% And here the Lua code for |bidi=basic|:
+%
+% \begin{macrocode}
+%<*basic>
+Babel = Babel or {}
+
+Babel.fontmap = Babel.fontmap or {}
+Babel.fontmap[0] = {} -- l
+Babel.fontmap[1] = {} -- r
+Babel.fontmap[2] = {} -- al/an
+
+function Babel.pre_otfload_v(head)
+ -- head = Babel.numbers(head)
+ head = Babel.bidi(head, true)
+ return head
+end
+
+function Babel.pre_otfload_h(head)
+ -- head = Babel.numbers(head)
+ head = Babel.bidi(head, false)
+ return head
+end
+
+require('babel-bidi.lua')
+
+local characters = Babel.characters
+local ranges = Babel.ranges
+
+local DIR = node.id('dir')
+local GLYPH = node.id('glyph')
+
+local function insert_implicit(head, state, outer)
+ local new_state = state
+ if state.sim and state.eim and state.sim ~= state.eim then
+ dir = ((outer == 'r') and 'TLT' or 'TRT') -- ie, reverse
+ local d = node.new(DIR)
+ d.dir = '+' .. dir
+ node.insert_before(head, state.sim, d)
+ local d = node.new(DIR)
+ d.dir = '-' .. dir
+ node.insert_after(head, state.eim, d)
+ end
+ new_state.sim, new_state.eim = nil, nil
+ return head, new_state
+end
+
+local function insert_numeric(head, state)
+ local new
+ local new_state = state
+ if state.san and state.ean and state.san ~= state.ean then
+ local d = node.new(DIR)
+ d.dir = '+TLT'
+ _, new = node.insert_before(head, state.san, d)
+ if state.san == state.sim then state.sim = new end
+ local d = node.new(DIR)
+ d.dir = '-TLT'
+ _, new = node.insert_after(head, state.ean, d)
+ if state.ean == state.eim then state.eim = new end
+ end
+ new_state.san, new_state.ean = nil, nil
+ return head, new_state
+end
+
+function Babel.bidi(head, ispar)
+ local d -- d is used mainly for computations in a loop
+ local prev_d = ''
+ local new_d = false
+
+ local nodes = {}
+ local outer_first = nil
+
+ local has_en = false
+ local first_et = nil
+
+ local save_outer
+ local temp = node.get_attribute(head, luatexbase.registernumber'bbl at attr@dir')
+ if temp then
+ temp = temp % 3
+ save_outer = (temp == 0 and 'l') or
+ (temp == 1 and 'r') or
+ (temp == 2 and 'al')
+ elseif ispar then
+ save_outer = ('TRT' == tex.pardir) and 'r' or 'l'
+ else
+ save_outer = ('TRT' == tex.textdir) and 'r' or 'l'
+ end
+ local outer = save_outer
+ local last = outer
+ -- 'al' is only taken into account in the first, current loop
+ if save_outer == 'al' then save_outer = 'r' end
+
+ local fontmap = Babel.fontmap
+
+ for item in node.traverse(head) do
+
+ -- three cases: glyph, dir, otherwise
+ if item.id == GLYPH then
+
+ local chardata = characters[item.char]
+ d = chardata and chardata.d or nil
+ if not d then
+ for nn, et in ipairs(ranges) do
+ if item.char < et[1] then
+ break
+ elseif item.char <= et[2] then
+ d = et[3]
+ break
+ end
+ end
+ end
+ d = d or 'l'
+
+ local temp = (d == 'l' and 0) or
+ (d == 'r' and 1) or
+ (d == 'al' and 2) or
+ (d == 'an' and 2) or nil
+ if temp and fontmap and fontmap[temp][item.font] then
+ item.font = fontmap[temp][item.font]
+ end
+
+ if new_d then
+ table.insert(nodes, {nil, (outer == 'l') and 'l' or 'r', nil})
+ attr_d = node.get_attribute(item,
+ luatexbase.registernumber'bbl at attr@dir')
+ attr_d = attr_d % 3
+ if attr_d == 1 then
+ outer_first = 'r'
+ last = 'r'
+ elseif attr_d == 2 then
+ outer_first = 'r'
+ last = 'al'
+ else
+ outer_first = 'l'
+ last = 'l'
+ end
+ outer = last
+ has_en = false
+ first_et = nil
+ new_d = false
+ end
+
+ -- Optimizar el caso en que el dir actual == anterior
+ -- eg, if prev_dir == d goto next_node end
+
+ elseif item.id == DIR then
+ d = nil
+ new_d = true
+
+ else
+ d = nil
+ end
+
+ -- AL << EN/ET/ES -- W2 + W3 + W6
+ if last == 'al' and d == 'en' then
+ d = 'an' -- W3
+ elseif last == 'al' and (d == 'et' or d == 'es') then
+ d = 'on' -- W6
+ end
+
+ -- EN + CS/ES + EN -- W4
+ if d == 'en' and #nodes > 2 then
+ if (nodes[#nodes -1][2] == 'es' or nodes[#nodes -1][2] == 'cs')
+ and nodes[#nodes -2][2] == 'en' then
+ nodes[#nodes -1][2] = 'en'
+ end
+ end
+
+ -- AN + CS + AN -- W4 too, because uax9 mixes both cases
+ if d == 'an' and #nodes > 2 then
+ if (nodes[#nodes -1][2] == 'cs')
+ and nodes[#nodes -2][2] == 'an' then
+ nodes[#nodes -1][2] = 'an'
+ end
+ end
+
+ -- ET/EN -- W5 + W7->l / W6->on
+ if d == 'et' then
+ first_et = first_et or (#nodes + 1) -- not yet added to nodes
+ elseif d == 'en' then
+ has_en = true
+ first_et = first_et or (#nodes + 1) -- not yet added to nodes
+ elseif first_et then -- d may be nil here !
+ if has_en then
+ if last == 'l' then
+ temp = 'l' -- W7
+ else
+ temp = 'en' -- W5
+ end
+ else
+ temp = 'on' -- W6
+ end
+ for e = first_et, #nodes do
+ if nodes[e][1].id == GLYPH then nodes[e][2] = temp end
+ end
+ first_et = nil
+ has_en = false
+ end
+
+ if d then
+ if d == 'al' then
+ d = 'r'
+ last = 'al'
+ elseif d == 'l' or d == 'r' then
+ last = d
+ end
+ prev_d = d
+ table.insert(nodes, {item, d, outer_first})
+ else
+ -- Not sure about the following. Looks too 'ad hoc', but it's
+ -- required for numbers, so that 89 19 becomes 19 89. It also
+ -- affects n+cs/es+n.
+ if prev_d == 'an' or prev_d == 'en' then
+ table.insert(nodes, {item, 'on', nil})
+ end
+ end
+
+ outer_first = nil
+
+ end
+
+ -- TODO -- repeated here in case EN/ET is the last node. Find a
+ -- better way of doing things:
+ if first_et then -- dir may be nil here !
+ if has_en then
+ if last == 'l' then
+ temp = 'l' -- W7
+ else
+ temp = 'en' -- W5
+ end
+ else
+ temp = 'on' -- W6
+ end
+ for e = first_et, #nodes do
+ if nodes[e][1].id == GLYPH then nodes[e][2] = temp end
+ end
+ end
+
+ -- dummy node, to close things
+ table.insert(nodes, {nil, (outer == 'l') and 'l' or 'r', nil})
+
+ --------------- NEUTRAL -----------------
+
+ outer = save_outer
+ last = outer
+
+ local first_on = nil
+
+ for q = 1, #nodes do
+ local item
+
+ local outer_first = nodes[q][3]
+ outer = outer_first or outer
+ last = outer_first or last
+
+ local d = nodes[q][2]
+ if d == 'an' or d == 'en' then d = 'r' end
+ if d == 'cs' or d == 'et' or d == 'es' then d = 'on' end --- W6
+
+ if d == 'on' then
+ first_on = first_on or q
+ elseif first_on then
+ if last == d then
+ temp = d
+ else
+ temp = outer
+ end
+ for r = first_on, q - 1 do
+ nodes[r][2] = temp
+ item = nodes[r][1] -- MIRRORING
+ if item.id == GLYPH and temp == 'r' then
+ item.char = characters[item.char].m or item.char
+ end
+ end
+ first_on = nil
+ end
+
+ if d == 'r' or d == 'l' then last = d end
+ end
+
+ -------------- IMPLICIT, REORDER ----------------
+
+ outer = save_outer
+ last = outer
+
+ local state = {}
+ state.has_r = false
+
+ for q = 1, #nodes do
+
+ local item = nodes[q][1]
+
+ outer = nodes[q][3] or outer
+
+ local d = nodes[q][2]
+
+ if d == 'nsm' then d = last end -- W1
+ if d == 'en' then d = 'an' end
+ local isdir = (d == 'r' or d == 'l')
+
+ if outer == 'l' and d == 'an' then
+ state.san = state.san or item
+ state.ean = item
+ elseif state.san then
+ head, state = insert_numeric(head, state)
+ end
+
+ if outer == 'l' then
+ if d == 'an' or d == 'r' then -- im -> implicit
+ if d == 'r' then state.has_r = true end
+ state.sim = state.sim or item
+ state.eim = item
+ elseif d == 'l' and state.sim and state.has_r then
+ head, state = insert_implicit(head, state, outer)
+ elseif d == 'l' then
+ state.sim, state.eim, state.has_r = nil, nil, false
+ end
+ else
+ if d == 'an' or d == 'l' then
+ state.sim = state.sim or item
+ state.eim = item
+ elseif d == 'r' and state.sim then
+ head, state = insert_implicit(head, state, outer)
+ elseif d == 'r' then
+ state.sim, state.eim = nil, nil
+ end
+ end
+
+ if isdir then
+ last = d -- Don't search back - best save now
+ elseif d == 'on' and state.san then
+ state.san = state.san or item
+ state.ean = item
+ end
+
+ end
+
+ return node.prev(head) or head
+end
+%</basic>
+% \end{macrocode}
+%
% \section{The `nil' language}
%
% This `language' does nothing, except setting the hyphenation patterns to
diff --git a/required/babel/babel.ins b/required/babel/babel.ins
index f33ccf2..682ec7d 100644
--- a/required/babel/babel.ins
+++ b/required/babel/babel.ins
@@ -26,7 +26,7 @@
%% and covered by LPPL is defined by the unpacking scripts (with
%% extension .ins) which are part of the distribution.
%%
-\def\filedate{2018/02/14}
+\def\filedate{2018/04/04}
\def\batchfile{babel.ins}
\input docstrip.tex
@@ -224,6 +224,7 @@
\nopostamble
\generate{\file{babel-bidi.lua}{\from{babel.dtx}{bidi}}}
\generate{\file{babel-bidi-basic-r.lua}{\from{babel.dtx}{basic-r}}}
+\generate{\file{babel-bidi-basic.lua}{\from{babel.dtx}{basic}}}
\Msg{***********************************************************}
\Msg{*}
diff --git a/required/babel/babel.pdf b/required/babel/babel.pdf
index d72692a..53f549d 100644
Binary files a/required/babel/babel.pdf and b/required/babel/babel.pdf differ
diff --git a/required/babel/bbcompat.dtx b/required/babel/bbcompat.dtx
index d99a4de..72bddd2 100644
--- a/required/babel/bbcompat.dtx
+++ b/required/babel/bbcompat.dtx
@@ -30,7 +30,7 @@
%
% \iffalse
%<*dtx>
-\ProvidesFile{bbcompat.dtx}[2018/02/14 v3.18]
+\ProvidesFile{bbcompat.dtx}[2018/04/04 v3.18.1236]
%</dtx>
%
%% File 'bbcompat.dtx'
diff --git a/required/babel/samples/lua-bidibasic.pdf b/required/babel/samples/lua-bidibasic.pdf
new file mode 100644
index 0000000..33c3354
Binary files /dev/null and b/required/babel/samples/lua-bidibasic.pdf differ
diff --git a/required/babel/samples/lua-bidibasic.tex b/required/babel/samples/lua-bidibasic.tex
new file mode 100644
index 0000000..291d316
--- /dev/null
+++ b/required/babel/samples/lua-bidibasic.tex
@@ -0,0 +1,151 @@
+% lualatex
+
+\documentclass{book}
+
+\setlength{\parindent}{0pt}
+
+\usepackage[english, bidi=basic]{babel}
+
+% Define two new languages:
+
+% \babelprovide[mapfont=direction]{english} %
+\babelprovide[mapfont=direction]{arabic}
+\babelprovide[mapfont=direction]{hebrew}
+
+\babelfont{rm}{Crimson}
+\babelfont[*arabic]{rm}
+ [Color=118811, ItalicFont=FreeSerif]
+ {FreeSerif} % also try Amiri
+\babelfont[*hebrew]{rm}[Color=774422]{FreeSerif}
+
+\babeltags{en = english, ar = arabic, he = hebrew}
+
+% ====================
+%
+% You may want to see what's going on. Use the following if you want.
+%
+% \directlua{
+%
+% function test (head)
+% texio.write_nl('====')
+% texio.write_nl('')
+% for item in node.traverse(head) do
+% if item.id == node.id'glyph' then
+% texio.write(' ' .. unicode.utf8.char(item.char)
+% .. '(' .. item.font
+% % .. font.getfont(item.font).name
+% % .. font.getfont(item.font).size
+% .. ')')
+% elseif item.id == node.id'dir' then
+% texio.write(' <' .. item.dir .. '>')
+% elseif item.id == node.id'glue' then
+% texio.write(' _ ')
+% end
+% end
+% return head
+% end
+%
+% luatexbase.add_to_callback("pre_linebreak_filter", test, "test")
+% luatexbase.add_to_callback("hpack_filter", test, "test")
+% }
+
+\begin{document}
+
+\tableofcontents
+
+\chapter{واحد two ثلاثة four خمسة}
+
+\section{واحد two ثلاثة four خمسة}
+
+efg \textar{شديد 123\% الإقبال abc على}
+\texthe{במשחק hij כדורגל 123\% בו} xyz
+
+Why just setting \verb|\textdir| is not enough — Arabic: \textar{123\%};
+Hebrew: \texthe{123\%}.
+
+Some Wikipedia articles are: \textar{الاتحاد الأوروبي},
+\textar{إسبانيا}, and \textar{لغة عربية}. Here the Unicode algorithm
+fails, so the Arabic texts are explicitly marked up. Another text from
+Wikipedia:
+\begin{quotation}\small
+ Most Arabic speakers consider the two varieties to be two registers
+ of one language, although the two registers can be referred to in
+ Arabic as فصحى العصر \textit{fuṣḥā l-ʻaṣr} (MSA) and
+ فصحى التراث \textit{fuṣḥā t-turāth} (CA).
+\end{quotation}
+
+\textar{(لغة عربية)}.
+
+\textar{(لغة عربية).}
+
+(\textar{لغة عربية.})
+
+\selectlanguage{arabic}
+
+\begin{quotation}\small
+وقد عرفت شبه جزيرة العرب طيلة العصر الهيليني
+(الاغريقي) بـ Arabia أو Aravia (بالاغريقية Αραβία)،
+استخدم الرومان ثلاث بادئات بـ”Arabia“ على ثلاث
+مناطق من شبه الجزيرة العربية، إلا أنها حقيقةً
+كانت أكبر مما تعرف عليه اليوم.
+\end{quotation}
+
+في 1085 تحولاً كبيراً في ميزان القوى لصالح الممالك المسيحية في
+أيبيريا. بعد العودة القوية للحكم الإسلامي في القرن الثاني عشر، سقطت
+المعاقل الإسلامية الرئيسية في الجنوب بيد الممالك المسيحية في القرن
+الثالث عشر - قرطبة 1236 وإشبيلية 1248 - ولم يتبقى سوى إمارة مسلمة في
+غرناطة في الجنوب. شهد القرن الثالث عشر أيضاً توسع تاج أراغون المتمركز
+في شمال شرق إسبانيا إلى جزر البحر الأبيض المتوسط إلى صقلية وصولاً إلى
+أثينا.\footnote{Payne، Stanley G. (1973). «A History of Spain and
+Portugal; Ch. 5 The Rise of Aragón-Catalonia». The Library of Iberian
+Resources Online. اطلع عليه بتاريخ 2008-08-09.} في هذه الفترة تقريباً
+تأسست جامعات بلنسية (1212/1263) وسالامانكا (1218/1254). دمر الموت
+الأسود البلاد بين عامي 1348 و1349 [30].
+
+\selectlanguage{english}
+
+\newpage
+
+The following lines only makes sense if you read the source code (and
+know the Unicode algorithm subtleties). Suggestion: copypaste it into
+another program and compare.
+
+\medskip
+
+את az בת bz גת 19 דת (29) הת
+
+את az 19 בת 29 bz גת cz 19 dz דת (ez) הת
+
+את (az בת) bz גת cz (דת dz) הת
+
+את £ בת 19£ גת (az bz) דת (29£) הת
+
+את £ az 19£ גת bz (cz) דת (dz) ez הת
+
+את 19-29 בת 39/49 גת 59£ az דת 69.79 הת
+
+\medskip
+
+١٩ ٢٩ ٣٩ ٤٩
+
+بى تى ثى سى شى صى ضى طى ظى
+
+بى az تى bz ثى 19 سى 29 cz شى dz 39 صى ez (ضى) fz (طى) ظى
+
+بى az 19 bz تى (cz) ثى dz (سى ez) شى (صى fz) ضى gz (hz) طى (iz jz) ظى
+
+بى 19 az 29 تى 19 29 39 ثى 49-59 سى 69/79 شى 19.29 صى 39,49 ضى 49.59-69/79 سى
+
+بى £ تى az £ ثى bz £ cz سى £ dz شى 19£ صى \$29 ضى ez 39£ طى 49£ fz ظى
+
+بى ١٩ تى az ٢٩ ثى ٣٩ bz سى cz ٤٩ dz شى ١٩-٢٩ صى ١٩/٢٩ ضى 49£-59£ طى ظى
+
+بى az (19) bz تى (29) ثى 39 (سى 49) شى (صى 59) ضى 69 (79) طى (89 19)
+ظى
+
+19 az 29 bz 19 (29) 39 cz 49-59 dz 69/79 ez 19.29 fz 39,49 gz
+
+ai 29 bj شى 19 (29) 39 ck 49-59 dl 69/79 em 19.29 fn 39,49 go
+49.59-69/79 hp
+
+\end{document}
\ No newline at end of file
More information about the latex3-commits
mailing list