texlive[59850] Master: inputnormalization (5jul21)
commits+karl at tug.org
commits+karl at tug.org
Mon Jul 5 22:58:36 CEST 2021
Revision: 59850
http://tug.org/svn/texlive?view=revision&revision=59850
Author: karl
Date: 2021-07-05 22:58:35 +0200 (Mon, 05 Jul 2021)
Log Message:
-----------
inputnormalization (5jul21)
Modified Paths:
--------------
trunk/Master/tlpkg/bin/tlpkg-ctan-check
trunk/Master/tlpkg/libexec/ctan2tds
trunk/Master/tlpkg/tlpsrc/collection-plaingeneric.tlpsrc
Added Paths:
-----------
trunk/Master/texmf-dist/doc/latex/inputnormalization/
trunk/Master/texmf-dist/doc/latex/inputnormalization/README.md
trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf
trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.tex
trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf
trunk/Master/texmf-dist/source/latex/inputnormalization/
trunk/Master/texmf-dist/source/latex/inputnormalization/inputnormalization.dtx
trunk/Master/texmf-dist/tex/latex/inputnormalization/
trunk/Master/texmf-dist/tex/latex/inputnormalization/inputnormalization.sty
trunk/Master/tlpkg/tlpsrc/inputnormalization.tlpsrc
Added: trunk/Master/texmf-dist/doc/latex/inputnormalization/README.md
===================================================================
--- trunk/Master/texmf-dist/doc/latex/inputnormalization/README.md (rev 0)
+++ trunk/Master/texmf-dist/doc/latex/inputnormalization/README.md 2021-07-05 20:58:35 UTC (rev 59850)
@@ -0,0 +1,24 @@
+# The `inputnormalization` package
+Unicode normlization for LuaTeX and XeTeX.
+
+Author: Marcel Krüger
+
+License: LPPL v1.3
+
+Provides a cross engine interface to normalizing input before it's read by TeX. Based on XeTeX's `\XeTeXinputnormalization` primitive and `lua-uni-algos` for LuaTeX.
+
+# Usage
+For LaTeX, just load the `inputnormalization` package with
+
+ \usepackage{inputnormalization}
+
+For plain TeX, use `\input` instead:
+
+ \input inputnormalization
+
+See the manual for more details.
+
+## Support
+For bug reports, please open an [issue on GitHub](https://github.com/zauguin/lua-uni-algos/issues).
+
+
Property changes on: trunk/Master/texmf-dist/doc/latex/inputnormalization/README.md
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf
===================================================================
(Binary files differ)
Index: trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf
===================================================================
--- trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf 2021-07-05 20:55:54 UTC (rev 59849)
+++ trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf 2021-07-05 20:58:35 UTC (rev 59850)
Property changes on: trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.pdf
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/pdf
\ No newline at end of property
Added: trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.tex
===================================================================
--- trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.tex (rev 0)
+++ trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.tex 2021-07-05 20:58:35 UTC (rev 59850)
@@ -0,0 +1,65 @@
+%%
+%% This is file `inputnormalization.tex',
+%% generated with the docstrip utility.
+%%
+%% The original source files were:
+%%
+%% inputnormalization.dtx (with options: `tex-package')
+%%
+%% IMPORTANT NOTICE:
+%%
+%% For the copyright see the source file.
+%%
+%% Any modified versions of this file must be renamed
+%% with new filenames distinct from inputnormalization.tex.
+%%
+%% For distribution of the original source see the terms
+%% for copying and modification in the file inputnormalization.dtx.
+%%
+%% This generated file may be distributed as long as the
+%% original source files, as listed above, are part of the
+%% same distribution. (The sources need not necessarily be
+%% in the same archive or directory.)
+%% Copyright (C) 2020--2021 by Marcel Krueger
+%%
+%% This file may be distributed and/or modified under the
+%% conditions of the LaTeX Project Public License, either
+%% version 1.3 of this license or (at your option) any later
+%% version. The latest version of this license is in:
+%%
+%% http://www.latex-project.org/lppl.txt
+%%
+%% and version 1.3 or later is part of all distributions of
+%% LaTeX version 2005/12/01 or later.
+\ifx\directlua\undefined
+\ifx\XeTeXinputnormalization\undefined
+ \begingroup
+ \ifx\PackageError\undefined
+ \def\PackageError#1#2#3{\errhelp{#3}\errmessage{#1: #2}}
+ \fi
+\PackageError{inputnormalization}{LuaTeX or XeTeX required}%
+ {inputnormalization requires LuaTeX or XeTeX.
+ Maybe you forgot to switch the engine in your editor?}
+ \endgroup
+\else
+ \let\Uinputnormalization\XeTeXinputnormalization
+\fi
+\else
+ \ifx\newluafunction\@undefined
+ \input ltluatex
+ \fi
+ \newcount\Uinputnormalization
+ \directlua{
+ local getcount = tex.getcount
+ local function ident(buf) return buf end
+ local uni_normalize = require'lua-uni-normalize'
+ local normalize = {[0] = ident, uni_normalize.NFC, uni_normalize.NFD}
+ luatexbase.add_to_callback('process_input_buffer', function(buf)
+ return normalize[getcount(\the\allocationnumber)](buf)
+ end, 'inputnormalization')
+ }
+\fi
+\Uinputnormalization=1
+\endinput
+%%
+%% End of file `inputnormalization.tex'.
Property changes on: trunk/Master/texmf-dist/doc/latex/inputnormalization/inputnormalization.tex
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf
===================================================================
(Binary files differ)
Index: trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf
===================================================================
--- trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf 2021-07-05 20:55:54 UTC (rev 59849)
+++ trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf 2021-07-05 20:58:35 UTC (rev 59850)
Property changes on: trunk/Master/texmf-dist/doc/latex/inputnormalization/plain.pdf
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/pdf
\ No newline at end of property
Added: trunk/Master/texmf-dist/source/latex/inputnormalization/inputnormalization.dtx
===================================================================
--- trunk/Master/texmf-dist/source/latex/inputnormalization/inputnormalization.dtx (rev 0)
+++ trunk/Master/texmf-dist/source/latex/inputnormalization/inputnormalization.dtx 2021-07-05 20:58:35 UTC (rev 59850)
@@ -0,0 +1,225 @@
+% \iffalse meta-comment
+%
+%% Copyright (C) 2020--2021 by Marcel Krueger
+%%
+%% This file may be distributed and/or modified under the
+%% conditions of the LaTeX Project Public License, either
+%% version 1.3 of this license or (at your option) any later
+%% version. The latest version of this license is in:
+%%
+%% http://www.latex-project.org/lppl.txt
+%%
+%% and version 1.3 or later is part of all distributions of
+%% LaTeX version 2005/12/01 or later.
+%
+%<*batch>
+%<*gobble>
+\ifx\jobname\relax\let\documentclass\undefined\fi
+\ifx\documentclass\undefined
+\csname fi\endcsname
+%</gobble>
+\input docstrip.tex
+\keepsilent
+\generate{%
+ \file{inputnormalization.sty}{\from{inputnormalization.dtx}{package}}
+ \file{inputnormalization.tex}{\from{inputnormalization.dtx}{tex-package}}}
+\endbatchfile
+%</batch>
+%<*gobble>
+\fi
+\expandafter\ifx\csname @currname\endcsname\empty
+\csname fi\endcsname
+%</gobble>
+%<*driver>
+\RequirePackage{inputnormalization}
+\documentclass{article}
+\usepackage{csquotes,doc,metalogo,hyperref,luacolor,tikzducks,pict2e}
+\RecordChanges
+\MakeShortVerb\|
+\begin{document}
+\DocInput{inputnormalization.dtx}
+\PrintIndex
+\PrintChanges
+\end{document}
+%</driver>
+%<*gobble>
+\fi
+%</gobble>
+% \fi
+%
+% \GetFileInfo{inputnormalization.sty}
+% \title{The inputnormalization package\thanks{This document
+% corresponds to inputnormalization~\fileversion, dated~\filedate.}}
+% \author{Marcel Kr\"uger \\
+% \href{mailto:tex at 2krueger.de}{tex at 2krueger.de}}
+%
+% \maketitle
+% Add support for normalising input files for LuaTeX and provide a common interface for \LuaTeX\ and \XeTeX.
+%
+% \section{Motivation}
+% Modern \TeX\ engines like \XeTeX\ or \LuaTeX\ natively accept Unicode input.
+% Unicode is a rather special encoding since many characters can be encoded in
+% different ways which are officially considered equivalent. This can sometimes
+% lead to surprising behavior since many parts of \TeX\ are not aware of this
+% equivalences and therefore treats different encodings as different strings.
+% This can show itself during rendering when some the same text might appear
+% in different ways depending on the input, but it might also show itself in
+% macro or option names: When e.g.\ non-English macro names are in use, the
+% different encodings of the same name can name different macros, leading to
+% hard to understand and solve errors.
+%
+% Unicode defines a mechanism to solve such issues: It defines the normalization
+% forms NFC and NFD. When text is normalized to one of these forms, then two
+% equivalent strings are always encoded in the same way, leading to unique names
+% and consistent rendering.
+%
+% This package provides a uniform way to enable input normalization to either
+% of these forms in both \XeTeX\ and \LuaTeX.
+%
+% \section{Usage}
+% Just loading the package is enough to enable NFC normalization. This is the
+% right option for almost all users:
+%
+% \begin{verbatim}
+% \documentclass{article}
+% \usepackage{inputnormalization}
+% \begin{document}
+% Everything here gets normalized before it's processed by \TeX.
+% \end{document}
+% \end{verbatim}
+%
+% If you are a plain \LuaTeX/\XeTeX\ user, you can use
+%
+% \begin{verbatim}
+% \input inputnormalization
+% Everything here gets normalized before it's processed by \TeX.
+% \bye
+% \end{verbatim}
+% instead.
+%
+% \section{Advanced usage}
+% In addition to enabling NFC normalization by default, the package makes
+% \verb|\Uinputnormalization| available as a cross engine version of
+% \verb|\XeTeXinputnormalization| to make the normalization controllable.
+% See the \XeTeX\ documentation for detailed usage. E.g.\ you could write
+% \begin{verbatim}
+% \documentclass{article}
+% \usepackage{inputnormalization}
+% \begin{document}
+% Everything here gets normalized to NFC before it's processed by \TeX.
+%
+% \Uinputnormalization=0
+% Now normalization is disabled.
+%
+% \Uinputnormalization=2
+% Here we normalize to NFD instead.
+% \end{document}
+% \end{verbatim}
+%
+% \paragraph{Warning:} It is almost never a good idea to use different kinds
+% of normalization in the same document, therefore you should set one kind of
+% normalization directly after loading the package and not modify it afterwards.
+%
+% Additionally NFC works much better in a \TeX\ context than NFD, so you should
+% not set this at all unless you know exactly what you are doing.
+%
+% \StopEventually{}
+% \section{The implementation}
+% \changes{0.0.1}{2020-05-17}{Initial release}
+% \iffalse
+%<*package|tex-package>
+% \fi
+% \begin{macrocode}
+%<*package>
+\NeedsTeXFormat{LaTeX2e}
+\ProvidesPackage
+ {inputnormalization}
+ [2021/07/05 v0.2 Unicode input normalization]
+%</package>
+% \end{macrocode}
+% Only \LuaTeX\ and \XeTeX\ are supported.
+% For other engines we show an error.
+% \begin{macrocode}
+\ifx\directlua\undefined
+\ifx\XeTeXinputnormalization\undefined
+% \end{macrocode}
+% \iffalse
+%<*gobble>
+\iffalse
+%</gobble>
+% \fi \fi
+% \begin{macrocode}
+%<*tex-package>
+ \begingroup
+ \ifx\PackageError\undefined
+ \def\PackageError#1#2#3{\errhelp{#3}\errmessage{#1: #2}}
+ \fi
+%</tex-package>
+% \end{macrocode}
+% \iffalse \iffalse
+%<*gobble>
+\fi
+%</gobble>
+% \fi
+% \begin{macrocode}
+\PackageError{inputnormalization}{LuaTeX or XeTeX required}%
+ {inputnormalization requires LuaTeX or XeTeX.
+ Maybe you forgot to switch the engine in your editor?}
+% \end{macrocode}
+% \iffalse
+%<*gobble>
+\iffalse
+%</gobble>
+% \fi \fi
+% \begin{macrocode}
+%<*tex-package>
+ \endgroup
+%</tex-package>
+% \end{macrocode}
+% \iffalse \iffalse
+%<*gobble>
+\fi
+%</gobble>
+% \fi
+% \begin{macrocode}
+\else
+% \end{macrocode}
+% First deal with \XeTeX: Define |\Uinputnormalization| as an alias for |\XeTeXinputnormalization|.
+% Make sure that \texttt{ltluatex} is loaded.
+% \begin{macrocode}
+ \let\Uinputnormalization\XeTeXinputnormalization
+\fi
+\else
+% \end{macrocode}
+% In \LuaTeX\ we emulate |\Uinputnormalization| using a |process_input_buffer| callback.
+% First ensure that |ltluatex| is loaded to have proper callback handling:
+% \begin{macrocode}
+ \ifx\newluafunction\@undefined
+ \input ltluatex
+ \fi
+% \end{macrocode}
+% We need a integer register to control the normalization and then the actual implementation of the callback.
+% Nothing particularly interesting is happening here, the actual normalization is handled by \texttt{lua-uni-algos}.
+% \begin{macrocode}
+ \newcount\Uinputnormalization
+ \directlua{
+ local getcount = tex.getcount
+ local function ident(buf) return buf end
+ local uni_normalize = require'lua-uni-normalize'
+ local normalize = {[0] = ident, uni_normalize.NFC, uni_normalize.NFD}
+ luatexbase.add_to_callback('process_input_buffer', function(buf)
+ return normalize[getcount(\the\allocationnumber)](buf)
+ end, 'inputnormalization')
+ }
+\fi
+% \end{macrocode}
+% \changes{0.2}{2021-07-05}{Enable NFC by default}
+% Finally we enable NFC normalization as a reasonable default:
+% \begin{macrocode}
+\Uinputnormalization=1
+\endinput
+% \end{macrocode}
+% \iffalse
+%</package|tex-package>
+% \fi
+% \Finale
Property changes on: trunk/Master/texmf-dist/source/latex/inputnormalization/inputnormalization.dtx
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: trunk/Master/texmf-dist/tex/latex/inputnormalization/inputnormalization.sty
===================================================================
--- trunk/Master/texmf-dist/tex/latex/inputnormalization/inputnormalization.sty (rev 0)
+++ trunk/Master/texmf-dist/tex/latex/inputnormalization/inputnormalization.sty 2021-07-05 20:58:35 UTC (rev 59850)
@@ -0,0 +1,64 @@
+%%
+%% This is file `inputnormalization.sty',
+%% generated with the docstrip utility.
+%%
+%% The original source files were:
+%%
+%% inputnormalization.dtx (with options: `package')
+%%
+%% IMPORTANT NOTICE:
+%%
+%% For the copyright see the source file.
+%%
+%% Any modified versions of this file must be renamed
+%% with new filenames distinct from inputnormalization.sty.
+%%
+%% For distribution of the original source see the terms
+%% for copying and modification in the file inputnormalization.dtx.
+%%
+%% This generated file may be distributed as long as the
+%% original source files, as listed above, are part of the
+%% same distribution. (The sources need not necessarily be
+%% in the same archive or directory.)
+%% Copyright (C) 2020--2021 by Marcel Krueger
+%%
+%% This file may be distributed and/or modified under the
+%% conditions of the LaTeX Project Public License, either
+%% version 1.3 of this license or (at your option) any later
+%% version. The latest version of this license is in:
+%%
+%% http://www.latex-project.org/lppl.txt
+%%
+%% and version 1.3 or later is part of all distributions of
+%% LaTeX version 2005/12/01 or later.
+\NeedsTeXFormat{LaTeX2e}
+\ProvidesPackage
+ {inputnormalization}
+ [2021/07/05 v0.2 Unicode input normalization]
+\ifx\directlua\undefined
+\ifx\XeTeXinputnormalization\undefined
+\PackageError{inputnormalization}{LuaTeX or XeTeX required}%
+ {inputnormalization requires LuaTeX or XeTeX.
+ Maybe you forgot to switch the engine in your editor?}
+\else
+ \let\Uinputnormalization\XeTeXinputnormalization
+\fi
+\else
+ \ifx\newluafunction\@undefined
+ \input ltluatex
+ \fi
+ \newcount\Uinputnormalization
+ \directlua{
+ local getcount = tex.getcount
+ local function ident(buf) return buf end
+ local uni_normalize = require'lua-uni-normalize'
+ local normalize = {[0] = ident, uni_normalize.NFC, uni_normalize.NFD}
+ luatexbase.add_to_callback('process_input_buffer', function(buf)
+ return normalize[getcount(\the\allocationnumber)](buf)
+ end, 'inputnormalization')
+ }
+\fi
+\Uinputnormalization=1
+\endinput
+%%
+%% End of file `inputnormalization.sty'.
Property changes on: trunk/Master/texmf-dist/tex/latex/inputnormalization/inputnormalization.sty
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Modified: trunk/Master/tlpkg/bin/tlpkg-ctan-check
===================================================================
--- trunk/Master/tlpkg/bin/tlpkg-ctan-check 2021-07-05 20:55:54 UTC (rev 59849)
+++ trunk/Master/tlpkg/bin/tlpkg-ctan-check 2021-07-05 20:58:35 UTC (rev 59850)
@@ -400,7 +400,8 @@
imfellenglish impnattypo import imsproc imtekda
incgraph includernw inconsolata index indextools infwarerr
initials inkpaper inline-images inlinebib inlinedef innerscript
- inputenx inputtrc inriafonts insbox install-latex-guide-zh-cn installfont
+ inputenx inputnormalization inputtrc
+ inriafonts insbox install-latex-guide-zh-cn installfont
intcalc inter interactiveworkbook
interchar interfaces interpreter interval intopdf
intro-scientific
Modified: trunk/Master/tlpkg/libexec/ctan2tds
===================================================================
--- trunk/Master/tlpkg/libexec/ctan2tds 2021-07-05 20:55:54 UTC (rev 59849)
+++ trunk/Master/tlpkg/libexec/ctan2tds 2021-07-05 20:58:35 UTC (rev 59850)
@@ -2966,6 +2966,7 @@
'hycolor' => 'tex',
'hypdestopt' => 'etex',
'hyperbar' => 'tex',
+ 'inputnormalization' => 'etex -8bit',
'intopdf', => 'tex',
'icite', => 'lualatex',
'isodoc' => 'luatex',
Modified: trunk/Master/tlpkg/tlpsrc/collection-plaingeneric.tlpsrc
===================================================================
--- trunk/Master/tlpkg/tlpsrc/collection-plaingeneric.tlpsrc 2021-07-05 20:55:54 UTC (rev 59849)
+++ trunk/Master/tlpkg/tlpsrc/collection-plaingeneric.tlpsrc 2021-07-05 20:58:35 UTC (rev 59850)
@@ -47,6 +47,7 @@
depend gtl
depend hlist
depend hyplain
+depend inputnormalization
depend insbox
depend js-misc
depend kastrup
Added: trunk/Master/tlpkg/tlpsrc/inputnormalization.tlpsrc
===================================================================
More information about the tex-live-commits
mailing list.