texlive[74841] Master/texmf-dist: tokcycle (3apr25)
commits+karl at tug.org
commits+karl at tug.org
Thu Apr 3 21:52:07 CEST 2025
Revision: 74841
https://tug.org/svn/texlive?view=revision&revision=74841
Author: karl
Date: 2025-04-03 21:52:07 +0200 (Thu, 03 Apr 2025)
Log Message:
-----------
tokcycle (3apr25)
Modified Paths:
--------------
trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.pdf
trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.tex
trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.pdf
trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.tex
trunk/Master/texmf-dist/tex/generic/tokcycle/tokcycle.tex
Modified: trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.pdf
===================================================================
(Binary files differ)
Modified: trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.tex
===================================================================
--- trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.tex 2025-04-03 19:51:18 UTC (rev 74840)
+++ trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-doc.tex 2025-04-03 19:52:07 UTC (rev 74841)
@@ -28,7 +28,7 @@
\begingroup
\catcode`|=\active
\gdef|{\textbackslash}
-\gdef\xmacaux#1{\ttfamily\char92#1\endgroup}
+\gdef\xmacaux#1{\ttfamily\char92#1\endgroup{}}
\endgroup
\gdef\macname{\begingroup\catcode`|=\active \xmacaux}
\newcommand\thevbox{\medskip\theverbbox\par\medskip\noindent}
@@ -884,6 +884,14 @@
$\!\!\!$.
\end{sloppypar}
+As of version 1.5 of the package, the specification of the escape character
+ has been generalized.
+In addition to ASCII tokens such as ``$|\mkern1mu$'', unused macro tokens may be
+ specified, such as \begin{verbbox}[\vbdelim]
+\settcEscapechar{\z}\end{verbbox}
+ \theverbbox $\!\!\!$.
+
+
\subsection{Flagged tokens}
Certain token types are trapped and flagged via true/false
@@ -908,6 +916,14 @@
If an active token's substitution is governed by a \macname{def}, the
text substitution will have occurred before reaching the token cycle.
+The disposition of active tokens will depend on their actual definition. They could
+ be directed to the Character, Macro, or Space directive as a result.
+As an example, whereas non-active UNICODE characters would show up in the
+ Character directive when using lualatex or xelatex,
+ multi-byte UNICODE characters may be discerned (in pdflatex) by
+ looking for and testing active tokens in the Macro directive.
+An example of this is shown in the companion document depicting examples of use.
+
\paragraph{\macname{ifactivetokunexpandable}:}
This flag is similar to \macname{ifactivetok}, in that a token must
@@ -1197,17 +1213,16 @@
opportunity to interact with the leading developers and
practitioners of \TeX{} and \LaTeX{}.
-\vspace{.6in}
+%\vspace{1.6in}
\section*{Source Code}
%\vspace{-0.1in}
\Large\texttt{tokcycle.sty}
\verbfilenobox[\footnotesize]{tokcycle.sty}%
-\vspace{1.5in}
+\vspace{0.5in}
\noindent\texttt{tokcycle.tex}
\verbfilenobox[\footnotesize]{tokcycle.tex}
-
\end{document}
Modified: trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.pdf
===================================================================
(Binary files differ)
Modified: trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.tex
===================================================================
--- trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.tex 2025-04-03 19:51:18 UTC (rev 74840)
+++ trunk/Master/texmf-dist/doc/generic/tokcycle/tokcycle-examples.tex 2025-04-03 19:52:07 UTC (rev 74841)
@@ -1,6 +1,6 @@
\documentclass[10pt]{article}
\usepackage[margin=1.3in]{geometry}
-\usepackage[T1]{fontenc}
+\usepackage[T2A,T1]{fontenc}
\usepackage{tokcycle,xcolor,manfnt,lmodern,mathtools}
%%%%TRIAL PACKAGE CHANGES (MAKE SURE EXAMPLE FILE COMPILES CORRECTLY)
\makeatletter
@@ -50,6 +50,16 @@
arc=1mm,boxrule=1pt,
title=#2,#1}
+\newtcblisting{exampleD}[2][]{%
+ colframe=red!70!yellow!50!black,
+ colback=red!20!yellow!5!white,
+ coltitle=red!50!yellow!3!white,
+ fonttitle=\sffamily\bfseries,
+ text only,
+ top=0mm,bottom=0mm,left=0mm,right=0mm,
+ arc=1mm,boxrule=1pt,
+ title=#2,#1}
+
\def\altdecytoks{\par\medskip\begingroup\noindent\macname{cytoks}
\ \textit{alt}detokenization:\\\raggedright\footnotesize\ttfamily
\expandafter\altdetokenize\expandafter{\the\cytoks}\normalsize\par
@@ -57,6 +67,10 @@
\newcommand\TokCycle{\textsf{tokcycle}}
+\fboxsep=1.5pt
+\DeclareUnicodeCharacter{1D60}{$\varphi$}%{\fbox{U+1D60}}
+\DeclareUnicodeCharacter{1D131}{$\sharp$}%{\fbox{U+1D131}}
+
\parindent0pt
\parskip0pt
\begin{document}
@@ -392,8 +406,10 @@
The default escape character is $|$.
One can change the token-cycle escape character by way of the
\macname{settcEscapechar} macro.
+As of version 1.5, the secape character need no longer be restricted to
+ ASCII characters, but can now include an otherwise unutilized macro name.
Below, the character codes are incremented by one; however, the
- arguments of \macname{rule} are protected against that.
+ arguments of \macname{rule} as well as other text are protected against that.
\begingroup
@@ -414,6 +430,10 @@
of |\rule{1em}{.5em}|
{\bfseries mine}.}
\the\cytoks
+\par\settcEscapechar{\Esc}
+\tokcyclexpress{%
+Now we \Esc test a new escape token \Esc here.}
+\the\cytoks
\end{exampleA}
\endgroup
@@ -1486,7 +1506,7 @@
\subsubsection{Count in advance that which follows}
-Specify the point values for questions and let tokcycle perform the cumulative
+Specify the point values for questions and let \TokCycle{} perform the cumulative
tallies in advance, using a single-pass compilation.
\begin{exampleC}[]{The \texttt{pointtracker} environment to count point values
@@ -1585,7 +1605,7 @@
\begingroup
Tokcycle provides the escape character $|$ to delimit tokens to be protected
- from processing by tokcycle directives.
+ from processing by \TokCycle{} directives.
But that requires the input stream to be adjusted to include these delimiters.
Here's an alternative, not requiring input stream adjustment.
The Macro directive protects all \macname{rule} arguments against contamination
@@ -1789,7 +1809,10 @@
\endgroup
-\section{Advanced topics: implicit tokens and catcode changes}
+\let\svtilde~
+\let~\\
+\section{Advanced topics: implicit and active tokens, catcode~changes}
+\let~\svtilde
\subsection{Trap active characters (catcode 13)}
@@ -1920,6 +1943,135 @@
\end{exampleA}
\endgroup
+\subsection{Digest/discern Unicode characters in pdflatex\label{s:unicode}}
+
+When using the lualatex or xelatex engines, the presence of Unicode characters
+ poses no problem.
+In the context of a token cycle, each such character is digested and passed as
+ an individual token to the Character directive.
+
+\medskip
+In pdflatex, however, which digests input one byte at a time, Unicode characters
+ are represented in UTF-8 encoding as multi-byte sequences of two to four bytes
+ in length.
+These tokens are made active and the leading bits of the first byte can be used
+ to decipher the length of the Unicode character encoding.
+Since tokens in pdflatex are composed of single bytes, a token cycle in the
+ context of pdflatex only gets to digest one byte at a time of a given Unicode
+ character.
+The key to digesting and decoding Unicode characters therefore, is to look
+ for active tokens coming into the Macro directive.
+If such a byte is found, its slot should be determined.
+Slots 192--223 indicate a 2-byte Unicode character and so one additional byte
+ needs to be absorbed to compose the full Unicode character.
+If the first byte resides in slots 224--239, a 3-byte encoding is called for,
+ so that two additional bytes need to be absorbed to reconstitute the full
+ character.
+Finally, if the slot of the active byte exceeds 239, a 4-byte encoding is called
+ for and three additional bytes must be absorbed to reformulate the Unicode
+ character.
+The following code shows exactly how this is done using an example.
+
+\begin{exampleC}[]{Setting up a token cycle to digest/discern Unicode characters
+ in pdflatex}
+% PREAMBLE SETUP FOR NEEDED UNICODE TOKENS
+\DeclareUnicodeCharacter{1D60}{$\varphi$}
+\DeclareUnicodeCharacter{1D131}{$\sharp$}
+%
+\newif\ifUnicode
+\resettokcycle
+\Macrodirective{%
+ \ifactivetok\digestactivechar{#1}%
+ \ifUnicode\addcytoks[1]{\expandafter\makered\expandafter{\tcactivechar}}%
+ \else\addcytoks{#1}\fi
+ \else
+ \addcytoks{#1}%
+ \fi}
+\newcommand\tcdigestbyte{\tcpopappto\tcactivechar}
+\newcommand\digestactivechar[1]{\Unicodefalse
+ \def\tcactivechar{#1}%
+ \ifnum`#1>239 \Unicodetrue\tcdigestbyte\tcdigestbyte\tcdigestbyte\else
+ \ifnum`#1>223 \Unicodetrue\tcdigestbyte\tcdigestbyte\else
+ \ifnum`#1>191 \Unicodetrue\tcdigestbyte\fi\fi\fi}
+\newcommand\makered{\textcolor{red}}
+\end{exampleC}
+
+\begingroup
+\begin{exampleD}[]{Executing a pdflatex token cycle able to digest/discern
+ Unicode characters}
+\small
+\newif\ifUnicode
+\resettokcycle
+\Macrodirective{%
+ \ifactivetok\digestactivechar{#1}%
+ \ifUnicode\addcytoks[1]{\expandafter\makered\expandafter{\tcactivechar}}%
+ \else\addcytoks{#1}\fi
+ \else
+ \addcytoks{#1}%
+ \fi}
+\newcommand\tcdigestbyte{\tcpopappto\tcactivechar}
+\newcommand\digestactivechar[1]{\Unicodefalse
+ \def\tcactivechar{#1}%
+ \ifnum`#1>239 \Unicodetrue\tcdigestbyte\tcdigestbyte\tcdigestbyte\else
+ \ifnum`#1>223 \Unicodetrue\tcdigestbyte\tcdigestbyte\else
+ \ifnum`#1>191 \Unicodetrue\tcdigestbyte\fi\fi\fi}
+\newcommand\makered{\textcolor{red}}
+\begingroup
+\Macrodirective{%
+ \ifactivetok\digestactivechar{#1}%
+ \ifUnicode\addcytoks{\begingroup\fontencoding{T2A}\selectfont}%
+ \addcytoks[1]{\expandafter{\tcactivechar}\endgroup}
+ \else\ifx~#1\addcytoks{\string#1}\else\addcytoks{#1\mbox{}}\fi\fi
+ \else
+ \addcytoks{\string#1}
+ \fi}
+\obeylines\obeyspaces
+\ttfamily
+\string\tokencyclexpress \tokencyclexpress On \today, we are running input through a token cycle
+that highlights Unicode characters in red:
+
+Testing an active tilde: (~) and a Euro sign: €.
+
+Accented Latin Characters:
+á, é, ó, ú, ü, ñ, ç, à, è, ò, ù, â, ê, ô, û, ã,
+õ, ä, ö, ü, ß, œ.
+
+Cyrillic Characters:
+А, Б, В, Г, Д, Е, Ё, Ж, З, И, Й, К, Л, М, Н, О, П,
+Р, С, Т, У, Ф, Х, Ц, Ч, Ш, Щ, Ъ, Ы, Ь, Э, Ю, Я.
+
+Пример текста на кириллице: Здравствуйте, Latin, как дела?
+
+Higher level (3, 4-byte) Unicode (unsupported by font):
+ᵠ, and 𝄱
+\endtokencyclexpress\string\endtokencyclexpress\endgroup
+
+\vspace{-7pt}\hrulefill
+\fontencoding{T2A}\selectfont
+%\includegraphics{UNICODE DECODE}
+
+\tokencyclexpress On \today, we are running input through a token cycle
+that highlights Unicode characters in red:
+
+Testing an active tilde: (~) and a Euro sign: €.
+
+Accented Latin Characters:
+á, é, ó, ú, ü, ñ, ç, à, è, ò, ù, â, ê, ô, û, ã,
+õ, ä, ö, ü, ß, œ.
+
+Cyrillic Characters:
+А, Б, В, Г, Д, Е, Ё, Ж, З, И, Й, К, Л, М, Н, О, П,
+Р, С, Т, У, Ф, Х, Ц, Ч, Ш, Щ, Ъ, Ы, Ь, Э, Ю, Я.
+
+Пример текста на кириллице: Здравствуйте, Latin, как дела?
+
+Higher level (3, 4-byte) Unicode (unsupported by font):
+ᵠ, and 𝄱
+\endtokencyclexpress
+\end{exampleD}
+
+\endgroup
+
\subsection{Trap catcode 6 (explicit \& implicit) tokens}
\begingroup
@@ -2103,6 +2255,158 @@
in the input stream as duplicate, e.g. \texttt{\#\#}.
+\subsection{Using the \TokCycle{} traps to understand a token's nature}
+
+With the use of all the \TokCycle{} traps, one can do a great deal to
+ understand the nature of the more arcane tokens found in an input stream.
+We already saw in section \ref{s:unicode} how the \macname{ifactivetok}
+ trap can be used to decipher multi-byte Unicode tokens in pdflatex.
+In the example below, we define various tokens outside of \TokCycle{}, and then
+ pass them, successively, to a token cycle to see what can be discerned about
+ their nature and if the result matches what we know externally about how each
+ token was created.
+
+\medskip
+First, we define the \TokCycle{} macro \macname{identAI} that is used to test
+ the token which is passed as the argument.
+Traps employed include \macname{ifactivetok}, \macname{ifimplicittok},
+ \macname{ifactivetokunexpandable}, and \macname{ifcatSIX}.
+
+\begin{exampleC}[]{Setting up a token cycle to discern the characteristics of
+ a token}
+\newcommand\identAI[1]{\begingroup\color{red}
+ \tokcycle
+ {\findAI{Character}{##1}}
+ {\processtoks{##1}}
+ {\findAI{Macro}{##1}}
+ {\findAI{Space}{##1}}
+ {#1}\endgroup\par}
+\newcommand\findAI[2]{%
+ \ifactivetok
+ \ifimplicittok\showtok{#1}{#2}{active and implicit}%
+ \else\showtok{#1}{#2}{active}\fi
+ \else
+ \ifimplicittok\showtok{#1}{#2}{implicit}\else
+ \showtok{#1}{#2}{conventional}\fi
+ \fi}
+\newcommand\showtok[3]{
+ \ifcatSIX
+ #1 \expandafter\string#2 is #3, of catcode-6%
+ \else
+ #1 \string#2 is #3,
+ \ifimplicittok let to ``#2''%
+ \else
+ \ifactivetokunexpandable but unexpandable%
+ \else replaces with ``\expandafter\string#2''\fi
+ \fi%
+ \fi.}
+\def\lettok#1#2{\let#1= #2\empty}
+\end{exampleC}
+
+Once the above macros are set up, we define the active tilde token (\texttt{\string~})
+ in a variety of ways and pass it, successsively, to \macname{identAI}.
+Then, we define the macro \macname{z} in the same variety of ways and perform the
+ tests on successive incarnations of \macname{z}.
+Finally, we pass a catcode-6 \string# to \macname{identAI} just to see how it
+ reacts.
+
+\medskip
+Note that the \macname{ifactivetokunexpandable} trap only applies to active tokens.
+Therefore, for non-active tokens that are likewise unexpandable, such as when
+ \macname{z} is let to \macname{relax}, the \macname{identAI} macro instead
+ indicates that the analyzed token (\macname{z}) and its replacement code
+ are identical.
+This, of course, is another way to say ``unexpandable''.
+
+\medskip
+In the output below, the code is shown at the top, and in the output shown at the
+ bottom, the setup external to \macname{identAI} is shown in black, while the
+ output of the \macname{identAI} token cycle is shown in red.
+The first word of the \macname{identAI} output, either ``Character'', ``Macro'',
+ or ``Space'', indicates the particular \TokCycle{} directive to which the
+ token was delivered for examination.
+As we can see, the token cycle, when presented with a wide variety of token types,
+ including combinations of active, implicit, and unexpandable, is able to
+ discern the conditions under which the token was created.
+
+
+\begin{exampleD}[]{Discerning the characteristics of unknown tokens (token cycle
+ output in red)}
+\small
+\newcommand\identAI[1]{\begingroup\color{red}
+ \tokcycle
+ {\findAI{Character}{##1}}
+ {\processtoks{##1}}
+ {\findAI{Macro}{##1}}
+ {\findAI{Space}{##1}}
+ {#1}\endgroup\par}
+\newcommand\findAI[2]{%
+ \ifactivetok
+ \ifimplicittok\showtok{#1}{#2}{active and implicit}%
+ \else\showtok{#1}{#2}{active}\fi
+ \else
+ \ifimplicittok\showtok{#1}{#2}{implicit}\else
+ \showtok{#1}{#2}{conventional}\fi
+ \fi}
+\newcommand\showtok[3]{
+ \ifcatSIX
+ #1 \expandafter\string#2 is #3, of catcode-6%
+ \else
+ #1 \string#2 is #3,
+ \ifimplicittok let to ``#2''%
+ \else
+ \ifactivetokunexpandable but unexpandable%
+ \else replaces with ``\expandafter\string#2''\fi
+ \fi%
+ \fi.}
+\def\lettok#1#2{\let#1= #2\empty}
+\begin{verbatim}
+\def\SayFoo{Foo!}
+\lettok~{Q}\string~ let to Q: \identAI{~}
+\def~{Q}\string~ def'ed to Q: \identAI{~}
+\lettok~{\relax}\string~ let to \string\relax: \identAI{~}
+\def~{\relax}\string~ def'ed to \string\relax: \identAI{~}
+\lettok~{\SayFoo}\string~ let to \string\SayFoo: \identAI{~}
+\def~{\SayFoo}\string~ def'ed to \string\SayFoo: \identAI{~}
+\lettok~{ }\string~ let to space tok: \identAI{~}
+\def~{ }\string~ def'ed to space tok: \identAI{~}
+\lettok~{#}\string~\ let to \string#: \identAI{~}
+\lettok\z{Q}\string\z\ let to Q: \identAI{\z}
+\def\z{Q}\string\z\ def'ed to Q: \identAI{\z}
+\lettok\z{\relax}\string\z\ let to \string\relax: \identAI{\z}
+\def\z{\relax}\string\z\ def'ed to \string\relax: \identAI{\z}
+\lettok\z{\SayFoo}\string\z\ let to \string\SayFoo: \identAI{\z}
+\def\z{\SayFoo}\string\z\ def'ed to \string\SayFoo: \identAI{\z}
+\lettok\z{ }\string\z\ let to space tok: \identAI{\z}
+\def\z{ }\string\z\ def'ed to space tok: \identAI{\z}
+\lettok\z{#}\string\z\ let to \string#: \identAI{\z}
+\string# itself: \identAI{#}
+\end{verbatim}
+\vspace{-15pt}\hrulefill
+
+\def\SayFoo{Foo!}
+\lettok~{Q}\string~ let to Q: \identAI{~}
+\def~{Q}\string~ def'ed to Q: \identAI{~}
+\lettok~{\relax}\string~ let to \string\relax: \identAI{~}
+\def~{\relax}\string~ def'ed to \string\relax: \identAI{~}
+\lettok~{\SayFoo}\string~ let to \string\SayFoo: \identAI{~}
+\def~{\SayFoo}\string~ def'ed to \string\SayFoo: \identAI{~}
+\lettok~{ }\string~ let to space tok: \identAI{~}
+\def~{ }\string~ def'ed to space tok: \identAI{~}
+\lettok~{#}\string~\ let to \string#: \identAI{~}
+\lettok\z{Q}\string\z\ let to Q: \identAI{\z}
+\def\z{Q}\string\z\ def'ed to Q: \identAI{\z}
+\lettok\z{\relax}\string\z\ let to \string\relax: \identAI{\z}
+\def\z{\relax}\string\z\ def'ed to \string\relax: \identAI{\z}
+\lettok\z{\SayFoo}\string\z\ let to \string\SayFoo: \identAI{\z}
+\def\z{\SayFoo}\string\z\ def'ed to \string\SayFoo: \identAI{\z}
+\lettok\z{ }\string\z\ let to space tok: \identAI{\z}
+\def\z{ }\string\z\ def'ed to space tok: \identAI{\z}
+\lettok\z{#}\string\z\ let to \string#: \identAI{\z}
+\string# itself: \identAI{#}
+\end{exampleD}
+
+
\subsection{Changing grouping tokens (catcodes 1,2)}
\begingroup
Modified: trunk/Master/texmf-dist/tex/generic/tokcycle/tokcycle.tex
===================================================================
--- trunk/Master/texmf-dist/tex/generic/tokcycle/tokcycle.tex 2025-04-03 19:51:18 UTC (rev 74840)
+++ trunk/Master/texmf-dist/tex/generic/tokcycle/tokcycle.tex 2025-04-03 19:52:07 UTC (rev 74841)
@@ -1,7 +1,7 @@
\def\tcname {tokcycle}
-\def\tcver {1.42}
+\def\tcver {1.5}
%
-\def\tcdate {2021/08/25}
+\def\tcdate {2025/04/02}
%
% Author : Steven B Segletes, Christian Tellechea (contributor)
% Maintainer : Steven B Segletes
@@ -61,6 +61,7 @@
\long\def\count at toks#1{\the\numexpr-1\count@@toks#1.\tc at endcnt}
\long\def\count@@toks#1#2\tc at endcnt{+1\tc at ifempty{#2}{\relax}{\count@@toks#2\tc at endcnt}}
\def\sv at hash{##}
+\let\tc at svtilde=~
\def\def at popname{\expandafter\def\tc at popname}
% EXTERNAL TOOLS
\let\tcsptoken= \tc at sptoken
@@ -111,7 +112,8 @@
\long\def\can at absorb#1{\tc at tok{#1}\trapcatSIX{#1}\expandafter\can at absorb@
\the\tc at tok}
\long\def\can at absorb@#1{\tctestifnum{\count at stringtoks{#1}>1}%
- {\tctestifx{\endtokcycraw#1}{#1}{\backslashcmds#1\@tokcycle}}%
+ {\tctestifx{\endtokcycraw#1}{#1}{\tc at trapescape#1{\tc at escapecytoks\empty}%
+ {\backslashcmds#1\@tokcycle}}}%
{\trapactives#1\tc at trapescape#1{\tc at escapecytoks\empty}{\can at absorb@@#1}}}
\long\def\can at absorb@@#1{\let\@tmp=#1\test at ifmacro\@tmp{\implicittokfalse
\@macT#1}{\trapimplicitegrp#1\implicitgrpfork#1}\@tokcycle}
@@ -151,8 +153,8 @@
\@implicitgroupingcase\or\addcytoks{#1}\or\@chrT{#1}\fi}{\@chrT#1}}
% SET UP ESCAPE MECHANISM
\def\settcEscapechar#1{\let\@tcEscapeptr#1%
- \def\tc at escapecytoks##1#1{\addcytoks[1]{##1}\@tokcycle}}
-\def\tc at trapescape#1{\tctestifx{\@tcEscapeptr#1}}
+ \long\def\tc at escapecytoks##1#1{\addcytoks[1]{##1}\@tokcycle}}
+\long\def\tc at trapescape#1{\tctestifx{\@tcEscapeptr#1}}
% TRAP CAT-6
\long\def\trapcatSIX#1{\tctestifcatnx#1\relax{}{\trapcatSIXb#1}}
\def\trapcatSIXb#1{\expandafter\tctestifcatnx\sv at hash#1{\catSIXtrue\trapcatSIXc#1}{}}
@@ -185,10 +187,12 @@
% TRAP ACTIVE TOK
\def\trapactives#1{\trapactivechar{#1}\trapactivetok{#1}}
\def\trapactivechar#1{\tctestifnum{\number\catcode`#1=13}{\activechartrue}{}}
-\def\trapactivetok#1{\tctestifcatnx~#1{\activetoktrue}{\trapactivetokunexpandable#1}}
+\def\trapactivetok#1{\begingroup\catcode`~=13\let~=\tc at svtilde\tctestifcatnx~#1%
+ {\endgroup\activetoktrue\trapactivetokunexpandable#1}{\endgroup}}
%% WILL ALSO TRAP ACTIVE \let TO PRIMITIVES AS IMPLICIT; UNDO LATER IN \can at absorb@@
-\def\trapactivetokunexpandable#1{\tctestifcon{\expandafter\if
- \detokenize{#1}#1}{}{\activetoktrue\activetokunexpandabletrue\implicittoktrue}}
+\def\trapactivetokunexpandable#1{\edef\@tmpX{\detokenize\expandafter{#1}}%
+ \edef\@tmpNX{\detokenize{#1}}\tctestifcon{\ifx\@tmpX\@tmpNX}%
+ {\activetokunexpandabletrue\implicittoktrue}{}}
% FEATURES TO LOOK-AHEAD INTO THE INPUT STREAM (INTRODUCED v1.4)
\long\def\tcpeek#1#2\@tokcycle{\def\tc at tmp{\ifx#1\endtokcycraw
\let#1=\empty\fi#2\@tokcycle}\futurelet#1\tc at tmp}%___________________________PEEK_
@@ -341,3 +345,11 @@
- Bug fix in \tc at escapecytoks (if escaped text was a single brace group, the
braces were lost)
+v1.5 2025/04/02
+ -Macro \can at absorb@ adjusted to enable not just characters, but also macros
+ as the tokcycle escape token.
+ -Made several definitions \long to accommodate multi-paragraph escapes:
+ \tc at escapecytoks, \tc at trapescape.
+ -Fixed several bugs in the code which traps active, implicit, and
+ active-unexpandable tokens: \trapactivetok, \trapactivetokunexpandable
+
More information about the tex-live-commits
mailing list.