texlive[71378] Master/texmf-dist: markdown (28may24)
commits+karl at tug.org
commits+karl at tug.org
Tue May 28 21:27:46 CEST 2024
Revision: 71378
https://tug.org/svn/texlive?view=revision&revision=71378
Author: karl
Date: 2024-05-28 21:27:46 +0200 (Tue, 28 May 2024)
Log Message:
-----------
markdown (28may24)
Modified Paths:
--------------
trunk/Master/texmf-dist/doc/generic/markdown/CHANGES.md
trunk/Master/texmf-dist/doc/generic/markdown/README.md
trunk/Master/texmf-dist/doc/generic/markdown/VERSION
trunk/Master/texmf-dist/doc/generic/markdown/markdown.html
trunk/Master/texmf-dist/doc/generic/markdown/markdown.pdf
trunk/Master/texmf-dist/scripts/markdown/markdown-cli.lua
trunk/Master/texmf-dist/source/generic/markdown/markdown.dtx
trunk/Master/texmf-dist/tex/generic/markdown/markdown.tex
trunk/Master/texmf-dist/tex/luatex/markdown/markdown.lua
Modified: trunk/Master/texmf-dist/doc/generic/markdown/CHANGES.md
===================================================================
--- trunk/Master/texmf-dist/doc/generic/markdown/CHANGES.md 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/doc/generic/markdown/CHANGES.md 2024-05-28 19:27:46 UTC (rev 71378)
@@ -1,10 +1,32 @@
# Changes
+## 3.6.0
+
+Development:
+
+- Add Lua options `unicodeNormalization` and `unicodeNormalizationForm` for
+ normalizing markdown text using Unicode normalization.
+ (#434, #442, sponsored by @istqborg)
+
+Fixes:
+
+- Make `\markdownInput` search in `\l_file_search_path_seq`.
+ (#443, reported by @robertjlee)
+
+Documentation:
+
+- Add a link to a work-in-progress article for TUGboat 45:2 to `README.md`.
+ (7a1a7ea0)
+
+Fixes:
+
+- Correctly sort YAML map keys with mixed types. (#433, #441)
+
## 3.5.0 (2024-04-29)
Development:
-- Add `\markinline` plain TeX command. (#300, #439)
+- Add `\markinline` plain TeX command. (#300, #439, [matrix.org][matrix-439])
- Support incremental definitions of token renderers and renderer prototypes
in the `\markdownSetup` command.
(#232, #435, 540a83c0, [matrix.org][matrix-435], #437, #438,
@@ -12,12 +34,13 @@
[on TeX StackExchange][tse-716400])
[matrix-435]: https://matrix.to/#/!UeAwznpYwwsinVTetR:matrix.org/$k4ky6I-uvxdp8ipVlHvef5JXfIfPQvFtXOAD_ogF2uU?via=matrix.org&via=im.f3l.de
+ [matrix-439]: https://matrix.to/#/!pznomuvubVyxElflTe:matrix.org/$Gp9CYKWP3h_gyFyArnW3Y3CJzVXefWYThjlyB58zvP8?via=matrix.org&via=im.f3l.de
[tse-716400]: https://tex.stackexchange.com/questions/716362/convert-control-sequence-with-a-variable-number-of-parameters-into-a-token-list/716400#716400
Default Renderer Prototypes:
- Add default renderers for unnumbered sections in LaTeX.
- (#401, istqborg/istqb_product_base#23, #430, reported by @felinecrp,
+ (#401, istqborg/istqb_product_base#23, #430, reported by @felipecrp,
sponsored by @istqborg)
## 3.4.3 (2024-04-04)
Modified: trunk/Master/texmf-dist/doc/generic/markdown/README.md
===================================================================
--- trunk/Master/texmf-dist/doc/generic/markdown/README.md 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/doc/generic/markdown/README.md 2024-05-28 19:27:46 UTC (rev 71378)
@@ -192,8 +192,9 @@
- [Markdown 2.15.0: What's New?][tb133],
- [Markdown 2.17.1: What's New, What's Next?][tb135],
- [Attributes in Markdown][tb136],
- - Markdown 3 at TUG 2023: Reflections from the Q&A session ([preprint][tb138]), and
- - Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching ([preprint][tb139]).
+ - Markdown 3 at TUG 2023: Reflections from the Q&A session ([preprint][tb138]),
+ - Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching ([preprint][tb139]), and
+ - Markdown Themes in Practice ([work in progress][tb140]).
5. Journal articles published by [CSTUG Bulletin][csbul] (in Czech and Slovak):
- [Rendering Markdown inside TeX Documents][10.5300/2016-1-4/78],
- [Markdown 2.8.1: Boldly Unto the Throne of Lightweight Markup in TeX][10.5300/2020-1-2/48],
@@ -219,15 +220,16 @@
[overleaf-5]: https://www.overleaf.com/latex/examples/writing-posters-with-markdown/jtbgmmgqrqmh "Writing Posters with Markdown"
[overleaf-6]: https://www.overleaf.com/latex/examples/using-markdown-in-latex-documents/whdrnpcpnwrm "Using Markdown in LaTeX documents"
- [tb119]: https://www.tug.org/TUGboat/tb38-2/tb119novotny.pdf "Using Markdown inside TeX documents"
- [tb124]: https://www.tug.org/TUGboat/tb40-1/tb124novotny-markdown.pdf "Markdown 2.7.0: Towards lightweight markup in TeX"
- [tb129]: https://www.tug.org/TUGboat/tb41-3/tb129novotny-frozen.pdf "Making Markdown into a Microwave Meal"
- [tb131]: https://www.tug.org/TUGboat/tb42-2/tb131novotny-markdown.pdf "Markdown 2.10.0: LaTeX Themes & Snippets, Two Flavors of Comments, and LuaMetaTeX"
- [tb133]: https://www.tug.org/TUGboat/tb43-1/tb133novotny-markdown.pdf "Markdown 2.15.0: What's New?"
- [tb135]: https://www.tug.org/TUGboat/tb43-3/tb135novotny-markdown.pdf "Markdown 2.17.1: What's New, What's Next?"
- [tb136]: https://www.tug.org/TUGboat/tb44-1/tb136novotny-markdown-attr.pdf "Attributes in Markdown"
- [tb138]: https://www.overleaf.com/read/mjghwhrbgmfj "Markdown 3 at TUG 2023: Reflections from the Q&A session"
- [tb139]: https://github.com/Witiko/fast-regression-testing/releases/download/latest/tb139starynovotny-testing.pdf "Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching"
+ [tb119]: https://www.tug.org/TUGboat/tb38-2/tb119novotny.pdf "Using Markdown inside TeX documents"
+ [tb124]: https://www.tug.org/TUGboat/tb40-1/tb124novotny-markdown.pdf "Markdown 2.7.0: Towards lightweight markup in TeX"
+ [tb129]: https://www.tug.org/TUGboat/tb41-3/tb129novotny-frozen.pdf "Making Markdown into a Microwave Meal"
+ [tb131]: https://www.tug.org/TUGboat/tb42-2/tb131novotny-markdown.pdf "Markdown 2.10.0: LaTeX Themes & Snippets, Two Flavors of Comments, and LuaMetaTeX"
+ [tb133]: https://www.tug.org/TUGboat/tb43-1/tb133novotny-markdown.pdf "Markdown 2.15.0: What's New?"
+ [tb135]: https://www.tug.org/TUGboat/tb43-3/tb135novotny-markdown.pdf "Markdown 2.17.1: What's New, What's Next?"
+ [tb136]: https://www.tug.org/TUGboat/tb44-1/tb136novotny-markdown-attr.pdf "Attributes in Markdown"
+ [tb138]: https://www.overleaf.com/read/mjghwhrbgmfj "Markdown 3 at TUG 2023: Reflections from the Q&A session"
+ [tb139]: https://github.com/Witiko/fast-regression-testing/releases/download/latest/tb139starynovotny-testing.pdf "Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching"
+ [tb140]: https://github.com/witiko/markdown-themes-in-practice/releases/download/latest/tb140starynovotny-markdown.pdf "Markdown Themes in Practice"
[tb131-slides]: https://tug.org/tug2021/assets/pdf/tug2021-novotny-slides.pdf "Markdown 2.10.0: LaTeX Themes & Snippets, Two Flavors of Comments, and LuaMetaTeX"
[tb131-video]: https://youtu.be/i2GJMnLCZls "Markdown 2.10.0: LaTeX Themes & Snippets, Two Flavors of Comments, and LuaMetaTeX"
Modified: trunk/Master/texmf-dist/doc/generic/markdown/VERSION
===================================================================
--- trunk/Master/texmf-dist/doc/generic/markdown/VERSION 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/doc/generic/markdown/VERSION 2024-05-28 19:27:46 UTC (rev 71378)
@@ -1 +1 @@
-3.5.0-0-gfd01a252 (2024-04-29)
+3.6.0-0-g83c781b4 (2024-05-27)
Modified: trunk/Master/texmf-dist/doc/generic/markdown/markdown.html
===================================================================
--- trunk/Master/texmf-dist/doc/generic/markdown/markdown.html 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/doc/generic/markdown/markdown.html 2024-05-28 19:27:46 UTC (rev 71378)
@@ -93,7 +93,7 @@
<header id="title-block-header">
<h1 class="title">Markdown Package User Manual</h1>
<p class="author">Vít Starý Novotný</p>
-<p class="date">3.5.0-0-gfd01a252 2024-04-29</p>
+<p class="date">3.6.0-0-g83c781b4 2024-05-27</p>
</header>
<nav id="TOC" role="doc-toc">
<ul>
@@ -978,8 +978,83 @@
</blockquote>
<p>As you can see, the caching causes <code>convert1</code> and
<code>convert3</code> to be the same conversion function.</p>
-<h4 data-number="2.2.1.3" id="option-cachedir"><span
+<h4 data-number="2.2.1.3" id="option-unicodenormalization"><span
class="header-section-number">2.2.1.3</span> Option
+<code>unicodeNormalization</code></h4>
+<dl>
+<dt><code>unicodeNormalization</code> (default value:
+<code>true</code>)</dt>
+<dd>
+<dl>
+<dt>true</dt>
+<dd>
+<p>Markdown documents will be normalized using one of the four <a
+href="https://unicode.org/faq/normalization.html">Unicode normalization
+forms</a> before conversion. The Unicode normalization norm used is
+determined by option
+<strong><code>unicodeNormalizationForm</code></strong>.</p>
+</dd>
+</dl>
+</dd>
+<dd>
+<dl>
+<dt>false</dt>
+<dd>
+<p>Markdown documents will not be Unicode-normalized before
+conversion.</p>
+</dd>
+</dl>
+</dd>
+</dl>
+<h4 data-number="2.2.1.4" id="option-unicodenormalizationform"><span
+class="header-section-number">2.2.1.4</span> Option
+<code>unicodeNormalizationForm</code></h4>
+<dl>
+<dt><code>unicodeNormalizationForm</code> (default value:
+<code>nfc</code>)</dt>
+<dd>
+<dl>
+<dt>nfc</dt>
+<dd>
+<p>When option <strong><code>unicodeNormalization</code></strong> has
+been enabled, markdown documents will be normalized using Unicode
+Normalization Form C (NFC) before conversion.</p>
+</dd>
+</dl>
+</dd>
+<dd>
+<dl>
+<dt>nfd</dt>
+<dd>
+<p>When option <strong><code>unicodeNormalization</code></strong> has
+been enabled, markdown documents will be normalized using Unicode
+Normalization Form D (NFD) before conversion.</p>
+</dd>
+</dl>
+</dd>
+<dd>
+<dl>
+<dt>nfkc</dt>
+<dd>
+<p>When option <strong><code>unicodeNormalization</code></strong> has
+been enabled, markdown documents will be normalized using Unicode
+Normalization Form KC (NFKC) before conversion.</p>
+</dd>
+</dl>
+</dd>
+<dd>
+<dl>
+<dt>nfkd</dt>
+<dd>
+<p>When option <strong><code>unicodeNormalization</code></strong> has
+been enabled, markdown documents will be normalized using Unicode
+Normalization Form KD (NFKD) before conversion.</p>
+</dd>
+</dl>
+</dd>
+</dl>
+<h4 data-number="2.2.1.5" id="option-cachedir"><span
+class="header-section-number">2.2.1.5</span> Option
<code>cacheDir</code></h4>
<dl>
<dt><code>cacheDir</code> (default value: <code>"."</code>)</dt>
@@ -1111,8 +1186,8 @@
<code>cache</code> containing several cache files of the Markdown
package will also be produced as we requested using the
<code>cacheDir</code> option.</p>
-<h4 data-number="2.2.1.4" id="option-contentblockslanguagemap"><span
-class="header-section-number">2.2.1.4</span> Option
+<h4 data-number="2.2.1.6" id="option-contentblockslanguagemap"><span
+class="header-section-number">2.2.1.6</span> Option
<code>contentBlocksLanguageMap</code></h4>
<dl>
<dt><code>contentBlocksLanguageMap</code> (default value:
@@ -1276,8 +1351,8 @@
class="sourceCode tex"><code class="sourceCode latex"><span id="cb79-1"><a href="#cb79-1" aria-hidden="true" tabindex="-1"></a>This is an example code listing in <span class="fu">\ConTeXt</span>.</span></code></pre></div>
<p>This is a <em>transcluded markdown document</em>.</p>
</blockquote>
-<h4 data-number="2.2.1.5" id="option-debugextensionsfilename"><span
-class="header-section-number">2.2.1.5</span> Option
+<h4 data-number="2.2.1.7" id="option-debugextensionsfilename"><span
+class="header-section-number">2.2.1.7</span> Option
<code>debugExtensionsFileName</code></h4>
<dl>
<dt><code>debugExtensionsFileName</code> (default value:
@@ -1294,8 +1369,8 @@
have been applied.</p>
</dd>
</dl>
-<h4 data-number="2.2.1.6" id="option-frozencachefilename"><span
-class="header-section-number">2.2.1.6</span> Option
+<h4 data-number="2.2.1.8" id="option-frozencachefilename"><span
+class="header-section-number">2.2.1.8</span> Option
<code>frozenCacheFileName</code></h4>
<dl>
<dt><code>frozenCacheFileName</code> (default value:
@@ -1499,8 +1574,8 @@
the document without accessing the shell or invoking Lua, but the change
in the content of the markdown document from “Hello <em>world</em>!” to
“Hi <em>world</em>!” was not reflected.</p>
-<h4 data-number="2.2.1.7" id="option-autoidentifiers"><span
-class="header-section-number">2.2.1.7</span> Option
+<h4 data-number="2.2.1.9" id="option-autoidentifiers"><span
+class="header-section-number">2.2.1.9</span> Option
<code>autoIdentifiers</code></h4>
<dl>
<dt><code>autoIdentifiers</code> (default value:
@@ -1526,8 +1601,8 @@
</dl>
<p>See also the option
<strong><code>gfmAutoIdentifiers</code></strong>.</p>
-<h4 data-number="2.2.1.8" id="option-blankbeforeblockquote"><span
-class="header-section-number">2.2.1.8</span> Option
+<h4 data-number="2.2.1.10" id="option-blankbeforeblockquote"><span
+class="header-section-number">2.2.1.10</span> Option
<code>blankBeforeBlockquote</code></h4>
<dl>
<dt><code>blankBeforeBlockquote</code> (default value:
@@ -1720,8 +1795,8 @@
</blockquote>
<p>A paragraph > Not a quote.</p>
</blockquote>
-<h4 data-number="2.2.1.9" id="option-blankbeforecodefence"><span
-class="header-section-number">2.2.1.9</span> Option
+<h4 data-number="2.2.1.11" id="option-blankbeforecodefence"><span
+class="header-section-number">2.2.1.11</span> Option
<code>blankBeforeCodeFence</code></h4>
<dl>
<dt><code>blankBeforeCodeFence</code> (default value:
@@ -1925,8 +2000,8 @@
<pre><code>A fenced code.</code></pre>
<p>A paragraph. <code>Not a fenced code.</code></p>
</blockquote>
-<h4 data-number="2.2.1.10" id="option-blankbeforedivfence"><span
-class="header-section-number">2.2.1.10</span> Option
+<h4 data-number="2.2.1.12" id="option-blankbeforedivfence"><span
+class="header-section-number">2.2.1.12</span> Option
<code>blankBeforeDivFence</code></h4>
<dl>
<dt><code>blankBeforeDivFence</code> (default value:
@@ -1982,8 +2057,8 @@
<p>A paragraph.</p>
<p>::: {.identifier} Not a fenced div. :::</p>
</blockquote>
-<h4 data-number="2.2.1.11" id="option-blankbeforeheading"><span
-class="header-section-number">2.2.1.11</span> Option
+<h4 data-number="2.2.1.13" id="option-blankbeforeheading"><span
+class="header-section-number">2.2.1.13</span> Option
<code>blankBeforeHeading</code></h4>
<dl>
<dt><code>blankBeforeHeading</code> (default value:
@@ -2176,8 +2251,8 @@
<h1 id="a-heading.-3">A heading.</h1>
<p>A paragraph. Not a heading. ==============</p>
</blockquote>
-<h4 data-number="2.2.1.12" id="option-blankbeforelist"><span
-class="header-section-number">2.2.1.12</span> Option
+<h4 data-number="2.2.1.14" id="option-blankbeforelist"><span
+class="header-section-number">2.2.1.14</span> Option
<code>blankBeforeList</code></h4>
<dl>
<dt><code>blankBeforeList</code> (default value:
@@ -2370,8 +2445,8 @@
</ul>
<p>A paragraph. - not a list</p>
</blockquote>
-<h4 data-number="2.2.1.13" id="option-bracketedspans"><span
-class="header-section-number">2.2.1.13</span> Option
+<h4 data-number="2.2.1.15" id="option-bracketedspans"><span
+class="header-section-number">2.2.1.15</span> Option
<code>bracketedSpans</code></h4>
<dl>
<dt><code>bracketedSpans</code> (default value: <code>false</code>)</dt>
@@ -2443,8 +2518,8 @@
<blockquote>
<p>Here is some <span style="color: red">colored text</span>.</p>
</blockquote>
-<h4 data-number="2.2.1.14" id="option-breakableblockquotes"><span
-class="header-section-number">2.2.1.14</span> Option
+<h4 data-number="2.2.1.16" id="option-breakableblockquotes"><span
+class="header-section-number">2.2.1.16</span> Option
<code>breakableBlockquotes</code></h4>
<dl>
<dt><code>breakableBlockquotes</code> (default value:
@@ -2665,8 +2740,8 @@
<p>Another block quote.</p>
</blockquote>
</blockquote>
-<h4 data-number="2.2.1.15" id="option-citationnbsps"><span
-class="header-section-number">2.2.1.15</span> Option
+<h4 data-number="2.2.1.17" id="option-citationnbsps"><span
+class="header-section-number">2.2.1.17</span> Option
<code>citationNbsps</code></h4>
<dl>
<dt><code>citationNbsps</code> (default value: <code>true</code>)</dt>
@@ -2734,8 +2809,8 @@
Donald·Ervin Knuth. <em>The TeXbook, volume A of Computers and
typesetting.</em> Addison-Wesley, 1984.</p>
</blockquote>
-<h4 data-number="2.2.1.16" id="option-citations"><span
-class="header-section-number">2.2.1.16</span> Option
+<h4 data-number="2.2.1.18" id="option-citations"><span
+class="header-section-number">2.2.1.18</span> Option
<code>citations</code></h4>
<dl>
<dt><code>citations</code> (default value: <code>false</code>)</dt>
@@ -2815,8 +2890,8 @@
Donald Ervin Knuth. <em>The <span class="tex">T<sub>e</sub>X</span>book,
volume A of Computers and typesetting.</em> Addison-Wesley, 1984.</p>
</blockquote>
-<h4 data-number="2.2.1.17" id="option-codespans"><span
-class="header-section-number">2.2.1.17</span> Option
+<h4 data-number="2.2.1.19" id="option-codespans"><span
+class="header-section-number">2.2.1.19</span> Option
<code>codeSpans</code></h4>
<dl>
<dt><code>codeSpans</code> (default value: <code>true</code>)</dt>
@@ -3005,8 +3080,8 @@
span.’’</p>
<p>“This is a quote.” “This is another quote.”</p>
</blockquote>
-<h4 data-number="2.2.1.18" id="option-contentblocks"><span
-class="header-section-number">2.2.1.18</span> Option
+<h4 data-number="2.2.1.20" id="option-contentblocks"><span
+class="header-section-number">2.2.1.20</span> Option
<code>contentBlocks</code></h4>
<dl>
<dt><code>contentBlocks</code> (default value: <code>false</code>)</dt>
@@ -3179,8 +3254,8 @@
class="sourceCode tex"><code class="sourceCode latex"><span id="cb197-1"><a href="#cb197-1" aria-hidden="true" tabindex="-1"></a>This is an example code listing in <span class="fu">\ConTeXt</span>.</span></code></pre></div>
<p>This is a <em>transcluded markdown document</em>.</p>
</blockquote>
-<h4 data-number="2.2.1.19" id="option-contentlevel"><span
-class="header-section-number">2.2.1.19</span> Option
+<h4 data-number="2.2.1.21" id="option-contentlevel"><span
+class="header-section-number">2.2.1.21</span> Option
<code>contentLevel</code></h4>
<dl>
<dt><code>contentLevel</code> (default value: <code>block</code>)</dt>
@@ -3291,8 +3366,8 @@
</ul>
<p>- this is - a text</p>
</blockquote>
-<h4 data-number="2.2.1.20" id="option-debugextensions"><span
-class="header-section-number">2.2.1.20</span> Option
+<h4 data-number="2.2.1.22" id="option-debugextensions"><span
+class="header-section-number">2.2.1.22</span> Option
<code>debugExtensions</code></h4>
<dl>
<dt><code>debugExtensions</code> (default value:
@@ -3414,8 +3489,8 @@
<span id="cb209-37"><a href="#cb209-37" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<p>This output shows us that our user-defined syntax extension has been
correctly inserted to the grammar of markdown.</p>
-<h4 data-number="2.2.1.21" id="option-definitionlists"><span
-class="header-section-number">2.2.1.21</span> Option
+<h4 data-number="2.2.1.23" id="option-definitionlists"><span
+class="header-section-number">2.2.1.23</span> Option
<code>definitionLists</code></h4>
<dl>
<dt><code>definitionLists</code> (default value:
@@ -3531,8 +3606,8 @@
</dd>
</dl>
</blockquote>
-<h4 data-number="2.2.1.22" id="option-expectjekylldata"><span
-class="header-section-number">2.2.1.22</span> Option
+<h4 data-number="2.2.1.24" id="option-expectjekylldata"><span
+class="header-section-number">2.2.1.24</span> Option
<code>expectJekyllData</code></h4>
<dl>
<dt><code>expectJekyllData</code> (default value:
@@ -3631,8 +3706,8 @@
<blockquote>
<p>Jane Doe is 99 years old.</p>
</blockquote>
-<h4 data-number="2.2.1.23" id="option-extensions"><span
-class="header-section-number">2.2.1.23</span> Option
+<h4 data-number="2.2.1.25" id="option-extensions"><span
+class="header-section-number">2.2.1.25</span> Option
<code>extensions</code></h4>
<dl>
<dt><code>extensions</code> (default value: <code>{}</code>)</dt>
@@ -3691,8 +3766,8 @@
<blockquote>
<p>This is <del>a lunar roving vehicle</del> strike-through text.</p>
</blockquote>
-<h4 data-number="2.2.1.24" id="option-fancylists"><span
-class="header-section-number">2.2.1.24</span> Option
+<h4 data-number="2.2.1.26" id="option-fancylists"><span
+class="header-section-number">2.2.1.26</span> Option
<code>fancyLists</code></h4>
<dl>
<dt><code>fancyLists</code> (default value: <code>false</code>)</dt>
@@ -3772,8 +3847,8 @@
<li>third item</li>
</ol>
</blockquote>
-<h4 data-number="2.2.1.25" id="option-fencedcode"><span
-class="header-section-number">2.2.1.25</span> Option
+<h4 data-number="2.2.1.27" id="option-fencedcode"><span
+class="header-section-number">2.2.1.27</span> Option
<code>fencedCode</code></h4>
<dl>
<dt><code>fencedCode</code> (default value: <code>true</code>)</dt>
@@ -3908,8 +3983,8 @@
<span id="cb238-7"><a href="#cb238-7" aria-hidden="true" tabindex="-1"></a> <span class="dt"></</span><span class="kw">code</span><span class="dt">></span></span>
<span id="cb238-8"><a href="#cb238-8" aria-hidden="true" tabindex="-1"></a><span class="dt"></</span><span class="kw">pre</span><span class="dt">></span></span></code></pre></div>
</blockquote>
-<h4 data-number="2.2.1.26" id="option-fencedcodeattributes"><span
-class="header-section-number">2.2.1.26</span> Option
+<h4 data-number="2.2.1.28" id="option-fencedcodeattributes"><span
+class="header-section-number">2.2.1.28</span> Option
<code>fencedCodeAttributes</code></h4>
<dl>
<dt><code>fencedCodeAttributes</code> (default value:
@@ -3985,8 +4060,8 @@
<span id="cb242-2"><a href="#cb242-2" aria-hidden="true" tabindex="-1"></a><span class="fl">2.</span> <span class="fu">moveShip</span>(<span class="dv">5</span> <span class="op">*</span> gravity<span class="op">,</span> DOWN)<span class="op">;</span></span>
<span id="cb242-3"><a href="#cb242-3" aria-hidden="true" tabindex="-1"></a><span class="fl">3.</span> }</span></code></pre></div>
</blockquote>
-<h4 data-number="2.2.1.27" id="fenced-divs"><span
-class="header-section-number">2.2.1.27</span> Option
+<h4 data-number="2.2.1.29" id="fenced-divs"><span
+class="header-section-number">2.2.1.29</span> Option
<code>fencedDivs</code></h4>
<dl>
<dt><code>fencedDivs</code> (default value: <code>false</code>)</dt>
@@ -4041,8 +4116,8 @@
<blockquote>
<p>Here is a special paragraph.</p>
</blockquote>
-<h4 data-number="2.2.1.28" id="option-finalizecache"><span
-class="header-section-number">2.2.1.28</span> Option
+<h4 data-number="2.2.1.30" id="option-finalizecache"><span
+class="header-section-number">2.2.1.30</span> Option
<code>finalizeCache</code></h4>
<dl>
<dt><code>finalizeCache</code> (default value: <code>false</code>)</dt>
@@ -4173,8 +4248,8 @@
the document without accessing the shell or invoking Lua, but the change
in the content of the markdown document from “Hello <em>world</em>!” to
“Hi <em>world</em>!” was not reflected.</p>
-<h4 data-number="2.2.1.29" id="option-frozencachecounter"><span
-class="header-section-number">2.2.1.29</span> Option
+<h4 data-number="2.2.1.31" id="option-frozencachecounter"><span
+class="header-section-number">2.2.1.31</span> Option
<code>frozenCacheCounter</code></h4>
<dl>
<dt><code>frozenCacheCounter</code> (default value: <code>0</code>)</dt>
@@ -4214,8 +4289,8 @@
contain the text “Hello <em>world</em>!” A frozen cache with two entries
will also be produced as we requested using the
<code>frozenCacheCounter</code> option.</p>
-<h4 data-number="2.2.1.30" id="option-gfmautoidentifiers"><span
-class="header-section-number">2.2.1.30</span> Option
+<h4 data-number="2.2.1.32" id="option-gfmautoidentifiers"><span
+class="header-section-number">2.2.1.32</span> Option
<code>gfmAutoIdentifiers</code></h4>
<dl>
<dt><code>gfmAutoIdentifiers</code> (default value:
@@ -4242,8 +4317,8 @@
</dl>
<p>See also the option
<strong><code>autoIdentifiers</code></strong>.</p>
-<h4 data-number="2.2.1.31" id="option-hashenumerators"><span
-class="header-section-number">2.2.1.31</span> Option
+<h4 data-number="2.2.1.33" id="option-hashenumerators"><span
+class="header-section-number">2.2.1.33</span> Option
<code>hashEnumerators</code></h4>
<dl>
<dt><code>hashEnumerators</code> (default value:
@@ -4345,8 +4420,8 @@
<li>Parish</li>
</ol>
</blockquote>
-<h4 data-number="2.2.1.32" id="header-attributes"><span
-class="header-section-number">2.2.1.32</span> Option
+<h4 data-number="2.2.1.34" id="header-attributes"><span
+class="header-section-number">2.2.1.34</span> Option
<code>headerAttributes</code></h4>
<dl>
<dt><code>headerAttributes</code> (default value:
@@ -4375,8 +4450,8 @@
</dl>
</dd>
</dl>
-<h4 data-number="2.2.1.33" id="option-html"><span
-class="header-section-number">2.2.1.33</span> Option
+<h4 data-number="2.2.1.35" id="option-html"><span
+class="header-section-number">2.2.1.35</span> Option
<code>html</code></h4>
<dl>
<dt><code>html</code> (default value: <code>true</code>)</dt>
@@ -4613,8 +4688,8 @@
support.</p>
<p>There is support. There is support. There is support.</p>
</blockquote>
-<h4 data-number="2.2.1.34" id="option-hybrid"><span
-class="header-section-number">2.2.1.34</span> Option
+<h4 data-number="2.2.1.36" id="option-hybrid"><span
+class="header-section-number">2.2.1.36</span> Option
<code>hybrid</code></h4>
<dl>
<dt><code>hybrid</code> (default value: <code>false</code>)</dt>
@@ -4805,8 +4880,8 @@
<p><math><msqrt><mo>−</mo><mn>1</mn></msqrt></math> <em>equals</em>
<math><mi>i</mi></math>.</p>
</blockquote>
-<h4 data-number="2.2.1.35" id="option-inlinecodeattributes"><span
-class="header-section-number">2.2.1.35</span> Option
+<h4 data-number="2.2.1.37" id="option-inlinecodeattributes"><span
+class="header-section-number">2.2.1.37</span> Option
<code>inlineCodeAttributes</code></h4>
<dl>
<dt><code>inlineCodeAttributes</code> (default value:
@@ -4880,8 +4955,8 @@
<p>Here is some <span
style="color: red"><code>colored text</code></span>.</p>
</blockquote>
-<h4 data-number="2.2.1.36" id="option-inlinenotes"><span
-class="header-section-number">2.2.1.36</span> Option
+<h4 data-number="2.2.1.38" id="option-inlinenotes"><span
+class="header-section-number">2.2.1.38</span> Option
<code>inlineNotes</code></h4>
<dl>
<dt><code>inlineNotes</code> (default value: <code>false</code>)</dt>
@@ -4954,8 +5029,8 @@
<p>Here is an inline note.<a href="#fn2" class="footnote-ref"
id="fnref2" role="doc-noteref"><sup>2</sup></a></p>
</blockquote>
-<h4 data-number="2.2.1.37" id="option-jekylldata"><span
-class="header-section-number">2.2.1.37</span> Option
+<h4 data-number="2.2.1.39" id="option-jekylldata"><span
+class="header-section-number">2.2.1.39</span> Option
<code>jekyllData</code></h4>
<dl>
<dt><code>jekyllData</code> (default value: <code>false</code>)</dt>
@@ -5093,8 +5168,8 @@
<blockquote>
<p>Jane Doe is 99 years old.</p>
</blockquote>
-<h4 data-number="2.2.1.38" id="option-linkattributes"><span
-class="header-section-number">2.2.1.38</span> Option
+<h4 data-number="2.2.1.40" id="option-linkattributes"><span
+class="header-section-number">2.2.1.40</span> Option
<code>linkAttributes</code></h4>
<dl>
<dt><code>linkAttributes</code> (default value: <code>false</code>)</dt>
@@ -5165,8 +5240,8 @@
contain an example image (from <a href="https://ctan.org/pkg/mwe"
title="mwe – Packages and image files for MWEs">Martin Scharrer’s mwe
package</a>) displayed at size 5cm × 4cm.</p>
-<h4 data-number="2.2.1.39" id="option-lineblocks"><span
-class="header-section-number">2.2.1.39</span> Option
+<h4 data-number="2.2.1.41" id="option-lineblocks"><span
+class="header-section-number">2.2.1.41</span> Option
<code>lineBlocks</code></h4>
<dl>
<dt><code>lineBlocks</code> (default value: <code>false</code>)</dt>
@@ -5272,8 +5347,8 @@
I have spread my dreams under your feet;<br />
Tread softly because you tread on my dreams.</div>
</blockquote>
-<h4 data-number="2.2.1.40" id="option-mark"><span
-class="header-section-number">2.2.1.40</span> Option
+<h4 data-number="2.2.1.42" id="option-mark"><span
+class="header-section-number">2.2.1.42</span> Option
<code>mark</code></h4>
<dl>
<dt><code>mark</code> (default value: <code>false</code>)</dt>
@@ -5320,8 +5395,8 @@
<blockquote>
<p>This <mark>is highlighted text.</mark></p>
</blockquote>
-<h4 data-number="2.2.1.41" id="option-notes"><span
-class="header-section-number">2.2.1.41</span> Option
+<h4 data-number="2.2.1.43" id="option-notes"><span
+class="header-section-number">2.2.1.43</span> Option
<code>notes</code></h4>
<dl>
<dt><code>notes</code> (default value: <code>false</code>)</dt>
@@ -5444,8 +5519,8 @@
<p>This paragraph won’t be part of the note, because it isn’t
indented.</p>
</blockquote>
-<h4 data-number="2.2.1.42" id="pipe-tables"><span
-class="header-section-number">2.2.1.42</span> Option
+<h4 data-number="2.2.1.44" id="pipe-tables"><span
+class="header-section-number">2.2.1.44</span> Option
<code>pipeTables</code></h4>
<dl>
<dt><code>pipeTables</code> (default value: <code>false</code>)</dt>
@@ -5579,8 +5654,8 @@
</tbody>
</table>
</blockquote>
-<h4 data-number="2.2.1.43" id="option-preservetabs"><span
-class="header-section-number">2.2.1.43</span> Option
+<h4 data-number="2.2.1.45" id="option-preservetabs"><span
+class="header-section-number">2.2.1.45</span> Option
<code>preserveTabs</code></h4>
<dl>
<dt><code>preserveTabs</code> (default value: <code>true</code>)</dt>
@@ -5601,8 +5676,8 @@
</dl>
</dd>
</dl>
-<h4 data-number="2.2.1.44" id="option-rawattribute"><span
-class="header-section-number">2.2.1.44</span> Option
+<h4 data-number="2.2.1.46" id="option-rawattribute"><span
+class="header-section-number">2.2.1.46</span> Option
<code>rawAttribute</code></h4>
<dl>
<dt><code>rawAttribute</code> (default value: <code>false</code>)</dt>
@@ -5669,8 +5744,8 @@
<blockquote>
<p>H<sub>2</sub>O is a liquid.</p>
</blockquote>
-<h4 data-number="2.2.1.45" id="option-relativereferences"><span
-class="header-section-number">2.2.1.45</span> Option
+<h4 data-number="2.2.1.47" id="option-relativereferences"><span
+class="header-section-number">2.2.1.47</span> Option
<code>relativeReferences</code></h4>
<dl>
<dt><code>relativeReferences</code> (default value:
@@ -5734,8 +5809,8 @@
<p>In this paper, we have discovered that most grandmas would rather eat
dinner with their grandchildren than get eaten. Begone, wolf!</p>
</blockquote>
-<h4 data-number="2.2.1.46" id="option-shiftheadings"><span
-class="header-section-number">2.2.1.46</span> Option
+<h4 data-number="2.2.1.48" id="option-shiftheadings"><span
+class="header-section-number">2.2.1.48</span> Option
<code>shiftHeadings</code></h4>
<dl>
<dt><code>shiftHeadings</code> (default value: <code>0</code>)</dt>
@@ -5836,8 +5911,8 @@
<h2 id="a-section-7">A section</h2>
<h3 id="a-section-8">A section</h3>
</blockquote>
-<h4 data-number="2.2.1.47" id="option-slice"><span
-class="header-section-number">2.2.1.47</span> Option
+<h4 data-number="2.2.1.49" id="option-slice"><span
+class="header-section-number">2.2.1.49</span> Option
<code>slice</code></h4>
<dl>
<dt><code>slice</code> (default value: <code>^ $</code>)</dt>
@@ -6004,8 +6079,8 @@
<h2 id="act-v-2">Act V</h2>
<p>Hamlet dies.</p>
</blockquote>
-<h4 data-number="2.2.1.48" id="option-smartellipses"><span
-class="header-section-number">2.2.1.48</span> Option
+<h4 data-number="2.2.1.50" id="option-smartellipses"><span
+class="header-section-number">2.2.1.50</span> Option
<code>smartEllipses</code></h4>
<dl>
<dt><code>smartEllipses</code> (default value: <code>false</code>)</dt>
@@ -6188,8 +6263,8 @@
<p>These are just three regular dots …</p>
<p>. . . and this is a victorian ellipsis.</p>
</blockquote>
-<h4 data-number="2.2.1.49" id="option-startnumber"><span
-class="header-section-number">2.2.1.49</span> Option
+<h4 data-number="2.2.1.51" id="option-startnumber"><span
+class="header-section-number">2.2.1.51</span> Option
<code>startNumber</code></h4>
<dl>
<dt><code>startNumber</code> (default value: <code>true</code>)</dt>
@@ -6307,8 +6382,8 @@
<li>fifth item</li>
</ol>
</blockquote>
-<h4 data-number="2.2.1.50" id="option-strikethrough"><span
-class="header-section-number">2.2.1.50</span> Option
+<h4 data-number="2.2.1.52" id="option-strikethrough"><span
+class="header-section-number">2.2.1.52</span> Option
<code>strikeThrough</code></h4>
<dl>
<dt><code>strikeThrough</code> (default value: <code>false</code>)</dt>
@@ -6401,8 +6476,8 @@
<blockquote>
<p>This is <del>a lunar roving vehicle</del> strike-through text.</p>
</blockquote>
-<h4 data-number="2.2.1.51" id="option-stripindent"><span
-class="header-section-number">2.2.1.51</span> Option
+<h4 data-number="2.2.1.53" id="option-stripindent"><span
+class="header-section-number">2.2.1.53</span> Option
<code>stripIndent</code></h4>
<dl>
<dt><code>stripIndent</code> (default value: <code>false</code>)</dt>
@@ -6484,8 +6559,8 @@
class="sourceCode sh"><code class="sourceCode bash"><span id="cb380-1"><a href="#cb380-1" aria-hidden="true" tabindex="-1"></a><span class="ex">context</span> <span class="at">--luatex</span> document.tex</span></code></pre></div>
<p>A PDF document named <code>document.pdf</code> should be produced and
contain the text “Hello <em>world</em>!”</p>
-<h4 data-number="2.2.1.52" id="option-subscripts"><span
-class="header-section-number">2.2.1.52</span> Option
+<h4 data-number="2.2.1.54" id="option-subscripts"><span
+class="header-section-number">2.2.1.54</span> Option
<code>subscripts</code></h4>
<dl>
<dt><code>subscripts</code> (default value: <code>false</code>)</dt>
@@ -6550,8 +6625,8 @@
<blockquote>
<p>H<sub>2</sub>O is a liquid.</p>
</blockquote>
-<h4 data-number="2.2.1.53" id="option-superscripts"><span
-class="header-section-number">2.2.1.53</span> Option
+<h4 data-number="2.2.1.55" id="option-superscripts"><span
+class="header-section-number">2.2.1.55</span> Option
<code>superscripts</code></h4>
<dl>
<dt><code>superscripts</code> (default value: <code>false</code>)</dt>
@@ -6616,8 +6691,8 @@
<blockquote>
<p>2<sup>10</sup> is 1024.</p>
</blockquote>
-<h4 data-number="2.2.1.54" id="option-tableattributes"><span
-class="header-section-number">2.2.1.54</span> Option
+<h4 data-number="2.2.1.56" id="option-tableattributes"><span
+class="header-section-number">2.2.1.56</span> Option
<code>tableAttributes</code></h4>
<dl>
<dt><code>tableAttributes</code> (default value:
@@ -6739,8 +6814,8 @@
</tbody>
</table>
</blockquote>
-<h4 data-number="2.2.1.55" id="table-captions"><span
-class="header-section-number">2.2.1.55</span> Option
+<h4 data-number="2.2.1.57" id="table-captions"><span
+class="header-section-number">2.2.1.57</span> Option
<code>tableCaptions</code></h4>
<dl>
<dt><code>tableCaptions</code> (default value: <code>false</code>)</dt>
@@ -6888,8 +6963,8 @@
</tbody>
</table>
</blockquote>
-<h4 data-number="2.2.1.56" id="option-tasklists"><span
-class="header-section-number">2.2.1.56</span> Option
+<h4 data-number="2.2.1.58" id="option-tasklists"><span
+class="header-section-number">2.2.1.58</span> Option
<code>taskLists</code></h4>
<dl>
<dt><code>taskLists</code> (default value: <code>false</code>)</dt>
@@ -6978,8 +7053,8 @@
<li>Yes I can!</li>
</ul>
</blockquote>
-<h4 data-number="2.2.1.57" id="option-texcomments"><span
-class="header-section-number">2.2.1.57</span> Option
+<h4 data-number="2.2.1.59" id="option-texcomments"><span
+class="header-section-number">2.2.1.59</span> Option
<code>texComments</code></h4>
<dl>
<dt><code>texComments</code> (default value: <code>false</code>)</dt>
@@ -7065,8 +7140,8 @@
class="sourceCode sh"><code class="sourceCode bash"><span id="cb410-1"><a href="#cb410-1" aria-hidden="true" tabindex="-1"></a><span class="ex">context</span> <span class="at">--luatex</span> document.tex</span></code></pre></div>
<p>A PDF document named <code>document.pdf</code> should be produced and
contain the text “Hello <em>world</em>!”</p>
-<h4 data-number="2.2.1.58" id="option-texmathdollars"><span
-class="header-section-number">2.2.1.58</span> Option
+<h4 data-number="2.2.1.60" id="option-texmathdollars"><span
+class="header-section-number">2.2.1.60</span> Option
<code>texMathDollars</code></h4>
<dl>
<dt><code>texMathDollars</code> (default value: <code>false</code>)</dt>
@@ -7249,8 +7324,8 @@
<p><span
class="math display"><em>f̂</em>(<em>ξ</em>) = ∫<sub>−∞</sub><sup>∞</sup><em>f</em>(<em>x</em>)<em>e</em><sup>−<em>i</em>2<em>π</em><em>ξ</em><em>x</em></sup><em>d</em><em>x</em></span></p>
</blockquote>
-<h4 data-number="2.2.1.59" id="option-texmathdoublebackslash"><span
-class="header-section-number">2.2.1.59</span> Option
+<h4 data-number="2.2.1.61" id="option-texmathdoublebackslash"><span
+class="header-section-number">2.2.1.61</span> Option
<code>texMathDoubleBackslash</code></h4>
<dl>
<dt><code>texMathDoubleBackslash</code> (default value:
@@ -7435,8 +7510,8 @@
<p><span
class="math display"><em>f̂</em>(<em>ξ</em>) = ∫<sub>−∞</sub><sup>∞</sup><em>f</em>(<em>x</em>)<em>e</em><sup>−<em>i</em>2<em>π</em><em>ξ</em><em>x</em></sup><em>d</em><em>x</em></span></p>
</blockquote>
-<h4 data-number="2.2.1.60" id="option-texmathsinglebackslash"><span
-class="header-section-number">2.2.1.60</span> Option
+<h4 data-number="2.2.1.62" id="option-texmathsinglebackslash"><span
+class="header-section-number">2.2.1.62</span> Option
<code>texMathSingleBackslash</code></h4>
<dl>
<dt><code>texMathSingleBackslash</code> (default value:
@@ -7620,8 +7695,8 @@
<p><span
class="math display"><em>f̂</em>(<em>ξ</em>) = ∫<sub>−∞</sub><sup>∞</sup><em>f</em>(<em>x</em>)<em>e</em><sup>−<em>i</em>2<em>π</em><em>ξ</em><em>x</em></sup><em>d</em><em>x</em></span></p>
</blockquote>
-<h4 data-number="2.2.1.61" id="option-tightlists"><span
-class="header-section-number">2.2.1.61</span> Option
+<h4 data-number="2.2.1.63" id="option-tightlists"><span
+class="header-section-number">2.2.1.63</span> Option
<code>tightLists</code></h4>
<dl>
<dt><code>tightLists</code> (default value: <code>true</code>)</dt>
@@ -7717,8 +7792,8 @@
<li><p>third item</p></li>
</ul>
</blockquote>
-<h4 data-number="2.2.1.62" id="option-underscores"><span
-class="header-section-number">2.2.1.62</span> Option
+<h4 data-number="2.2.1.64" id="option-underscores"><span
+class="header-section-number">2.2.1.64</span> Option
<code>underscores</code></h4>
<dl>
<dt><code>underscores</code> (default value: <code>true</code>)</dt>
Modified: trunk/Master/texmf-dist/doc/generic/markdown/markdown.pdf
===================================================================
(Binary files differ)
Modified: trunk/Master/texmf-dist/scripts/markdown/markdown-cli.lua
===================================================================
--- trunk/Master/texmf-dist/scripts/markdown/markdown-cli.lua 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/scripts/markdown/markdown-cli.lua 2024-05-28 19:27:46 UTC (rev 71378)
@@ -58,7 +58,7 @@
-- those in the standard .ins files.
--
local metadata = {
- version = "3.5.0-0-gfd01a252",
+ version = "3.6.0-0-g83c781b4",
comment = "A module for the conversion from markdown to plain TeX",
author = "John MacFarlane, Hans Hagen, Vít Starý Novotný",
copyright = {"2009-2016 John MacFarlane, Hans Hagen",
@@ -69,6 +69,8 @@
local defaultOptions = {}
defaultOptions.eagerCache = false
defaultOptions.singletonCache = true
+defaultOptions.unicodeNormalization = true
+defaultOptions.unicodeNormalizationForm = "nfc"
defaultOptions.cacheDir = "."
defaultOptions.contentBlocksLanguageMap = "markdown-languages.json"
defaultOptions.debugExtensionsFileName = "debug-extensions.json"
Modified: trunk/Master/texmf-dist/source/generic/markdown/markdown.dtx
===================================================================
--- trunk/Master/texmf-dist/source/generic/markdown/markdown.dtx 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/source/generic/markdown/markdown.dtx 2024-05-28 19:27:46 UTC (rev 71378)
@@ -2574,9 +2574,110 @@
% \iffalse
%</lua>
%<*manual-options>
+
+#### Option `unicodeNormalization`
+
+`unicodeNormalization` (default value: `true`)
+
% \fi
% \begin{markdown}
%
+% \Optitem[true]{unicodeNormalization}{\opt{true}, \opt{false}}
+%
+: true
+
+ : Markdown documents will be normalized using one of the four [Unicode
+ normalization forms][unicode-normalization] before conversion. The
+ Unicode normalization norm used is determined by option
+ \Opt{unicodeNormalizationForm}.
+
+: false
+
+ : Markdown documents will not be Unicode-normalized before conversion.
+
+ [unicode-normalization]: https://unicode.org/faq/normalization.html
+
+% \end{markdown}
+% \iffalse
+%</manual-options>
+%<*tex>
+% \fi
+% \begin{macrocode}
+\@@_add_lua_option:nnn
+ { unicodeNormalization }
+ { boolean }
+ { true }
+% \end{macrocode}
+% \iffalse
+%</tex>
+%<*lua,lua-cli>
+% \fi
+% \begin{macrocode}
+defaultOptions.unicodeNormalization = true
+% \end{macrocode}
+% \par
+% \iffalse
+%</lua,lua-cli>
+%<*manual-options>
+
+#### Option `unicodeNormalizationForm`
+
+`unicodeNormalizationForm` (default value: `nfc`)
+
+% \fi
+% \begin{markdown}
+%
+% \Optitem[nfc]{unicodeNormalizationForm}{\opt{nfc}, \opt{nfd}, \opt{nfkc}, \opt{nfkd}}
+%
+: nfc
+
+ : When option \Opt{unicodeNormalization} has been enabled, markdown documents
+ will be normalized using Unicode Normalization Form C (NFC) before
+ conversion.
+
+: nfd
+
+ : When option \Opt{unicodeNormalization} has been enabled, markdown documents
+ will be normalized using Unicode Normalization Form D (NFD) before
+ conversion.
+
+: nfkc
+
+ : When option \Opt{unicodeNormalization} has been enabled, markdown documents
+ will be normalized using Unicode Normalization Form KC (NFKC) before
+ conversion.
+
+: nfkd
+
+ : When option \Opt{unicodeNormalization} has been enabled, markdown documents
+ will be normalized using Unicode Normalization Form KD (NFKD) before
+ conversion.
+
+% \end{markdown}
+% \iffalse
+%</manual-options>
+%<*tex>
+% \fi
+% \begin{macrocode}
+\@@_add_lua_option:nnn
+ { unicodeNormalizationForm }
+ { string }
+ { nfc }
+% \end{macrocode}
+% \iffalse
+%</tex>
+%<*lua,lua-cli>
+% \fi
+% \begin{macrocode}
+defaultOptions.unicodeNormalizationForm = "nfc"
+% \end{macrocode}
+% \par
+% \iffalse
+%</lua,lua-cli>
+%<*manual-options>
+% \fi
+% \begin{markdown}
+%
%### File and Directory Names
%
% \end{markdown}
@@ -25705,8 +25806,12 @@
local buffer = {}
local prev_space = false
local letter_found = false
+ local normalized_s = s
+ if not options.unicodeNormalization or options.unicodeNormalizationForm ~= "nfc" then
+ normalized_s = uni_algos.normalize.NFC(normalized_s)
+ end
- for _, code in utf8.codes(uni_algos.normalize.NFC(s)) do
+ for _, code in utf8.codes(normalized_s) do
local char = utf8.char(code)
-- Remove everything up to the first letter.
@@ -25761,8 +25866,12 @@
local buffer = {}
local prev_space = false
local letter_found = false
+ local normalized_s = s
+ if not options.unicodeNormalization or options.unicodeNormalizationForm ~= "nfc" then
+ normalized_s = uni_algos.normalize.NFC(normalized_s)
+ end
- for _, code in utf8.codes(uni_algos.normalize.NFC(s)) do
+ for _, code in utf8.codes(normalized_s) do
local char = utf8.char(code)
-- Remove everything up to the first non-space.
@@ -29916,6 +30025,25 @@
return function(input)
% \end{macrocode}
% \begin{markdown}
+% Unicode-normalize the input.
+% \end{markdown}
+% \begin{macrocode}
+ if options.unicodeNormalization then
+ local form = options.unicodeNormalizationForm
+ if form == "nfc" then
+ input = uni_algos.normalize.NFC(input)
+ elseif form == "nfd" then
+ input = uni_algos.normalize.NFD(input)
+ elseif form == "nfkc" then
+ input = uni_algos.normalize.NFKC(input)
+ elseif form == "nfkd" then
+ input = uni_algos.normalize.NFKD(input)
+ else
+ error(format("Unknown normalization form %s", form))
+ end
+ end
+% \end{macrocode}
+% \begin{markdown}
% Since the Lua converter expects \acro{unix} line endings, normalize the
% input. Also add a line ending at the end of the file in case the input file
% has none.
@@ -32359,13 +32487,38 @@
for k, _ in pairs(d) do
table.insert(keys, k)
end
- table.sort(keys)
+% \end{macrocode}
+% \begin{markdown}
+%
+% For reproducibility, sort the keys. For mixed string-and-numeric keys, sort
+% numeric keys before string keys.
+%
+% \end{markdown}
+% \begin{macrocode}
+ table.sort(keys, function(first, second)
+ if type(first) ~= type(second) then
+ return type(first) < type(second)
+ else
+ return first < second
+ end
+ end)
if not p then
table.insert(buf, "\\markdownRendererJekyllDataBegin")
end
- if #d > 0 then
+ local is_sequence = false
+ if #d > 0 and #d == #keys then
+ for i=1, #d do
+ if d[i] == nil then
+ goto not_a_sequence
+ end
+ end
+ is_sequence = true
+ end
+ ::not_a_sequence::
+
+ if is_sequence then
table.insert(buf, "\\markdownRendererJekyllDataSequenceBegin{")
table.insert(buf, self.identifier(p or "null"))
table.insert(buf, "}{")
@@ -32420,7 +32573,7 @@
end
end
- if #d > 0 then
+ if is_sequence then
table.insert(buf, "\\markdownRendererJekyllDataSequenceEnd")
else
table.insert(buf, "\\markdownRendererJekyllDataMappingEnd")
@@ -34065,6 +34218,42 @@
%
% \end{markdown}
% \begin{macrocode}
+\ExplSyntaxOn
+\cs_new:Npn
+ \markdownInput
+ #1
+ {
+% \end{macrocode}
+% \begin{markdown}
+%
+% If the file does not exist in the current directory, we will search for it in
+% the directories specified in \mref{l_file_search_path_seq}. On \LaTeX, this
+% also includes the directories specified in \mref{input at path}.
+%
+% \end{markdown}
+% \begin{macrocode}
+ \file_get_full_name:nNTF
+ { #1 }
+ \l_tmpa_tl
+ {
+ \exp_args:NV
+ \markdownInputRaw
+ \l_tmpa_tl
+ }
+ {
+ \msg_error:nnnV
+ { markdown }
+ { markdown-file-does-not-exist }
+ { #1 }
+ }
+ }
+\msg_new:nnn
+ { markdown }
+ { markdown-file-does-not-exist }
+ {
+ Markdown~file~#1~does~not~exist
+ }
+\ExplSyntaxOff
\begingroup
% \end{macrocode}
% \begin{markdown}
@@ -34076,7 +34265,7 @@
\catcode`|=0%
\catcode`\\=12%
\catcode`|&=6%
- |gdef|markdownInput#1{%
+ |gdef|markdownInputRaw#1{%
% \end{macrocode}
% \begin{markdown}
% Change the category code of the percent sign (`\%`) to other, so that a user
Modified: trunk/Master/texmf-dist/tex/generic/markdown/markdown.tex
===================================================================
--- trunk/Master/texmf-dist/tex/generic/markdown/markdown.tex 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/tex/generic/markdown/markdown.tex 2024-05-28 19:27:46 UTC (rev 71378)
@@ -329,6 +329,14 @@
{ boolean }
{ true }
\__markdown_add_lua_option:nnn
+ { unicodeNormalization }
+ { boolean }
+ { true }
+\__markdown_add_lua_option:nnn
+ { unicodeNormalizationForm }
+ { string }
+ { nfc }
+\__markdown_add_lua_option:nnn
{ cacheDir }
{ path }
{ \markdownOptionOutputDir / _markdown_\jobname }
@@ -572,8 +580,8 @@
{ boolean }
{ true }
\ExplSyntaxOff
-\def\markdownLastModified{2024-04-29}%
-\def\markdownVersion{3.5.0-0-gfd01a252}%
+\def\markdownLastModified{2024-05-27}%
+\def\markdownVersion{3.6.0-0-g83c781b4}%
\let\markdownBegin\relax
\let\markdownEnd\relax
\let\markinline\relax
@@ -3403,12 +3411,37 @@
{ markinline-peek-failure }
{ Use~of~\iow_char:N \\ markinline~doesn't~match~its~definition }
{ The~macro~should~be~followed~by~inline~markdown~text~in~curly~braces }
+\cs_new:Npn
+ \markdownInput
+ #1
+ {
+ \file_get_full_name:nNTF
+ { #1 }
+ \l_tmpa_tl
+ {
+ \exp_args:NV
+ \markdownInputRaw
+ \l_tmpa_tl
+ }
+ {
+ \msg_error:nnnV
+ { markdown }
+ { markdown-file-does-not-exist }
+ { #1 }
+ }
+ }
+\msg_new:nnn
+ { markdown }
+ { markdown-file-does-not-exist }
+ {
+ Markdown~file~#1~does~not~exist
+ }
\ExplSyntaxOff
\begingroup
\catcode`|=0%
\catcode`\\=12%
\catcode`|&=6%
- |gdef|markdownInput#1{%
+ |gdef|markdownInputRaw#1{%
|begingroup
|catcode`|%=12
|catcode`|#=12
Modified: trunk/Master/texmf-dist/tex/luatex/markdown/markdown.lua
===================================================================
--- trunk/Master/texmf-dist/tex/luatex/markdown/markdown.lua 2024-05-28 19:27:35 UTC (rev 71377)
+++ trunk/Master/texmf-dist/tex/luatex/markdown/markdown.lua 2024-05-28 19:27:46 UTC (rev 71378)
@@ -58,7 +58,7 @@
-- those in the standard .ins files.
--
local metadata = {
- version = "3.5.0-0-gfd01a252",
+ version = "3.6.0-0-g83c781b4",
comment = "A module for the conversion from markdown to plain TeX",
author = "John MacFarlane, Hans Hagen, Vít Starý Novotný",
copyright = {"2009-2016 John MacFarlane, Hans Hagen",
@@ -120,6 +120,8 @@
convert = nil,
options = nil,
}
+defaultOptions.unicodeNormalization = true
+defaultOptions.unicodeNormalizationForm = "nfc"
defaultOptions.cacheDir = "."
defaultOptions.contentBlocksLanguageMap = "markdown-languages.json"
defaultOptions.debugExtensionsFileName = "debug-extensions.json"
@@ -2971,8 +2973,12 @@
local buffer = {}
local prev_space = false
local letter_found = false
+ local normalized_s = s
+ if not options.unicodeNormalization or options.unicodeNormalizationForm ~= "nfc" then
+ normalized_s = uni_algos.normalize.NFC(normalized_s)
+ end
- for _, code in utf8.codes(uni_algos.normalize.NFC(s)) do
+ for _, code in utf8.codes(normalized_s) do
local char = utf8.char(code)
-- Remove everything up to the first letter.
@@ -3020,8 +3026,12 @@
local buffer = {}
local prev_space = false
local letter_found = false
+ local normalized_s = s
+ if not options.unicodeNormalization or options.unicodeNormalizationForm ~= "nfc" then
+ normalized_s = uni_algos.normalize.NFC(normalized_s)
+ end
- for _, code in utf8.codes(uni_algos.normalize.NFC(s)) do
+ for _, code in utf8.codes(normalized_s) do
local char = utf8.char(code)
-- Remove everything up to the first non-space.
@@ -6164,6 +6174,20 @@
inlines_no_link_or_emphasis_t.EndlineExceptions = parsers.EndlineExceptions - parsers.eof
parsers.inlines_no_link_or_emphasis = Ct(inlines_no_link_or_emphasis_t)
return function(input)
+ if options.unicodeNormalization then
+ local form = options.unicodeNormalizationForm
+ if form == "nfc" then
+ input = uni_algos.normalize.NFC(input)
+ elseif form == "nfd" then
+ input = uni_algos.normalize.NFD(input)
+ elseif form == "nfkc" then
+ input = uni_algos.normalize.NFKC(input)
+ elseif form == "nfkd" then
+ input = uni_algos.normalize.NFKD(input)
+ else
+ error(format("Unknown normalization form %s", form))
+ end
+ end
input = input:gsub("\r\n?", "\n")
if input:sub(-1) ~= "\n" then
input = input .. "\n"
@@ -8004,13 +8028,30 @@
for k, _ in pairs(d) do
table.insert(keys, k)
end
- table.sort(keys)
+ table.sort(keys, function(first, second)
+ if type(first) ~= type(second) then
+ return type(first) < type(second)
+ else
+ return first < second
+ end
+ end)
if not p then
table.insert(buf, "\\markdownRendererJekyllDataBegin")
end
- if #d > 0 then
+ local is_sequence = false
+ if #d > 0 and #d == #keys then
+ for i=1, #d do
+ if d[i] == nil then
+ goto not_a_sequence
+ end
+ end
+ is_sequence = true
+ end
+ ::not_a_sequence::
+
+ if is_sequence then
table.insert(buf, "\\markdownRendererJekyllDataSequenceBegin{")
table.insert(buf, self.identifier(p or "null"))
table.insert(buf, "}{")
@@ -8065,7 +8106,7 @@
end
end
- if #d > 0 then
+ if is_sequence then
table.insert(buf, "\\markdownRendererJekyllDataSequenceEnd")
else
table.insert(buf, "\\markdownRendererJekyllDataMappingEnd")
More information about the tex-live-commits
mailing list.