[tex-hyphen] Unicode patterns for Unicode in Pdflatex/Babel?

[Dominik Wujastyk]

Hi, there are several good compliant unicode UTF8 Devanagari fonts available
nowadays, e.g., Nakula,
Sahadeva<http://bombay.indology.info/software/fonts/index.html>,
and even Sanskrit
2003<http://salrc.uchicago.edu/resources/fonts/available/sanskrit/sanskrit2003.shtml>,
with 1000 ligatures, which looks very nice to those used to reading books
published by the legendary Nirnayasagara Press in Bombay.

Typing Devanagari input is a local matter; I find iBus on Linux works very
well, and already comes with a convenient keyboard entry setup.  However, it
is trivially easy to set up one's own keystrokes using iBus, so that's a
non-issue, really.  There are good keyboard handlers for Windows too, like
Tavultesoft that some of my colleagues are happy with.  Mac, I don't know.

There's another important issue here, though.  With most languages, there is
a one-to-one correspondence between script and language.  Russian uses
Cyrillic script, Greek uses Greek script, Arabic the Arabic script, English
the Latin script.  There are some variations here and there, e.g., Greek is
sometimes written in Latin script in scholarly publications, for
convenience.  Other languages also use Cyrillic and Latin scripts, etc. etc.

With Sanskrit, this goes further.  Historically, Sanskrit has always been
written down phonetically in the local script, whatever that happened to
be.  There are between ten and twenty major script areas in South Asia.
Even today, books in Sanskrit published in Mysore may be printed in the
Kannada script, not in Devanagari.  Almost all books in Sanskrit published
in Calcutta - a major publishing centre - are printed in the Bengali
script.  And so on.  So Devanagari, while widespread, is by no means the
only script used for writing or printing Sanskrit.

Because of the cultural looseness of the language-script link in S. Asia,
European scholars since the nineteenth century have felt particularly free
to use Latin script for Sanskrit, Pali, and related languages.  The
transliteration scheme was settled at an international congress in 1898, and
has remained more or less stable since then.  This situation is quite
different from the case for Greek, Arabic or Chinese, where Latin
transcription has never been widely accepted, and is subject to competing,
incompatible standards.

Final point.  The rules for hyphenating Sanskrit written in Indian scripts
was never explicitly articulated within the tradition itself.  Unspoken
usages from manuscript scribes was carried over into nineteenth-century
printing, where - broadly speaking - hyphenation breaks were permitted after
any vowel.  But Europeans working in Latin script transliteration have
developed hyphenation usages that follow word etymology.  For example, like
German, Sanskrit makes many compound words.  So, the words in a compound
make natural hyphenation points, like "Gaṅgā-tīre," "on the bank (tīre) of
the Ganges".  Within words, hyphenation points offer themselves at
grammatically meaningful points.  So "bhavati" "he is" would be
"bhav-a-ti".

The existing hyphenation points for Sanskrit,
hyph-sa.tex<http://ctan.org/search/?search=hyph-sa.tex&search_type=filename>,
offer both Devanagari and Latin patterns, and several other SA scripts.
This is great, and the Right Approach.  The Devanagari ones are more or less
okay, though But the Latin patterns especially need development, just
because of the European scholarly hyphenation habits are more demanding.
I've started a list of explicitly hyphenated Skt words in Latin script, for
eventually processing through Patgen, but it's just a beginning.

By the way, gloss-sanskrit.ldf doesn't have a \def\fontsetup at sanskrit@Latin,
which it should have really.

Best,
Dominik

Dr Dominik Wujastyk
Institut für Südasien-, Tibet- und Buddhismuskunde
Universität Wien
Spitalgasse 2-4, Hof 2, Eingang 2.1
A-1090 Vienna
Austria
Project: http://www.istb.univie.ac.at/caraka/
--
Long-term email address: wujastyk at gmail.com
PGP key: http://wujastyk.net/pgp.html
--
Sign-up to Dropbox using the following link and get 2.25 gigabytes of
storage free: https://www.dropbox.com/referrals/NTIzNzI2MTY5



On 12 June 2011 23:25, Mojca Miklavec <mojca.miklavec.lists at gmail.com>wrote:

> On Sun, Jun 12, 2011 at 20:38, Arthur Reutenauer wrote:
> >    Hi Daniel,
> >
> >> I am trying to make myself clear: I would like to know if it wouldn't be
> possible to employ a custom
> >> Unicode hyphenation rules/pattern file also for Pdflatex/Babel when the
> text there is Unicode, too
>
> As Arthur pointed out, you cannot simply load hyph-foo.tex file with
> UTF-8 patterns without loading other macros to handle UTF-8 *before*
> patterns themselves. (You may try and see what happens.)
>
> But we tried to make sure that all patterns that can reasonably work
> in pdfTeX do work. Indic languages, Sanskrit, "Ethiopic", Lao etc.
> were disabled in pdfTeX because we had no idea how to handle them.
>
> >  However, I'm going to venture a wild guess and assume that the language
> you're interested in is Sanskrit, a language which actually has patterns
> disabled for pdfTeX, because we couldn't determine what font encoding was
> appropriate when the patterns were submitted: for the vast majority of
> languages that had patterns when Mojca and I took over work on hyphenation
> files three years ago, there was one single 8-bit encoding, that was used by
> both the pattern file and the Babel support files.  Several languages,
> though, have been added in the mean time, including Sanskrit, that had no
> dedicated 8-bit encoding that we could use(*).  We thus decided to make them
> available for Unicode-aware TeX engines only; hence, you don't have access
> to them from pdfTeX.  But if you have a reason to want to use them, we'll
> gladly make them available as well.  That won't be a problem at all; we only
> never considered the issue because we didn't think it would come up --
> Mojca, what do you t!
>  hink?
>
> I would really like to know what language Daniel is referring to
> before any further discussion.
>
> If it is really about Sanskrit ... I have no idea how one can type it
> in UTF-8 in pdfTeX. All the examples about Devanagari that I found
> were using ASCII and were pointing to Velthuis' package claiming:
>
> % Hyphenation
> % ~~~~~~~~~~~
> % The responsibility for hyphenating Devanagari text is taken over
> % completely by the preprocessor, devnag.c. The preprocessor inserts
> % discretionary hyphenation points (\-) in all the places it thinks are
> % appropriate.
>
> or whatever other older package.
>
> I would need more input to provide any reasonable answer.
>
> Mojca
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/tex-hyphen/attachments/20110613/94eb3aaf/attachment-0001.html>