<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
On 6/15/2011 11:44 AM, Gerrit wrote:
<blockquote cite="mid:4DF8FD82.3050400@gmx.de" type="cite">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
Hello again, everyone,<br>
<br>
I am currently writing an article, in which I also have some
romanization of Japanese. Until now, I have to define the
hyphenation manually, which I think is a little bit of a nuisance.<br>
<br>
[snip]<br>
<br>
What do you think about that?<br>
</blockquote>
<br>
Since phonetic guide texts for CJKV are tied to characters, I would
consider the most logical split one where the guide text is dictated
by the character boundaries, and the language used. Hyphenation for
guide text would be strongly tied to the original text splits, as
pronunciation guide text does not significantly run past the
character boundary (more creative uses of top text such as the
common Japanese practice of treating it as a 'thinking space', using
the real text to express what is said and the guide text what is
thought wouldn't be convered by this of course. Nor should they,
probably).<br>
<br>
To my knowledge, this is already automatically the case for
(Mandarin) Chinese, as every character only has a single syllable
pronunciation, so hyphenation is unlikely to even matter; whether
it's romanised or bopomofo, the guide text won't run past the
character.<br>
<br>
For Japanese this is also true for the most part, with a very small
number of special words that consist of multiple characters that
only have a single syllable pronunciation (like 所為, romanised as
"sei", which cannot be decomposed as [se]-[i]. In Japanese the
furigana for this is never split up over multiple lines either).
Aside from these words, there are some "ateji" readings for words,
where some originally character-less word has been assigned a set of
characters that do not normally "spell" that word. For these, you
would also need special hyphenation rules. However, the vast
majority of Japanese words follow the rules of compositional
reading, so 天国(tengoku) would split up as 天(ten-)//国(-goku) and
腹切り(harakiri) would split up as 腹(hara-)//切り(-kiri), with optional
guide text over the syllable り(ri) depending on the target audience.<br>
<br>
I do not know about character guide texts in other Asian languages
that borrowed Chinese characters.<br>
<br>
The main challenge would be to build the "which character maps to
which reading in which word" dataset, which will be quite vast. For
western languages grammars can be constructed that fairly accurately
describe when a word would be allowed to split, based on its written
form. For CJK languages that approach goes straight out the window,
because you can split anywhere in a sentence. This means that there
is no concept of "hyphenation", and it will only apply to western
guide text, which for chinese character words requires knowing the
pronunciation of these words (or taking a really good guess and
allowing the author to override guesses). Particularly for Chinese
and Japanese this leads to huge datasets; the first because even
though most characters are complete words, and typically only have
one pronunciation, there are easily ten thousand characters in daily
use (although of course not all as frequent), the second because
even though there are fewer characters to contend with in Japanese,
some 3500, the actual pronunciations depend on the words characters
are used in, and unlike Chinese most Japanese words are actually
compound character words, still leaving you with over ten thousands
distinct combinations for which you can't really abstract
pronunciation rules because most characters in Japanese have three
or four readings (at least). To get automate hyphenation right, you
first need to tackle automatic guessing of pronunciation (even
lexical analysers for Japanese like MeCab, ChaSen or YamCha can't
get around this) and you'll end up with quite a few MB of data just
to hyphenate guide text, and then only when it's western guide text.<br>
<br>
That's not to discourage anyone from taking a stab at it, it's just
quite a mountain of work.<br>
<br>
- Mike "Pomax" Kamermans<br>
nihongoresources.com<br>
</body>
</html>