<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'><br><br><div><br><br><br>> <br>> Message: 4<br>> Date: Fri, 10 Oct 2014 08:54:04 +0700<br>> From: Nathan Wells <sungkhum@gmail.com><br>> To: "About TeX hyphenation patterns." <tex-hyphen@tug.org><br>> Cc: Unicode-based TeX for Mac OS X and other platforms <xetex@tug.org><br>> Subject: Re: [tex-hyphen] Help with UTF-8 Language<br>> Message-ID:<br>> <CAFSe7HTPaagZyr4OcP5Bn3davp+MW30_qi-TmK70XkfoPN0HbQ@mail.gmail.com><br>> Content-Type: text/plain; charset="utf-8"<br>> <br>> Thank you all for your replies!<br>> My programming abilities are quite limited and I realize there aren't many<br>> people who need to make hyphenation dictionaries, hence the lack of good<br>> Unicode support. But would someone be willing to help with a little more<br>> step-by-step help? I am a little confused as how best to map the Khmer<br>> Unicode characters to 8-bit values.<br>> I think it would be quite useful to post a tutorial of the process once I<br>> am done so others can more easily create hyphenation dictionaries for<br>> languages that don't have them yet (I have yet to find a good tutorial<br>> anywhere).<br>> Thanks again for your help,<br>> Nathan<br>Hi Nathan<br><br>step 1.<br>First you need word database in your language.<br>I wrote small program for my case which accepts text file(it can be text with mixed scripts) and<br>gets from there words in some "Lang" sorts them and outputs in file.<br>another code merges this wordlists.<br>finally you need something like this:<br><br>Aggressive<br>Animal<br>Alphabet<br>Dosimeter<br>Guard<br><br>if you can make such list in other way thats fine.<br><br>Step 2<br>after this you need to know hyphenation rules.<br>This can be different from language to language<br>In example for my case i can hyphenate word on after vowel,<br>if there are two or more consonants after vowel one stays on same line others<br>go on next line, but there are some consonant pairs which can not be splitted.<br><br>After doing this with your wordlist you get something like this:<br>splitted_word_list.txt<br>Ag-gres-si-ve<br>A-ni-mal<br>Al-pha-bet<br>Do-si-me-ter<br>Gu-ard<br><br>Step 3.<br>after this comes patgen and you pass splitted_word_list.txt as dictionary file<br>for 'hyph_start' and 'hyph_finish' left hypmin righthypmin you can use 3*N. "3" because<br>Khmer is 3 bytes long. Using this trick i made patgen to work with utf-8. <br><br>I used wordlist from step 1 and generated patterns from step 3 to test hyphenation using hyph-utf8 and luatex and<br>compared it to splitted wordlist from step 2.<br><br>For step 1-2 i have wrote program which does all work. Unfortunately script codes(Language script detection, hyphenation rules, vowels) are "hardwired" in code.<br><br>I can send you codes and you can modify them or send me textfiles with your language /text can me mixed with some other languages or with html murkup, but not word files please :) / , vowels list /unicode codes/ and hyphenation rule set.<br>I'll try modify my code in way program can accept: script_code_ranges, vowels_set, consonant_pairs_which_cannotbesplited<br><br></div> </div></body>
</html>