[pdftex] Extending pdftex

[Hans Hagen]

At 01:46 01/07/2003 +0200, Andreas Matthias wrote:

>And since Hans said today `there is no real need to add functionality',
>I know why you ignored me. But why didn't you tell that before? If
>specially features are not desired, why don't you tell us? If I knew
>that before I wouldn't be working on that extension. But since it
>always seems that pdftex development is just lacking volunteers I
>gave it a try. Now I know better. I still don't know what is
>going wrong in pdftex development, but it is definitely not a lack of
>volunteers. Or do you think it is correct to just ignore volunteers?

although i greatly dislike the way you let this discussion evolve, i'll try 
to reply

there are several kind of extensions:

(1) turning typeset text into pdf
(2) providing pdf (viewer) specific functionality
(3) adding pdf file manipulation features

Concerning (1): the possible extensions in this area (unicode and other 
fonts, multi-main-stream, etc) demand extensions to the tex engine and such 
extensions take place in a controlled way (thanh doing a thesis, etex, 
omega, mltex/enctex etc); since pdfetex is currently the main stream tex 
binary, such extensions are normally discussed in detail (for instance, 
during the last dante meeting language deoendent ligature extensions came 
up in the process of the development of the latin modern font family); for 
instance adding support for structured pdf would demands extensions to tex 
and much discussion beforehand since macropackages that would support it 
differ in their approach; it would also be quite unstable and experimental 
for a while and mess up the already not that readable codebase of pdftex.

(one reason for removing encryption for instance was that it crept in all 
over the code, did some parsing on object content, lacked control over 
specific object/literal situations, and afaik, was not actively maintained 
once there; also: one can do such things quite well with other software)

Just to mention a few low level extensions on the list: access to the page 
object number (needed for certain pdf structuring features and certain 
hyper features, but kind of tricky since t he number of pages in often 
unknown), access to the trailer (needed for pdf/x), etc.

Developments in this area not always take place in a large audience; i 
still remember the moment when cooltype came around and some pdf docs were 
problematic: these issues were solved (mainly by tom k) fast and 
efficiently, lots of bulk testing was done, and the new version made it in 
time for tex live; also, bugs were reported and taken care of by adobe 
(actually, a group of texies visited adobe around that time), etc.

Concerning (2): this is the category that i meant when i commented 'there 
is no real need'; you can bet that i (and i guess martin and others as 
well) went through the 1.5 specs and looked for new thingies; it looks like 
everything new can comfortably be implemented; some recent developments 
with regards to pdf/x will be covered in future versions of pdftex (most of 
pdf/x is covered).

Concerning (3): pdftex does a pretty good job in including pdf code thanks 
to derek's work; it greatly helps that we're dealing with a separate 
library and even then, martin needs to keep things in sync (can be tricky 
with extensions that influence both xpdf and pdftex). If someone has an 
extension -for instance the one with regards to merging annotations- this 
needs (a) coordination, (b) testing , (c) inclusions in the code bed, (d) 
discussion. Rougly spoken this comes down to:

- martin (who currently is in charge of code integration) tries to 
integrate the code and tests
- others prepare binaries for testing (e.g. fabrice for windows) [i can 
only do testing when i have binaries]
- then testing can start
- when things run ok (and are decided to be useful and maintainable) 
discussion can start keeping it there
- normally a new tex live version determins such a decission point
- when needed, in the meantime maintainers of pdf-related macro code need 
to become in sync (often pre-sync)

in the meantime, there is nothing against users having their own version 
with extensions, which is one of the nice things about tex!

Now, another aspect is the time frame. There is no way that new code, when 
submitted, will make it into pdftex directly. For that, too many people 
depend on stability. Especially when new functionality is involved, we need 
to be careful. For instance: if suddenly included pdf code will obey 
hyperlinks, this is not what used to be the case, so it should be under 
control of macropackages, actually be off by default. You will be surprised 
how users can intermix direct \pdfprimitives with macro code that also uses 
them and what hard to support errors can result from that.

Related to time is that it happens that some of those involved in pdftex 
development from the beginning attend user group meetings, meet each other 
etc, which takes time, but also introduces delays in answering mails.

In any case: pdftex is actively maintained, but not in terms of many lines 
of code added per day. Bugs are fixed, viewer anomalities are catched, 
primitives are added, but only when really needed i.e. functionality not 
provided by other primitives, the code is optimized. If you have input on 
new functionality, it's welcome, but don;t expect pdftex to be a 
each-day-new-feature-and-version program, stability is the key-word.

Personally, i'm always willing to test new things, but only in a separate 
binary (to be provided by others, since i don't maintain a source tree 
here). This has happened many times before -)

Hans

PS. In most cases, if someone wants to volunteer, and feels that something 
got lost in mail, it helps more to mail with persons involved directly than 
to start complaining on lists like this; it may also cross your mind that 
some involved in pdftex development spent a huge amount of time to (pdf)tex 
support and have to divide their time over more than one person/mail
-------------------------------------------------------------------------
                                   Hans Hagen | PRAGMA ADE | pragma at wxs.nl
                       Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
  tel: +31 (0)38 477 53 69 | fax: +31 (0)38 477 53 74 | www.pragma-ade.com
-------------------------------------------------------------------------
                        information: http://www.pragma-ade.com/roadmap.pdf
                     documentation: http://www.pragma-ade.com/showcase.pdf
-------------------------------------------------------------------------