[tex-eplain] \sidx not catcode-clean

[Oleg Katsitadze]

On Wed, Apr 19, 2006 at 05:41:14PM -0400, Dorai Sitaram wrote:
> By the way, what is the strike against writing to an external file and
> reading it back (other than the fact that it's I/O and therefore has the
> whiff of inefficiency)?

At first I though it would be very inefficient, but after
doing some tests I agree with Stepan that it wouldn't be a
problem on a modern computer.  Plus we can use eTeX's
\scantokens when it's available.  We'll have to reserve
another filename (.tmp), but this probably wouldn't be a
problem.

However, after thinking some more about this, I still
believe that changing the default behavior of the indexing
macros would have the problem of compatibility.  For
example, now these three commands yield the same index
entry:

  \idx{foo bar}
  \idx{foo  bar}
  \idx{foo
       bar}

but if we start reading the arguments verbatim, they will
all become separate entries.  Shucks.

So I guess we have two options left:

  1) leave \?idx* macros alone and define a simplistic
     \index command;

  2) leave \?idx* macros alone by default but give the user
     an option to convert them into `verbatim argument'
     macros.

The problem with 1) is that Eplain won't be able to create
hyperlinks for index terms created this way, as it won't
know how to stick in its hyperlink encapsulator into a
completely arbitrary argument of \index (and parsing it
would be no fun).

This leaves us with 2).  So here's what I think can be done.

We patch all indexing macros to handle specially all TERM
and SUBTERM arguments.  Instead of reading them directly,
like this:

  \def\foo#1{...}

they would use a call like this:

  \def\foo{\idx at getarg\bar}
  \def\bar#1{...}

For the optional args, they would use \idx at getoptarg instead
of \@getoptionalarg.

By default, \idx at getarg will be \relax and \idx at getoptarg
will be \@getoptionalarg.  But for the indexes which the
user declares as `verbatim' (e.g., by saying
\verbatimindex{i}, where `i' is the index's letter),
\idx at getarg and \idx at getoptarg will be \let to these macros:

  % \@getverbatimarg#1{ARG} reads ARG verbatim and then calls #1 with
  % that argument.
  \def\@getverbatimarg#1{%
    \def\@getverbatimarg at cmd{#1}% Save the command.
    \bgroup
      \uncatcodespecials
      \catcode`\{=1
      \catcode`\}=2
      \@fingetverbatimarg
  }%
  \def\@fingetverbatimarg#1{\egroup\@getverbatimarg at cmd{#1}}%
  % \@getverboptarg#1[ARG] reads ARG verbatim, saves it as \@optionalarg
  % and then calls #1.  Spaces between #1 and [ARG] are no longer
  % allowed because of the catcode changes.
  \def\@getverboptarg#1{%
    \def\@getverboptarg at cmd{#1}% Save the command.
    \bgroup
      \uncatcodespecials
      \catcode`\[=\other
      \catcode`\]=\other
      \@getoptionalarg\@fingetverboptarg
  }%
  \def\@fingetverboptarg{%
      \global\let\@getverboptarg at tmp\@optionalarg
    \egroup
    \let\@optionalarg\@getverboptarg at tmp
    \@getverboptarg at cmd
  }%

Such setup of \idx at getarg and \idx at getoptarg will have to be
done every time in a call to \idx* and \sidx* (in case there
are several indexes with different `verbatim' flags), but
it's only two \let's, so it shouldn't be a big deal.

I'll try to implement this, but maybe someone has a better
idea or any suggestions?

Thanks,
Oleg