[metapost] make_text related scanning improvements

[Marcel Krueger]

Hi,

some time ago I found the undocumented but very useful `make_text` callback in mplib.
Sadly the code for scanning the text between `btex/verbatimtex` and `etex`
is not compatible with normal `btex...etex` scanning:
Especially `etex` at the end of a line is not accepted because of an off-by-one error and only a single line of TeX material is allowed.

This is a problem when using this with `verbatimtex`, where `etex` often comes at the end of a line and sometimes multiple lines are used.

To fix these problems I implemented a new version, which also adds some error reporting and more consistent detection of the etex token.
I think mplib would really benefit from merging this code into the official sources.
The corresponding CWEB change file is

@x
@ @<Pass btex ... etex to script@>=
{
    int first ;
    while ((loc < limit - 4) && (mp->buffer[loc] == ' ')) {
        incr(loc);
    }
    first = loc ;
    if (mp->buffer[loc-1] == ' ') {
        decr(loc);
    }
    while (loc < limit - 5) {
        if (mp->buffer[loc] == ' ') {
            incr(loc);
            if (mp->buffer[loc] == 'e') {
                incr(loc);
                if (mp->buffer[loc] == 't') {
                    incr(loc) ;
                    if (mp->buffer[loc] == 'e') {
                        incr(loc) ;
                        if (mp->buffer[loc] == 'x') {
                            /* start action */
                            char *s, *txt ;
                            int size ;
                            mp_value new_expr;
                            size = loc - first + 1 - 4 ;
                            if (size < 0) {
                                size = 0 ;
                            } else {
                                while ((size > 1) && (mp->buffer[first+size-1] == ' ')) {
                                    decr(size);
                                }
                            }
                            txt = malloc(size+1);
                            if (size > 0) {
                                (void) memcpy (txt, mp->buffer + first, size);
                            }
                            txt[size] = '\0';
                            incr(loc);
                            s = mp->make_text(mp,txt,(cur_mod() == verbatim_code)) ; /* we could pass the size */
                            @<Run a script@>
                            /* done */
                            free(txt);
                            break ;
                        } else {
                      //      decr(loc) ;
                        }
                    }
                }
            }
        } else {
            incr(loc);
        }
    }
}
@y
@ @<Pass btex ... etex to script@>=
{
    char *s, *txt, *orig_txt ;
    int size, first ;
    quarterword cclass; /* the |char_class| of previous token */
    mp_value new_expr;
    old_info = line;
    size = 0;
    first = loc ;
    cclass = mp->char_class[mp->buffer[loc - 1]] ;
    while (   cclass == letter_class
           || loc + 4 > limit
           || memcmp("etex", mp->buffer + loc, 4)
           || (loc + 4 < limit && letter_class == mp->char_class[mp->buffer[loc + 4]])) {
        cclass = mp->char_class[mp->buffer[loc]];
        if (loc == limit) {
            if (size) {
                txt = realloc(txt, size + limit - first + 1);
            } else {
                txt = malloc(limit - first + 1);
            }
            (void) memcpy (txt + size, mp->buffer + first, limit - first);
            size += limit - first + 1;
            txt[size - 1] = '\n';
            if (move_to_next_line(mp)) {
                char msg[256];
                const char *hlp[] = {
                       "The file ended while I was looking for the `etex' to",
                       "finish this TeX material.  I've inserted `etex' now.",
                        NULL };
                mp_snprintf(msg, 256, "TeX mode didn't end; all text was ignored after line %d", (int)old_info);
                mp_error (mp, msg, hlp, false);
                goto READING_FINISHED;
            }
            first = loc;
            cclass = percent_class;
        } else {
            incr(loc);
        }
    }
    if (size) {
        txt = realloc(txt, size + loc - first + 1);
    } else {
        txt = malloc(loc - first + 1);
    }
    (void) memcpy (txt + size, mp->buffer + first, loc - first);
    size += loc - first + 1;
    loc += 4;
READING_FINISHED:
    if (cur_mod() == btex_code) {
        while ((size > 1) && (cclass == space_class || txt[size - 2] == '\n')) {
            decr(size);
            cclass = mp->char_class[(ASCII_code) txt[size - 2]];
        }
    }
    orig_txt = txt;
    cclass = mp->char_class[(ASCII_code) txt[0]];
    while ((size > 1) && (cclass == space_class || txt[0] == '\n')) {
        incr(txt);
        decr(size);
        cclass = mp->char_class[(ASCII_code) txt[0]];
    }
    txt[size - 1] = '\0';
    s = mp->make_text(mp,txt,(cur_mod() == verbatim_code)) ; /* we could pass the size */
    @<Run a script@>
    free(orig_txt);
}
@z

Best regards
Marcel Krüger