[metapost] make_text related scanning improvements

[Marcel Krueger]

 ---- On Fri, 18 May 2018 10:03:38 +0200 Hans Hagen <j.hagen at xs4all.nl> wrote ---- 
 > [...]
 > Thanks for the patch / suggestion. The space before etex / verbatim tex  
 > is deliberate (mandate) and that wil not change. Keep in mind that full  
 > compatibility is not our aim, definitely not with extensions (in mp it  
 > was kind of a kludge anyway) and the library has its own logic  
 > sometimes. The text gets passed to a lua function and people can 
 > manipulate there. 

IMO  the bigger problem than the space before etex is the currently required character after etex.
I think

        while (loc < limit - 5) {

should be replaced by

          while (loc < limit - 4) {

to allow the x from etex, at this point the character at loc + 4, to be the last charater of the line, limit - 1.

 >  
 > Incidentely you patch can and will break other macro packages. We take  
 > context as reference here. 

That's sad. I there a short example for breaking code?
I must be missing something here, because the only incompatibility I could find
was detecing etex followed by another letter which looks weird anyway.

 >  
 > The newline issue you sketch can probably be dealt with but then we  
 > might also will look at strings in general and it might become an option  
 > controlled by some flag to honor spaces and newlines (basically any  
 > character then). We'll add it to our todo list to look at it in more detail. 

That would be great.

Marcel