[luatex] slow io.popen

[Stephan Hennig]

Hi,

I'm observing io.popen in texlua being slower by an order of magnitude
compared to PUC lua.  The attached code opens a zipped or gzipped text
file (given as argument) through a pipe and reads in all lines in the
file.  Large (g)zipped text files can be found at
<URL:http://storage.googleapis.com/books/ngrams/books/datasetsv2.html>.

Here are the results of my timings on an Intel T8300 (Core 2 Duo at 2.4
GHz) on Windows 7 x64 with this file,
<URL:http://storage.googleapis.com/books/ngrams/books/googlebooks-ger-all-1gram-20090715-0.csv.zip>:

  $ time lua.exe test-pipe.lua googlebooks-ger-all-1gram-20090715-0.csv.zip
  real    0m21.437s
  user    0m0.015s
  sys     0m0.015s

  $ time texlua.exe test-pipe.lua
googlebooks-ger-all-1gram-20090715-0.csv.zip
  real    4m53.810s
  user    0m0.015s
  sys     0m0.015s

texlua is more than 13 times slower than lua.  Is this a known issue?

Best regards,
Stephan Hennig
-------------- next part --------------
--- Test piped input.
--
local io = require('io')
-- get file name
local fname = arg[1]
-- get file extension
local cmd = string.match(fname, '[^.]+$')
if cmd == 'zip' then
  cmd = 'unzip -p '
elseif cmd == 'gz' then
  cmd = 'gunzip -c '
else
  io.stderr:write('unknown file extension\n')
  os.exit()
end
io.stderr:write(cmd .. fname, '\n')
-- open input file
local f = io.popen(cmd .. fname)
-- read lines
for line in f:lines() do
end