[XeTeX] xelatex pdf files much larger than latex dvipdfm pdf files

[Daniel Greenhoe]

When I compile a book of mine using xelatex, the resulting pdf file is about 1.8 times as large as when I compile using latex together with dvipdfm. The xelatex pdf is ~3.7Mbytes and the latex-dvipdfm pdf is ~2Mbytes. Why would this be?

The book is about 200 pages long, and contains a very large number of equations and many illustrations using LaTeX drawing commands. A very similar version is at
  http://banyan.cm.nctu.edu.tw/~dgreenhoe/order/index.html

I tried to produce a very simple test case using simple text from a Project Guttenberg text file "The Story of Doctor Dolittle" available at
  http://www.gutenberg.org/etext/501

The tex test file I tried was similar to this (with the dolit10.txt somewhat modified for compatibility with tex):

\documentclass{book}
\begin{document}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\input{dolit10.txt}
\end{document}

However, in this case, the xelatex pdf was actually a little smaller than the latex-dvipdfm pdf file, with both being about 1.4Mbytes.

I can keep trying to comment lines out of my could to try to isolate the problem --- but it is a time consuming and somewhat tedious process. Could anyone give me some insight into where I could start looking as to why my original (not the Dr. Doolittle one) xelatex pdf file is so much larger than my latex-dvipdfm pdf file?

Many thanks in advance,
Dan