Problem getting different-sized fences in Y&Y TeX

[Jim Wilson]

The following PDF files

   ftp://ftp.ncsu.edu/pub/eos/pub/jwilson/test11.pdf

   ftp://ftp.ncsu.edu/pub/eos/pub/jwilson/test12.pdf

show the result of typesetting in Y&Y TeX the LaTeX files
test11.tex and test12.tex that are contained in the .zip file

   ftp://ftp.ncsu.edu/pub/eos/pub/jwilson/fence-test.ZIP

The typeset output of test11.tex is set in 11-point Times Roman type; and
as you can see in equation (1), the nested fences

  \Big  (  ... \big \{ ... [ ... ] \big \} ... \Big )

show up in progressively smaller sizes as the nesting gets deeper.
This is what I want to indicate visually the level of nesting
of conditional expectations in this situation.

On the other hand, when we typeset this same LaTeX file in 12-point
type, the result seen in test12.pdf is clear not what was intended---
\big \{  seems to produce a curly left brace that is the same size
as the normalsize left square bracket [.   Similarly, the \big |
produces exactly the same size vertical bar | as the normalsize |.

If the intended effect is achieved in 11-point type, then it seems it
should work in a similar way in 12-point type.

Is there some way to fix this?

Jim Wilson