The parenthesis don't grow in this example:
\input luaotfload.sty
\font\test={name:XITS Math:mode=base;script=math} at 10pt
%\font\test="[xits-math.otf]:script=math" at 10pt % for xetex
\textfont0=\test
$$ a \overwithdelims() b $$
\bye
--
Khaled Hosny
Arabic localiser and member of Arabeyes.org team
Free font developer