[XeTeX] internal error using opentype font on mathaccent

[David Carlisle]

\font\f="TeX Gyre Termes"

\textfont0=\f
\scriptfont0=\f
\scriptscriptfont0=\f


$ \mathaccent"0017 c $

\bye

This is a plain version of a latex example posted by a user to
tex.stackexchange.
It shouldn't really do anything sensible (if the font load is adjusted
for luatex
you get the standard missing character warning)

But it gives a fatal internal error

This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021)
(preloaded format=xetex)
 restricted \write18 enabled.
entering extended mode
(./aa539a.tex
! Internal error: bad native font flag in `map_char_to_glyph'


It doesn't seem to matter much which font is used, I just used TeX
Gyre above as a specific example.

Could this be adjusted to give a normal TeX warning (or error) ?

David