TeX (= tau epsilon chi, and pronounced similar to "blecch", not to the state known for `Tex-Mex' chili) is a computer language and program designed for use in typesetting; in particular, for typesetting math and other technical (from Greek "techne" = art/craft, the stem of `technology') material.
In short, the TeX program is a composition engine for typesetting entire documents. Strictly speaking, it is an interpreter, not a compiler. It is essentially a batch engine, although a limited amount of interactivity is possible when processing a file, to allow error recovery and diagnosis.
History: in the late 1970s, Donald E. Knuth was revising the second volume of his multivolume magnum opus The Art of Computer Programming, got the galleys, looked at them, and said (approximately) "blecch"! He had just received his first samples from the new typesetting system of the publisher's, and its quality was so far below that of the first edition of Volume 2 that he couldn't stand it. Around the same time, he saw a new book (Artificial Intelligence, by Patrick Winston) that had been produced digitally, and ultimately realized that typesetting meant arranging 0's and 1's (ink and no ink) in the proper pattern, and said (approximately), "As a computer scientist, I really identify with patterns of 0's and 1's; I ought to be able to do something about this", so he set out to learn what were the traditional rules for typesetting math, what constituted good typography, and (because he wanted to determine his own books, down to the pixel) as much as he could about type design. He figured this would take about 6 months. Ultimately, it took nearly 10 years, and along the way he had lots of help from some people who are well known to many readers here—Hermann Zapf, Chuck Bigelow, Kris Holmes, Matthew Carter and Richard Southall, to list those acknowledged in the introduction to Volume E, Computer Modern Typefaces, of the Addison-Wesley Computers & Typesetting book series.
A year or so after he started, Knuth was invited by the American Mathematical Society (AMS) to present one of the principal invited lectures at their annual meeting. This honor is awarded to significant academic researchers who (mostly) were trained as mathematicians, but who have done most of their work in not strictly mathematical areas (there are a number of physicists, astronomers, etc., in the annals of this lecture series as well as computer scientists); the lecturer can speak on any topic s/he wishes, and Knuth decided to speak on computer science in the service of mathematics. The topic he presented was his new work on TeX (for typesetting) and Metafont (for developing fonts for use with TeX). He presented not only the roots of the typographical concepts, but also the mathematical notions (e.g., the use of Bezier splines to shape glyphs) on which these two programs are based. The programs sounded like they were just about ready to use, and quite a few mathematicians, including the chair of the Math Society's board of trustees, decided to take a closer look. As it turned out, TeX was still a lot closer to a research project than to an industrial strength product, but there were certain attractive features:
Mathematicians are traditionally, shall we say, frugal; their budgets have not been large (before computer algebra systems, pencils, paper, chalk and blackboards were the most important research tools). TeX came along just before the beginnings of the personal computer; although it was developed on one of the last of the "academic" mainframes (the DECsystem ("Edusystem")-10 and -20), it was very quickly ported to some early HP workstations and, as they emerged, the new personal systems. From the start, it has been popular among mathematicians, physicists, astrophysicists, astronomers, any research scientists who were plagued by lack of the necessary symbols on typewriters and who wanted a more professional look to their preprints.
To produce his own books, Knuth had to tackle all the paraphernalia of academic publishing—footnotes, floating insertions (figures and tables), etc., etc. As a mathematician/computer scientist, he developed an input language that makes sense to other scientists, and for math expressions, is quite similar to how one mathematician would recite a string of notation to another on the telephone. The TeX language is an interpreter. It accepts mixed commands and data. The command language is very low level (skip so much space, change to font X, set this string of words in paragraph form, …), but is amenable to being enhanced by defining macro commands to build a very high level user interface (this is the title, this is the author, use them to set a title page according to AMS specifications). The handling of footnotes and similar structures are so well behaved that "style files" have been created for TeX to process critical editions and legal tomes. It is also (after some highly useful enhancements in about 1990) able to handle the composition of many different languages according to their own traditional rules, and is for this reason (as well as for the low cost), quite widely used in eastern Europe and with other scripts outside of western Europe/North America.
Some of the algorithms in TeX have not been bettered in any of the composition tools devised in the years since TeX appeared. The most obvious example is the paragraph breaking: text is considered a full paragraph at a time, not line-by-line; this is the basic starting algorithm used in the HZ-program by Peter Karow (and named for Hermann Zapf, who developed the special fonts this program needs to improve on the basics). The "microtypography" enhancements (font expansion, margin kerning) invented by Karow and Zapf have been implemented in pdfTeX.
In summary, TeX is a special-purpose programming language (and program that interprets the language) that is the centerpiece of a typesetting system that produces publication quality mathematics (and surrounding text), available to and usable by individuals.
Metafont, Knuth's font creation program, is independent of TeX. It generates only bitmap fonts (although internally it creates outlines on which the bitmaps are based). There is still research to be done on combining overlapping outlines into a single outline that can be used like the outline of a Type 1 font; as with TeX, Knuth has "frozen" Metafont, so any further research and development will be done by others, and the result will not be called "Metafont". (It's possible to use fonts in Type 1, TrueType, OpenType and other formats with TeX.)
The first version of Metafont (MF79), used only a pen metaphor for drawing glyphs. After years of experience and collaboration with Hermann Zapf, Richard Southall, and others, the final Metafont (MF84) is based on outlines, with the ductus of a pen being a particular case of drawing outlines. Pens continue to be used extensively throughout the Metafont implementation of the Computer Modern fonts.
It does not require applications like PageMaker, Quark Express, Fontographer or FontLab. A TeX system stands on its own, provided all the fonts one needs are available. TeX uses only the metrics, and produces a "device independent" output file—.dvi—that must then be translated to the particular output device being used (an imagesetter, laser printer, inkjet printer; in the "old days" even daisy-wheel printers were used). The DVI translator actually accesses the font shapes, either as bitmaps, Type 1 fonts, or pointers to fonts installed in a printer with the shapes not otherwise accessible. PostScript and PDF are two of the most popular "final" output forms for TeX.
One of the major areas where TeX holds its own is as a "back end" to SGML and XML systems, where no human intervention is expected between data input (structured, not WYSIWYG) and removing the output from the printer or viewing it on a screen. Granted, this isn't "creative" in the sense most often discussed, but it's still important to the readability and usefulness of such documents that care is taken with the design and typography, and the flexibility and programmability of TeX makes that possible.
As an aside, TeX can be the cause of religious wars. For those of us who need the capabilities of TeX—for production of books and journal articles in research mathematics—no other current composition tool, proprietary or otherwise, can handle the material and produce high-quality, publication-worthy output, and simultaneously be usable by the writer of the document. We'll be glad to provide test material to anyone who wants to prove us wrong. (It hasn't happened yet; the audience is much too small, and the problem too complex for a Microsoft or Quark or Adobe to be interested.) On the other hand, if you want a tool for producing a slick advertisement or a letter to Aunt Henrietta, unless you're already familiar with it, TeX is not the tool for you.
For information on where to report bugs in the TeX system software or related publications, please see the errata section on Knuth's Computers and Typesetting web page, and our TeX bug reporting pages here.
DEK rewards the first finder of each typo or software bug with a check based on the source, age, and type of the bug. Since his books have numerous reprintings and editions, he does have a chance to correct errors. Typos and other errors in books typically yield $2.56 each once a book is in print (pre-publication "bounty-hunter" editions are less), and program bugs rise by powers of 2 each year from $1.28 or so to a maximum of $327.68. Knuth's name is so valued that very few of his checks—even the largest ones—are actually cashed, but instead framed. Informal surveys of past check recipients have shown that this holds overwhelmingly for nearly everyone but starving students. (2008 update: Knuth no longer writes checks on a bank in the mundane world, but instead personal certificates of deposit to his Bank of San Serriffe.)
If you want to install a running TeX for use on your computer, please consult the resources mentioned on the TUG home page.
If you want to inspect Knuth's own sources for educational or other such purposes, without any of the scaffolding and enhancements that have come to surround them in modern systems, you can get them from Stanford; the material is also mirrored on CTAN.