It is possible to install and use TeX Live with binaries that are not part of the original distribution. The most common case for this is when you are on a platform which the original TL distribution does not support.
Some of Nelson's binary sets that may be of particular interest: 64-bit CentOS 6, 32-bit CentOS 7, 32-bit CentOS 6, 32-bit CentOS 5. These include all binaries that were possible to build on these platforms; the most likely to be missing are xetex, dvisvgm, biber, asymptote, and luajit. Precise lists for each platform are in Nelson's README.
64-bit Windows binaries are not included in TL this year, but can be downloaded separately and used in parallel with the regular binaries. See the Windows on TeX Live page. get them as natively-compiled binaries from Akira Kakuto (has its own instructions).
Given a set of binaries, here are the steps to use it:
Regarding subsequent TL package updates from the net: ensure that you have wget, xz, and xzdec available in your PATH; otherwise, TL will have no way to download or decompress packages. Also, there will be no updates to the platform-specific packages in TL, since the custom platform doesn't exist in the canonical TL repository.
Therefore, you may have to manually adjust symlinks in your custom dir. Symlinks (on Unix) in the binary directories point to scripts that can run on any platform. When and if the target of a symlink changes location for whatever reason, you would need to update the symlink in your own directory.
If you want to install a second set of custom binaries (from, say, /tmp/barbin), you first have to manually rename the first set, like this: