That's not 'all', that's technology that we don't have, or anything like - though my son had an idea for a system that might do that. It was interesting enough for his company to investigate it, and whilst the theory was sound, there was no way of actually realising it at present. So, what we settle for is a description of the scene, that allows us to create stimuli that the human visual system perceives as something quite similar to looking at the original scene. As photographers we play around creatively with the several ways that optical illusion can fail.
The problem is that we rarely view images in such a way that the image forms enough of the visual environment that our visual will adapt solely to it. When 'colour science' was developed it was about printed reproduction. You view a print by reflected light, in an environment to which your visual system has adapted. The lighting conditions in that environment will rarely be the same as the original scene. In a print the colorimetry assumes daylight viewing conditions and our visual system adapts when the image is viewed in other lighting conditions - up to a point. For critical assessment of prints it was necessary to use the right D series illuminant (usually D65 or D50)
Metamerism is the condition where objects issuing different spectral distributions are perceived as the same hue. It's an inevitable consequence of an colour vision that does not have infinitely many different stimuli covering the whole visual spectrum(in classical theory - on quantum theory and the Heisenberg uncertainty principle would provides a computable upper bound) The number of stimuli required to do away with metamerism would certainly be a lot above three.
Since metamerism is about the spectral distribution, it depends on both the objects in the scene and the spectral distribution of the illuminant. Thus sometimes paints will look to be different colours in different illuminants. That's not strictly metamerism failure, which is when a reproduction system presents spectral distributions which should be metamers as different hues. In practice we tend to call 'metamerism failure' when things in our photos turn out unexpected colours.
The key thing here is that colour photographe does not try to capture the spectral disributions of the objects in the scene. What it's trying to capture is the three human visual stimuli that scene would produce, or at least sufficient information to allow them to be reproduced.
