Definitely, if you are shooting JPEG, the ISO setting is essential. We agree on this, yes?
If you are shooting RAW, the ISO setting is necessary unless your sensor is ISO invariant. I would say that the definition of an ISO invariant sensor is one in which you can get the exact same result using you RAW editor’s Brightness or “Exposure” slider as you can by you can by adjusting ISO in the camera.
I tried using Canon’s DPPro software to adjust the brightness in my RAW images to equal the effect of the camera’s ISO setting, but it did not work. I took fixed manual exposure shots in daylight, using a tripod and adjusting ISO. The ISO 800+ shots were blown out. Unrecoverable.
You may be correct in theory; I have no idea. But in practice, on the Canon platform, the ISO setting definitely matters for both JPEG and RAW.