Do you take into account intended purpose? Not all photography is done to create large prints. Some purposes have very modest magnification requirements. Images generally appear less noisy, the less you magnify them. The noise that is decreasing by ~0.25e- in quadrature is post-gain read noise, which generally contains the most spatially-correlated and lowest-frequency noises of all noise sources, and spatially-correlated noise survives low magnification very well, in visibility. That is in quadrature, of course, and the actual decrease in post-gain read noise (in isolation) from ISO 500 to ISO 12800 is actually much greater than 0.25e-.
In a quick mental analysis of your graph, it seems like post-gain read noise is about 4.1e- at ISO 100, so at ISO 500 (assuming the camera gives the same headroom), I expect post-gain read noise to be about 0.82e-, and at 12800, 0.032e-. So, any large-scale correlated noise in the post-gain read noise is going to be about 25.6x greater at ISO 500 than at ISO 12800.
You also have the issue that the metadata in the raw file is most likely giving a more incorrect black point, in absolute terms (electrons), at ISO 500 than at ISO 12800, which causes false colors to become visible in the pushing process. We can see that even with pushes of only 4 or 5 stops with some cameras. Converters that allow you to move the blackpoints are nice, but they tend to have limited precision, with granularity dictated by the size of a raw level step.