I feel like better calibration patterns could largely address those issues. Since it’s temporal you could even have a frame that just shows every single color combination, and then have maybe something in the timing pattern for example to correct for deviations after that. Compression might be a large issue though, since chroma is always compressed more than luminance. I feel like multiple bits per pixel might even be possible, depending on how good your camera is.

Have you considered an RGB mode?