So being a long time fan of protocol buffers specifically for the fact that they use varints, I saw this article today and thought it was interesting.
https://lemire.me/blog/2017/09/27/stream-vbyte-breaking-new-speed-records-for-integer-compression/
What I’m specifically curious about is if game engines could make use of this. We throw around a ton of floats. In well optimized games I’d say 95% or better of the data on the hot path is floats.
What if you applied this approach to data passing between cpu/gpu also?
It’s extremely rare that you ever actually need the precision a float gives. And if you look at the total memory cost you are paying for all that precision you never need, and throw in the savings you would get from optimized varint approaches, that’s a lot of memory bandwidth.