"What we did with weight sharing in neural architecture search is rather than train a separate model for every accuracy latency configuration, we use weight sharing in NAS to basically train thousands of subnetwork configurations in parallel with a single training run. That was a huge freaking unlock for us to be able to use our compute budget efficiently."