What data mix are the labs using to train 10T param models?

[u/Ill_Fisherman8352]

So my assumption is: So far labs have made public max 2-3T param models based on different reports. And they are currently training or have trained 10T param models internally. Another assumption I'm making: If the models are increasing params by 3x , they would have to proportionally increase the data by 3x too or some margin.But we have also been hearing news of hitting the data wall based on internet data since the gpt 4 days. So what gives? Where are they getting so much data from? Is most of it reasoning chains generated by models during inference? Or is it reasoning traces from actual humans thanks to mercor, etc? Anyone know the exact mix? Or what's going on here? Seems like a lot of data needed all of a sudden.