Does it matter? Evolution is the brain's very own "pre-training". Hundreds of millions of years of priors hardwired.
We can do that for AIs too - pre-train on pure low Kolmogorov complexity synthetics. The AI then "knows things" before it sees any real data. Advantageous sometimes. Hard to pick compute efficient synthetics though.
