Model Collapse

It's being reported that OpenAI's next version won't be the big leap forward that previous versions have been. The rate of improvement is declining which readers will know is not a huge shock to me. The effectiveness of these models is directly proportional to the amount of data and compute they have available to them - both of which are starting to dwindle. What is interesting is that they seem to be countering the lack of new data with training the model on synthetic data. Now these people know a truckload more about AI than I do, but does that lead to model collapse?

I found this interview with Dave Anthony from the Dollop on the Serious Danger podcast cathartic. They discuss the US election and the failure of the Democrats. I also got a lot out of this episode of You're Wrong about with Kelly Carter Jackson - a good reminder that the revolution needs to continue.

I really enjoyed Angus Hervey's piece on truth, inevitability and the need to keep struggling against injustice, bullies and an emerging Fascist US (my words, not his).