Back to feeds

Chirotpal Das

Stealth • 2d

🤔 𝐎𝐩𝐞𝐧𝐀𝐈 𝐨𝟏 - 𝐢𝐬 𝐢𝐭 𝐦𝐨𝐫𝐞 𝐛𝐢𝐠𝐠𝐞𝐫 𝐨𝐫 𝐦𝐨𝐫𝐞 𝐟𝐢𝐧𝐞-𝐭𝐮𝐧𝐞𝐝? We're all excited about OpenAI's o1 model and many other such bigger models, but here's what keeps me up at night: Are we witnessing a genuinely larger, more advanced LLM, or is this the result of brilliant engineering and fine-tuning of existing architectures? 𝐓𝐡𝐞 𝐫𝐞𝐚𝐥 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧 𝐢𝐬: Can we, as users and developers, ever truly distinguish between a massive pre-trained model and an expertly fine-tuned one by ourselves? It's like trying to tell if a master chef created a new recipe or perfectly refined an existing one. The taste might be extraordinary either way. What do you think? 🧠

4 replies13 likes

Replies (4)

Tanuj Sharma

Stealth • 1d

Hi, It is trained on more bigger dataset and with more gpu hours so its not fine tuned on earlier versions, its trained with more parameters than the earliest versions, if you have heard its training cost is around 50-100M $, while if you see thier latest offering o3, its training cost till now around 1-1.4B $, while training cost for chat gpt initial version was only around ~10M $, but as per the latest rumours if you heard they already have exhausted all available data on internet in training thier models till now, so future mai its going to be very hard for them to train their models on new data, then it would be mostly fine tuning on older versions with synthetic datasets.

0 replies