|
| Featured Contents |
| Question of the Day |
| The Voice of the DBA |
Finely Tuned Models I have no idea if this is true, but this post on X says that Thomson Reuters used information they've collected for decades to fine tune and train a model. They started with one of the qwen models and then spent $40 million to add their knowledge to the model. The post says this model is comparable to the Gpt5.5 and Sonnet 5 models. They have used 10% of their data, which spans over 100 years. I have many questions. But first, $40mm? How many tokens is USD$40mm? A quick calculation is trillions of tokens, and while I'm sure there is some payback if lots of their customers use the model for work, there's also a compute cost every time they use the model. Perhaps they can fine-tune and train the model more efficiently over time, but I can't see many individual companies spending this effort on training their own model. I also wonder what the time it took to train this. The story notes 2 years, but if I want to update this, then how much more effort. What's the cost? This certainly reduces the dependence on the frontier models from Antropic/OpenAI/etc., but is this something that makes sense for other companies? If I choose to use the TR model, would it be cheaper than Sonnet 5? Maybe it uses less power, which is always good, but will T-R charge me less than Anthropic? I do think that fine-tuned models might make sense, especially if you can use a small language model and train it for a reasonable cost. Like $100,000, not millions. In that case, would it make sense to you? Would your company fund this themselves? Is this something that a trade group could do with support from multiple organizations? Or is the competition between companies so high that they can't work together? I do think that AI has a lot of possibilities to assist humans, and focused models can be useful to solve specific problems, with a lot less cost than the best frontier models. I'm just not sure if the training effort is something that many are willing to put forth. It will be interesting to see how GenAI models evolve as the costs and resources needed by the frontier models continue to rise and smaller models become more capable. Steve Jones - SSC Editor Join the debate, and respond to today's editorial on the forums |
| Featured Contents |
