Researchers say they trained a foundation model from scratch for about $1,500
β’ Researchers at Sapient developed a 1B parameter reasoning model, called HRM-Text, by training it from scratch for approximately $1,500. β’ The model was trained on 40B tokens and achieved performance levels competitive with larger models ranging from 2B to 7B parameters. β’ This breakthrough demonstrates that foundational pretraining is no longer exclusive to wealthy institutions, allowing smaller organizations to build capable reasoning models affordably.
venturebeat.com
