Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern Machine Learning (ML). In this paper we study trends in the most readily quantified factor – compute. We show that before 2010 training compute grew in line with Moore’s law, doubling roughly every 20 months. Since the advent of Deep Learning in the early 2010s, the scaling of training compute has accelerated, doubling approximately every 6 months. In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute. Based on these observations we split the history of compute in ML into three eras: the Pre Deep Learning Era , the Deep Learning Era and the Large-Scale Era . Overall, our work highlights the fast-growing compute requirements for training advanced ML systems. Read More
Daily Archives: February 23, 2022
Report: 70% of orgs are spending $1M or more on AI
According to a new report by LXT, artificial intelligence (AI) spending is strong at mid-to-large U.S. organizations, and 40% rate themselves at the three highest levels of AI maturity, having already achieved operational to transformative implementations. A key component to success across all organizations is AI training data, in terms of both quality and investment.
The survey found that over a third of high-revenue companies are spending between $51 million to $100 million on AI, and seven in ten organizations are spending $1 million or more of their budget on AI. Enterprises are using AI to innovate, scale up and drive competitive advantage as well as gain internal efficiencies. Read More
Rick's Cafe AI 