Machine learning (ML) has become prominent in information technology, which has led some to raise concerns about the associated rise in the costs of computation, primarily the carbon footprint, i.e., total greenhouse gas emissions. While these assertions rightfully elevated the discussion around carbon emissions in ML, they also highlight the need for accurate data to assess true carbon footprint, which can help identify strategies to mitigate carbon emission in ML.
In “The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink”, accepted for publication in IEEE Computer, we focus on operational carbon emissions — i.e., the energy cost of operating ML hardware, including data center overheads — from training of natural language processing (NLP) models and investigate best practices that could reduce the carbon footprint. We demonstrate four key practices that reduce the carbon (and energy) footprint of ML workloads by large margins, which we have employed to help keep ML under 15% of Google’s total energy use. Read More
Tag Archives: Big7
DeepMind’s AI can control superheated plasma inside a fusion reactor
DeepMind’s streak of applying its world-class AI to hard science problems continues. In collaboration with the Swiss Plasma Center at EPFL—a university in Lausanne, Switzerland—the UK-based AI firm has now trained a deep reinforcement learning algorithm to control the superheated soup of matter inside a nuclear fusion reactor. The breakthrough, published in the journal Nature, could help physicists better understand how fusion works, and potentially speed up the arrival of an unlimited source of clean energy. Read More
FILM: Frame Interpolation for Large Scene Motion
Tensorflow 2 implementation of our high quality frame interpolation neural network. We present a unified single-network approach that doesn’t use additional pre-trained networks, like optical flow or depth, and yet achieve state-of-the-art results. We use a multi-scale feature extractor that shares the same convolution weights across the scales. Our model is trainable from frame triplets alone. Read More
Next Gen Stats: Intro to Passing Score metric
Next Gen Stats teamed up with the AWS Proserve data science group to develop a more comprehensive metric for evaluating passing performance: the Next Gen Stats Passing Score. Built off of seven different AWS-powered machine-learning models, the NGS Passing Score seeks to assess a quarterback’s execution on every pass attempt and transform that evaluation into a digestible score with a range between 50 and 99. The score can be aggregated on any sample of pass attempts while still maintaining validity in rank order.
… Instead of simply awarding all passing yards, touchdowns and interceptions to the quarterback, the NGS Passing Score equation leverages the outputs of our models to form the components that best
- Evaluate passing performance relative to a league-average expectation.
- Isolate the factors that the quarterback can control.
- Represent the most indicative features of winning football games.
- Encompass passing performance in a single composite score (ranging from 50 to 99).
- Generate valid scores at any sample size of pass attempts.
Don’t forget Microsoft
Despite its scale, Microsoft is one of the most overlooked companies in tech.
- It is not a beloved consumer brand like Apple, Facebook, Amazon, or Google.
- It was not a venture capital success story: Microsoft was too profitable to raise real VC money, so the founders owned 70% at IPO.
- It is the oldest of FAMGA, hidden away in a different state.
This piece undertakes a daunting set of tasks: 1) understand what Microsoft is, 2) chart a path for its global domination, and 3) apply learnings from the company to the startup ecosystem. Read More
#big7
Fake It Till You Make It
We demonstrate that it is possible to perform face-related computer vision in the wild using synthetic data alone.
The community has long enjoyed the benefits of synthesizing training data with graphics, but the domain gap between real and synthetic data has remained a problem, especially for human faces. Researchers have tried to bridge this gap with data mixing, domain adaptation, and domain-adversarial training, but we show that it is possible to synthesize data with minimal domain gap, so that models trained on synthetic data generalize to real in-the-wild datasets.
We describe how to combine a procedurally-generated parametric 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism and diversity. We train machine learning systems for face-related tasks such as landmark localization and face parsing, showing that synthetic data can both match real data in accuracy as well as open up new approaches where manual labelling would be impossible. Read More
Dataset
Introducing the First Self-Supervised Algorithm for Speech, Vision and Text
- We’re introducing data2vec, the first high-performance self-supervised algorithm that learns in the same way for speech, vision and text.
- With data2vec, we’re closer to building machines that learn about different aspects of the world around them without having to rely on labeled data. Read More
Meta has built an AI supercomputer it says will be world’s fastest by end of 2022
Social media conglomerate Meta is the latest tech company to build an “AI supercomputer” — a high-speed computer designed specifically to train machine learning systems. The company says its new AI Research SuperCluster, or RSC, is already among the fastest machines of its type and, when complete in mid-2022, will be the world’s fastest.
… The news demonstrates the absolute centrality of AI research to companies like Meta. Rivals like Microsoft and Nvidia have already announced their own “AI supercomputers,” which are slightly different from what we think of as regular supercomputers. RSC will be used to train a range of systems across Meta’s businesses: from content moderation algorithms used to detect hate speech on Facebook and Instagram to augmented reality features that will one day be available in the company’s future AR hardware. And, yes, Meta says RSC will be used to design experiences for the metaverse — the company’s insistent branding for an interconnected series of virtual spaces, from offices to online arenas. Read More
Meta’s new learning algorithm can teach AI to multi-task
If you can recognize a dog by sight, then you can probably recognize a dog when it is described to you in words. Not so for today’s artificial intelligence. Deep neural networks have become very good at identifying objects in photos and conversing in natural language, but not at the same time: there are AI models that excel at one or the other, but not both.
Part of the problem is that these models learn different skills using different techniques. This is a major obstacle for the development of more general-purpose AI, machines that can multi-task and adapt. It also means that advances in deep learning for one skill often do not transfer to others.
A team at Meta AI (previously Facebook AI Research) wants to change that. The researchers have developed a single algorithm that can be used to train a neural network to recognize images, text, or speech. The algorithm, called Data2vec, not only unifies the learning process but performs at least as well as existing techniques in all three skills. “We hope it will change the way people think about doing this type of work,” says Michael Auli, a researcher at Meta AI. Read More
Microsoft forms new coalition for AI in healthcare
Microsoft has created the Artificial Intelligence Industry Innovation Coalition (AI3C) to drive the use of artificial intelligence (AI) in healthcare by providing recommendations, tools and best practices.
Member organisations include The Brookings Institution, Cleveland Clinic, Duke Health, Intermountain Healthcare, Novant Health, Plug and Play, Providence, UC San Diego, and University of Virginia.
“The goal of the newly created AI3C is to establish a pragmatic coalition with public and private organisations to advance health by identifying and addressing significant societal and industry barriers,” said Patty Obermaier, vice president of US health and life sciences at Microsoft. “I am excited about the launch of AI3C and working with its distinguished board as we continue the momentum towards serving the needs of patients and communities through AI innovation.” Read More
Rick's Cafe AI 