31^st IEEE International Symposium on Computer Arithmetic

ARITH 2024

Main page Program Social Program Committees Call for Papers Registration Venue About Málaga Archives Contacts

Sponsors:






  
  

ARITH 2024 Keynotes

“Report from IEEE WG P3109: Arithmetic Formats for Machine Learning”

Dr Andrew Fitzgibbon, Co-editor P3109 Working Documents

In the field of machine learning, there is increasing interest in and deployment of low-bit-width floating point formats. These formats, typically eight-bit, but increasingly even smaller, have proved valuable in reducing computational requirements while maintaining accuracy and utility of the machine learning algorithms on which our society increasingly depends for security, medicine, and numerous industrial applications in agriculture, entertainment, transport, and many others. The aim of WG P3109 is to synthesize existing practice and to define a set of arithmetic formats which best serve this growing community of practice. This talk will represent the discussions and key ideas and rationales behind the working group's current decisions, as represented in the interim report of September 2023. (See link at https://github.com/P3109/Public/blob/main/Shared%20Reports/P3109%20WG%20Interim%20Report.pdf

Andrew Fitzgibbon has contributed to some of the most significant machine learning and computer vision based products and innovations of the last few decades. At the University of Oxford, he co-founded the company "2d3", which developed the world's first commercially available automated camera tracking system, which was given the Emmy Award for "revolutionary impact on the creation of complex visual effects". At Microsoft, he contributed to the machine learning system behind the human motion capture system in Kinect for Xbox 360, one of the first systems to use massive synthetic training data to deliver a real-world computer vision application. More recently he contributed to the hand tracking algorithms in Microsoft's Hololens, the first commercially available AR headset with fully articulated hand tracking. At Graphcore, his recent work has been on programming languages and molecular machine learning. He is also a highly cited researcher, having authored numerous papers with over a thousand citations, and is inventor on dozens of patents.

“Immense-scale Machine Learning: The Big, the Small, and the Not Right At All”

Norman P. Jouppi, Google VP and Engineering Fellow.

We start with an introduction to the decade-long evolution of Google's Tensor Processing Systems, including the evolution of their numerics over time. We also discuss the unique requirements arising from serving billions of daily users across multiple products in production environments.

Training Large Language ML Models (LLMs) can require 100,000 accelerators working together in synchrony for months. And LLM inference can require exaFLOP/OP speeds with response times of under a second. This has driven adoption of lower-precision formats across the industry. Finally, since random errors from various sources are likely to occur at this immense scale during months-long training runs, fault tolerance is also a key feature of immense-scale ML systems. All these factors provide interesting challenges for computer arithmetic.

Norman P. Jouppi is a Google VP and Engineering Fellow. He joined Google in 2013 to lead the design of Google's Tensor Processing Units (TPUs). He is known for his innovations in computer memory systems, and was the principal architect and lead designer of several microprocessors. His innovations in microprocessor design have been adopted in many high-performance microprocessors.

Norm received his Ph.D. in electrical engineering from Stanford University in 1984. While at Stanford he was one of the principal architects and designers of the MIPS microprocessor, and developed techniques for MOS VLSI timing verification. He joined HP in 2002 through its merger with Compaq, where he was previously a Staff Fellow at Compaq's Western Research Laboratory (formerly DECWRL) in Palo Alto, California. In 2010 he was named an HP Senior Fellow. From 1984 through 1996 he was a consulting assistant/associate professor in the electrical engineering department at Stanford University where he taught courses in computer architecture, VLSI, and circuit design.

Norm holds more than 125 U.S. patents. He has published over 125 technical papers, with several best paper awards and two International Symposium on Computer Architecture (ISCA) Influential Paper Awards. He is the recipient of the 2014 IEEE Harry H. Goode Award and the 2015 ACM/IEEE Eckert-Mauchly Award. He is a Fellow of the ACM, IEEE, and AAAS, and a member of the National Academy of Engineering.

Symposium at a glance

Detailed Program — Subject to Change

Practical information

All coffee breaks and lunches are included in the registration fee, as well as Welcome Reception and Banquet.

Dinners on Monday and Wednesday are not covered by the conference.

More information on the social program can be found in the dedicated page.

Disclaimer: The opinions expressed in the papers on this website are the opinions of the authors/speakers and not necessarily the opinions of the IEEE or of the conference and its organizers.

Sessions:

ARITH 2024 ACCEPTED PAPERS

ARITH 2024 Accepted Papers with Abstracts