SC11 Jen-Hsun Huang Keynote

NVIDIA Confidential
Exascale
An Innovator’s Dilemma
Jen-Hsun Huang, CEO
SC11, Seattle, Washington | Nov. 15, 2011

"Generally, disruptive innovations were
technologically straightforward,
consisting of off-the-shelf components
put together in a product architecture
that was often simpler than prior
approaches. They offered less of what
customers in established markets
wanted and so could rarely be initially
employed there. They offered a
different package of attributes valued
only in emerging markets remote from,
and unimportant to, the mainstream.”

A History of Amazing Advances
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW

Advances with Dennard Scaling
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
½ L every 4 yrs
8X – same P
1.68X per/W CAGR

End of Dennard Scaling
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
½ L every 4 yrs
8X @ 4X P
1.19X per/W CAGR

Supercomputing is Power Limited
1988 1998 2009
ZETTAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
100 PF, 20 MW
2012 2035
EXAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
70 PF, 20 MW
1 EF, 20 MW
CPU-only
“Titan”
6 PF, 8.6 MW
2019 2022

CPUs Fast But Complex
Optimized for single-threaded
performance
~50X energy to schedule
instruction than the operation
~20X energy to move data across
chip than the calculation

Super Efficient Processors Needed
Many simple processors with
minimal overhead
Locality reduces data movement
energy
Poor single-threaded performance
Innovator’s Dilemma!

PRINCIPLE #1
Companies depend on customers
and investors for resources.
Clayton M. Christensen
(1997) The Innovator’s Dilemma:
When New Technologies Cause Great Firms to Fail

PRINCIPLE #2
Small markets don’t solve growth
needs of large companies.

GPU Computing
“New Market” Disruption

Disruptive technologies underperform
established products in mainstream markets.
Cheaper, smaller, and frequently more
convenient.

Nagasaki University
Professor Tsuyoshi Hamada’s
Homemade supercomputer

GPU Computing
>350,000,000 GPUs with CUDA
>1,000,000 Toolkit Downloads
>120,000 Active Developers
>475 Universities Teaching GPU Computing
100% HPC OEMs offer GPU Clusters

World’s First
Whole H1N1 Virus Simulation

Lifecycles of fish
in Australia
University of Melbourne
Stars and galaxies
12.5B years ago
University of Groningen
Neural networks in a
self-learning robot
The University of Plymouth
Directives
65x in 2 Days 5.6x in 5 Days 4.7x in 4 Hours

The Road to Exascale
70 PF, 20 MW
100 PF, 20 MW
GPU-Accelerated
“Titan”
20 PF, 8.6 MW
2012 2022
PETAFLOPS
1 EF, 20 MW
2019 2035
CPU-only
“Titan”
6 PF, 8.6 MW
1 EF, 20 MW
ZETTAFLOPS

GPU-Accelerated
“Titan”
20 PF, 8.6 MW
2012
ZETTAFLOPS
PETAFLOPS
2035
70 PF, 20 MW
100 PF, 20 MW
1 EF, 20 MW
CPU-only
“Titan”
6 PF, 8.6 MW
1 EF, 20 MW
1 EF, 20 MW
2019 2022

GPU-Accelerated 1 EF, 20 MW
2012
ZETTAFLOPS
PETAFLOPS
2035
70 PF, 20 MW
100 PF, 20 MW
“Titan”
20 PF, 8.6 MW
1 EF, 20 MW
100 EF, 20 MW
2019 2022
CPU-only
“Titan”
6 PF, 8.6 MW

Huge Markets Support GPU
Workstation
5M
Console
20M
PCs
400M
Mobile
1,000M

“Teraflops” 5 Watts
ASCI Red @ Sandia Labs
1997 2019

“Tens” of Teraflops 100 Watts
Red Storm @ Sandia Labs
2004 2019

1000 Watts
“Hundreds” of Teraflops
Blue Gene @ LLNL
2006 2019

One Disruption Enables Another

‘Super’ Computing
From Super Computers to Super Phones

SC11 Jen-Hsun Huang Keynote

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to SC11 Jen-Hsun Huang Keynote

Similar to SC11 Jen-Hsun Huang Keynote (20)

More from NVIDIA

More from NVIDIA (20)

Recently uploaded

Recently uploaded (20)

SC11 Jen-Hsun Huang Keynote