[News] Google benchmarks its Tensor Processing Unit (TPU) chips

**StyM** · 04-06-2017, 05:53 AM

http://hexus.net/tech/news/industry/...nit-tpu-chips/

As Google relies heavily on compute-intensive machine learning for its core activities it has designed and rolled out its own Tensor Processing Unit (TPU) accelerator chips in recent years. The need for such a chip became evident about six years ago with the growth in computational expense due to technologies like Google voice search taxing the deep neural net speech recognition systems.

On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs.
The TPU also achieves much better energy efficiency than conventional chips, achieving 30x to 80x improvement in TOPS/Watt measure (tera-operations [trillion or 1012 operations] of computation per Watt of energy consumed).
The neural networks powering these applications require a surprisingly small amount of code: just 100 to 1500 lines. The code is based on TensorFlow, our popular open-source machine learning framework.

**StAndrew** · 04-06-2017, 07:36 AM

Comparing against a K80? Tumb on the scale or elbow?

**dave_graham** · 04-06-2017, 07:47 AM

Given the point in which these devices were determined to be made, K80 and Haswell v3 were top tier. The whitepaper is quite interesting in its approach to WHY, within a given thermal envelope, a TPU makes more sense. the ancillary discussion of FPGA vs. ASIC is interesting as well.

**AliG** · 04-06-2017, 08:27 AM

It also helps that the logic instructions were probably highly optimized to the type of work Google does. Nothing wrong with that (in fact it's really smart), but it doesn't necessarily mean a TPU is all that great for other applications.

**xlink** · 04-06-2017, 09:54 AM

So nVidia's growth is mainly primed on doing AI/ML work...
What's to stop something like this from impacting nVidia's server segment? I can see something like this dominating when it comes to doing DNNs.

**AliG** · 04-06-2017, 10:03 AM

Originally Posted by xlink

So nVidia's growth is mainly primed on doing AI/ML work...
What's to stop something like this from impacting nVidia's server segment? I can see something like this dominating when it comes to doing DNNs.

My best guess is lack of driver support and dev environment/toolboxes. CUDA is finally well established; switching to a TPU without a quality dev environment/toolbox would probably not be worth the headache right now.