19
$\begingroup$

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 standard introduced both quadruple and half precisions.

$\endgroup$
  • 2
    $\begingroup$ This was tagged MATLAB, but MATLAB doesn't (yet) support 16 bit floating point. $\endgroup$ – Brian Borchers May 21 at 3:46
  • $\begingroup$ See this stack exchange question which has some detailed information on this: stackoverflow.com/questions/49995594/… $\endgroup$ – Brian Borchers May 21 at 3:47
  • 3
    $\begingroup$ It would be worth clarifying whether you just mean CPUs, or both CPUs and GPUs. $\endgroup$ – Max Barraclough May 21 at 9:30
  • $\begingroup$ Thanks for your comment Max Barraclough. I did not know that you could alter the precision in just the graphics processing units (GPUs); I thought it had to be done in both. Your comment is very helpful. $\endgroup$ – Asad Mehasi May 21 at 15:57
20
$\begingroup$

Intel support for IEEE float16 storage format

Intel supports IEEE half as a storage type in processors since Ivy Bridge (2013). Storage type means you can get a memory/cache capacity/bandwidth advantage but the compute is done with single precision after converting to and from the IEEE half precision format.

Intel support for BFloat16

Intel has announced support for BF16 in Cooper Lake and Sapphire Rapids.

I work for Intel. I’m citing official sources and will not comment on rumors etc.

It is good to be curious about the relative merits of IEEE FP16 vs BF16. There is a lot of analysis of this topic, e.g. https://nhigham.com/2018/12/03/half-precision-arithmetic-fp16-versus-bfloat16/.

Non-Intel Hardware Support

The following is information on other processors. Please verify with the vendors as necessary.

http://on-demand.gputechconf.com/gtc/2017/presentation/s7676-piotr-luszcek-half-precision-bencharking-for-hpc.pdf lists the following hardware support:

  • AMD - MI5, MI8, MI25
  • ARM - NEON VFP FP16 in V8.2-A
  • NVIDIA - Pascal and Volta

NVIDIA Ampere has FP16 support as well (https://devblogs.nvidia.com/nvidia-ampere-architecture-in-depth/).

| cite | improve this answer | |
$\endgroup$
  • 10
    $\begingroup$ The first portion of this answer is a bit misleading. While Ivy Bridge did indeed introduce support for F16C, and that does technically mean that half-precision floats are "supported", the only instructions provided are those to convert to and from half-precision floats. No operations can be done on them whatsoever, so that hardly meets the requirement in the question: "I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is." It is merely a storage format. You're trading ALU conversion cycles for memory bandwidth/footprint. $\endgroup$ – Cody Gray May 21 at 5:25
  • 8
    $\begingroup$ I literally said it’s a storage type, all the savings comes from narrower data, and compute is done with single precision. It’s not misleading in the slightest. I said what support was present, which is half of the support possible. $\endgroup$ – Jeff May 21 at 13:30
  • $\begingroup$ Thank you all. I actually found each of these comments helpful. The initial reason for asking the question was due to listening to others arguing about this exact thing. $\endgroup$ – Asad Mehasi May 21 at 16:02
  • 1
    $\begingroup$ @AsadMehasi: also related on Stack Overflow: Half-precision floating-point arithmetic on Intel chips - yes on integrated GPUs since Skylake, but in the IA cores only for conversion to/from float on store/load. (Until BFloat support in upcoming HW) $\endgroup$ – Peter Cordes May 22 at 15:55
7
$\begingroup$

In my opinion, not very uniformly. Low precision arithmetic seems to have gained some traction in machine learning, but there's varying definitions for what people mean by low precision. There's the IEEE-754 half (10 bit mantissa, 5 bit exponent, 1 bit sign) but also bfloat16 (7 bit mantissa, 8 bit exponent, 1 bit sign) which favors dynamic range over precision, and a variety of other formats (NVidia's 19-bit TensorFloat, AMD's fp24, maybe more?). Most of this stuff is running on special purpose GPGPU-type hardware.

In contrast, float and double have generally agreed-upon meaning, as IEEE-754 compliant, 32-bit (23/8/1) and 64-bit (52/11/1) representations.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ While definitely non-standard, optical computing effectively uses half-precision (or less) floating-point operations. $\endgroup$ – Richard May 21 at 3:57
6
$\begingroup$

The accepted answer provides an overview. I'll add a few more details about support in NVIDIA processors. The support I'm describing here is 16 bit, IEEE 754 compliant, floating point arithmetic support, including add, multiply, multiply-add, and conversions to/from other formats.

Maxwell (circa 2015)

The earliest IEEE 754 FP16 ("binary16" or "half precision") support came in cc (compute capability) 5.3 devices which were in the Maxwell generation, but this compute capability was implemented only in the Tegra TX1 processor (SoC, e.g. Jetson).

Pascal (circa 2016)

Pascal family members have either "full rate" (cc 6.0, 6.2) or "low rate" (cc 6.1) FP16 throughput. cc6.2 was again a Tegra family product, TX2. cc 6.0 and 6.1 found use in a variety of processors in various product families such as GeForce, Quadro, and Tesla. "full rate" here refers to a rate that is equivalent to twice the IEEE 754 FP32 ("binary32" or "single precision") rate for the processor in question, when operations were done using a half2 data type (two half quantities handled in the same register and instruction).

Volta, Turing (2017, 2018)

Volta and Turing family members (cc 7.x) support FP16 at "full rate", and in addition use the format in TensorCore operations.

Ampere (May, 2020)

The recently announced Ampere architecture A100 GPU also supports FP16 in a fashion similar to Volta and Turing, and introduces additional capability for a TF32 datatype, which is a format where the mantissa is the same size (number of bits) as a FP16 mantissa, and the exponent is the same size as a FP32 exponent. Bfloat16 capability was also announced in Ampere.

Apart from the Ampere architecture processor recently announced, support and throughputs for 16-bit floating point operations (and other operations) across compute capabilities (including architectures) can be found in table 3 of the CUDA programming guide. The throughputs are per clock, per multiprocessor, so need to be scaled accordingly for the GPU type and specifications. These throughputs are not for TensorCore operations, and the peak throughputs are generally only applicable when processing on half2 datatypes (two half quantities packed together in a single 32-bit word).

| cite | improve this answer | |
$\endgroup$
4
$\begingroup$

You can find out if your hardware supports half-precision via:

$ lscpu | grep Flags | grep f16c
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req md_clear flush_l1d

The f16c instruction is documented here.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ Note that the C in F16C stands for Conversion; it only helps with memory bandwidth / cache footprint, not SIMD ALU throughput. You have to convert to float to do any math, so the number of elements per SIMD vector for FMA operations isn't improved. $\endgroup$ – Peter Cordes May 22 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.