In a previous blog post, we mentioned that for all its amazing successes, there is a dark hidden secret of AI – its energy consumption. In order to deploy efficient AI everywhere, which is Embedl’s goal, we need to cut the energy consumption of AI drastically.
Analog computing to the rescue!
In conventional digital computers, data is transferred from the machine’s memory to its central processing unit (CPU) with every computation. Semiconductor technology has now advanced to a point at which computation is dominated by the energy dissipated during this data transfer – this can be anywhere between 3 times and 10,000 times that required for the actual computation, depending on where the memory is located relative to the processing unit.
Placing the processing units close to or inside the memory is the best way to improve the efficiency of AI computations. But this is difficult to accomplish with standard digital circuits, because ‘multiply–accumulate’ operations (which form the majority of neural-network computations) typically require chips comprising hundreds or thousands of transistors. GPUs that were originally designed for computer-game graphics improve efficiency by enabling computations to be performed in parallel across multiple processing units.
Analog computing makes a dramatic change by locating the compute inside memory! A group from IBM has designed an analog AI chip based on a technology known as phase-change memory, which relies on a material that switches between amorphous and crystalline phases when it is hit with electrical pulses. The two phases are analogous to the 1s and 0s in digital computers, but the device can also encode a state that sits somewhere between the two. This value is known as its synaptic weight, and it allows multiply–accumulate operations to be encoded in simple combinations without needing to move a single bit!
The new chip comprises 35 million phase-change memories that can store a total of 45 million synaptic weights — each of which is preprogrammed before the computations begin. The chip is able to achieve 12.4 trillion operations per second for each watt of power, an energy efficiency that is tens or even hundreds of times higher than for the most powerful CPUs and GPUs!
Although analog chips show enormous promise for combating the carbon footprint of AI, the technology is still in its infancy. To make it commercially viable needs more research and innovation. One part has to do with the hardware - to design the memory technology itself, the circuit connectivity and the architecture on the chip.
On top of this, one needs a compiler to translate code from programming frameworks like PyTorch and TensorFlow to the hardware level, algorithms that are optimized for this hardware and applications where this energy efficiency gain will be crucial.
Designing algorithms optimized for these analog chips will be crucial to be able to deploy the new technology in commercial products. Analog computing is inherently prone to generating errors unlike digital computing because it is vulnerable to problems such as thermal noise, manufacturing imperfections and variations in the thermal and electrical environment of the device. Future innovation will need to develop algorithm optimization techniques that are robust to such errors. Bayesian statistics and robust optimization methods offer a rich set of techniques to handle such uncertainties, and these are part and parcel of Embedl’s toolkit already, so we are in pole position for the future with analog computing!