For decades we relied mostly on CPUs (Central Processing Units) to perform computations. Their performance was growing rapidly thanks to increasing clock frequencies and doubling the number of transistors approximately every two years as stated by the Moore’s Law. When the frequencies could not go any higher due to heat dissipation issues, we continued scaling with the Moore’s Law by creating multi-core processors. This scaling has slowed down in around 2012.
During the first decade of the second millennium many people, including myself, have realised that CPUs are excellent for serial tasks but GPUs (Graphics Processing Units) are more suitable for parallel tasks. I have started programming GPUs in 2008 and developed the first multiple time-scales artificial recurrent neural networks and multi-GPU back-propagation through time GPU algorithms that would be used to develop action and language acquisition for the humanoid robot iCub. This PhD research would eventually get me to work at NVIDIA research in Santa Clara where I was probably the first person to evaluate the performance of artificial neural networks and the training algorithms for GPUs. I say this because I had to convince NVIDIA to let me work on these topics and needed to explain why I believed they were so important. This was just before the deep learning took off and just before NVIDIA started investing heavily into this area.
GPUs are now used pretty much everywhere for running the training of deep learning models as this involves sustaining some serious processing power usually over a period of several days. However, once the model is trained it can be deployed on various other architectures that can run the inference much more efficiently and in situations where standard GPUs would not be feasible.
Recent advancements in chip technology gave birth to novel architectures designed to perform incredibly complex computations using very little power. This in turn enabled novel solutions for ever more popular domain of edge computing. Unlike cloud computing, edge computing is capable of processing data right at its source resulting in real-time or near real-time data analysis, lower operating costs, reduced network traffic and improved application performance.
One such architecture, which I decided to evaluate, was Intel Movidius Myriad 2 chip offering high-performance at ultra-low power. I came across news about a very new mini-PCI Express module developed by UP featuring Myriad 2 processor. I have previously used Intel Movidius Neural Compute Stick that has the same processor but having a mini PCI-e board opens up a whole new list of possibilities, so I decided to test it out. This new PCI-e module called AI Core enables Artificial Intelligence at the Edge. Thanks to its form factor it can be seamlessly integrated in most of the industrial computing edge devices. It is powered by ultra-low power high-performance Intel® Movidius™ Myriad™ 2 2450 VPU with 512MB.
For this demonstration, a single AI Core module was plugged into a mini-PCI Express port of UP Squared, the world’s fastest x86 maker board based on latest Intel platforms Apollo Lake and the successor of 2015 Kickstarter supported UP board. UP Squared was setup with Ubuntu 16.04 Linux distribution and the AI Core module was used to run convolutional neural networks trained to recognise and localise various objects. These neural networks are based on the Caffe implementation of the mobilenet-ssd architecture. The following video shows three different models running on this setup in sequence at around 10 frames per second. None of this model is particularly well trained or optimised. The main point of the video was to demonstrate how this setup can be used for various edge applications.
As a Head of Innovation at Cortexica, I look into emerging applications that I believe hold a lot of promise for the future. I am convinced that Edge Computing and particularly the Intel Myriad family chips on the AI Core module will enable many exciting applications. Cortexica is already working with Cisco on some cool projects (https://emear.thecisconetwork.com/site/content/lang/en/id/8533) I already see countless applications of edge computing in our business.