Training Solutions
AI Training Solutions
The Training Appliance. Training is one of the most computationally intensive applications in AI. Most low cost solutions are inference only. Their CPU or PIM based engines don’t have the horsepower to do the calculationsin a reasonable amount of time, or they simply cannot. For example PIM solutions use resistive elements like memristors ornon-volatile memory structures which are very slow to change and can only accept a limited number of modifications. This is a major limitation to most non-digital AI solutions. Normally training is done in the cloud or on large systems and the weights provided to an edge application. Of course this is not ideal because connectivity is required and information does not stay in the secure edge element.
AIStorm’s Charge Domain IP is Training Capable. AIStorm charge domain technology is very difficult. We can update on weights in 10’s to 100’s of picoseconds (a mechanism we use to recursively reuse our neurons). We can also perturb each neuron using a small charge change and use back propagation or some other goal function. AIStorm’s charge domain neurons are unique in that they can execute at speeds that would take several hundred GHz equivalent in a digital system to match. As such our inference chips can be made to perform localized training without the need for send data off chip. This is a very unique capability.
Analog Training. The figure above provides the equations of backpropagation which is a method of online training that is compatible with AIStorm’s charge domain analog neurons. For edge systems AIStorm utilizes its charge domain (analog) neurons to perform fast training. IoT and Edge devices can continually monitor and adjust to their environment. This type of training is typically used for systems with less than 16 bit precision, and preferrably 4-8 bit precision such as imaging monitoring systems, keyword spotting, sound spotting or sensors. Please see Patent #11,604,906 and other patents issued or pending.
Charge Domain Digital Training. For larger training systems charge domain digital can synthesize popular training models using charge domain digital gates that can significantly outperform transistor based solutions. Performance can increase by up to 10x for the same area in the same lithography, power can fall by up to 117x and area by 30x. Best of all only the most performance limiting areas of the circuits need be modified and can coexist with transistor based digital usually with no process modificaiton required.
Cost Functions & Error Contours. The equations above illustrate a potential cost function and contour generations that can be used for training an on chip neural network. These functions are implemented by direct perturbation of the network rather than mathematical multi-cycle simulation which speeds up the process significantly. AIStorm can craft custom on chip training solutions that can work in parallel with or during specific training periods or trigger on changes or user inputs.