AIStorm’s new Mantis Family of AI-in-Imager & AI-in-Audio chips are the first and only AI-in-Sensor solutions capable of accepting pixel charge data or audio MEMs charge data directly in its native charge form. The result is the world’s only family of solutions capable of image or audio based smart AI wakeup on a person, face, object, behavior, sound or word. Mantis includes CNN and FC capability with the flexibility to implement a variety of popular machine learning models.
AIStorm’s “Always On” Mantis family of AI-in-Sensors devices cover a broad range of applications while drawing negligible power until they detect template based triggers such as a person, face, action, behavior, gesture, object, sound or word. Mantis offers “always on” ultra-low latency and power consumption of <15uW enabling a new class of products with extended battery life. The family includes various imager resolutions, integrated switch charge ISP macros, and on board microcontroller as well as audio solutions with and without integrated microphone, audio processing and post processing. Audio processing is done in the charge domain and does not require a digital input stream.
Although Mantis uses charge domain processing and communicates with pulses between neurons, it still uses a standard TensorFlow development methodology and a bridge allows the results to be downloaded to Mantis ICs. Mantis can accept input data from its imager inputs , but also accepts information through the SPI interfaces. Data can be output using the digital interfaces as well to create training sets. Training is done on the PC with the weights and execution information loaded to Mantis which provides inference execution.
All cluster inputs and output can be directed to a time to digital converter to interrogate any data in the neural network and these outputs can be dumped to SRAM and sent to a host PC during the debug process.
People Counting · Face Detection · IR Beam Replacement · Behavior Monitoring · Activity Monitoring ·
Wellness Monitoring · Smart Cities · Eye Tracking · Key Word Spotting · Pupil Dilation · Noise Monitoring · Sound Identification · Vibration Monitoring · Object Detection
Mantis is a unique machine learning solution that includes all the components required to implement a low cost camera solution. Mantis includes an on board imager, “always on” neural network functionality, an LED driver, in chip optics, power management and exposure control functions, clocking, and a powerful AI engine capable of convolution and fully connected implementation of popular imaging networks. Certain variants also include ISP and a microcontroller. Mantis wakes up on an AI template, with the pixels being the input layer to a charge based neural network capable of accepting pixel charge directly. This is different to all competitive solutions which cannot work directly with the pixel charge information and the result is low power and negligible latency. Mantis Audio provides spectral pre-processing and similarly allows wake up on audio at very low power. After wakeup powerful CNN and FC constructs may be stitched together to create arbitrary networks and weights may be loaded several layers at a time for deep networks.
Mantis C100A-EVP includes a Hardware Evaluation board and a Software Development Kit (SDK). It features Mantis C100A AI-in-Sensor Imager wakeup imager chip & its accompanying MantisNet architecture. The SDK is divided into two portions for an easy-to-use and familiar development environment which includes:
The AIS-C100A is supported by Mantis SDK and Hardware Development Platform. Mantis SDK offers a graphical development tool which uses TensorFlow to train and evaluate AI model implementations on a PC. Once satisfied the weights and flow may be downloaded to the C100A “Mantis” chip. Alternatively, the Hardware Development Platform may be used.