Flir’s Firefly development camera taps Intel’s Movidius Myriad 2 for on-device object detection
Flir, a Wilsonville, Oregon-based firm that’s among the largest producers of thermal imaging cameras and sensors in the world, teamed up with Intel to create an open camera platform — the Flir Firefly — designed for artificial intelligence (AI) systems development. Flir’s pitching it as an autonomous building block for a range of industries, including robotic guidance, automated quality inspection, biometric recognition, medical imaging, and agriculture analytics.
It’ll be available in 2019, the company says, and on display at the Vision conference in Stuttgart, Germany in early November.
“Automated analysis of images captured by machines is a key part of our day-to-day lives that few of us think about,” explained Flir CEO and president James Cannon. “The quality, affordability, and speed to market of items like our smartphones or the food on our tables are made possible by systems using cameras doing both inspection and automated production.”
The Flir Firefly, which boasts four general-purpose input/output GPIO lines and a USB port and measures roughly the size of a quarter, is built on Intel’s Movidius Myriad 2 vision processing unit (VPU) — the same chip inside Google’s Clip camera and Project Tango augmented reality prototypes and DJI’s Phantom 4 drone. It’s optimized for image signal processing and inference — the point at which a trained AI model makes predictions — on-device, minimizing the need for internet connectivity, while simultaneously boosting battery life.
Above: The Flir Firefly’s architecture.
Image Credit: Flir
Specifically, Myriad 2 comprises (1) hardware chips for dedicated imaging and computer vision tasks, (2) a dozen 128-bit streaming hybrid architecture vector engine (SHAVE) processor cores that speed up the sort of vector data processing that’s at the core of neural networks, and (3) general-purpose RISC processor cores that parse and schedule workload processing on the SHAVE cores and execute inferences. It can handle input from up to six HD cameras and perform gesture tracking and object classification, according to Intel.
That sets the Flir Firefly apart from development kits like Amazon’s DeepLens, which use board computers attached to camera modules — in DeepLens’ case, one containing an Intel Atom processor — rather than AI-tailored systems-on-chip like Myriad 2.
On the imaging side of the equation, Flir tapped Sony for the Flir Firefly’s 1.6-megapixel Pregius CMOS sensor, which can shoot at 60 frames per second. The company says future iterations of the Flir Firefly will offer additional sensor options.
The Flir Firefly’s secret sauce might be its firmware. It supports the UBS3 Vision protocol — a protocol for USB-based image-processing devices that draws on the GenICam standard, a programming interface for industrial cameras — and selectable regions of interest, plus pixel binning (a technique that reduces noise and improves frame rate) and 8- and 16-bit raw pixel formats. Additionally, it offers control of the camera’s GPIO ports, enabling it to trigger devices.
The Flir Firefly allows developers to load trained neural networks onto the VPU via Flir’s Spinnaker software development kit (SDK), and, if they’re using an Intel Movidius Neural Compute Stick, deploy existing networks directly onto the camera. Intel launched the $79 Neural Compute Stick — which works by translating convolutional neural networks (CNN) developed on Facebook’s Caffe framework into an embedded neural network on the VPU — in July 2017 and says it can reach up to 100 GFLPOPs of performance within 1W of power consumption.
”The Intel Movidius Neural Compute Stick enabled Flir to rapidly prototype, streamlining the early development of machine learning in the [Flir] Firefly,” said Adam Burns, director of computer vision products at Intel. “Now the Flir Firefly uses the compact, efficient Intel Movidius Myriad 2 VPU to perform real-time inference in the camera, without compromising the amazing levels of miniaturization that Flir has achieved in this device.”
Flir, which was founded in 1978 and has more than 3,500 employees, says more than 500,000 of its thermal sensors are built into the driver warning systems from General Motors, BMW, Audi, Mercedes-Benz, and Volkswagen. In June, it released an open source dataset of 10,000 infrared light images to jump-start development of autonomous car systems. And in September 2017 it announced a VGA sensor designed for self-driving cars. Flir also offers a software developer kit — the Autonomous Driving Kit (ADK) — with tools to build and train the sort of neural networks used in collision avoidance systems.
Source: VentureBeat
To Read Our Daily News Updates, Please visit Inventiva or Subscribe Our Newsletter & Push.
