The "KI-Oszilloskop" - Performance Evaluation of Edge AI Hardware and Software

Edge-AI has emerged as a promising paradigm for deploying artificial intelligence algorithms directly on edge devices, offering numerous advantages including reduced latency, enhanced privacy, and bandwidth efficiency. However, its adoption is accompanied by a set of challenges, particularly concerning the performance limitations of small hardware devices such as Raspberry Pi, Jetson Nano, Orin, or Xavier. Our analysis focuses on the performance of Edge-AI on these platforms, comparing a diverse set of model architectures including Fully Connected, LSTM, Conv1D, and Conv2D networks.
We explore the implications of utilizing model languages such as PyTorch, Keras, TensorFlow Lite, and ONNX, and analyze their suitability for deployment in resource-constrained environments. Through comprehensive benchmarking and experimentation, we aim to provide insights into the suitability of these model architectures for deployment on resource-constrained edge devices. By examining factors such as inference speed, memory usage, and power efficiency, we seek to identify the most effective model configurations for Edge-AI applications in real-world scenarios. This enables us to assist customers in informed decisions-making and finding suitable solutions in this rapidly expanding field.
Our work aims to develop a tool named KI-Oszilloskop that simplifies the analysis of hardware and software properties, relevant to Edge-AI environments. It helps to examines various hardware and software configurations to effectively navigate the complexities of deploying AI at the edge.