Lamin from Infineon recommended that we talk to Jon Linden from Ekkono. The Swede explains his ML on the edge approach and why Volvo, Siemens and ABB work with him.
The podcast is growing and we want to keep growing. That’s why our German-language podcast is now available in English. We are happy about new listeners.
We thank our new partner [Hannover Messe]
(https://www.hannovermesse.de/en/)
Our guest: https://www.linkedin.com/in/jon-linden/
this podcast is supported by Hanover Messa your meeting place for the industrial [Music] Community hello everybody and welcome to new episode of our industrial AI podcast my name is Robert bber and we have a new partner many thanks to hanova Messa we are already looking looking forward to April and our AI event at the fair so we start into February I take a brief look at the topics for this month we start today with John from eono in Sweden next week we talked to an ex Audi employee who wants to bring reinforcement learning into production ABB and Marcus Bor from code scene follow at the end of the months as an announced we start today with John uh a special Thanks goes out to lamin from infinion episode 163 he gave us the hint to talk to John John’s promise machine learning on the edge okay we already heard about that using our SDK you can rapidly develop and deploy self-learning predictive and personalized smart features to your iot products he explains how he does it and what his customers Volvo or Seamans are doing with it in the podcast so enjoy listening and greetings to Peter to Thailand enjoy your holidays and my guest is John Linton from eono AI swedish-based company hello John how are you hi good morning I’m I’m good please introduce yourself and your company in a few sentences sure so as you said John Lynden CEO co-founder of econ Solutions a swedish-based edge machine learning software company I have a background as a serial entrepreneur I spent a significant number of years in the Telecom space before starting eono and for the last four or five years in that space we we saw that everyone was talking about iot and what’s going to happen when things get connected and stream constantly streaming data and that was the foundation when when I met my co-founder our CTO R king who had been leading research on predictive modeling so machine learning and found a way to actually run machine learning onboard connected devices which is a way of approaching that that challenge of everything being connected our last guest from infinion gave us the advice to do an interview with you yeah do you know why why he gave us this advice do you have a special approach on machine learning we do uh and and we work with infinion and for natural reason since we are an all software solution and of course you need a host for this which is normally a a a standard processor and then two things our unique approach is based on the fact that we found early on that traditional machine learning is all about collecting data from thousands of devices and look for common denominators but that is pretty rare I mean we our very first customer especially in industrial use cases absolutely I mean the application where they’re being used the environment the surrounding climate all these different things are different from machine to machine and then learning generic insights is is not very useful so we flip things around and learn individually by actually doing machine learning out on the device so we learn individually per unit and we can do this even on very constrained processes down to just more mcus like a cortex m0 plus so you learn on one machine but you can’t compare with the machine then well you can and and the thing is you always do your homework before you do the implementation so you generate a machine learning model which is based on the collected data where you figure out things what are the input values what’s the frequency of that in those input values what is the target value that we’re trying to predict what’s the prediction Horizon you set up pipelining like sliding averages and and lags and so forth you generate a model uh that is played on that this is generic model that you apply and then you can start training it individually at the edge meaning that you adjust it to those local conditions which actually makes a big difference because that way without telling the machine learning model what deviates for more that specific for example the the climate the temperature the altitude whatever it might be that is different you don’t have to tell the machine learning model that is automatically incorporated as it learns in operations and then you can train it and you can do inference on the high frequency sensor data that’s being generated on the de device rather than the blunt averages meaning second average or minute average that are typically sent to the cloud and you do this without any manual Intervention which improves the the whole data Integrity since you can send more onized insights to the cloud rather than sending all the sensitive or data that sounds to me like an Federated learning approach absolutely as an extension based on the fact that we can do individual learning we can do Federated learning where you do aggregated learning from many but but the benefit is that instead of sending all this huge data to the cloud and then processing it you actually do the learning in step one on the device can send the model the insights which are a fraction in insights to the cloud and and then combine them through Federated learning in the cloud and still they’ve been trained on this high frequency realtime sensor data like 12 KZ of of vibration data instead of like a 1 second average uh so you actually get better insights and better granularity with way way way less data we often talk about deep learning and uh neural networks we always talk about big data but in the industry field we do not have these big data how much data do you need to train your model on the edge that’s a very good question and first of all let me emphasize that we don’t focus on deep learning and that’s for a few different reasons I mean everyone’s predicting billions of connected devices and very few of them will be able to afford a camera as a sensor and deep learning is perfect if you’re doing like like leadar radar computer vision that kind of stuff but most products all out of these billions of connected things will have regular traditional sensors and and that is typically time serious data and that’s what we focus on to learn based on that and deep learning the computer vision they will get output that we can use for example there’s a human being going to the right and and and so forth that can be valuable input to the more Dynamic kind of machine learning we do where we combine it with other sensor data so so that’s our focus and and for that this approach of of doing individual training and then applying maybe a shallow neural network instead and we also support linear regression and Rand Force we’re quite pragmatic about what is best to actually solve the case is really useful to uh to do that what you deliver to your customer is it a software tool is it a application is it a application with the added Hardware or can I use my Edge device I have by my own or what’s the product it’s a software tool sport on it’s 100% software it’s platform agnostic we do all the development in C++ to make it as agnostic as possible and then we generate a native C version which is how we can scale down to very small devices there’s an API so you you can use Python and and C when you do the development of these models and then you can use the same model and incorporate it into your CM uh implementation there’s a whole SDK with auto ml functionality that helps you with evaluating which algorithm might be the most suitable for the data set and so forth It’s a comprehensive tool to help expedite the implementation of edge machine learning that’s what we provide and everyone runs the same runtime that you can compile you actually get the source code that you compile into your your Edge firmware Edge product firmware and then the model is separated which means that the customer always owns the result that’s being generated based on the data and you can even hot swap it if you do Federate learning and come to conclusions where you want to upload a new super model with new learning so it’s perfectly designed for this purpose of implementing a lot of different machine learning based functionality of features on board connected devices it sounds very interesting my question is the most difficult part for our industrial AI users or applications is the engineering to bring the software on the edge to run the software on the edge how difficult is that with your solution the the thing is at the end of the day when when you do the actual implementation that’s between 10 and 15 lines of code but of course that that’s based on doing the homework and and we are on a mission to make this as available and as simple as possible because your spot on I mean the thing is every company is not sitting with a data scientist some do and and they’re pretty exhausted with all the different things you’re going to do in in all the different projects but we are looking at the combination between three parties you have the data scientists that have to help with understanding the analytics and and the insights could be a machine learning engineer someone who has a good insight into what it takes to to draw insights from data data driven decisions you have the engineer which is very very important who possesses the domain expertise because I mean we can do all kinds of insights using machine learning but if we don’t understand what it actually says it’s pretty useless so someone who can really understand the problem and interpret the output that we get in terms of like change uh deviation scores and and and anomal detection that kind of stuff and then we have the software developer that needs to implement this because this goes into firmware so it’s not like like you do a PowerPoint production and just just just email it it has to go through a proper validation and implementation so three users and with those three combined if we can make it simple enough for them to use it Standalone then this can really happen then we can see a big big increase in number of smart devices out there instead of just connected devices out there and we we come pretty far in making this simple we’re still Pioneers in in in an emerging space that that people are not that familiar with so it’s not just click and go but we we’re getting there we made significant progress as the global Innovation leader in this space how long does does it take with the initial learning when you install the software and how long does the initial learning of the model takes time oh yeah I realized I didn’t ask your question before about how much data we need and and and what it takes and that’s that’s a highly relevant question unfortunately the answer is is pretty much how long is a strain it’s it’s very hard to say it kind of comes back to Case by case but what we have seen is that we need surprisingly little data we have one case where they were actually lacking sensors in total and and it was not possible to do any kind of intrusive implementation of sensors and and this is for heat exchanger where you have fluids going in through different directions and and meeting and and you want to measure it so we had to do the measurements outside and the only measurement we could get was the temperature inbound temperature outbound on two different streams and the pressure and with that alone we could actually see how the effect and and the performance of the heat exchanges started deteriorating and use that to calculate remaining time to service how long does it take before you actually have to stop and do cleaning of the slates in the the heat exchanger to make it run efficiently so you need surprisingly little amount of input streams and still get some some interesting results there are cases where we have thousands of input streams but that’s the more the merrier in many cases but you don’t need a lot of input data and the time it takes depends very much on what is the cycle so if you have something that that that needs in this case like the cleaning if it needs cleaning every second week or every fourth week then then you need some repetitive patterns to actually learn but if it takes five years then the training time takes longer but you can still find some insights on the way of getting there but there are some characteristics that are favorable to machine learning and there are some that are not and and this is something we’re helping our customer is understanding when can I expect a certain result and what does it take to really train this but still independent doing taking this approach of doing incremental learning on the device in operations would always be more efficient because otherwise you’re going to start collecting data and you might need two years so you get the seasonal changes and compare between year one and year two and by the time you get there in two years you’re going to have a new product new configuration new sensors and then it’s going to be obsolete so then you have to start all over again so many companies actually fail because they they try to collect all the data they need and then make decisions and then it should be constant while they operate in an Ever Changing environment where they have to adjust to it and and then this kind of incremental approach where you do lifelong learning on the device for how the device is being used is very applicable can you go a little bit deeper what are you doing with the data on the software especially from the cleaning use case sure what happens is that first of all the way we can do this on very constraint devices is that we do everything in memory don’t you reduce a lot of read and write you don’t do batch training which is extremely resource demanding for short period of time but we do it for every instance of data so we do streaming analytics we do incremental learning on the data so we do everything in a process it starts with doing the pipelining that I mentioned to calculate a new sliding average or a lag and so forth then we do the inference then we do the training then we do the change in anomaly detection then we do sensitivity analysis we can even do form of prediction so all of that can be done in memory which is very eff efficient so compared to just doing inference you need a very small time series data base right yeah and then thing is it basically builds up over time if the pipeline is being updated you don’t really need the database so to say in particular the the pipeline is being updated on the slide for the training you typically need some kind of historical data but we take what you have which is in many cases even with the large corporations that we work with quite limited and make the most out of it the best out of it then we implement it and then we refine it in operation so you’re right you don’t need a very extensive database to start with do you use time series you mentioned right yeah what kind of Time series do you use so I mean in most cases what we have seen is that a lot of the the things we work with and we have been doing project in industrial equipment so it’s very suitable for us things like pumps and and electrical motors and and compressors and that kind of St Automotive Energy building automation so forth what we have seen is that most cases they have pretty traditional sensors it’s a question of temperature on things that are rotating it’s a temperature of pressure in things that like pumps and so forth it’s a it’s a question of of RPM it’s a question of voltage that goes into a process that kind of input variables are very common input to to to our machine learning do you use a special time series database there no not really since we are doing the the processing in memory we we don’t let’s talk about two use cases you mentioned the cleaning use case but you have one special use cases in the medical industry what you did there yeah the first one you mentioned that is a typical maintenance case and typically boils down to two kinds of of primary categories of of of use cases maintenance and optimization so the second one it’s it’s for a ventilator and and that has been on everyone’s mind due to the pandemic all of a sudden and what we learned in the process is that every human breathe differently so it’s is unique how we breathe now if you apply a ventilator that helps you breathe and you do it the same for everyone that’s not necessarily very healthy for your lungs because you might apply too much pressure too little pressure Etc so since we’ve taken this approach to individual learning at the edge we applied a solution where we actually learned the the a profile of a typical breathing stroke for the patient the user in in L than five breathing strokes and applied that to optimize the ventilator accordingly so it actually replicated that to actually be an healthy approach to how you do ventilation so very interesting case and and and a typical one where it comes to learning and applying to optimize how you use it to to run better I think it’s very interesting because you have individual Models All Around The Factory or the hospital where you use the ventilators and then you can use this indiv idual models to build a generic or a baseline model for the start when you have a new ventilator or when you have a new machine or when you do a optimization over the whole Factory right yeah and the thing is more and more if you look at the business side and the commercial side more and more companies get reluctant to share data we have customers that that tell us that that their customers say no no no you’re not taking any data off the premises which makes sense that’s sensitive data that’s production data so the only place where you can learn is that The Edge and I think it makes a lot of sense when everything is starting to get smart so the the ball bearing manufacturer wants to provide intelligence the motor manufacturer that uses the ball bearing wants to provide intelligence the machine that uses the motor wants to provide intelligence it makes much much more sense that you feed that kind of intelligence locally so the bull bearing tells the the motor that tells the machine because that’s also the customer relationship you have rather than the bull bearing extracting data to the cloud processing it and sending it back to the motor manufacturer doesn’t really make sense so this kind of edge approach approach driven by the fact that you don’t actually you will not have access to all the data you want in the future because customers get more and more educated and reluctant to share it will drive a very big change in in how we look at data data management in the future what are your next steps with your company with the technology well we’re still a small company we’re pioneering this space and we have a few steps to go before we have world global domination but I mean our ambition is to provide a very easy to ous comprehensive toolkit for implementing smart features on your device any kind of device so a very broad uh approach to it we now start to see that the the market is catching up with us and our way to reach the market is to go through partners because as I said we’re a small company the only way we can scale is that we work with those that build the solutions for the customer the comprehensive solution because we don’t do Hardware we don’t do sensors we don’t do Cloud connectivity and everything else so we work with system integrators and that kind of Partners who can build those so we can reach a much much more Global Market that’s why we’re putting a lot of emphasis right now to get the product out there at SK and there there are a lot of famous ABB you work with ABB and uh can you name a little bit more ABB and zens absolutely vvo among our customers a lot of prominent customers and and the the reason is that the DAT actually we were aiming originally for the the the like the the tier two because we thought thought that they were going to be faster but they were typically suppliers to the tier ones and were getting instructions from those so that’s why we had to pursue customers like Volvo ABB alpal Laval husa cus energy eh so different companies in in different vertical but but very prominent customers are very very demanding customers which is something we’re thankful for this stage so you are satisfied with a position as a second tier supplier you are happy with this this situation absolutely I mean the thing is that that our objective is that as many as possible can use this and I’d be happy to take a position where it’s Econo inside and where someone else can can actually provide that solution I’m perfectly fine with that as long as we can reach as many users and devices as possible and is it a license model or how do you sell it y it is this it’s software it’s all license model you you license the tools at an annual subscription and then you pay a per copy C when you start Distributing it what about the next technology steps what are the next steps in technology well you hit the nail on the head when when you asked about Federated learning before because that is kind of the next step now the customers have not been there I mean first I have to buy into the concept and the idea of doing individual learning because that is the turnkey to to to doing the aggregated learning that we provide uh but that’s starting to to to really understand that okay this is really interesting now we can learn and automate on an individual basis but we want to get that feedback loop where we can use this for generating better starting models or super models and for for product development so then we need Federated learning so there there stting to understand and request it so so that is probably the next big step in in in my opinion when it comes to to product development I heard some Rumors in the market that there’s a big patent war on Federated learning do you recognize that too we haven’t been in involved in that at this point in time and I think what I have seen so far and with some disclaimer is that where when it comes to this it’s kind of like when we talk about machine learning it comes in very many different flavors we have one way of approaching and how we use Federated learning in many cases people or a lot of companies are using it more for the purpose of doing like distributed computing using capacity in in in Edge devices to do learning so very different approaches I hate that kind of it’s that’s not the best best used time to pursue different infingement and patent cases I I hope that we can can stay away from it but we’ll see you’re BAS in Sweden and in the west of Sweden at the west west coast correct correct in VAR Bay on the west coast yeah because we had an episode with a guys from HMS maybe you know them yeah yeah yeah absolutely and they are also developing Edge devices do you work together with the edge devices uh guys because that would be very interesting for them too to add the technology to their to their device yeah absolutely we know HMS for for natural reasons we know it quite well it’s not very far from here I actually live probably a pretty good golf drive away from the from the CEO but that is kind of the next step for us that that right now we have been focusing on reaching the end customers helping them understand how they can really make use of this the business case the user case user case and everything else and prove it and then that will generate demand but of course at the at the end of the day just like we mentioned inion at the beginning of the episode we will be Implement that as a standard feature on chipsets on Hardware platforms on edge devices so you can activate it our hold back to our ambition to make it as simple as possible being pre-installed being pre- available and being able to activate it is a way to to to help with that so that is definitely part of our long-term strategy to have technology Partnerships like that okay so greetings to Stefan dstr absolutely it was a pleasure to talk to you I think it was at the beginning of this podcast I think in the early 50s or something episode it was a pleasure John for talking to you about your your company your product and your approach I think it’s very interesting it was a pleasure to talk to you and all the best to you and greetings to Sweden Mutual thank you very much [Music] [Music]