Podcast Host – Vinayak Joglekar, CTO at Synerzip

Podcast Guest – Parag Vaidya, Technical Lead and IoT Expert at Synerizp

Summary

Industrial IoT is professed to open up tremendous opportunities for industrial automation, analytics, quality control, and predictive maintenance. Tune in to this episode where Vinayak, CTO at Synerzip, discusses a case study on industrial IoT with our guest, Parag Vaidya, Technical Lead and IoT expert. They talk about how industrial IoT solved a real-world challenge in the automobile industry. They also discuss the use of tools like Raspberry Pi cameras, python, and AWS Greengrass.

Podcast Transcript

Madhura Gaikwad 

Hello everyone. I’m Madhura Gaikwad and you are listening to ZipRadio podcast powered by Synerzip. In this episode, we are going to discuss a case study on industrial IoT. The term internet of things was first coined in 1999 by Kevin Ashton. Since then, IoT has revolutionized the way we live and work. Today, our CTO, Vinayak, will chat with Parag Vaidya, an IoT expert, about an interesting case study on industrial IoT. So let’s get started with this session. Welcome onboard guys!

Vinayak Joglekar:

Thank you, Madhura. Thanks for the kind words of introduction. We are going to talk about a very interesting subject that is industrial IoT. IoT has a wide range of applications, and synerzip has been doing things for the hospitality sector, as well as the industrial IoT. And today we have our guest Parag, and Parag has actually worked on industrial IoT in a very interesting case. We are going to hear from him on how he was able to solve a very interesting problem. It’s been our focus to take real world real problems because it’s not the tensors that we keep. And, uh, you know, we have been having sensors in industries and sensors have been picking up signals and they have been interacting with actuators and there has already been some level of automation that already exists in the industries, but IoT has enabled these sensors to work and communicate with the cloud and also with each other. And then in a very coordinated way, even use modern tools and modern methods like machine learning. So we are going to see all of that today, in our chat with Parag. Welcome to this Zipradio episode. So I would like you to first start telling a little bit about the business and the business problem that you’re trying to solve.

Parag Vaidya:

Yeah. So the problem the client was facing was his job, the entire batch got rejected because only one forging job failed in their test. And the problem was with the drill. The drill was not proper.

Vinayak Joglekar:

Before we go there, can you tell a little bit about the business? I mean, I understand that this was a forging shop with which they are supplying to the automotive industry, is that correct?

Parag Vaidya

Yeah, correct. So it’s a forging company, wherein we have deployed the solution. So this forging company, the main work was creating heavyweight jobs. These are forging jobs, so they have the metal blocks, blocks that get fit into some of their engines, heavy-weight engines. So that was the main role of that forging company. Another thing is their customers are in Italy. So the batch, which was rejected at client-side means it got rejected at Italy and, you know, they incurred a heavy fine against that as well.

Vinayak Joglekar

As well. Yeah. So that is what makes this pain point even more painful for the client that not only it got rejected, but they had to transport it back and forth between India and Italy. How heavy was this job? I mean, heavy is like, is it in kilos or is it tons or is it…

Parag Vaidya

The one job weighs between 20 to 40 kg? There are different jobs which weigh between 20 or 40. Yeah.

Vinayak Joglekar

And also is this in a state that is very hot? Like I understand that forgings are hot. So is there a temperature that is a constraint in this job or is this job in a cold state?

Parag Vaidya 

No. So the temperature was not the constraint. The constraint was the QC part, which comes on the conveyor belt’s assembly line. So these forging companies refer to them as a gantry. So the gantries are a bit hot during the work on that actual job. So when we implemented the solution, there is one cooling period for each job, which is 10 seconds. So we have utilized that time to test this job.

Vinayak Joglekar

Okay. So this is a smart thing to do, right? I mean, as the job is cooling for 10 seconds simultaneously, you’re using the same 10 seconds to also do the quality control. So you don’t add any additional time for quality control. So that is interesting.

Parag Vaidya

That was one of their major concerns, again, like, you know, because this is the first time they’re doing some automation work, otherwise everything was manual in that industry before that. So this is the first time wherein they’re implementing automation. This was very difficult for us to convince them initially. So earlier their QC was supposed to take three to four hours of, you know, and that is again after three to four hours per person and after production of those jobs. So once they’re in a particular QC room, then the man comes in and, you know, he checks it manually.

Vinayak Joglekar

Okay. So why was it difficult? I mean, for someone to convince, because instead of doing it manually and taking three to four hours of the shift time, doing it in parallel, in an automated way and simultaneously as the job is cooling would add no time at all. So what was the, was it a kind of mental block because they had never done automation before?

Parag Vaidya

Exactly. Correct. So that was a mental block. And to clear out that mental block, we initially created a POC and we deployed it and tested it for a couple of months. And after that, like they are now in production.

Vinayak Joglekar

I’m not sure that I, when we did this POC, was the customer ready to pay for the POC? Or were they like wanting you to take the risk on your side?

Parag Vaidya

Frankly speaking, the POC was completely free. It’s our risk completely.

Vinayak Joglekar 

So you were a hundred percent sure that it will work. I mean, you had to be 100% sure that it would work, because it was at your cost.

Parag Vaidya

Yes, yes. I have to make it work because it’s, you know, I was investing into it.

Vinayak Joglekar

Because you’re spending out of your pocket. So can you describe how technically it was configured? How was it done?

Parag Vaidya

Yeah, there are different parts. Let me explain what we did in the POC part. So we took a raspberry PI as a hardware Stripe. So we took a Raspberry PI and Raspberry Pi camera, a couple of load cells, industrial digital load cells, because those jobs are pretty heavy for measuring their weight. Again, some relays and some other hardware components along with this. So then after that in the software stack, we have taken Python as a primary language, open CV for template matching. And the major part is a green grass. AWS Greengrass, we have selected the green grass for future use as well here and along with this, we are taking the flask in a flask is a very lightweight, uh, Python web server. So we have created one interface where you can provide a positive team to the system so that you can upload it or different templates into the system at any time. So we don’t want to take your support post for that. That’s why we created that interface using flask and Python. The entry point for QC was the load cells where in that job comes in into that cooling area of that conveyor belt. It is a triggering point for our lambda functions. So we got, put it into using green grass and deployed it on Rasp Pi. And those were triggering the camera for taking the picture of the job. So the distance between the camera and that job is approximately 12 inch.

Vinayak Joglekar

Hey, let me stop you there. So as I understand it, the whole lot that you already said, so as I understand it, there was a load cell that would trigger, or that would start the event of capturing the photo or the snapshot of the job. So the load cell will only detect when the job comes in on the conveyor. Right? I mean, that was the primary function of the load cell, there was, it was not actually recording the way that there was no quality check of the weight of the job.

Parag Vaidya

Yeah, no. So there is actually, so there is a small deviation of, let’s say 20 grams, max 20-30, it depends on the job size actually. 20 to 30 grams of deviation was expected in the load. So that is the main purpose of measuring the weight of the job.

Vinayak Joglekar

Let’s say, I mean, for the sake of argument, the job did not meet the weight specification. It would not trigger the camera, right? It would just right straight away reject, if it is outside the 20 grams tolerance band, if it was too light or too heavy, you just reject without turning on the camera.

Parag Vaidya

Yes. Correct. Without spending much time, we just straight up reject it.

Vinayak Joglekar

And how were you informed that the job is rejected? I mean, would somebody be observing the on the cloud? So as I understand, there was a web server, which was running on the raspberry PI, which was using flask and somebody was browsing that web server using his browser somewhere remotely and that person would see that the job is rejected. But then what action would that person take or near the conveyer there some indication that the job was rejected?

Parag Vaidya

Yeah. So that’s a good question. That was again, the big challenge for us to integrate their PLC control. So this conveyor belt, this gantry program using PLCs, and we have to integrate our, you know, the solution to their appeal. So we send some signals to their PLC controllers, so that the entire conveyor belt stops. So that entire assembly line will be stopped. One buzzer was brought in by us and some LEDs also. So along with that flask interface, we are stopping the entire assembly line.

Vinayak Joglekar

So now let’s dig a little bit more into detail as to how about raspberry PI and that is communicating with two things. One is the LED and second, is it a system with, you have to send via UART, or how was the communication between the raspberry PI and their system PLC? So their PLCs would accept UART kind of communication?

Parag Vaidya

Yes, it’s a UART,  it’s a serial based communication. So they provided us with a couple of commands to stop the process and the reason for the stopping. So first we need to provide the reason for stopping it. And then after that, we need to, you know, uh, fire up that command for stopping the entire assembly line.

Vinayak Joglekar

So there was a command going from your raspberry PI to their PLC controller and then PLC controller would stop, that is straightforward, but simultaneously you also had to display something. You said there was a led display or was it just a bulb?

Parag Vaidya

No, that’s a small red colored LED bulb, so that, you know, because there are multiple assembly lines in parallel, all of them have the same solution deployed. So which of the assembly lines got the faulty job? To detect that,  we have provided the red LED bulbs and the hooter.

Vinayak Joglekar

Right. Yeah. So there was a hooter also, right? I mean, one of the GPIO pins would send out signals and trigger off a relay, right? There was one relay probably that would trigger up both your hooter as well as LED to go off simultaneously.

Parag Vaidya

Yes. So, uh, for that thing, like we provided one    hardware acknowledgement button because that was the requirement. Again, someone used to come there, maybe a plant manager or the assembly line head, and he will acknowledge by using that button and he will check their job as it got rejected. As it’s a PLC based stuff, we cannot start it from our rasp PI directly.

Vinayak Joglekar

Sorry to interrupt. There was no command. Just the way you had a command that you could send  from your Rasp PI to stop the assembly line. There was no way to start by giving a command?

Parag Vaidya

There is actually, but as these assembly lines have multiple CNC machines, So which CNC machine design was faulty, so there is no mechanism to verify which CNC machine was being worked as we are just, you know, our role was to perform the QC only. So this is something which is a serial process. The person, the plant head comes in and he manually checks, which CNC machine design was faulty, after that he starts the machine manually actually.

Vinayak Joglekar

Let’s say there was a problem in one of the holes that was drilled improperly. Which hole, or drilled by which CNC machine was determined manually. There was no, only thing what we could tell was it was not matching the specification. Either it was not  of the right diameter or whatever was the specification, or underweight or overweight or whatever would be the problem, but there was no way your process could correlate that to the machine, which was the cause, or which resulted into that defect.

Parag Vaidya

No, as of no, you’re right. As of now the role of this device is to just quality check the job, not to, you know, monitor the entire process, actually. That may be the future scope, I can say wherein, we can provide an interface wherein they can, you know, provide the inputs to their CNC machines as well from our interface directly.

Vinayak Joglekar

We will go there. But even before that, I have a question: you have a buzzer going up and then you have a man coming and taking care of everything that needs to be taken care of, and then pressing a button so that he has taken care of it to restart the process. So where is the need for flask and where is the need for a web interface. What  was the purpose of all that?

Parag Vaidya

Yeah. So the main problem here is like, there are multiple different assembly lines wherein they have different jobs going on and they might change job design as well. Once the production of a particular specification, what ended then there to land new job specifications. So they need to provide the positive template for us for matching. So it will be part of open CV. So we were providing that template as an input for open CV. So providing that template is the main purpose of providing this web interface on our Rasp PI.

Vinayak Joglekar

Okay. So on the rasp PI, you have open CV software running that matches the photo image of the job with the template that is uploaded by somebody remotely. And someone remotely knows what is running on which assembly line. And based on that the template would change or the raspberry PI would have all the templates and then it would just pick up one of them by prompting the user to select.

Parag Vaidya

No, no. So it is their responsibility to update that template. So there is only one positive template at a time.

Vinayak Joglekar

True. But then you don’t have like raspberry PI I think, as a 32 gig card, right. So you can store quite a few images there. It has a repository of templates from which the user was picking.

Parag Vaidya

So during this, you know, while creating this POC part, uh, they didn’t have any such templates. So it’s us, we luckily got couple of their jobs. And, uh, know we took the photos of that. So those photos are also taken manually by us.

Vinayak Joglekar

Oh. So the template is nothing but a picture taken by you using your camera manually. And then you’re trying to match that picture with what is in front of the camera. When the QC process is on, is that what you say template matching?

Parag Vaidya

Initial positive templates were taken by different cameras actually then after that per matching, it is taken by our Rasp Pi camera.

Vinayak Joglekar

So you had a, probably a better camera than the rasp PI camera to take the initial, I would say baseline photos which were used as the standard. Yeah. And then, uh, those photos were the templates with which you were matching. So can you tell a little bit about open CV for our listeners? Like what is open CV?

Parag Vaidya

OpenCV is an open source computer vision library. Mainly it’s a library, it provides multiple functions for doing real-time computer vision functions, like comparing images and image processing.

Vinayak Joglekar

Okay. So you had this open CV library and it was loaded on your raspberry PI or was it something in the cloud because you also mentioned there would be some cloud enablement. I’m not sure OpenCV is available as a library in Python or something?

Parag Vaidya

Yeah. It does support Python as well. I have an OpenCV library ported on raspberry PI. It was running on raspberry PI completely.

Vinayak Joglekar

It would do the work of matching the template with the actual image and you, or, I mean, what would be the nature of output? Was it yes, no,  or what was the nature? Input I understood. There would be two images to compare, right, or a set of images to compare with a set of images, but how do you see the output?

Parag Vaidya:

Yeah. So the output is in the form of some ratios. So it gives us the matching ratio, like how much it got mashed, like 80%, 90%, 95%, that kind of ratio we get actually.

Vinayak Joglekar:

And you had a cutoff that it had to, you know, be within a certain tolerance, maybe 90% plus, and they don’t lead or get accepted because otherwise be rejected.

Parag Vaidya:

Right. Right. As we mashed, like, uh, it was 96%, 96 to 97% was the acceptance criteria.

Vinayak Joglekar:

Yeah. And then you also mentioned that there are multiple such lines parallely running. So each one of them will have their own Rasp Pi, or, you know, I suppose so. Because otherwise it would be difficult, right? So you had multiple rasp pi’s and each one of them would be running their own instance of flask.

Parag Vaidya:

Yes. So it’s a part of a POC, right now we have deployed it in multiple incidents. Uh, five of them are running our POC. This is a complete POC right now. So, uh, the future scope might be like, you know, we’ll be having one centralized unit of processing for this one. And, uh, so that is the main purpose of having AWS GreenGrass here from the beginning actually. So it was not connected.

Vinayak Joglekar:

Yeah. So can you explain more, like what is Greengrass and what it has to do with managing multiple lines?

Parag Vaidya:

Yes. So Greenglass is a solution provided by AWS. It’s an IoT solution mainly. So they have provided the capability of running lambda functions on electronic bolts, or maybe small devices like Rasp pi, or I’m a six kind of bolts can deploy Docker containers, or we can secure lambda functions and things like that from onboard itself. We don’t need to be connected to the cloud for executing those Lambda functions. So it’s more or less, it’s a kind of, uh, environment where you can execute Lambda functions offline.

Vinayak Joglekar:

Oh, so that is interesting. So then why do you need AWS if you can do it offline?

Parag Vaidya:

Yeah. We need AWS for, you know, taking help of other capabilities or other services, other IoT services provided by AWS core  like, let’s take an example of, you know, if we want it to interact on multiple devices, then we can make use of MQTT, the broker and the subscriber provided by the Amazon. There’s a part of IoT core.

Vinayak Joglekar:

So for the information of our listeners, MQTT is the asynchronous messaging, very simple asynchronous messaging by which an IoT device can communicate with the cloud. They can use the same to communicate with each other as well. So if a device A  wants to leave a message for device B, the same can be done using MQTT, and then device B  we could pick up that message from MQTT, use it and respond back to devices so on and so forth. So there are topics. So it’s like a pub sub model like there are topics. And the topic name itself is the name of the queue. And, you know, it’s a very simple system using which devices can talk to each other. So what you’re saying is that’s one thing that you would need because there are multiple simultaneous, uh, assembly lines, and then they will need to communicate with the central iMac  that you are going to put there, and even if you have a Lambda there, that’s not going to get the message, unless you put it on MQTT on the cloud. That’s why you require AWS, right? Yes. Yeah. Okay. So you’re already there or that’s your future plan because you just mentioned that you did POC for five workstations or five assembly lines?

Parag Vaidya:

Yeah. So the POC was, uh, they are not connected to each other. They are running in parallel and there is no connectivity between them. The future is like, uh, having one central communication device wherein we’ll be having the same green grass at the same lamba functions, which we are running on these, uh, Rasp pis, uh, wherein we can have, you know, different camera units on those assembly lines, along with their flashlights or, uh, you know, irrespective weather conditions we have provided flashlights as well. And that one unit will be controlled centrally so that we can update templates at runtime and, you know, can manage it from a central location, you know, having different assembly lines.

Vinayak Joglekar:

So, these will be communicating with AWS, all the Rasp Pi’s are communicating with AWS using wifi?

Parag Vaidya:

Uh, yes. Correct.

Vinayak Joglekar:

There is no challenge in terms of using Wi-Fi? There’s a lot of electromagnetic noise with the assembly line. So many PLC controllers and part forging machines running and all that. So no problem?

Parag Vaidya:

No problem. Because our units are deployed in the cooling area, which is kind of a different room. I must say. There is no distortion of those signals.

Vinayak Joglekar:

Okay. Okay. But, uh, it will be interesting to see what happens in the case of if it’s, if the QC has to be done in a place where there’s a lot of electromagnetic noise, right. I mean, you would have to rely on, uh, probably wired devices. They need to be wired to their location. Yeah. So anything else that was interesting and that I didn’t ask you in the last few minutes that we chatted?

Parag Vaidya:

No. Like, uh, we covered most of the part. So there were two different challenges. One is the automated QC part,, saving a lot of time and also the energy on the people, manual QC people. And the other was  the cost effectiveness of this solution. They wanted it within the range of 10K or something, because initially changing of the mindset was one of the big challenges.

Vinayak Joglekar :

So which model of rasp pi did you use? I mean, did you use Rasp Pi zero?

Parag Vaidya 

No, it’s a 3B.

Vinayak Joglekar

So zero would be cheap, right? I mean, I think zero is under a thousand rupees.

Parag Vaidya 

Yes, yes. Correct. As it was a POC like that unit was available with me, so the problem with the rasp zero was, again, uh, you know, executing those Greengrass and, you know, things like that. So that may be a big challenge for that time.

Vinayak Joglekar 

Because Rasp Pi zero also uses the same SD card,  only thing I think Rasp Pi zero has a slower speed. That’s the only clock speed is more, that’s the major challenge. So that would be a future exploration for you.

Parag Vaidya 

Yeah. The main challenge with zero would be like the execution of Greengrass, because it needs higher processing.

Vinayak Joglekar 

Right, right. Yeah. I suppose so. Yeah. So thanks. Thanks. Thanks for, uh, giving us all this wonderful information and taking us down this trip in the world of IoT. And it was very interesting. I’m sure it was interesting for our listeners as well. So thanks once again, and, uh, we will be continuing our various episodes in this ZipRadio. Thank you all listeners for being patient and listening to us. Thank you.

Parag Vaidya 

Yeah. Thanks.

Madhura Gaikwad:

Thanks Vinayak and thank you for sharing this case study with Parag. Thank you everyone for joining this episode. If you are looking to accelerate your product roadmap, visit our website, www.synerzip.com for more information. Stay tuned to future ZipRadio episodes for more insights on technology, and agile trends. Thank you!

ZipRadio is available on these platforms