Edge AI combined with Agile means project success!

It's Monday morning and a client is calling you to discuss a new project.

The client wants to design a custom electronic device that recognises and counts cars on the road. The client tells you the device has to dissipate less than 0.5 Watts and, of course, he wants to have the solution ASAP.

Implementing an AI algorithm on a physical device in the field instead of in the cloud is what we call Edge AI. In this particular case we design a car recognition algorithm and implement it on an electronic device that is physically close (and connected) to a camera. Running AI algorithms on the Edge of the network comes with lots of benefits like data privacy, real-time behaviour, reduced power consumption and a significant decrease is data transmission costs.

You tell the client that your company can do it. You estimate the timeline of the project to be 9 months with a certain budget. A contract is signed and the project starts.

After 6 months you decide to show the first version of the product to the client. However, he doesn't like it. Your AI model running on the device is not accurate enough in recognising cars. The artificial intelligence (AI) model is also outdated and your electronic board is way too big. Oh and by the way, the client wants to further reduce power dissipation.

As it turns out, it seems you need to start over again from scratch. You and your client have burned a lot of time and money in developing the first product which eventually was not the product the client expected.

It's the year 2020 and traditional design methodology does not work for the new Edge AI market.

State of the art today, outdated tomorrow

The entire industry revolving around artificial intelligence is in constant evolution. AI algorithms which are considered state of the art today, are rendered obsolete tomorrow. In the below table you can see a list of names of the most famous open source (freely available) AI models.

These models can do various things like object recognition, sound classification and data pattern recognition. I sorted them according to release date of the academic paper.

As you can see, several different models were invented with just a few months in between release dates. Even in the technology landscape of today, this rate of innovation is something rarely seen before.

The problem with the high release rate is that a new AI model usually comes with a new computational scheme. Meaning that the new model is computed differently as the one before and thus previous hardware cannot efficiently process the new model anymore.

Designing hardware takes several months to complete. This makes it notoriously hard to develop custom Edge AI hardware using traditional design methods. We need some kind of project framework that gives us more flexibility and the ability to adapt to change.

Agile Methodology

Agile Development refers to iterative development. Deliver the solution incrementally instead of all at once at the end of the project. Agile processes generally promote a disciplined project management process that encourages frequent inspection and adaptation, a leadership philosophy that encourages teamwork, self-organization and accountability, a set of engineering best practices intended to allow for rapid delivery of high-quality work, and a business approach that aligns development with customer needs and company goals.

Divide and conquer

Agile is a concept from the software industry. However, in this blog we show you that an Agile framework is also possible in the hardware industry. This brand new way of thinking is tremendously efficient when it comes to delivering the solution to the client without losing time or money.

An agile project is divided into sub-projects which we need to tackle in order to deliver the full solutions. We call this sub-project a "sprint".

A "sprint" literally means a short race at full speed. Accordingly, teams usually define a short duration of a Sprint up to 2-4 weeks. Each sprint has a goal and implements a subset of the features necessary in the final product. In the planning phase every team member receives a number of tasks / features to implement. After planning, the race starts and the team works together to complete the planned features.

Sprints make projects more manageable, allow teams to ship high-quality work faster and more frequently, and gives them more flexibility to adapt to change.

The meat of the game: How we do it

At Edgise we solve problems using Edge AI algorithms with various type of hardware devices. Depending on the project we use:

1. Readily available hardware development boards (Jetson Nano, Google Coral, Raspberry Pi, ....) .
2. FPGA's (basically empty computer chips with configurable hardware within)
3. ASIC's (custom made computer chips specifically designed for the project)

As the edge computing market and artificial intelligence research community is rapidly evolving, we cannot use traditional design methods. We need to deploy solutions fast and constantly adjust intermediate goals to meet customer requirements.

We primarily focus on:

1. quickly delivering small testable pieces of software/hardware
2. intensive communication / feedback moments with the client

By focussing on these two points, we are able to quickly pivot during the project and thus adapt to changing situations, clients and markets. It also helps us completely removing unclarities of what exactly the client can expect at the end of each sprint.

Which is exactly what our project should do, right? Meet the expectations of the client!

Let's take our Edge AI project from the introduction and take the Agile approach instead of the traditional one.

SPRINT 0: Algorithm POC

First and foremost we need to develop a proof of concept (POC) to demonstrate the core functionality of the end device. It is essential to develop this in the early stage of the project so that the client can have a feeling of how the solution will work. Below you can see a car detection algorithm we developed at Edgise.

We won't think about hardware just yet. We just develop a model in a high-level programming language (like Python) to show how the system can work.

During this sprint a lot of unforeseen design problems can arise, which is normal. It is essential to quickly receive feedback from the client to solve the issues. Frequent communication with the client is necessary in this stage of the project and will benefit the further implementation of the solution.

SPRINT 1: Breadboarding

After the algorithm POC phase, it's time to think about designing the hardware prototype with off-the-shelf components.

Mind the "off-the-shelf" here. It's not the time yet to design custom hardware. Using readily available prototyping components, like development boards, we can save time and money for us as well as for the client.

Saving time and money, sounds good right?

A development board is a printed circuit board containing a microprocessor and a lot of different sensors which we can use. The processor is the main part and needs to be carefully chosen to be able to support the features that our algorithm needs. But also the constraints of the client should be taken into account (power dissipation, area, data throughput, connectivity ...).

Whether or not your development board is open source (design files available) might also play a big role in selecting the board or not. Especially if you need to go custom in a later stage of the project.

NEXT SPRINTS: Custom hardware design

It's perfectly possible that the client is happy with the prototype and does not want any further change. In other words, the story can end after sprint 1. It's also possible that our prototype is a good fit, but that the client demands to further reduce the power dissipation, decrease the size of the hardware object or increase performance in general. In that case, we might need to go custom.

We don't change the functionality of the device. The same input will generate the same output. If the client demands functional changes, we plan a separate sprint for that.

Going custom means you will design your own printed circuit board and/or computerchip. Here we take an already existing design and start stripping away all the components we don't need. Afterwards, we add some modules that help us in reaching the performance goal. Starting from something that already exists, makes sure we don't reinvent the wheel so that we can advance more quickly!

Conclusion

The world of Edge AI and Edge Computing in general is changing fast. New AI models are released on a frequent basis. This makes it hard to design hardware to run an AI algorithm efficiently on-device. Edge AI projects using the agile design method have a much higher success rate and helps tremendously in keeping projects manageable and prevents budgets/deadlines from going through the roof.

It is important to notice that dividing the goal in several sprints makes you conquer the entire project!

Got an idea you'd like to discuss ? Simply schedule a call with me here: https://nick.edgise.com