ML algorithms and AI systems

Do we need a new software development cycle?

AI systems work really well and solve lots of problems. But are we ready for debugging it? Or is it gonna leave a scorched land behind?

Hype

Artificial Intelligence is the hype now. You hear about it everywhere: chatbots, NLP, tensorflow, deep learning, translation, Smart IoT, AlphaGo and self-driving cars (Yay!). One thing that has been bothering me is: how do you approach the cycle around software development in AI? Does anything change? Our concepts, the state-of-the-art around Continuous Integration, Automation, Continuous Delivery and all other DevOps ideas. Are they enough for an AI system? Do we need to change the approach?

When a client asks me to implement a Jenkins pipeline and I realise the ML algorithm is on Jupiter Notebook that his data scientist has…

My goal here is to open a discussion, a “call for help”, so we can start looking at how we build AI systems from a software development cycle perspective. People talk a lot about Narrow and General AI, Singularity and so on. If this was your goal, sorry, this is not that kind of text.

No, no Singularity talk here. Sorry to disappoint you like RDJ disappointed this kid expecting Iron Man.

AI Systems

First, we must reach a common ground. What do you consider an AI system? Based on the implementations that I have done, the talks, articles and posts I have seen, I have a concept that it has:
(i) Perception — how it can perceive the world it lives. The part of gathering data, doing a data pipeline and so on.
(ii) Learning — lots of people think the AI is only this part, the Machine Learning itself. Several well-known algorithms are here: Support Vector Machines, Artificial Neural Networks, Ensemble, General Linear Models and many others. Do not forget; there are several types of learning (spatial, relational, stimulus response and many others), but all of them can be represented in mathematical problems.
(iii) Reasoning — it is the part where you define and reason what the system is trying to learn. The algorithms in ML need a measurement of how good they are doing, how well they can predict, classify or cluster things. How do you define the reasoning behind it?
(iv) Planning — based on the output, you reason about the outcome. You have an action take and respond to it, but you need to plan how you will execute it.
(v) Execution — a lot associated with robotics, it is the actual implementation of proposed action due to a reasoning phase. However, in a recommendation system, for example, there is no robotics — just an API call that will expose products to a user.

Debugging ML

One of the trickiest things in implementing Machine Learning algorithms is debugging. Why? I learned in college that I should think about an algorithm and apply the logic first in these binary graphs below. They are associated with If/Loops, clauses, try/catch and so on.

Ohh, how I found boring these Algorithm classes! Now I can only thank it, because I had to learn it and it is so simple to debug now!

When there is a bug in your application, you will look at logs, trace the error and you can usually pinpoint to a particular sequence of code that you can fix. You patch it up and problem solved!

When I patch something — Eye of the tiger!

Hyperparameters & AI tweaking

Now let’s think a bit different. You built an ML algorithm that captions image — just like you can see on this Google Research article.
If a user is paying for your service and he sees a mistake in a picture, he may have the interest to open a ticket for you and ask you to fix this problem. Now the million dollar question: can you do it? Can you solve the problem of this specific user? If there is a tensorflow implementation with an LSTM Deep Learning, you can not tweak the ML algorithm for just one case. Here there are no if/else clauses or loops to point out exactly which sequence is making the mistake. You use an API to program in tensorflow, define the layers, activation function, loss function and all that is needed. You have the flexibility to change the hyperparameters of the tensorflow implementation; you can modify the base algorithm that tells the machine how it should learn, so it can optimise the gradients and build its matrix of weights. You do not explicitly define the algorithm, the loops and sequences as the regular software system we have seen above.

Build the data pipeline, define hyperparameters for you ML implementation and you have an outcome! *disclaimer: It is not as easy as it sounds!

You provide data to this algorithm, it can measure its learning and will improve on it. As a result, the machine will have its algorithm to implement and predict, classify and reason on what it learned. So, in an ML algorithm, you build a data pipeline, define hyperparameters to this “black box” that will give you an outcome. The important is, if there is a bug on the result of the ML algorithm, can you change one case without touching the others?

AI Expectations

This is a different concept of debugging and it has a lot to do with expectations. You will never have an ML algorithm that is always right, just like humans. So when that nice marketing guy come and ask if you can build a recommendation system only if you can guarantee 100% of something, just go grab a coffee and throw back at him. Moreover, development has been building up the concept of microservices and modularity, so an AI system may seem counterintuitive. ML is the best we have now for solving problems that need external data input, a problem that would be impossible to solve with algorithms because of the infinite possibilities. Thus, now, our best solution is not modular. It can be a microservice on its own, but you can not touch one point of the algorithm without making sure you will not change all. It will evolve to that with reinforcement learning, removing unused neurons, deepdreaming concepts, but it is needed to ensure: ML algorithms have modularity issues.

Change one change all. ML has a pre-built domino effect in its regular algorithms. This will change!

Build/Test/Release & Online AI

Another important point is the Online Learning VS Build/Test/Release concept. In companies I worked when you release a story you follow this steps: submit a pull request, gets evaluated, merged into master, build and package the application, do your deployment testing, functional testing and finally, release to production. I hope you are aware of this automation process, pair programming, necessary validations for merging and is using Jenkins/Travis to have all this — do not forget all the unit tests before you do your PR, please!

Just merge to production and let Ops team take care of it, right? =O

If you have an online machine learning model, a model that learn “on the fly” with new data streaming into it, that is brilliant! Nevertheless, every time your model recalculates itself it changes its algorithm (because it changed the weights). Thus, is a new Build necessary so you can run all your tests and your CI/CD pipeline again? Or is this overreacting? Are there any other tests we should include in this case? Are the concepts we have today for Package/Build/Release ready for ML algorithms? This is the “cry for help” part… I do not know and I have some ideas in how to deal with it. But I would like to hear what you guys think of this!

AI is here to stay

I am sure of one thing: AI systems are here, and they can solve what other algorithms can not solve. The evolution of image recognition, Natural Language Processing and translation (points for the Google Interlingua) are just few examples that bring to light all the benefits of using Deep Learning. So we must start paying more attention to its development cycle, deployments, test, how it goes to production, how to package and build it. When we have a clearer vision around this, we will be able to better integrate data scientists in dev teams, leave the Jupiter Notebooks behind to move into a more scalable/integrated platform. The maturity level of such systems will bring benefits to all dev teams. To do so, we need to discuss more and try more new things.

Do you have any experience to share, any ideas to try or you simply do not agree? Leave your comment, heart this article, share and let’s stir this discussion.

This is based on the presentation “Robots in a scorched land — AI legacy in uncertain systems” on devops Amsterdam Meetup sponsored by LINKIT.

Originally published at www.linkit.nl.