From Software Engineer to Writing a Python Deep Learning Book

Ivan Vasilev started working on the first open source Java Deep Learning library with GPU support in 2013. He has also worked as a machine learning engineer and researcher in the area of medical image classification and segmentation with deep neural networks. Since 2017 he has focused on financial machine learning.

We are very glad to be presented with the opportunity to interview the author of Python Deep Learning: Exploring deep learning techniques and neural network architectures with PyTorch, Keras, and TensorFlow, 2nd Edition. More about Ivan’s path from Software Engineer to an author of a Deep Learning book you can read below.

Tell us how you started your path as a Data Scientist, how long ago was this?

I’ve been interested in data science (and more specifically in ML) ever since high school.  My first contact with ML was my bachelor’s thesis – I implemented a neural network known as self-organizing map (or Kohonen net) to help navigate an agent in a simulated maze environment. Then I got my MSc in AI, but at the time I was still working non-ML related software engineering job. At one point I decided I want to do ML full-time. As a way to learn neural networks and do something useful at the same time, I implemented one of the first GPU deep learning libraries I’ve been working in the ML field ever since. My current focus is on ML in finance.

What did motivate you to start in this field and continue with Deep Learning?

I’ll try to answer with an analogy. One of the goals of contemporary physics is to find the theory of everything – a framework, which unifies the four fundamental interactions of nature: gravitation, strong interaction, weak interaction, and electromagnetism. Candidates for such theory include String theory and M-theory. As a software engineer, I’ve always strived to write code, which is simple, elegant, and at the same time covers all possible use cases. Of course, I often ended up with the opposite, but that’s a whole other topic 🙂 For me, neural networks seem like the ultimate generalization – one day (not soon though), they might help us build artificial general intelligence. And this is the most fundamental thing we can achieve.

Is Deep Learning at a plateau at the moment?

That’s a great question. There are arguments in either direction, but ultimately I believe the answer is no. On one hand, it seems like we’ve picked the low-hanging fruit. For example, one of the major improvements compared to “classic” neural networks have been the use of new activation functions like rectified linear units (ReLU) instead of sigmoids. Solutions like these have made it possible to train truly deep networks, which in turn led to big improvements in accuracy. It becomes less and less likely that we’ll see such “cheap” improvements in the future. On the other hand, the current advances have opened many doors, which were closed before. Fields like reinforcement learning, attention mechanisms, memory augmented networks, and neural architecture search are just picking up now. Even more exciting is that deep learning algorithms are not limited to academia, but are increasingly adopted by the industry as well.

What have you seen in the field of Deep Learning that really excites you?

I’m really excited by the prospects of finding more generalized ML algorithms. Current neural networks have some limitations. For example, they tend to forget what they’ve already learned once we train them with a new type of data. They also seem to work best on unstructured data like text, speech, or images, but don’t perform as well on structured data. It would be great if we are able to solve these problems.

python deep learningYou have written a book on Deep Learning with Python – tell us why Python for Deep learning and what other techniques are used?

Python is the de-facto ML language. Major libraries like TensorFlow, PyTorch, and Keras are Python based. So, it made sense to write a book combining these two topics.

Tell us more about the book – what is different in the second edition and do you plan translating the book into other languages?

The second edition is very different than the first. I tried to improve some the existing content and I added many new things. The new topics include generative adversarial networks (GANs), variational autoencoders, object detection, image segmentation, advanced convolutional network architectures, the latest in reinforcement learning, and even a chapter on autonomous vehicles.

As much as I wish, I think there won’t be enough demand to translate the book to other languages because Еnglish is the largest market.

What potential applications of Deep learning do you see in the next 10 years and which industry domains will thrive because of it?

I believe one of the major applications will be in autonomous vehicles. On one hand, they can use deep learning for computer vision to build an accurate reconstruction of their surrounding environment. On the other, they can implement driving policy (that is, what the car does once it’s familiar with the environment) with reinforcement learning.

Another major area is the medical field. Currently, medical test results are interpreted by highly-trained professionals, which use them to identify a diagnosis. These specialists are often scarce and their time is valuable. This could lead to long waiting times for test results and general difficulty for conducting tests. Deep learning algorithms won’t replace these professionals, but it can aid them in their work. Deep learning algorithms could identify warning signs and alert the doctors early. In this way, screening and diagnostics will become more accessible. In turn, this will help preventative care, which is often underestimated, but it’s one of the most important aspects of healthcare.

We might also see deep learning applied in automated manufacturing. Imagine an industrial robot grasping apart with random shape and weight without dropping or breaking it, then navigating the factory floor to a location on the assembly line, and finally mounting the part on the assembled product.

Finally, I also think the voice activated personal assistants like Alexa and Google Now could see major improvements (not necessarily in the speech recognition part, which already works well). They’ll become smarter and will allow us to communicate with computers in even more natural way.

What comes after deep learning, more biological inspired approaches?

That’s a great question. Classic neural networks were more inspired by the biological neurons, compared to the algorithms we currently use. In the past, mimicking the brain was a goal in and of itself. Nowadays, the approach is more pragmatic – we just select the best tool for the job. So, I think the future direction of ML will be decided by the types of problems we’re trying to solve.

What is the next step for you after writing the book?

I hope to focus more on ML for finance.

What would you encourage students considering a career in Data Science to start doing?

When I want to learn something, I usually find a related task and try to solve it. A great way to do this is to participate in competitions and datathons, like the ones organized by Data Science Society. I think you can benefit from such experience in two major ways. First, the competition deadline helps you concentrate and be productive. But even more important is the teamwork and your interaction with other participants – this is how you make the most progress.

If you are interested in deep and reinforcement learning, neural, convolutional and recurrent networks, you might want to check the book Python Deep Learning at Amazon. 

Share this

One thought on “From Software Engineer to Writing a Python Deep Learning Book

  1. Playing guitar and writing are two of my most cherished hobbies. As long as I can remember, I’ve always wanted to be a good writer. As a result, I often turned to services like BestWritersOnline – to acquire the best writing aid I could get. Since I’ve started reading more, my writing has become better. My favorite method to have fun is to read a book on something I’m interested in.

Leave a Reply