How I Conquered Analysis Paralysis to Kickstart My Journey

Hello again, fellow explorers! Welcome back to our journey through the intriguing and sometimes overwhelming world of artificial intelligence. In our last post, we talked about rebooting our careers and embracing the ever-evolving field of AI. Today, we dive into the AI landscape and discuss how to navigate its many pathways to find the one that suits your goals. More specifically I will talk about what I did and why. Today I am actually working on my very first AI project through my consulting company. I think this is a direct result of some of the choices I’ve made. Let me explain.

Feeling Overwhelmed? You’re Not Alone!

When I first looked into AI, I felt like a kid in a candy store—there were so many exciting directions to explore, but I had no idea where to start. It’s all moving so fast! I wanted to build fundamental skills, but I didn’t want to spend so much time doing it I would miss the boat on some of the latest and greatest new things happening. I also wanted to have a strong understanding of the concepts in AI, but that didn’t mean I wanted to be an AI researcher or a mathematician or a full time AI software developer. We’ll talk more about that below, but suffice it to say I found it hard to sift through all the online content. If you’ve ever felt the same way, you’re in good company. The AI landscape is vast, and it’s easy to get lost in the jargon and multitude of specializations.

So early on, I embraced two very key and core beliefs:

Belief #1 : I am not the only one working furiously to learn AI.

This was a very important thing for me to realise and accept. When I was looking around and consuming learning content, it felt everyone was an expert in AI except me. But the opposite was the case. There is a large community of learners but a relatively small community of experts. When I realised this, I went on Meetup.com and I joined all the local AI meet-ups I could find which not only put me in touch with others who shared my interest, but also started to expose me to the vocabulary, the problem spaces, and solution spaces. It gave me access to people to whom I could ask questions and who would stimulate my own learning. And it’s through these meet-ups that I landed my first project!

So just do it. Join at least one local meet-up and make yourself a member of the community of enthusiasts.

Belief #2: I should not only learn about AI, I should learn to USE AI in my work.

Remember the days before ubiquitous GPS on mobile phones? Nowadays, we can fly to another country, land at the airport, enter our hotel address into our phones and drive ourselves there in complete confidence. We arrive in a new environment and operate with the proficiency of an expert.

Similarly, in tech work, I’ve been a decent coder for decades, but I confess that I am a Google/StackOverflow coder. Meaning that when I write code I am constantly interacting with search engines to remind me how to accomplish one particular thing or another.

AI is like GPS and it’s like search engines. It will radically change how we work, and those of us who get used to using AI tools as a part of our work will very quickly develop a massive advantage over others. Those who stick with the way things have been done and don’t adopt this new way of doing things will be like people who don’t know how to use search engines or GPS.

Finding My Starting Point

I had a strong desire to get started. I had connected with my local community, and I had begun to train myself to use AI to augment me. But I had to actually start somewhere. My approach was to answer a couple of key questions.

Question 1: What are my goals?

What do I want to do with the knowledge I gain?

I just wanted to know how this stuff basically worked, what the complexities were and what the technical tradeoffs were for various architectures and implementations. I wanted to be able to develop a strong understanding of various AI approaches to technical and business challenges.

I wanted to know:

• enough that I could identify good and bad use cases and have a pretty good idea of how to tackle them
• what would be difficult and what would be easy
• which problems are solvable and which are still fanciful thinking
• what’s expensive to do and what’s cheap to do

Most importantly I wanted to do with AI what I’ve done with many other technologies in my career: I wanted to be able to sit in a room of people who are much smarter than me on the topic and not only follow the discussion but contribute my own unique value to it. There is no way that I would be able to get where I wanted to go without getting my hands dirty. And that’s ok.

Question 2: What kind of AI work excites me?

AI encompasses numerous fields, from machine learning and natural language processing to computer vision and robotics and so many other things. Now perhaps you have seen this diagram:

Where did I want to focus? Here’s how I ended up breaking this down.

Prompt Engineering

If you read about Generative AI it won’t take long before you start running into articles extolling the virtues of good prompt engineering. When I saw these articles my initial reaction was to yawn. Yes, of course, if you want to get the most out of a tool you should try to get good at using it. However, as I engaged with the AI community and began using AI tools in my work, I realized I had underestimated its importance. Effective prompt engineering can drastically improve the results you get from AI models. For example, instead of asking ChatGPT to perform a complex task directly, you can first ask it to outline the steps needed to accomplish the task and then follow those steps. This approach often yields far better results. Through meet-ups and practice, I’ve learned valuable prompting techniques that have significantly enhanced my AI projects.

I learned quite a few tips from my meet-ups, and applying them to my usage has made a tremendous difference. And remember – if you use AI tools to augment yourself as you delve into this new world, you will achieve far more with far less. Take this very blog, for example. I started writing it by giving an LLM a copy of my previous blog entry and a summary of what I wanted to cover in today. I asked the LLM to generate the blog entry. I then heavily edited what the LLM provided. And then I asked the LLM to tighten it up. One final edit by me and here we are.

Data Science

Initially I felt a little hopeless because I had been basically ignoring most of the “big data” craze that swept through tech in the early 2010s. I have some data science skills because my work has always involved capturing and making sense of large amounts of data. But I knew my data science knowledge was a little brutish. I resolved to at least spend a little time on the basics of data cleanup and profiling and analysis. The reason is that there is a massively important group of activities that undergirds AI: data. I wanted to strengthen my knowledge of data preprocessing, feature engineering and exploratory data analysis. So I revisited an old course I had taken a while back on Udemy called Python for Data Science and Machine Learning. I’m going to have an entire post on this blog that goes over the high-level lessons I learned.

Machine Learning

Although not all Machine Learning is Neural Networks (eg: decision trees, support vector machines, etc), I’m mostly talking about Neural Networks here. What was making me uncomfortable was that these things just seemed a little too magical. You see these hand-wavy diagrams where you have a bunch of data on the left, then a “neural network” in the middle, and spectacular results on the right. What in the world is going on here? Could I really be an AI professional if I didn’t at least understand the basics of how this works? I firmly believed the answer was no. Take this list of terms:

‘weight’, ‘bias’, ‘activation function’, ‘input shape’, ‘regression’, ‘sentiment analysis’, ‘back propagation’, ‘loss function’, ‘parameters’

In the context of AI, do you know what they mean? I could have guessed at some of them, but I wanted to know from experience. So I took a class on building artificial neural networks, using a framework called Tensorflow. I could have used a different framework called PyTorch, but I was less interested in the framework than the actual neural networks themselves. Let the developers worry about the pros and cons of a development framework. I just wanted to know what it would really take to make a neural network perform image recognition or sentiment analysis.

In this blog series, I will also have a post that goes over the concepts I learned here.

Generative AI

It was incredibly tempting to dive straight into Generative AI because it’s where all the excitement is. Watching friends and colleagues make rapid progress triggered a strong Fear of Missing Out (FOMO) in me. However, I knew I needed to build a solid foundation first. One of the most thrilling aspects of Generative AI is its low barrier to entry. The capabilities of Large Language Models (LLMs) and their development toolkits are so advanced yet accessible that almost anyone can start creating impressive applications. You could even learn about things like LangChain and Autogen that lets you skip right past the low level stuff.

In my opinion, two fascinating applications of LLMs are Retrieval-Augmented Generation (RAG), which enhances responses by referencing specific data sets, and AI Agents, where multiple LLM instances collaborate to solve complex problems. My current project leverages these concepts, and while I can’t share all the details just yet, I can say it’s incredibly exciting and promising.

Leveraging My Background

One issue I really needed to grapple with, and frankly something I am still grappling with, is how to leverage my background. Probably the easiest way for me to execute my pivot to AI would be to find (or start) a company focused on using AI to deliver value to Network Infrastructure companies. Most likely this would be a company that would be more focused on classical Machine Learning than Generative AI. By this I mean that the most obvious applications of AI to the Network Infrastructure space involve things like predictive analytics, anomaly detection and network event profiling. Generative AI applications seem limited right now to things like query interfaces and configuration copilots. I’m sure other use cases for GenAI will emerge in the near future, but if I’m just looking at the opportunity out there now, it seems like Machine Learning and Analytics is still the best bet. And so this influenced my order of operations. I am by no means locked into this specific sector – what I care about is that I get to work with customers on tough technical challenges. So I don’t intend to limit myself. But pragmatically – I have to think not only of what I want to get, but what I have to give, so focusing on buttressing my own area of expertise with some AI knowledge seems wise.

My Chosen Path

After much consideration, my approach was essentially this:

1. Join AI meetups and expose myself to other enthusiasts and professionals.
2. Refresh my knowledge of data science concepts and grease up my Python coding skills which had gotten rusty.
3. Dive into artificial neural networks by taking a very intense class that took me through all the foundational models in a practical way.
4. Improve my prompt engineering game so that I can use LLMs as a copilot in almost everything I do (LLMs are an amazing teaching assistant)
5. Find a Generative AI project that gives me a real objective and set to work conquering the obstacles one by one.

Wrapping Up

So this is basically also giving you somewhat of a roadmap of what this blog will look like. I am learning in the open. I’m sharing my thoughts, my experience and as always, I am inviting you to chime in with you own thoughts.

Next time, we’ll dive deeper into one of the core areas I’ve chosen, and I’ll share some hands-on projects to get you started. Until then, happy learning, and don’t hesitate to reach out with your thoughts and questions!

Here’s to making sense of the AI maze together. See you in the next post!