"Exploring what should organization considering when choosing to adopt LLMs" with guest Nick Brady, Senior Program Manager at Microsoft Azure Open AI Service

AI has been at the forefront of transformation for more than a decade now. Still, the Open AI launch of chat GPT in November 2022 will be noted as a historical moment – the scale of which even Open AI did not expect – in the history of technological innovations. Most people don’t realize or fully appreciate the magnitude of the shift that we’re in. Now, we’re able to directly express to a machine a problem that we need to have solved; equipping these technologies with the right reasoning engines and the right connectivity could bring the biggest technology leapfrog not just for enterprises but even in everyday lives.

The onset of leapfrog does bring out a few questions for enterprises looking to adopt GenAI as a part of their strategy, operations and way ahead, like:

if we equip these technologies with the right reasoning engines and the right connectivity to this work and the things that we do every single day, this could be the single most democratising force of technology like the world has ever seen, not just in enterprise but even in our personal lives.rn

– Nick Brady

What use cases are best suited to adopt the models?

While most customers are looking for how this could reduce business costs in their organizations, the true value is when it is used to maximize business value productivity and downstream that could lead to employee satisfaction and customer satisfaction. Any place where there’s language – programming or natural language – is a good use case for generative AI, and that probably would be the most profound shift. So, if you have language, if you have a document, if you have big data where you’re trying to sort of synthesize, understand what that content and what the content is, generative AI models can do this ad nauseam without any delay.

The interesting thing about parameters is it’s actually not a direct correlation to how powerful the model might be. I mean, the parameter size refers to essentially the number of values that the model can change independently as it learns from data.

– Nick Brady

The most common metric used across the world to describe LLMs is the number of parameters; in the case of GPT 3, it is trained on 175 billion parameters, but what does this mean?

Parameter size refers to essentially the number of values that the model can change independently as it learns from data and stores all information in the vast associative ray of memory as its model weights. What’s perhaps more important for these models, and it speaks to more of their capability, is their vocabulary size.

How does one decide and evaluate which would be the best-suited model for the selected use cases?

The best practice really is to start with the most powerful and advanced language model like GPT 4.0 to test, if it’s even possible, with your use case. Post confirming the possibility of use case trickle down to simpler models to find its efficacy and efficiency. If the simpler model can probably achieve 90% of the way, with just a little bit of prompt engineering, then you could optimize for costs.

But many of LLMs are English only and that’s a real problem, especially for multinational enterprises and organisations that have diverse employees and diverse customers that speak in many different languages.

– Nick Brady

Organizations would have to define what quality means to them. It could be the model’s output, its core response, or performance in terms of latency, where the quality of the output may not be as important as how quickly we can respond back to the user.

The key for leaders is to pay close attention to the potential use cases, test them with the best model and then optimize the model to balance the cost, efficacy and efficiency factors.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

“Exploring Reinforcement Learning” with guest Vivek Farias, Professor, MIT, discusses what role reinforcement learning has to play in this world of Artificial Intelligence.

Learning systems with humans date back to almost 5,000 years ago. And these learning systems have what allowed us to progress as a society. Being able to teach other people what we know and share knowledge has been the foundational pillars of our evolution and civilization. And interestingly, these learning systems are not unique to just humans. Animals also have these learning systems. When you look at orcas, dolphins, the higher-order intelligent animals spend time training and teaching their young ones. In the last 50 to 60 years, we have not just been teaching humans how to learn, but we have been teaching machines how to learn. And this artificial intelligence area has benefited from our understanding of these learning systems.

The guest started with highlighting the importance of acknowledging uncertainty and balancing between exploiting what is known and exploring to learn more about the environment. This problem is referred to as a "multi-arm bandit problem" and is considered fundamental in reinforcement learning, where the goal is to optimize actions in an environment.

When looking at it specifically for Large Language Models (LLMs) the role of Reinforcement Learning. RL has played the central role in building general purpose chatbots that are based on LLMs. Because the resulting model that has been trained on data might not give you the refined output that you are expecting from it.

When discussing about rewards and losses in reinforcement learning phase, it came out that the way we structure rewards and penalties for AI models greatly influences their reliability, how they interact with public and the accountability.

Overall deploying AI involves a balance. Backend deployment offers some level of predictability, while front-end deployment is uncertain. Successful business must experiment and capitalize in both aspects.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

Join us in exploring the evolving space of GenAI tools and infrastructure, featuring Rajat Monga, Co-Founder of TensorFlow and Google Brain.

A good power tool can make the difference of easily six to seven hours of work when you're doing woodworking. And the world of AI is no different. The types of tools and infrastructure that are developed to help you as enterprise leaders build artificial intelligence products and features are very, very important. In this episode, with our guest's help, we will unpack the infrastructure that surrounds you and the tooling that will help you as enterprise leaders build AI products and services.

We will look at how tooling and infrastructure needs are changing in the world of AI with the increasing adoption of GenAI. One key change that emerged from the talk is that now things have evolved to the extent that we don’t need to train the models from scratch. We already have foundation models available that know our world up to some level, reducing the burden of training the model with tonnes of data. However, models today have become so large that they sometimes don’t fit in a single machine.

As connecting your database with these models is very important, we also discussed the trade-offs between the open-source and private libraries. Should companies manage data on their own or outsource it? When you are not training your model, the easiest and fastest way is to use API, and if you want your data on-prem, then it will mostly cost you more. In the end, it boils down to what core to you is, and often, not all part of the infrastructure is core to companies. So, if your data is not your core strength, then better outsource it.

This episode also uncovered the current tools and infrastructure available for GenAI. The current tools and infrastructure available for large-level deployments are going through a rapid evolution; they are not very hard to rebuild or replace, and new companies are emerging in the tooling and infrastructure for GenAI space.

When looking at the talent and skills needed for GenAI implementation in your organization, it is important to have technically sound people with domain expertise in the organization's particular area.

For the differentiation in the market domain knowledge in your area, relationship with the customers, distribution channel, your execution, etc., today plays an even bigger role. However, in this data-driven world, having proprietary data and knowing how to leverage it can be an added advantage. To find out more, tune in to the full podcast.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha