“Exploring Reinforcement Learning” with guest Vivek Farias, Professor, MIT, discusses what role reinforcement learning has to play in this world of Artificial Intelligence.

Learning systems with humans date back to almost 5,000 years ago. And these learning systems have what allowed us to progress as a society. Being able to teach other people what we know and share knowledge has been the foundational pillars of our evolution and civilization. And interestingly, these learning systems are not unique to just humans. Animals also have these learning systems. When you look at orcas, dolphins, the higher-order intelligent animals spend time training and teaching their young ones. In the last 50 to 60 years, we have not just been teaching humans how to learn, but we have been teaching machines how to learn. And this artificial intelligence area has benefited from our understanding of these learning systems.

Reinforcement Learning is the agent interacts with the world, the world does something to the agent.

– Vivek Farias

The guest started with highlighting the importance of acknowledging uncertainty and balancing between exploiting what is known and exploring to learn more about the environment. This problem is referred to as a “multi-arm bandit problem” and is considered fundamental in reinforcement learning, where the goal is to optimize actions in an environment.

When looking at it specifically for Large Language Models (LLMs) the role of Reinforcement Learning. RL has played the central role in building general purpose chatbots that are based on LLMs. Because the resulting model that has been trained on data might not give you the refined output that you are expecting from it.

The idea is that, listen, there are so many uncertain things in my environment. If I, I don’t acknowledge uncertainty altogether, I may get into this trap where I never learn.

– Vivek Farias

When discussing about rewards and losses in reinforcement learning phase, it came out that the way we structure rewards and penalties for AI models greatly influences their reliability, how they interact with public and the accountability.

Overall deploying AI involves a balance. Backend deployment offers some level of predictability, while front-end deployment is uncertain. Successful business must experiment and capitalize in both aspects.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

This podcast offers valuable insights about the limitations of current AI reasoning, including common sense reasoning, and debunks the hype around AGI, which viewers can apply in their field.

In this podcast, Arvind interviews Dr. Srinivas Padmanabhuni, who has a PhD in common sense reasoning from the University of Alberta and is a founder of an AI startup testAIng.com.

Dr. Srinivas explains that the concept of the old Aristotelian logic is just not enough to capture human-level reasoning. The traditional logic that is taught in schools is based on zero one, which is just one part of our reasoning. Human beings reason a lot more than that, and the way AI reasons today is a very narrow, myopic way of reasoning.

Dr. Srinivas then shares his thoughts on AGI (Artificial General Intelligence). He thinks that it is too premature to talk about AGI, and it’s only marketing. He says that AI is not able to invent anything new unless it is actually coming from somewhere, some large corpus existing somewhere. He says that unless you really have a common sense level agent, which can think about reasoning, you cannot just think about AGI.

The podcast also discusses the limitations of AI and what we can expect from it in the future. Dr. Srinivas explains that AI is not capable of inventing anything new, but is only capable of processing information and making predictions based on existing data. He suggests that AI is best used for tasks that involve large amounts of data processing, such as financial analysis, medical diagnosis, and weather forecasting.

Dr. Srinivas also talks about the importance of common sense reasoning and how it is an essential part of human-level reasoning, which is missing in AI today. He explains that common sense reasoning is necessary for tasks such as understanding jokes, sarcasm, and metaphors.

Towards the end of the podcast, Arvind and Dr. Srinivas discuss the future of AI and its potential applications. They suggest that AI has the potential to revolutionize industries such as healthcare, retail, and transportation. However, they also caution that the development of AI should be done in an ethical and responsible manner, with consideration for the potential risks and consequences.

Overall, the podcast is very informative for anyone who wants to learn about the field of AI. It gives a good understanding of the limitations of AI and what we can expect from it in the future. The podcast also highlights the importance of common sense reasoning and how it is an essential part of human-level reasoning, which is missing in AI today. Lastly, the podcast emphasizes the need for ethical and responsible development of AI, with consideration for the potential risks and consequences.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

"Exploring what should organization considering when choosing to adopt LLMs" with guest Nick Brady, Senior Program Manager at Microsoft Azure Open AI Service

AI has been at the forefront of transformation for more than a decade now. Still, the Open AI launch of chat GPT in November 2022 will be noted as a historical moment – the scale of which even Open AI did not expect – in the history of technological innovations. Most people don't realize or fully appreciate the magnitude of the shift that we're in. Now, we're able to directly express to a machine a problem that we need to have solved; equipping these technologies with the right reasoning engines and the right connectivity could bring the biggest technology leapfrog not just for enterprises but even in everyday lives.

The onset of leapfrog does bring out a few questions for enterprises looking to adopt GenAI as a part of their strategy, operations and way ahead, like:

What use cases are best suited to adopt the models?

While most customers are looking for how this could reduce business costs in their organizations, the true value is when it is used to maximize business value productivity and downstream that could lead to employee satisfaction and customer satisfaction. Any place where there's language – programming or natural language – is a good use case for generative AI, and that probably would be the most profound shift. So, if you have language, if you have a document, if you have big data where you're trying to sort of synthesize, understand what that content and what the content is, generative AI models can do this ad nauseam without any delay.

The most common metric used across the world to describe LLMs is the number of parameters; in the case of GPT 3, it is trained on 175 billion parameters, but what does this mean?

Parameter size refers to essentially the number of values that the model can change independently as it learns from data and stores all information in the vast associative ray of memory as its model weights. What's perhaps more important for these models, and it speaks to more of their capability, is their vocabulary size.

How does one decide and evaluate which would be the best-suited model for the selected use cases?

The best practice really is to start with the most powerful and advanced language model like GPT 4.0 to test, if it's even possible, with your use case. Post confirming the possibility of use case trickle down to simpler models to find its efficacy and efficiency. If the simpler model can probably achieve 90% of the way, with just a little bit of prompt engineering, then you could optimize for costs.

Organizations would have to define what quality means to them. It could be the model's output, its core response, or performance in terms of latency, where the quality of the output may not be as important as how quickly we can respond back to the user.

The key for leaders is to pay close attention to the potential use cases, test them with the best model and then optimize the model to balance the cost, efficacy and efficiency factors.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha