“Exploring Reinforcement Learning” with guest Vivek Farias, Professor, MIT, discusses what role reinforcement learning has to play in this world of Artificial Intelligence.

Learning systems with humans date back to almost 5,000 years ago. And these learning systems have what allowed us to progress as a society. Being able to teach other people what we know and share knowledge has been the foundational pillars of our evolution and civilization. And interestingly, these learning systems are not unique to just humans. Animals also have these learning systems. When you look at orcas, dolphins, the higher-order intelligent animals spend time training and teaching their young ones. In the last 50 to 60 years, we have not just been teaching humans how to learn, but we have been teaching machines how to learn. And this artificial intelligence area has benefited from our understanding of these learning systems.

Reinforcement Learning is the agent interacts with the world, the world does something to the agent.

– Vivek Farias

The guest started with highlighting the importance of acknowledging uncertainty and balancing between exploiting what is known and exploring to learn more about the environment. This problem is referred to as a “multi-arm bandit problem” and is considered fundamental in reinforcement learning, where the goal is to optimize actions in an environment.

When looking at it specifically for Large Language Models (LLMs) the role of Reinforcement Learning. RL has played the central role in building general purpose chatbots that are based on LLMs. Because the resulting model that has been trained on data might not give you the refined output that you are expecting from it.

The idea is that, listen, there are so many uncertain things in my environment. If I, I don’t acknowledge uncertainty altogether, I may get into this trap where I never learn.

– Vivek Farias

When discussing about rewards and losses in reinforcement learning phase, it came out that the way we structure rewards and penalties for AI models greatly influences their reliability, how they interact with public and the accountability.

Overall deploying AI involves a balance. Backend deployment offers some level of predictability, while front-end deployment is uncertain. Successful business must experiment and capitalize in both aspects.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

Join us in exploring the evolving space of GenAI tools and infrastructure, featuring Rajat Monga, Co-Founder of TensorFlow and Google Brain.

A good power tool can make the difference of easily six to seven hours of work when you're doing woodworking. And the world of AI is no different. The types of tools and infrastructure that are developed to help you as enterprise leaders build artificial intelligence products and features are very, very important. In this episode, with our guest's help, we will unpack the infrastructure that surrounds you and the tooling that will help you as enterprise leaders build AI products and services.

We will look at how tooling and infrastructure needs are changing in the world of AI with the increasing adoption of GenAI. One key change that emerged from the talk is that now things have evolved to the extent that we don’t need to train the models from scratch. We already have foundation models available that know our world up to some level, reducing the burden of training the model with tonnes of data. However, models today have become so large that they sometimes don’t fit in a single machine.

As connecting your database with these models is very important, we also discussed the trade-offs between the open-source and private libraries. Should companies manage data on their own or outsource it? When you are not training your model, the easiest and fastest way is to use API, and if you want your data on-prem, then it will mostly cost you more. In the end, it boils down to what core to you is, and often, not all part of the infrastructure is core to companies. So, if your data is not your core strength, then better outsource it.

This episode also uncovered the current tools and infrastructure available for GenAI. The current tools and infrastructure available for large-level deployments are going through a rapid evolution; they are not very hard to rebuild or replace, and new companies are emerging in the tooling and infrastructure for GenAI space.

When looking at the talent and skills needed for GenAI implementation in your organization, it is important to have technically sound people with domain expertise in the organization's particular area.

For the differentiation in the market domain knowledge in your area, relationship with the customers, distribution channel, your execution, etc., today plays an even bigger role. However, in this data-driven world, having proprietary data and knowing how to leverage it can be an added advantage. To find out more, tune in to the full podcast.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha

“Developing GenAI Strategy” with guest Omid Bakhshandeh, AI Scientist with a PhD in Artificial Intelligence, discusses how organizations can foray into adoption of GenAI.

Whether you are the company's CEO or leading a business unit, if you're asking yourself the question, should I develop an AI strategy? That's the wrong question because today, we know that if you don't have an AI strategy, the odds of you being successful in the next couple of years will diminish. So, the right question is, what is my AI strategy, and how fast can I deploy this strategy? To answer this question, large language models are at the heart of every company's AI strategy. In a previous episode with Professor Anum Datta, we unpacked LLMs and explored what LLMs are. In this episode, that conversation was taken to the next level, and we discussed the key things you need to know about LLMs that'll help you develop your company's AI strategy.

Looking at the current landscape of Large Language Models (LLMs), these LLMs capture vast amounts of knowledge and serve as repositories of knowledge that have given rise to foundational models. With this concept, there's no need to initiate the training of an LLM from the ground up. Instead, existing LLMs available in the market, which have already encapsulated knowledge, can be harnessed and seamlessly integrated into applications. It is beneficial for companies in most cases to follow this strategy. The inherent trade-off pertains to the risk of foregoing the utilization of established LLMs, which could result in a delay in promptly reaching the market.

On the contrary, some companies, characterized by their possession of significant volumes of unique and customized data, may contemplate the development of proprietary foundational models and specific LLMs. This strategic manoeuvre facilitates the integration of such models into their respective industries and provides avenues for potential monetization opportunities.

The key for leaders is to pay close attention to the potential use cases, data, and the support system available when building the AI strategy.

Production Team
Arvind Ravishunkar, Ankit Pandey, Chandan Jha