Cadmore media player playing video The Coordinated Dance Between AI and Human-Intelligence for Optimal Outcomes
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
The Coordinated Dance Between AI and Human-Intelligence for Optimal Outcomes
Description:
The Coordinated Dance Between AI and Human-Intelligence for Optimal Outcomes
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/15352883-21d7-48bd-bb3b-1163affad9d7/videoscrubberimages/Scrubber_1.jpg
Duration:
T00H30M54S
Embed URL:
https://stream.cadmore.media/player/15352883-21d7-48bd-bb3b-1163affad9d7
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/15352883-21d7-48bd-bb3b-1163affad9d7/industry_breakout__tnq_tech (1080p).mp4?sv=2019-02-02&sr=c&sig=aiQ%2BSEJolHFJRSivsmW7yRzPcut%2By9IXf0cnRx5QwNE%3D&st=2025-04-29T21%3A31%3A34Z&se=2025-04-29T23%3A36%3A34Z&sp=r
Upload Date:
2024-12-03T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
Last year, we were at Portland. ChatGPT had just been released and a few months ago and we did a session to talk about the basics of a large language model. What is it. How is it different to a model. And we got a very good feedback. So that has incentivized us to come back and talk about a similar topic for those of you who do not know about.
We've been around for 25 years. We deliver post acceptance publishing services and technology products to scholarly societies, commercial, and open access publishers spread across North America, Europe, and the UK. We are based out of a very hot and exotic city of Chennai in the South of India. I am Abby. I am the CEO of tech and have the pleasure and privilege of representing 3,000 publishing and technology experts who form tech do drop by our booth to know more about us.
For today's session, as we were preparing for it, we realized we had picked a very difficult topic only because these questions that we are posing have so many different responses that it is impossible to cover anything conclusively in 30 minutes. So we have picked some of those that are very relevant. The underlying technology is evolving so swiftly that it is extremely difficult to keep pace. Nevertheless, I want to acknowledge that we have borrowed from views of academicians and business leaders, but adjusted those ideas to our experience so far.
The session has a very catchy title, the coordinated dance between and humans. And to present this session, you have me someone born with two left feet and not even the faintest idea of a dance. So I thought I will enlist support from Neil, who heads our technology and clearly knows more about these things than I do. But apparently he's as poorly endowed with dancing skills as I am.
At a broad level. This is a discussion of how we are looking at leveraging AI and humans for optimal business outcome in a manner that is sustainable, ethical, and practical. I recognize this is a heavy topic and we wanted to talk about something that isn't the conventional conversation at this conference. We recognize it could be a complete hit or a complete miss, but we will try and do our best and live with the consequences.
Let me first tell you what we are not going to talk about. Enough is being discussed about the impact of AI on scholarly publishing. So we are staying away from it beyond using some examples. Enough people are talking about AI powered products that they have developed. We will talk about products when you meet us at our booth and not at this session. Enough people are predicting the future of AI.
We are most definitely not. If I can predict the future of AI with any level of certainty I would not be at. We are also not talking about the ethics of AI, largely because that is a very specialized area and there are better people who are trying to figure that out. But partly also because ethical considerations are so fundamental to the use of AI that I'll be repeating that after every statement or every framework.
So please assume that ethical considerations form the backbone of anything and everything to do with AI, and it is implicit in today's talk. So what are we talking about. You will agree when I say that one should not use technology just because it is available. We should use technology when we can be sure that it is useful and safe to use and when the value is clear and apparent.
Today, we'll be sharing our experience with you of how we determine value in the world of AI, both from technology as well as from humans and multiple combinations of both these resources. And a large part of using something effectively is to know what it is capable of. But also what it is not capable of when to use it and when not to use it. Today's session is in four segments.
The first and the most important is explaining what is it about humans that makes us indispensable in the world of AI. It seems obvious, but it is nevertheless something we should keep talking about. Then Neil will talk about LLMs and what makes them unique. More importantly, what does this unique uniqueness mean in terms of the context of work that we do. How should we compare LLMs with conventional technologies, basically.
I will then talk about a mental framework that we have been using to determine when to use AI and how to use AI. And then Neil will take the final segment to talk about the human dance, the synchronization of it, and in that process derive an outcome that is meaningful, useful and safe. So human value. The first thing to explore is this In a world where the relevance of AI is increasing, the question of human relevance becomes even more pressing.
So the question is, why is human involvement as yet indispensable? An easy answer to this question is to look at the difference between I and human. This quote from Albert Einstein beautifully captures the distinction between the intuitive and rational mind, which in my understanding is the most significant difference between. And humans lacks creativity and intuition lacks emotional intelligence.
And I wasn't even a term as it is today when Einstein was alive. Underpinning statistics, making predictions based on data and patterns, finding logic and reasons that exists within that data and pattern is rational. It is no doubt powerful, but it lacks nuances and it lacks the ability to make decision that combines data or fact with context and intuition.
Basically, it lacks out of box thinking quite literally. While it is great with data processing, pattern recognition and future enhancement will only make more sophisticated with better rational outcome, it is unlikely to replace humans. Well, all of that is very generic though, and that's something that you all know already. There is nothing new in what I have said so far. So thought we will pose a more difficult question, which is the relevance in the context of work to understand the value of humans in the world of air, which is a simple matter of trust.
The question to ask, therefore, is, can we trust in a manner we trust humans. If the answer to that is Yes, that is a huge endorsement for AI. If we can't, we need to understand why we can't trust AI. I will use two arguments to explain the prevalent view on this matter. The matter of trust.
Please remember ethics and everything to do with it is implicit and is not being discussed here. The first is the matter of bias. Everyone talks about the bias that plagues the system because and because of those biases you can't trust. Well, as humans, we also have our own biases. All of those biases are represented by our views. All of those views form content that goes on the internet, and that's the content that has used for his training.
So essentially, the bias that the system has is a reflection of our own biases. So if we have bias and has bias, what is the problem. The problem is that whilst humans have biases, humans have bias. We also have the ability to recognize them, reflect on them and adjust through conscious effort. We have the ability to consciously review data in a completely different context, add emotional intelligence and intuition to it, add subjectivity, empathy.
We have the humility to accept that our learning may be flawed and then do something that may not have a precedence, something that may not follow a data based pattern. Can't do that, at least not without human intervention. Which gives you a clue of where we are headed with this session will need a lot more data to change their decision, even in a conventional, rule based technology that we are all so used to could just add a line of code to alter a decision.
But a model needs a huge amount of data to override their current bias, and preparing that data is expensive and challenging. And who knows. This new data that is trained on can potentially lead to a completely different kind of bias. And more importantly, biases without corrective measures may just keep getting amplified, which is a real risk that many people have already experienced, and that can lead to unintended consequences.
So the first issue of trust is with the bias. The second matter that we can discuss is what is commonly known as explainability. Humans have the ability to explain their decisions. We may choose not to but we have the ability, at least most models, especially the deep learning models, are Black boxes. We know what goes in. We know what comes out.
But the how is not entirely clear. And the more billions of parameters you have, the models decision making process becomes even more complicated. And even more difficult to predict. So the model will provide you with an output, but without clearly explaining a clear explanation of how it got to that output. LLMs recently have added what they call as reference ability, which is a step further, but not enough at all.
They can tell you where they got the information from that help them produce the output, but how still remains a black box. So the humans ability to explain their decision making builds trust and accountability. And conversely, AI'S inability to do so takes trust and accountability away from it. With these two things biases and explainability being valid issue, I think it is safe to assume that trust in AI is output is likely to remain a questionable aspect in the foreseeable future, which is very important in determining the way we use AI and our work today.
So we had set out to answer the question of why are humans yet indispensable in the world of AI, intuition and creativity, ability to reflect on our bias and the ability to explain our decision making are essentially what makes humans indispensable. This was an important thing to establish before we move forward, and there are many other reasons, but I think these three impact us on the work front the most. And before I bring Neil to talk about LLMs, I want to pose another interesting question.
Humans make mistake all the time. Why then do we accept that we can make mistakes but has to be perfect. I had thought of this question but never really thought about it. And again, it is very important to understand this from a work perspective. There was a long list of pointers to this, but in the interest of time, I've trimmed it down.
We talked about biases and we talked about explainability. Both of these make it more acceptable to accept human mistakes, but not that of AI. When humans make mistakes, we have accountability. We can think about our decision making, apologize, take corrective actions. Unlike humans, I operate without fear of job loss, without contractual obligation, without legal accountability.
Basically, it acts with impunity. That makes it OK and more acceptable to accept human mistakes, but not that of humans are considered to be subjective, whereas AI is expected to be objective or at least perceived to be objective. That perception of objectivity also leads to the expectation of a flawless performance, which in a convoluted manner allows us to trust human mistakes but not that of AI.
And finally, a quote from history that sums it all. Historically, humans have been the decision makers and societal structures are built around human authority and discretion. Being a relatively new entrant is held to a higher standard to prove its reliability and safety. So the distinction between I and human that leads relevance to humans in the world of AI is what we have discussed so far. But now let me ask Neil to come and talk about what is unique about them.
That is obvious but not that obvious. Neil great. So we heard from Abby about the relevance of humans. Now, let me share our perspective on how we see AI, especially in the context of LLMs and their unique attributes and capabilities. So if you think about conventional technologies that we have been using so far, it always had a very clear input and a very clear output.
Its input was objective and largely predictable was the output. Think of a website, a workflow, a tool. Everything had a range of predetermined scenarios. Of course, there is complicated programming in between and calculations, but input and output both remained largely objective. And in the world of AI, this notion has been challenged and to understand human interaction, it is very important to understand why particularly LLMs are not the same as conventional technology.
Otherwise, we run the risk of getting taking our traditional understanding of making technology work for us and apply it to AI with potentially disastrous outcomes. So let me present to you a framework that has helped us understand better. It is called dialogue dynamics grid. In simple terms, it creates a broad classification based on input and output of the system and whether they are subjective or objective.
Like I mentioned, conventional technologies excelled in scenarios where both input and output were objective, and that's why we have become fairly adept at managing these situations. So we call it deterministic output. But the large language models, for example, ChatGPT, they are designed to handle subjectivity in both input and output, which puts it at the other end of the spectrum called creative synthesis.
Think about problems we typically solve today. They usually have objective outcomes, which conventional technologies handle well. But these technologies lack the ability to process subjective inputs and outputs effectively. So our experience with them has always been very objective, and the emergence of LLMs has fundamentally shifted this very paradigm. And with LLMs interactions are no longer confined to predefined parameters and hence creative synthesis.
Or simply put, brainstorming with AI. So previously brainstorming required human collaboration. Now it can be done with LLMs and I'm sure you would have experienced LLMs are generally pretty good at it. And because that and that happens because that's the design of the system. But it may not be as good in giving you an objective output to your objective input because that's not what it is meant for.
So in expecting LLMs that it is, which is very capable of creative synthesis and to behave like traditional technology and still provide a deterministic output, we are trying to apply an expectation to that. It isn't designed for. And when the outcome isn't what we expect, we don't get that. It causes doubts in our mind about a technology. And in that process, we as the users of this technology are setting it up for failure.
So if we have to get LLMs to produce a deterministic output, we have to make it work differently. And that's where humans come in. To get an to maximize its subjective assessment capability, but guide it to produce a deterministic output using human intervention. Hold on to that thought. That's what I'll come back and discuss. But before that, I'll let Avi talk about his mental model of how and in what scenarios is useful and how to use it.
So we all acknowledge that technology is changing and therefore our approach to deal with technology must also change. Neil gave you an idea of what's different in Chennai compared with conventional technologies. Conventional technology was a brute force that applied to all scenarios. AI isn't not just because of the subjectivity of input and output, but also because of the trust and explainability that we had discussed before.
Before we go further, I want to talk about two things that we have experienced that might be helpful when you are thinking about technology in general and in particular. The first is automation. For the longest we have considered automation as the final goal. But as we started understanding at a deeper level than simply using it as a tool, the thing that we realized was that automation is perhaps not the best goal to have.
Automation improves efficiency to a degree, but it is not always synonymous with optimal result or optimal outcome. An efficient system is basically optimized for doing things with minimal waste. That's a very manufacturing industry term that we have all adopted for the longest. It isn't effectiveness, on the other hand, or having an effective system is doing the right things to achieve the desired outcome.
And in a new world where we have to balance AI and human strength, we have to understand and define the outcome. If we still keep thinking about AI in terms of automation, we will always hesitate in using it because AI is not about automation. It does not follow the same conventional patterns of technologies that we are currently used to. This can be best illustrated by what we call as error awareness, the ability of a system to determine where it has gone wrong and therefore ask humans for help.
Our entire technology vision is based on this one fundamental principle. We don't have time here, and I'll be happy to discuss what error awareness means if you find us outside this room. So with that background, let's look at a mental framework we use to determine when and how to use AI. I believe you will immediately relate to it and it is important to state that this isn't prescriptive.
It is only indicative. In this approach, we look at repeatability of a task or an action, a repeatability in the context of means. There will be a huge amount of data from where patterns can be determined, which makes it easier to train an AI system. On the other hand, if the task or action isn't repeatable, well, the data availability might be sparse and the outcome of the train system will be subjective and inaccurate.
On the other axis, we have the criticality or impact of the outcome of any task or action. So let's look at these four quadrants, one after the other. If the repeatability of the task is very low, meaning the task does not follow a pattern. And the criticality or impact of the outcome is very high. This is a very risky scenario in which should be we should consider using. And I'll give you an example, which can be controversial, but consider peer review.
That is not an exact science. It has nuances that cannot be captured in data and patterns. Again, focus on the overall activity of peer review and not individual. Particular tasks can most definitely be used to produce an output because it has the ability to analyze huge amount of data and huge amount of content in a scalable manner. Human can't, but the use of AI must be initiated by a human and the outcome must always be reviewed and acted on by the human.
This grid allows or expects a trust system that should be near zero. The second is if the repeatability of the task is high, meaning the task does follow a pattern. But the criticality or impact is also very, very high. Nope we go back. This is one of the scenarios of human coordination that has got great potential. This is a trust but verify scenario where we let do its magic and insert a human in the loop, either in the middle or at the top to verify the output.
Whether you verify 100% of the output or you verify a lower percentage depends on your perception of the criticality or the risk to your business. I think aspects of research integrity falls under this category. We have data and patterns that are emerging. AI is able to predict pattern based behavior but to take action on the output of AI without human oversight remains very risky.
This is a scenario where AI is supplemented to human. Human should always be at the end of the loop to make the final decision. The third scenario is where the repeatability of the task is high, meaning the task does follow a pattern, but the criticality or impact of the outcome is relatively low. This is an ideal human conversation or coordination scenario. This is the low hanging fruit.
This is a trust and accept scenario, but with the realization that I might mistakes make mistakes and that should be OK, just like humans do. You can have a trust but verify mechanism for this too. But on a sample basis, I would say copyediting falls in this category. Maybe not the language part of it. But the mechanical part of it, because the impact of capitalization going wrong is perceived not real, but the impact of a language error.
Changing the meaning of research is very, very high. Hence, this distinction is necessary. These are scenarios where human can be at the middle of the workflow, at the end of the workflow, and sometimes with very careful consideration, humans might not be needed at all. This is where AI is very complementary to humans. And the fourth scenario is where repeatability of the task is low, meaning the task does not follow a pattern.
And the criticality or outcome is also low. This is a very confusing scenario. The only thing that I'll say to this scenario is consider using AI only if it is a lot cheaper than humans. And even that would require a lot of testing and verification because the exploration of AI itself is a very expensive beginning. So to be honest for the moment, anything that comes in this area, we are happy to ignore and concentrate on other areas.
We don't have to solve all problems at the same time. If we can solve the 20% of the problem that have 80% impact, I can assure you that none of those ideas fall within this particular quadrant. All of this looks simple and straightforward in a framework, but that's it's not easy to incorporate. Still, the reason why this is it is important to discuss this is at least this gives us a basis to have a preliminary conversation for elimination.
We know what we should not concentrate on when thinking about our strategy and why we should not concentrate on that in our world in this quadrant that we are concentrating on. And we recommend that you also explore scenarios in your business that fits this particular quadrant and you are likely to find very low hanging fruits where can be used that done.
Let's look at the actual human interaction. What kind of interactions are possible for that. Let me ask Neil to come back again. So let's delve into the roles of human AI and LLMs in our workflows as we explore this dynamic partnership. We have found some key approaches that help us leverage the strength of both of these entities, human and AI.
And their expertise. Now the key problem to solve is what is the optimal mix of human and AI in a workflow for most effective business outcome. Let's do so in the context of having deterministic output, which is mostly relevant for the enterprise businesses. If you take a moment and think about all you want within publishing, a large part of that will fit in deterministic output.
And that quadrant, which I had spoken about, meaning when the input is objective and the output expectation is also objective. Subjectivity exists in many areas, but those are the ones where ethical considerations are very high. And so until those are sorted, let's still look at the deterministic output areas to drive effective outcome.
But before we talk about how to use humans in the workflow, let's talk about two kinds of human interactions, which have a lot of nuances that a Harvard professor aptly terms as cyborg and centaur. And don't be scared by the terms. Although there is an element of futuristic dystopia associated with the term cyborg and who has told us that cyborg is akin to dystopian world. The true and authentic source of scientific information Hollywood movies.
The centaur approach involves a clear division of labor, where humans and AI each handle the tasks they are best suited for. For instance, I might decide on the statistical technique for an analysis and then let produce the graphs and charts. This strategic division of labor ensures that strengths of both human and machine are utilized optimally. This can happen multiple times in a workflow where the human decides on a direction, executes it, and again human decides on another direction and executes it all over again until the final outcome is achieved.
Here the human is largely in control, in the control of input and output, and takes a supportive role where AI is only directed by humans. Now the other one is the cyborg approach. Here, human and AI work in a deeply integrated manner, with tasks being shared and passed back and forth. Let's take the scenario of creating a complex of creating a complex research discovery.
The cyborg approach would have AI and humans working together throughout the process, and it would be closely fused together with a lot of interdependencies in between them and human will ask an initial research question. For example, here is an PhD thesis that I'm thinking about. What do you think. I will look at the million of research material. It has at its disposal and it will propose some direction. Now, the process for humans will take months or years, but then, depending on output, human can engage and provide for the instruction and keep iterating.
And all of that process becomes much shorter. It takes otherwise, which would have taken much longer to achieve. Now let's take another daily use example. All word editors. Ms word. Google Docs are phone editors. They all come with auto suggestions and autocomplete where I might start a sentence and let the complete it continuously refining the output together and over a period of time the boundary of who is helping whom plus.
And both these scenarios present a significant risk. What is that risk. Research indicates that when works really well, it boosts average human performance. This is a significant source of excitement about AI. But when performs exceptionally well, humans may start relying on it blindly, which is in some ways similar to highway hypnosis, where drivers on a familiar road zone out and they rely more on their mental predictions than on actual visual feedback, where people can become overly dependent on AI, paying less attention and trusting and blindly.
So Abby mentioned the lack of trust in AI, and I'm talking about the opposite overreliance and blind trust in AI. So now, with the understanding of cyborg and center concept as a broad approach to look at human interaction, let me take you back to the model. And there with a focus on the segment where deterministic output is required considering the multiple parameters like the nature of input, task, complexity, criticality, cost, et cetera.
We look at designing the workflow. The variables in such workflow could be who triggers the task human or I. If I has triggered a task, can it finish on its own. End to end. Possible, but mostly not the case. Where exactly is the human in the loop needed. Who validates the completion of the task. Human or AI.
The ideal situation is knowing where will perform better, faster or cheaper than humans. And if I can recognize when to ask humans for help. We spoke about error awareness, which is our approach of doing this with technology in general, but in particular it enables the models to prompt for human intervention when it's not sure of the correctness of its output based on a predefined confidence score threshold.
So depending on the purpose of placing a human in the loop, the human with the respective skill needs to be there. And we know from our experience, exception handling often requires people with exceptional skills. Addressing these questions help us creating the workflow that balances human and contributions effectively. So let's put all of this together. We touched upon the irreplaceable value of humans in driven workflow, focusing on their strengths that can be leveraged.
The value of human, conversely, also tells us the limitations of AI. We then explored how large language models handle subjectivity and therefore the need for us to understand that before making it work in situations that requires deterministic output. Horses for courses and for which we looked at the framework. And we also looked at the cyborg center concept to illustrate how to navigate and optimize human interaction for the most optimal outcome.
There is no specific workflow of human interaction. You will have to accept and assume that multiple such workflows will exist and will eventually be needed to achieve any effective outcome for your work in the world. Focus should be on effectiveness of the solution. Efficiency will come as a byproduct when integrating AI and human intelligence. We also need to consider broader organizational context.
The success of implementation hinges on us doing the hard work of changing our culture. This change requires a first mindset from every person in the organization with the understanding that it is only with human coordination that we will be able to effectively use AI. This is what we are aiming for in our new brand identity of fostering an AI first mindset, ensuring that the value of human and the value of AI is clearly articulated.
So the original question of how we coordinate and balance the dance between humans and what we wanted to convey from the session is that there are multiple choices and multiple approaches. The frameworks we presented should give you a guideline, but that must be tailored to your specific environment, specific use case and specific business risk appetite. Just remember, the best dance happens when both partners know when to lead and when to follow.
And in this human interaction, we have to acknowledge that we will sometimes lead. And we will sometimes follow. But what matters most is that the human coordination leads to an effective outcome. Thank you very much. There are any questions. We can also be outside and.
Discuss this. I understand this is a very unconventional topic, but I think as we all look at just using and consuming in multiple conversations, what we have found is people do not understand when not to use AI, and it just appears to be a hammer to every nail. And we thought it is important to tackle that scenario of helping people understand the contrast between and conventional technology and humans.
I hope it gave some value. Thank you very much.
Segment:0 .
Last year, we were at Portland. ChatGPT had just been released and a few months ago and we did a session to talk about the basics of a large language model. What is it. How is it different to a model. And we got a very good feedback. So that has incentivized us to come back and talk about a similar topic for those of you who do not know about.
We've been around for 25 years. We deliver post acceptance publishing services and technology products to scholarly societies, commercial, and open access publishers spread across North America, Europe, and the UK. We are based out of a very hot and exotic city of Chennai in the South of India. I am Abby. I am the CEO of tech and have the pleasure and privilege of representing 3,000 publishing and technology experts who form tech do drop by our booth to know more about us.
For today's session, as we were preparing for it, we realized we had picked a very difficult topic only because these questions that we are posing have so many different responses that it is impossible to cover anything conclusively in 30 minutes. So we have picked some of those that are very relevant. The underlying technology is evolving so swiftly that it is extremely difficult to keep pace. Nevertheless, I want to acknowledge that we have borrowed from views of academicians and business leaders, but adjusted those ideas to our experience so far.
The session has a very catchy title, the coordinated dance between and humans. And to present this session, you have me someone born with two left feet and not even the faintest idea of a dance. So I thought I will enlist support from Neil, who heads our technology and clearly knows more about these things than I do. But apparently he's as poorly endowed with dancing skills as I am.
At a broad level. This is a discussion of how we are looking at leveraging AI and humans for optimal business outcome in a manner that is sustainable, ethical, and practical. I recognize this is a heavy topic and we wanted to talk about something that isn't the conventional conversation at this conference. We recognize it could be a complete hit or a complete miss, but we will try and do our best and live with the consequences.
Let me first tell you what we are not going to talk about. Enough is being discussed about the impact of AI on scholarly publishing. So we are staying away from it beyond using some examples. Enough people are talking about AI powered products that they have developed. We will talk about products when you meet us at our booth and not at this session. Enough people are predicting the future of AI.
We are most definitely not. If I can predict the future of AI with any level of certainty I would not be at. We are also not talking about the ethics of AI, largely because that is a very specialized area and there are better people who are trying to figure that out. But partly also because ethical considerations are so fundamental to the use of AI that I'll be repeating that after every statement or every framework.
So please assume that ethical considerations form the backbone of anything and everything to do with AI, and it is implicit in today's talk. So what are we talking about. You will agree when I say that one should not use technology just because it is available. We should use technology when we can be sure that it is useful and safe to use and when the value is clear and apparent.
Today, we'll be sharing our experience with you of how we determine value in the world of AI, both from technology as well as from humans and multiple combinations of both these resources. And a large part of using something effectively is to know what it is capable of. But also what it is not capable of when to use it and when not to use it. Today's session is in four segments.
The first and the most important is explaining what is it about humans that makes us indispensable in the world of AI. It seems obvious, but it is nevertheless something we should keep talking about. Then Neil will talk about LLMs and what makes them unique. More importantly, what does this unique uniqueness mean in terms of the context of work that we do. How should we compare LLMs with conventional technologies, basically.
I will then talk about a mental framework that we have been using to determine when to use AI and how to use AI. And then Neil will take the final segment to talk about the human dance, the synchronization of it, and in that process derive an outcome that is meaningful, useful and safe. So human value. The first thing to explore is this In a world where the relevance of AI is increasing, the question of human relevance becomes even more pressing.
So the question is, why is human involvement as yet indispensable? An easy answer to this question is to look at the difference between I and human. This quote from Albert Einstein beautifully captures the distinction between the intuitive and rational mind, which in my understanding is the most significant difference between. And humans lacks creativity and intuition lacks emotional intelligence.
And I wasn't even a term as it is today when Einstein was alive. Underpinning statistics, making predictions based on data and patterns, finding logic and reasons that exists within that data and pattern is rational. It is no doubt powerful, but it lacks nuances and it lacks the ability to make decision that combines data or fact with context and intuition.
Basically, it lacks out of box thinking quite literally. While it is great with data processing, pattern recognition and future enhancement will only make more sophisticated with better rational outcome, it is unlikely to replace humans. Well, all of that is very generic though, and that's something that you all know already. There is nothing new in what I have said so far. So thought we will pose a more difficult question, which is the relevance in the context of work to understand the value of humans in the world of air, which is a simple matter of trust.
The question to ask, therefore, is, can we trust in a manner we trust humans. If the answer to that is Yes, that is a huge endorsement for AI. If we can't, we need to understand why we can't trust AI. I will use two arguments to explain the prevalent view on this matter. The matter of trust.
Please remember ethics and everything to do with it is implicit and is not being discussed here. The first is the matter of bias. Everyone talks about the bias that plagues the system because and because of those biases you can't trust. Well, as humans, we also have our own biases. All of those biases are represented by our views. All of those views form content that goes on the internet, and that's the content that has used for his training.
So essentially, the bias that the system has is a reflection of our own biases. So if we have bias and has bias, what is the problem. The problem is that whilst humans have biases, humans have bias. We also have the ability to recognize them, reflect on them and adjust through conscious effort. We have the ability to consciously review data in a completely different context, add emotional intelligence and intuition to it, add subjectivity, empathy.
We have the humility to accept that our learning may be flawed and then do something that may not have a precedence, something that may not follow a data based pattern. Can't do that, at least not without human intervention. Which gives you a clue of where we are headed with this session will need a lot more data to change their decision, even in a conventional, rule based technology that we are all so used to could just add a line of code to alter a decision.
But a model needs a huge amount of data to override their current bias, and preparing that data is expensive and challenging. And who knows. This new data that is trained on can potentially lead to a completely different kind of bias. And more importantly, biases without corrective measures may just keep getting amplified, which is a real risk that many people have already experienced, and that can lead to unintended consequences.
So the first issue of trust is with the bias. The second matter that we can discuss is what is commonly known as explainability. Humans have the ability to explain their decisions. We may choose not to but we have the ability, at least most models, especially the deep learning models, are Black boxes. We know what goes in. We know what comes out.
But the how is not entirely clear. And the more billions of parameters you have, the models decision making process becomes even more complicated. And even more difficult to predict. So the model will provide you with an output, but without clearly explaining a clear explanation of how it got to that output. LLMs recently have added what they call as reference ability, which is a step further, but not enough at all.
They can tell you where they got the information from that help them produce the output, but how still remains a black box. So the humans ability to explain their decision making builds trust and accountability. And conversely, AI'S inability to do so takes trust and accountability away from it. With these two things biases and explainability being valid issue, I think it is safe to assume that trust in AI is output is likely to remain a questionable aspect in the foreseeable future, which is very important in determining the way we use AI and our work today.
So we had set out to answer the question of why are humans yet indispensable in the world of AI, intuition and creativity, ability to reflect on our bias and the ability to explain our decision making are essentially what makes humans indispensable. This was an important thing to establish before we move forward, and there are many other reasons, but I think these three impact us on the work front the most. And before I bring Neil to talk about LLMs, I want to pose another interesting question.
Humans make mistake all the time. Why then do we accept that we can make mistakes but has to be perfect. I had thought of this question but never really thought about it. And again, it is very important to understand this from a work perspective. There was a long list of pointers to this, but in the interest of time, I've trimmed it down.
We talked about biases and we talked about explainability. Both of these make it more acceptable to accept human mistakes, but not that of AI. When humans make mistakes, we have accountability. We can think about our decision making, apologize, take corrective actions. Unlike humans, I operate without fear of job loss, without contractual obligation, without legal accountability.
Basically, it acts with impunity. That makes it OK and more acceptable to accept human mistakes, but not that of humans are considered to be subjective, whereas AI is expected to be objective or at least perceived to be objective. That perception of objectivity also leads to the expectation of a flawless performance, which in a convoluted manner allows us to trust human mistakes but not that of AI.
And finally, a quote from history that sums it all. Historically, humans have been the decision makers and societal structures are built around human authority and discretion. Being a relatively new entrant is held to a higher standard to prove its reliability and safety. So the distinction between I and human that leads relevance to humans in the world of AI is what we have discussed so far. But now let me ask Neil to come and talk about what is unique about them.
That is obvious but not that obvious. Neil great. So we heard from Abby about the relevance of humans. Now, let me share our perspective on how we see AI, especially in the context of LLMs and their unique attributes and capabilities. So if you think about conventional technologies that we have been using so far, it always had a very clear input and a very clear output.
Its input was objective and largely predictable was the output. Think of a website, a workflow, a tool. Everything had a range of predetermined scenarios. Of course, there is complicated programming in between and calculations, but input and output both remained largely objective. And in the world of AI, this notion has been challenged and to understand human interaction, it is very important to understand why particularly LLMs are not the same as conventional technology.
Otherwise, we run the risk of getting taking our traditional understanding of making technology work for us and apply it to AI with potentially disastrous outcomes. So let me present to you a framework that has helped us understand better. It is called dialogue dynamics grid. In simple terms, it creates a broad classification based on input and output of the system and whether they are subjective or objective.
Like I mentioned, conventional technologies excelled in scenarios where both input and output were objective, and that's why we have become fairly adept at managing these situations. So we call it deterministic output. But the large language models, for example, ChatGPT, they are designed to handle subjectivity in both input and output, which puts it at the other end of the spectrum called creative synthesis.
Think about problems we typically solve today. They usually have objective outcomes, which conventional technologies handle well. But these technologies lack the ability to process subjective inputs and outputs effectively. So our experience with them has always been very objective, and the emergence of LLMs has fundamentally shifted this very paradigm. And with LLMs interactions are no longer confined to predefined parameters and hence creative synthesis.
Or simply put, brainstorming with AI. So previously brainstorming required human collaboration. Now it can be done with LLMs and I'm sure you would have experienced LLMs are generally pretty good at it. And because that and that happens because that's the design of the system. But it may not be as good in giving you an objective output to your objective input because that's not what it is meant for.
So in expecting LLMs that it is, which is very capable of creative synthesis and to behave like traditional technology and still provide a deterministic output, we are trying to apply an expectation to that. It isn't designed for. And when the outcome isn't what we expect, we don't get that. It causes doubts in our mind about a technology. And in that process, we as the users of this technology are setting it up for failure.
So if we have to get LLMs to produce a deterministic output, we have to make it work differently. And that's where humans come in. To get an to maximize its subjective assessment capability, but guide it to produce a deterministic output using human intervention. Hold on to that thought. That's what I'll come back and discuss. But before that, I'll let Avi talk about his mental model of how and in what scenarios is useful and how to use it.
So we all acknowledge that technology is changing and therefore our approach to deal with technology must also change. Neil gave you an idea of what's different in Chennai compared with conventional technologies. Conventional technology was a brute force that applied to all scenarios. AI isn't not just because of the subjectivity of input and output, but also because of the trust and explainability that we had discussed before.
Before we go further, I want to talk about two things that we have experienced that might be helpful when you are thinking about technology in general and in particular. The first is automation. For the longest we have considered automation as the final goal. But as we started understanding at a deeper level than simply using it as a tool, the thing that we realized was that automation is perhaps not the best goal to have.
Automation improves efficiency to a degree, but it is not always synonymous with optimal result or optimal outcome. An efficient system is basically optimized for doing things with minimal waste. That's a very manufacturing industry term that we have all adopted for the longest. It isn't effectiveness, on the other hand, or having an effective system is doing the right things to achieve the desired outcome.
And in a new world where we have to balance AI and human strength, we have to understand and define the outcome. If we still keep thinking about AI in terms of automation, we will always hesitate in using it because AI is not about automation. It does not follow the same conventional patterns of technologies that we are currently used to. This can be best illustrated by what we call as error awareness, the ability of a system to determine where it has gone wrong and therefore ask humans for help.
Our entire technology vision is based on this one fundamental principle. We don't have time here, and I'll be happy to discuss what error awareness means if you find us outside this room. So with that background, let's look at a mental framework we use to determine when and how to use AI. I believe you will immediately relate to it and it is important to state that this isn't prescriptive.
It is only indicative. In this approach, we look at repeatability of a task or an action, a repeatability in the context of means. There will be a huge amount of data from where patterns can be determined, which makes it easier to train an AI system. On the other hand, if the task or action isn't repeatable, well, the data availability might be sparse and the outcome of the train system will be subjective and inaccurate.
On the other axis, we have the criticality or impact of the outcome of any task or action. So let's look at these four quadrants, one after the other. If the repeatability of the task is very low, meaning the task does not follow a pattern. And the criticality or impact of the outcome is very high. This is a very risky scenario in which should be we should consider using. And I'll give you an example, which can be controversial, but consider peer review.
That is not an exact science. It has nuances that cannot be captured in data and patterns. Again, focus on the overall activity of peer review and not individual. Particular tasks can most definitely be used to produce an output because it has the ability to analyze huge amount of data and huge amount of content in a scalable manner. Human can't, but the use of AI must be initiated by a human and the outcome must always be reviewed and acted on by the human.
This grid allows or expects a trust system that should be near zero. The second is if the repeatability of the task is high, meaning the task does follow a pattern. But the criticality or impact is also very, very high. Nope we go back. This is one of the scenarios of human coordination that has got great potential. This is a trust but verify scenario where we let do its magic and insert a human in the loop, either in the middle or at the top to verify the output.
Whether you verify 100% of the output or you verify a lower percentage depends on your perception of the criticality or the risk to your business. I think aspects of research integrity falls under this category. We have data and patterns that are emerging. AI is able to predict pattern based behavior but to take action on the output of AI without human oversight remains very risky.
This is a scenario where AI is supplemented to human. Human should always be at the end of the loop to make the final decision. The third scenario is where the repeatability of the task is high, meaning the task does follow a pattern, but the criticality or impact of the outcome is relatively low. This is an ideal human conversation or coordination scenario. This is the low hanging fruit.
This is a trust and accept scenario, but with the realization that I might mistakes make mistakes and that should be OK, just like humans do. You can have a trust but verify mechanism for this too. But on a sample basis, I would say copyediting falls in this category. Maybe not the language part of it. But the mechanical part of it, because the impact of capitalization going wrong is perceived not real, but the impact of a language error.
Changing the meaning of research is very, very high. Hence, this distinction is necessary. These are scenarios where human can be at the middle of the workflow, at the end of the workflow, and sometimes with very careful consideration, humans might not be needed at all. This is where AI is very complementary to humans. And the fourth scenario is where repeatability of the task is low, meaning the task does not follow a pattern.
And the criticality or outcome is also low. This is a very confusing scenario. The only thing that I'll say to this scenario is consider using AI only if it is a lot cheaper than humans. And even that would require a lot of testing and verification because the exploration of AI itself is a very expensive beginning. So to be honest for the moment, anything that comes in this area, we are happy to ignore and concentrate on other areas.
We don't have to solve all problems at the same time. If we can solve the 20% of the problem that have 80% impact, I can assure you that none of those ideas fall within this particular quadrant. All of this looks simple and straightforward in a framework, but that's it's not easy to incorporate. Still, the reason why this is it is important to discuss this is at least this gives us a basis to have a preliminary conversation for elimination.
We know what we should not concentrate on when thinking about our strategy and why we should not concentrate on that in our world in this quadrant that we are concentrating on. And we recommend that you also explore scenarios in your business that fits this particular quadrant and you are likely to find very low hanging fruits where can be used that done.
Let's look at the actual human interaction. What kind of interactions are possible for that. Let me ask Neil to come back again. So let's delve into the roles of human AI and LLMs in our workflows as we explore this dynamic partnership. We have found some key approaches that help us leverage the strength of both of these entities, human and AI.
And their expertise. Now the key problem to solve is what is the optimal mix of human and AI in a workflow for most effective business outcome. Let's do so in the context of having deterministic output, which is mostly relevant for the enterprise businesses. If you take a moment and think about all you want within publishing, a large part of that will fit in deterministic output.
And that quadrant, which I had spoken about, meaning when the input is objective and the output expectation is also objective. Subjectivity exists in many areas, but those are the ones where ethical considerations are very high. And so until those are sorted, let's still look at the deterministic output areas to drive effective outcome.
But before we talk about how to use humans in the workflow, let's talk about two kinds of human interactions, which have a lot of nuances that a Harvard professor aptly terms as cyborg and centaur. And don't be scared by the terms. Although there is an element of futuristic dystopia associated with the term cyborg and who has told us that cyborg is akin to dystopian world. The true and authentic source of scientific information Hollywood movies.
The centaur approach involves a clear division of labor, where humans and AI each handle the tasks they are best suited for. For instance, I might decide on the statistical technique for an analysis and then let produce the graphs and charts. This strategic division of labor ensures that strengths of both human and machine are utilized optimally. This can happen multiple times in a workflow where the human decides on a direction, executes it, and again human decides on another direction and executes it all over again until the final outcome is achieved.
Here the human is largely in control, in the control of input and output, and takes a supportive role where AI is only directed by humans. Now the other one is the cyborg approach. Here, human and AI work in a deeply integrated manner, with tasks being shared and passed back and forth. Let's take the scenario of creating a complex of creating a complex research discovery.
The cyborg approach would have AI and humans working together throughout the process, and it would be closely fused together with a lot of interdependencies in between them and human will ask an initial research question. For example, here is an PhD thesis that I'm thinking about. What do you think. I will look at the million of research material. It has at its disposal and it will propose some direction. Now, the process for humans will take months or years, but then, depending on output, human can engage and provide for the instruction and keep iterating.
And all of that process becomes much shorter. It takes otherwise, which would have taken much longer to achieve. Now let's take another daily use example. All word editors. Ms word. Google Docs are phone editors. They all come with auto suggestions and autocomplete where I might start a sentence and let the complete it continuously refining the output together and over a period of time the boundary of who is helping whom plus.
And both these scenarios present a significant risk. What is that risk. Research indicates that when works really well, it boosts average human performance. This is a significant source of excitement about AI. But when performs exceptionally well, humans may start relying on it blindly, which is in some ways similar to highway hypnosis, where drivers on a familiar road zone out and they rely more on their mental predictions than on actual visual feedback, where people can become overly dependent on AI, paying less attention and trusting and blindly.
So Abby mentioned the lack of trust in AI, and I'm talking about the opposite overreliance and blind trust in AI. So now, with the understanding of cyborg and center concept as a broad approach to look at human interaction, let me take you back to the model. And there with a focus on the segment where deterministic output is required considering the multiple parameters like the nature of input, task, complexity, criticality, cost, et cetera.
We look at designing the workflow. The variables in such workflow could be who triggers the task human or I. If I has triggered a task, can it finish on its own. End to end. Possible, but mostly not the case. Where exactly is the human in the loop needed. Who validates the completion of the task. Human or AI.
The ideal situation is knowing where will perform better, faster or cheaper than humans. And if I can recognize when to ask humans for help. We spoke about error awareness, which is our approach of doing this with technology in general, but in particular it enables the models to prompt for human intervention when it's not sure of the correctness of its output based on a predefined confidence score threshold.
So depending on the purpose of placing a human in the loop, the human with the respective skill needs to be there. And we know from our experience, exception handling often requires people with exceptional skills. Addressing these questions help us creating the workflow that balances human and contributions effectively. So let's put all of this together. We touched upon the irreplaceable value of humans in driven workflow, focusing on their strengths that can be leveraged.
The value of human, conversely, also tells us the limitations of AI. We then explored how large language models handle subjectivity and therefore the need for us to understand that before making it work in situations that requires deterministic output. Horses for courses and for which we looked at the framework. And we also looked at the cyborg center concept to illustrate how to navigate and optimize human interaction for the most optimal outcome.
There is no specific workflow of human interaction. You will have to accept and assume that multiple such workflows will exist and will eventually be needed to achieve any effective outcome for your work in the world. Focus should be on effectiveness of the solution. Efficiency will come as a byproduct when integrating AI and human intelligence. We also need to consider broader organizational context.
The success of implementation hinges on us doing the hard work of changing our culture. This change requires a first mindset from every person in the organization with the understanding that it is only with human coordination that we will be able to effectively use AI. This is what we are aiming for in our new brand identity of fostering an AI first mindset, ensuring that the value of human and the value of AI is clearly articulated.
So the original question of how we coordinate and balance the dance between humans and what we wanted to convey from the session is that there are multiple choices and multiple approaches. The frameworks we presented should give you a guideline, but that must be tailored to your specific environment, specific use case and specific business risk appetite. Just remember, the best dance happens when both partners know when to lead and when to follow.
And in this human interaction, we have to acknowledge that we will sometimes lead. And we will sometimes follow. But what matters most is that the human coordination leads to an effective outcome. Thank you very much. There are any questions. We can also be outside and.
Discuss this. I understand this is a very unconventional topic, but I think as we all look at just using and consuming in multiple conversations, what we have found is people do not understand when not to use AI, and it just appears to be a hammer to every nail. And we thought it is important to tackle that scenario of helping people understand the contrast between and conventional technology and humans.
I hope it gave some value. Thank you very much.
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode