What Did Harvard Business School Teach Us About Using Generative AI?
This article was first published in the Wiser! newsletter on 5th Nov 2023.
I get asked a lot of questions about using AI. By far the biggest category is the “is it worth the effort?” group of questions. The short answer is “yes, you can.” But it’s a conditional answer because the slightly longer and more correct answer is “yes, you can, but be careful, know what it’s good at, what it’s not and remember that the buck stops with you!”
To support this advice I lean on the data from a recent working paper by Harvard Business School. The study involved 758 consultants from Boston Consulting Group and it’s purpose was to look at “the performance implications of AI on realistic, complex, and knowledge-intensive tasks.”
➜ In other words, how to get the best out ofAI tools in the workplace.
The headline finding is that when you train people to use AI, they produce higher quality results faster than people with no training and people who are not using AI at all.
The study reported that trained AI users “completed 12.2% more tasks on average and 25.1% more quickly.”
BUT (there’s a “but”)…this only applied for tasks within the AI’s core capabilities.
Which makes sense when you think about it. Because when you give AI a task it’s not trained to do, the human wins hands down, every time. It’s like using a calculator to write words. After “hello,” “bill,” and “boobies” you’re running out of options.
For the study, Harvard used GPT-4 for the test, which involved splitting the BCG consultants into 3 groups; those that had been given AI“prompt” training, those that hadn’t but were left to use the AI anyway, and a third group that had no access to ChatGPT at all.
When given tasks to do that were within ChatGPT’s range of capabilities, the study found:
- Consultants using AI produced higher quality results.
- They were also faster and more productive.
- Below-average performers got the biggest boost from using AI tools.
- Low performers + AI performed better than high performers without AI.
- Training in AI improved both the quality and the speed of output for all consultants.
When the consultants were asked to perform tasks outside of ChatGPT’s range of capabilities, the study found:
- Those who used AI were more likely to trust ChatGPT even when the output was incorrect.
- The AI was consistently wrong and also dangerously convincing and “confidently correct.”
- Training of the consultants in AI led to even greater levels of trust in the output even though it was consistently wrong.
- There was less variety in the results that were more generic and similar across the groups.
The study reported that consultants with below average performance levels increased their performance by 43%.
➜ Here’s The Thing: When you play to AI’s strengths, it can raise your performance and enable you to compete above your pay grade. It will also improve your productivity to do more with less.
On the flip side, the reverse is true. When you cross the line and take AI outside its comfort zone, you’re heading for trouble (see case of lawyer who used ChatGPT to prepare for a court case, didn’t check it, used it, and is now in a heap of trouble.)
The issue is knowing where the line is.
This applies to all of the AIs, not just ChatGPT. They’re convincing and confident when they’re wrong. That’s because they don’t know they’re wrong!
So, the key takeaway is: use AI to raise your game, but do so with a critical eye.
See AI as your multi-purpose Swiss Army productivity tool. It can do many things, but at the end of the day, it’s in your hands.
There’s no doubt that AI can both supercharge your output AND help you compete above your pay grade. Just be careful, that’s all.
➜ Download the Harvard Business School Study.
Further Reading
➜ The Rise of Generative AI: Exploring the Safer Alternative to ChatGPT with Claude by Anthropic
➜ What Do We Mean By “Large Language Models”?
➜ The Future of Work: Embracing AI’s Impact in Conference Calls
➜ How To Use ChatGPT To Enhance Your CV
➜ AI is Making Stars like Tom Hanks, Robert De Niro and Harrison Ford Look Younger in Movies
Supercharged Productivity Tips and Tricks
Raycast
If you’re a Apple user then you’re probably using Raycast on your Mac instead of the native Spotlight, why wouldn’t you be?? Raycast is free but the app has just added a paid layer that integrates with ChatGPT and GPT-4’s real-time web results. For $8/month you can now search the Internet from your Mac without opening a browser.-
Brave Browser
If privacy is your thing, then look at what Brave, the privacy-centric browser, have done with their new AI chatbot called Leo. It is designed to offer “anonymous and secure” assistance to its users, such as translating, answering questions, summarising web pages, and content generation. The unique selling point of Leo is that there is no recording of your conversations, nor is any of them used in AI training. Leo is based on Meta’s Llama 2 AI model and is free for all desktop users, but a premium version using Anthropic’s Claude Instant model (with faster responses) will be available for $15 per month. (NOTE: I’ve just ditched Microsoft Edge and moved back to Brave, I’ll let you know how I get on.)
Hal9
Are you an SME and want/need to get instant answers from your data but don’t have your own data analysts to do it for you? Then meet Hal9, an AI tool that lets you chat with your enterprise databases using secure generative AI. Where Hal9 is different to using ChatGPT’s Code Interpreter for data analysis is that your data never leaves your database. Where as ChatGPT requires that you upload your commercially sensitive data to be analysed, Hal9 brings the power of AI to you, not the other way round.
About The Author
Rick Huckstep has worked in technology his entire career, as a corporate sales leader, investor in tech startups and keynote speaker. From his home in Spain, Rick is thought leader in artificial intelligence, emerging technologies and the future of work.
🤔 Join The Mailing List and Get Wiser! every week (and your free eBooks and resources)
