Justin Sheehy on Being a Responsible Developer in the Age of AI Hype

Responsible AI Development in the Age of Hype

• The talk is aimed at software practitioners who might be feeling overwhelmed by the rapid developments and inflated expectations surrounding AI, with the goal of discussing how to be a responsible developer in the age of AI hype 16s.
• Developers, or software practitioners, have power and their decisions matter, as stated by tech industry analyst Steve O'Grady, and they need to know how to make good, responsible decisions 2m0s.
• The term "artificial intelligence" is broad and unhelpful, but it is still widely used, and it refers to computer programs that can be understood by developers 2m53s.
• Most AI systems have been built using either logic and symbol processing or statistics and mapping probability distributions, with recent attention focused on probabilistic systems 3m24s.
• The current generation of auto-regressive or AR LLMs (Large Language Models) is based on advances such as the Transformer concept, which allows for attention and statistical processing 4m12s.
• The talk emphasizes the importance of understanding the basics of AI systems, such as LLMs, in order to have a concrete conversation about their development and use 4m3s.
• The rapid progress in AI has led to an "age of AI hype," with some people believing that the hype is earned and that we are on the way to the singularity, while others may disagree 2m36s.
• The talk highlights the need for developers to learn from other fields, such as linguistics, philosophy, psychology, anthropology, art, and ethics, in order to make responsible decisions about AI development 1m21s.
• The speaker emphasizes that developers should not think that they can solve problems on their own, without input from other fields, as this is often a bad idea 1m31s.
• Language models, such as those from Google and Open AI, aim to predict and generate plausible language by predicting the next word or token in a sequence, essentially functioning as advanced autocomplete systems 4m52s.
• These models do not plan ahead, have knowledge, or understand meaning, and are not capable of being told to not give false answers, as stated by Google and Open AI 6m3s.
• Open AI has acknowledged in legal replies that their system's primary function is predicting the next most likely words in response to a prompt, and that asking it to tell the truth is an area of active research 6m36s.
• The type of AI being referred to in this context is the Autoregressive Language Model (AR LLM), such as ChatGPT, which is a powerful tool for its intended purpose but has limitations 6m58s.
• Despite their capabilities, these systems are not expected to magically change what they can do, and it is possible that other AI systems may be created in the future that do not have the same limitations 7m12s.
• The current era is being referred to as an "age of hype" rather than an "age of awesome AI" due to exaggerated claims being made about the capabilities of these systems, which is driven in part by the significant financial investments being made in AI research 7m40s.
• Similar hype surrounding AI has occurred in the past, approximately 60 years ago, but the main difference now is the large amount of money being invested in AI research, which adds to the incentive for exaggeration 8m11s.
• Prominent individuals are making claims about AI that are not supported by the current state of technology, and it is essential to be cautious of these claims and not fall for the hype 8m25s.
• To make better decisions as a responsible developer, it's essential to evaluate technology reasonably and not fall for hype or nonsense, which can lead to poor decisions about what technology to use and how to build future projects 8m38s.

The Current State of Large Language Models

• A significant portion of the current hype surrounding Large Language Models (LLMs) like ChatGPT, Palm, Llama, and Claud, claims that they are on a straightforward path to achieving General Artificial Intelligence (GAI), similar to human-like intelligence in science fiction, which is considered nonsense 9m7s.
• A paper from Microsoft Research suggests that LLMs like GP4 have "Sparks of general intelligence," but this claim is based on a flawed test that GPT passed, which was later found to be unreliable if the test was given slightly differently 9m30s.
• The authors of the paper used a test from psychology to evaluate GPT's theory of mind, but the results were not conclusive, and the LLM's ability to provide convincingly human-like text does not necessarily mean it has developed a sense of the beliefs of others 9m44s.
• Another article made an even more dramatic claim that GAI has already arrived, but this claim is not supported by any evidence and places the burden of proof on those who disagree 10m36s.
• The article's claim is not supported by science or reasonable discussion, and it relies on an unproven assertion that GAI exists, which is not a valid way to make a scientific claim 10m50s.
• While the article's core claim is disputed, it does raise important questions about who benefits from and who is harmed by the technology we build, and how we can impact the answers to those questions 11m33s.
• Some people argue that LLMs just need more information and computing power to continue improving and eventually achieve GAI, but this argument is based on a misunderstanding of how LLMs work, which is to synthesize text that looks like the text they were trained on, rather than thinking like a person 12m7s.
• The idea that more compute power will lead to significant advancements in AI is a common misconception, as it might improve performance but not lead to true intelligence, similar to how climbing a tree doesn't get you to the moon 12m45s.
• The Turing Test, also known as the Imitation Game, is often misunderstood as a measure of intelligence, when in fact it only tests a machine's ability to imitate human-like text, which is different from being generally intelligent 13m4s.
• The claim that humans do the same thing as Large Language Models (LLMs) like ChatGPT, probabilistically stringing along words, is misleading, as humans have actual knowledge and understanding, whereas LLMs do not 13m44s.
• The term "stochastic parrot" was coined in a 2021 paper to describe LLMs as probabilistic repeating machines that lack understanding of the meaning behind the words they produce 14m10s.
• The notion that humans are similar to LLMs in that we also probabilistically generate text is a fundamental misunderstanding of the fact that language is a tool used by humans to communicate meaning, which is not the case for LLMs 15m4s.
• Emily Bender, a computational linguist and author of the "stochastic parrot" paper, argues that LLMs like ChatGPT have no ideas, beliefs, or knowledge, and only synthesize text without intended meaning 15m44s.
• Even experts like Yan LeCun, the head of AI research at Meta, acknowledge that language alone is not enough for human-like intelligence, and that something trained only on form cannot develop a sense of meaning 16m28s.
• The face pareidolia effect, where humans see faces in images that are not actually there, is similar to how people perceive intention and meaning in AI-generated text that is structurally similar to human writing, even when they know the system has no intentions or meaning 16m54s.
• The term "hallucination" is misleading when used to describe AI systems, as it implies a disconnection from reality, but AI systems do not have a sense of truth, meaning, or observed reality, and are simply statistically predicting the next word 17m46s.
• AI systems are not capable of hallucination in the way humans are, and the use of this term is a trick that can lead people to believe there is meaning or intention behind the text 17m52s.
• The behavior of AI systems producing ungrounded text is not a bug, but rather what they are designed to do, and they are doing their job well 18m59s.

Examples of AI Hype and Misrepresentation

• The concept of arbitrary behavior emerging from AI systems is not supported by how these systems actually work, and is often encouraged by sci-fi talk about AGI 19m33s.
• Stories about AI systems learning new languages or abilities without training can be misleading, and it's essential to look for clear evidence before accepting such claims 19m58s.
• Huge claims about AI capabilities should come with huge, clear evidence, and it's essential to be skeptical and not simply swallow or dismiss such claims without evidence 20m35s.
• It's essential to require evidence before making outlandish claims about AI capabilities, and not to be fooled by misleading or exaggerated statements 21m1s.
• A YouTube video showcasing a language model, Gemini, received over 3 million views, demonstrating its ability to identify images in real-time conversations, but it was later revealed to be fake through video editing, highlighting the importance of verifying information 21m10s.
• The Mechanical Turk, a chess-playing machine from the 1770s, was able to beat human players, including Benjamin Franklin and Napoleon, but it was later discovered that a human chess player was inside the machine, making it a clever trick rather than true AI 21m59s.
• Similarly, Amazon's AI-powered checkout system and autonomous cars, such as those developed by Cruz, have been found to rely on human elements, with the latter having an average of more than one driver per car, despite claims of being fully autonomous 23m59s.
• The Tesla bot, a robot suit worn by a human, is another example of a company making exaggerated claims about their AI capabilities, highlighting the need for skepticism and verification when encountering seemingly impressive AI systems 24m20s.
• The importance of adversarial proof and transparency in AI systems is emphasized, as even if there is no single "man behind the curtain," human elements are often involved in the development and operation of AI systems 24m46s.
• The development of large language models (LLMs) and image generators, such as ChatGPT, relies on human breakthroughs and innovations, and it is essential to understand the human elements involved in these systems 25m0s.

The Human Element in AI Development

• Reinforcement learning through human feedback (RHF) is a method used to train AI systems, where people write answers to initial prompts to produce the original training set, and then more people interact with that to build a reward model, ultimately creating a system that statistically produces more text that people think is like what a person would write 25m9s.
• This process relies on an enormous amount of low-paid human labor, similar to how Amazon checkout lanes, cruise cars, and Tesla require multiple people per camera or car to function 25m44s.
• Using chat GPT or similar AI systems means utilizing the labor of thousands of people paid a couple of dollars an hour to do work that others may not be willing to do, raising a class of ethical problems 26m5s.
• To make informed choices about using these systems, it's essential to understand how they work and be aware of the labor involved 26m23s.

Responsible Use of AI Systems for Developers

• As developers, there are several things that can be done to use these systems responsibly, such as being cautious when using AI systems to build other systems, like GitHub co-pilot or chatbots on websites 26m50s.
• When using large language models (LLMs) or things that are wrappers around them directly for their output, it's crucial to consider the potential risks and consequences 27m18s.
• To ensure wise use of these systems, developers should be mindful of the data they input, just as they would with emailing confidential information to another company, and consider having a contract in place to protect that data 27m35s.
• Many large companies have policies against using systems like chat GPT or co-pilot for real work due to concerns about data protection and responsibility 28m1s.
• Developers should also be cautious when putting code, text, images, or other content that came from an LLM into their products or systems, as the legal issues surrounding property rights for works that have passed through an LLM are still being worked out 28m22s.
• Having "squeaky clean" answers to where everything comes from can make the process of discovery and diligence for startup acquisitions much smoother 28m41s.
• Hundreds of known cases of data leakage have occurred in both directions when using large language models (LLMs), so users should be cautious when sending data to these systems or using their output in work they wish to own 28m49s.
• Training one's own model can alleviate this concern, but it requires more work 29m3s.
• Using LLMs for tasks such as proofreading, building summaries, or as a debate partner can be beneficial, as long as the user is deeply involved in the process and edits the output 29m25s.
• LLMs can be useful for teaching developers debugging skills, as they can create plausible but buggy code 29m59s.
• The key success metric of LLMs is creating plausible text, which can be problematic when used for tasks that require accuracy, such as citing academic work or legal precedents 30m6s.
• LLMs can generate plausible but false information, such as citing non-existent academic papers or legal cases 30m40s.
• Using LLMs to write code that will be shipped is not recommended, as they lack the ability to reason and understand the context 31m32s.

Limitations of LLMs and Reasoning

• LLMs are not capable of reasoning, but rather generate text based on patterns and memorization 31m56s.
• A test was conducted to see if an LLM could reason by asking it to count the number of times the letter "e" appears in the name "Justin Sheehy", but it failed to provide the correct answer 32m11s.
• Prompt engineering was attempted to see if the LLM could figure out the correct answer, but it was unsuccessful 32m29s.
• AI models do not truly reason or account for information, but rather probabilistically generate text based on patterns in their training data, and may occasionally produce correct answers by chance 32m42s.
• When using AI models as components in larger systems, it is crucial to be aware of the content the model was trained on, as this can significantly impact the output and potentially perpetuate biases or undesirable content 33m37s.
• Training a model on one's own content can provide more control, but using pre-trained models trained on the entire internet can introduce a wide range of knowledge, including undesirable sources like 4chan and certain Reddit subs 33m52s.
• The concept of "bias laundering" refers to the tendency to view algorithmic answers as objective or better, despite the potential for biases in the training data, and developers should be aware of this issue 34m17s.

Addressing Bias and Misinformation in AI

• To address these concerns, developers can start by testing AI models for bias using tools like the one from IBM, which should be a basic expectation for any AI-powered system 35m2s.
• Irresponsible decisions are being made by embedding pre-trained language models into systems that make important decisions, leading to predictable results on issues like race, gender, and religion 34m45s.
• The practice of "AI washing" or adding AI components to a system solely for marketing purposes can be harmful, as it may divert resources away from more effective solutions and lead to dangerously worse decisions 35m32s.
• To mitigate these issues, developers should engage in discussions with CEOs, product managers, and other stakeholders to determine whether adding AI components will truly add value to the system, rather than simply adding hype 36m7s.

Accountability and Transparency in AI Development

• Being a responsible developer in the age of AI hype requires accountability, meaning that companies and developers must understand that they are accountable for what they ship, and take responsibility for the potential consequences of their actions 36m32s.
• Large language models (LLMs) cannot be forced to not hallucinate, so developers must be prepared to take accountability for any hallucinations that may occur when using LLMs in their apps or websites 37m1s.
• The use of AI chatbots may lead to financial losses for companies if they have to give out discounts or refunds due to the chatbot's inaccuracies, and it is the developer's responsibility to make sure that companies understand the limitations of the systems they develop 37m20s.
• To be a responsible developer, one must not lie and not make the hype problem worse by wildly over-promising what their systems can do 37m48s.
• Microsoft's Super Bowl commercial is an example of over-promising what their AI system can do, and the company should have represented their work more responsibly 38m17s.
• The US government's Federal Trade Commission (FTC) advises developers to ask themselves questions about their AI product, such as whether it is safe and legal, and whether it respects users' rights 38m40s.
• Developers should not do something that is not legally and safely possible, and should not prioritize the success of their product over the safety and rights of others 38m56s.
• Examples of irresponsible development include violating the rights of hundreds of thousands of people to train a large language model, and prioritizing the development of a product over the safety and well-being of users 39m24s.
• A starting place for being a responsible developer is to develop systems legally and safely, and to prioritize the safety and rights of others over the success of their product 39m51s.

Alignment and Ethical Considerations in AI

• The development of AI requires careful consideration and responsible decision-making to ensure a safe and beneficial outcome for humanity, and it's up to developers to make this happen safely 40m26s.
• The concept of "alignment" in AI development refers to ensuring that AI systems share human values, but this idea is still largely science fiction, as creating general-purpose AI is still multiple breakthroughs away 40m48s.
• A paper by Anthropic proposes a framework for alignment that defines an AI as "aligned" if it is helpful, honest, and harmless, which are values that can be applied to human developers as well 41m42s.
• Developers can use this framework to guide their own work by ensuring that what they build is helpful, has real value, and isn't just hype-chasing, but rather a solution to a real problem 43m8s.
• Developers should also be honest about what they build, avoiding overselling or misrepresenting their work, and minimizing harm caused by their creations 43m19s.
• To be a responsible developer, one should prioritize the perspectives and experiences of those who may be harmed by their work and strive to minimize harm 43m59s.
• By following these principles, developers can exercise great responsibility and help shape a beneficial future for humanity 44m28s.

Conclusion and Call to Action

• The InfQ Dev Summit conference in September will feature talks on critical topics, including responsible AI development, and more information can be found at devs.summit.inq.com 45m21s.