And a chatbot is not a human. And a linguist named Emily M. Bender is very worried what will happen when we forget this.
Nobody likes an I-told-you-so. But before Microsoft’s Bing started cranking out creepy love letters; before Meta’s Galactica spewed racist rants; before ChatGPT began writing such perfectly decent college essays that some professors said, “Screw it, I’ll just stop grading”; and before tech reporters sprinted to claw back claims that AI was the future of search, maybe the future of everything else, too, Emily M. Bender co-wrote the octopus paper.
Bender is a computational linguist at the University of Washington. She published the paper in 2020 with fellow computational linguist Alexander Koller. The goal was to illustrate what large language models, or LLMs — the technology behind chatbots like ChatGPT — can and cannot do. Read More
Tag Archives: ChatBots
More than you’ve asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models
We are currently witnessing dramatic advances in the capabilities of Large Language Models (LLMs). They are already being adopted in practice and integrated into many systems, including integrated development environments (IDEs) and search engines. The functionalities of current LLMs can be modulated via natural language prompts, while their exact internal functionality remains implicit and unassessable. This property, which makes them adaptable to even unseen tasks, might also make them susceptible to targeted adversarial prompting. Recently, several ways to misalign LLMs using Prompt Injection (PI) attacks have been introduced. In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes. Recent work showed that these attacks are hard to mitigate, as state-of-the-art LLMs are instruction-following. So far, these attacks assumed that the adversary is directly prompting the LLM. In this work, we show that augmenting LLMs with retrieval and API calling capabilities (so-called Application-Integrated LLMs) induces a whole new set of attack vectors. These LLMs might process poisoned content retrieved from the Web that contains malicious prompts pre-injected and selected by adversaries. We demonstrate that an attacker can indirectly perform such PI attacks. Based on this key insight, we systematically analyze the resulting threat landscape of Application-Integrated LLMs and discuss a variety of new attack vectors. To demonstrate the practical viability of our attacks, we implemented specific demonstrations of the proposed attacks within synthetic applications. In summary, our work calls for an urgent evaluation of current mitigation techniques and an investigation of whether new techniques are needed to defend LLMs against these threats. Read More
How will Language Modelers like ChatGPT Affect Occupations and Industries?
Recent dramatic increases in AI language modeling capabilities has led to many questions about the effect of these technologies on the economy. In this paper we present a methodology to systematically assess the extent to which occupations, industries and geographies are exposed to advances in AI language modeling capabilities. We find that the top occupations exposed to language modeling include telemarketers and a variety of post-secondary teachers such as English language and literature, foreign language and literature, and history teachers. We find the top industries exposed to advances in language modeling are legal services and securities, commodities, and investments. Read More
Microsoft now lets you change Bing’s chatbot personality to be more entertaining
Microsoft restricted Bing AI in recent days after wild responses, but a new toggle lets the chatbot get more creative once again.
Microsoft has added a new feature to its Bing chatbot that lets you toggle between different tones for responses. There are three options for the AI-powered chatbot’s responses: creative, balanced, and precise. The creative mode includes responses that are “original and imaginative,” whereas the precise mode favors accuracy and relevancy for more factual and concise answers.
Microsoft has set the default for the Bing chatbot to the balanced mode, which it hopes will strike a balance between accuracy and creativity. These new chat modes are rolling out to all Bing AI users right now, and around 90 percent of users should be seeing them already. Read More
Inside the ChatGPT race in China
A Chinese ChatGPT alternative won’t pop up overnight—even though many companies may want you to think so.
Every once in a while, there’s one thing that gets everybody obsessed. In the Chinese tech world last week, it was ChatGPT.
Maybe it was because of the holiday season, or maybe it was because ChatGPT is not currently available in China, but it took more than two months for the natural-language-processing chatbot to finally blow up in the country. (OpenAI, the company behind ChatGPT, told Reuters it wasn’t operating in China because “conditions in certain countries make it difficult or impossible for us to do so in a way that is consistent with our mission.”)
But in the span of the past week, a massive competition has developed, with almost every major Chinese tech company announcing plans to introduce their own ChatGPT-like products (even some that have never been known for artificial intelligence capabilities), while the Chinese public has been frantically trying out the service. Read More
Workers’ ChatGPT Use Restricted At More Banks—Including Goldman, Citigroup
CitiGroup, Bank of America, Deutsche Bank, Goldman Sachs and Wells Fargo have restricted employees’ use of ChatGPT, Bloomberg and Financial News reported Friday, joining JPMorgan Chase, as well as Amazon and multiple major public school districts to limit the use of OpenAI’s new chatbot, which has taken the internet by storm and raised concerns about sensitive information sharing. Read More
ChatGPT get-rich-quick schemes are coming for magazines, Amazon, and YouTube
One morning earlier this week, Neil Clarke, the editor of a prominent U.S.-based fantasy and science fiction magazine called Clarkesworld, was wading through the latest story submissions from authors hoping to be published. He determined that at least 50 submissions that day alone had been lazily drafted by artificial intelligence.
Of the 1,200 global submissions that Clarke received in the first 20 days of February, he deemed 500 of them to be AI-generated.
“It was picking up at a daily rate,” Clarke told Semafor. He was forced to close submissions because of the crush of ChatGPT-created content.
Clarke identified the likely culprit: Followers of online get-rich-quick scammers trying to make a quick buck, in the off-chance their AI-generated work gets published. Read More
Generative AI Is Coming For the Lawyers
Large law firms are using a tool made by OpenAI to research and write legal documents. What could go wrong?
David Wakeling, head of London-based law firm Allen & Overy’s markets innovation group, first came across law-focused generative AI tool Harvey in September 2022. He approached OpenAI, the system’s developer, to run a small experiment. A handful of his firm’s lawyers would use the system to answer simple questions about the law, draft documents, and take first passes at messages to clients.
The trial started small, Wakeling says, but soon ballooned. Around 3,500 workers across the company’s 43 offices ended up using the tool, asking it around 40,000 queries in total. The law firm has now entered into a partnership to use the AI tool more widely across the company, though Wakeling declined to say how much the agreement was worth. According to Harvey, one in four at Allen & Overy’s team of lawyers now uses the AI platform every day, with 80 percent using it once a month or more. Other large law firms are starting to adopt the platform too, the company says. Read More
ChatGPT for Robotics: Design Principles and Model Abilities
This paper presents an experimental study regarding the use of OpenAI’s ChatGPT [1] for robotics applications. We outline a strategy that combines design principles for prompt engineering and the creation of a high-level function library which allows ChatGPT to adapt to different robotics tasks, simulators, and form factors. We focus our evaluations on the effectiveness of different prompt engineering techniques and dialog strategies towards the execution of various types of robotics tasks. We explore ChatGPT’s ability to use free-form dialog, parse XML tags, and to synthesize code, in addition to the use of task-specific prompting functions and closed-loop reasoning through dialogues. Our study encompasses a range of tasks within the robotics domain, from basic logical, geometrical, and mathematical reasoning all the way to complex domains such as aerial navigation, manipulation, and embodied agents. We show that ChatGPT can be effective at solving several of such tasks, while allowing users to interact with it primarily via natural language instructions. In addition to these studies, we introduce an open-sourced research tool called PromptCraft, which contains a platform where researchers can collaboratively upload and vote on examples of good prompting schemes for robotics applications, as well as a sample robotics simulator with ChatGPT integration, making it easier for users to get started with using ChatGPT for robotics. Read More
Will Russian President Vladimir Putin use nuclear weapons in Ukraine? What ChatGPT thinks
NEW DELHI: When President Vladimir Putin ordered a full-scale invasion of Ukraine a year ago, most of the world expected Kyiv to fall within a few days and the superior Russian forces to prevail on the battlefield – similar to Taliban’s lightening quick takeover of Afghanistan.
But a resiliant Kyiv rewrote Putin’s script by putting up a brave front and eventually pushing back the Russian forces with the help of Western aid.
… For now, US thinks that Russia will not resort to nuclear use.
…Amid the raging ‘will he, won’t he’ debate, we asked ChatGPT about the possibility of Putin using nuclear weapons in Ukraine and when the war is likely to end. Here’s what it said… Read More
Rick's Cafe AI 