We propose Tied-LoRA, a simple paradigm utilizes weight tying and selective training to further increase parameter efficiency of the Low-rank adaptation (LoRA) method. Our investigations include all feasible combinations parameter training/freezing in conjunction with weight tying to identify the optimal balance between performance and the number of trainable parameters. Through experiments covering a variety of tasks and two base language models, we provide analysis revealing trade-offs between efficiency and performance. Our experiments uncovered a particular Tied-LoRA configuration that stands out by demonstrating comparable performance across several tasks while employing only 13~\% percent of parameters utilized by the standard LoRA method. — Read More
Recent Updates Page 147
Generative AI for Beginners — A Course
Learn the fundamentals of building Generative AI applications with our 12-lesson comprehensive course by Microsoft Cloud Advocates. Each lesson covers a key aspect of Generative AI principles and application development. Throughout this course, you will be building your own Generative AI startup so you can get an understanding of what it takes to launch your ideas. — Read More
Amid OpenAI Chaos, Rival Inflection AI Releases Model On GPT-4’s Heels
Inflection AI, the startup behind the conversational chatbot Pi, has unveiled a new AI model that the company claims can outperform two popular alternatives developed by Google and Meta — and is hot on the heels of OpenAI’s larger, flagship model GPT-4.
Called Inflection-2, the model performed better than Google’s PaLM Large 2 model previously announced in May on a number of standard benchmarks, Inflection said, while beating the open-source LLaMA 2 model largely developed by Meta on different measures. Overall, Inflection’s model is the top-performing of its size, the startup said. It only trails GPT-4, the flagship released model from OpenAI, thought to be significantly larger. — Read More
Putin to boost AI work in Russia to fight a Western monopoly he says is ‘unacceptable and dangerous’
Russian President Vladimir Putin on Friday announced a plan to endorse a national strategy for the development of artificial intelligence, emphasizing that it’s essential to prevent a Western monopoly.
Speaking at an AI conference in Moscow, Putin noted that “it’s imperative to use Russian solutions in the field of creating reliable and transparent artificial intelligence systems that are also safe for humans.”
“Monopolistic dominance of such foreign technology in Russia is unacceptable, dangerous and inadmissible,” Putin said. Read More
Oopsies: How OpenAI pulled off Sam Altman’s return? ‘72 intense hours of work,’ reveals interim CEO Shear
The Twitch co-founder was appointed as interim OpenAI CEO on November 20, three days after Altman’s ouster.
Twitch co-founder Emmett Shear, who joined OpenAI as interim CEO following Sam Altman’s ouster, on Wednesday revealed it took ‘72 very intense hours of work’ for the company to pull off Altman’s return, adding that he himself was ‘glad to have been a part of the solution.’ — Read More
OpenAI’s Misalignment and Microsoft’s Gain
I have, as you might expect, authored several versions of this Article, both in my head and on the page, as the most extraordinary weekend of my career has unfolded. To briefly summarize:
- On Friday, then-CEO Sam Altman was fired from OpenAI by the board that governs the non-profit; then-President Greg Brockman was removed from the board and subsequently resigned.
- Over the weekend rumors surged that Altman was negotiating his return, only for OpenAI to hire former Twitch CEO Emmett Shear as CEO.
- Finally, late Sunday night, Satya Nadella announced via tweet that Altman and Brockman, “together with colleagues”, would be joining Microsoft.
This is, quite obviously, a phenomenal outcome for Microsoft. The company already has a perpetual license to all OpenAI IP (short of artificial general intelligence), including source code and model weights; the question was whether it would have the talent to exploit that IP if OpenAI suffered the sort of talent drain that was threatened upon Altman and Brockman’s removal. Indeed they will, as a good portion of that talent seems likely to flow to Microsoft; you can make the case that Microsoft just acquired OpenAI for $0 and zero risk of an antitrust lawsuit. — Read More
ChatGPT Is Apparently a Great Surveillance Tool
This week, Forbes reported that a Russian spyware company called Social Links had begun using ChatGPT to conduct sentiment analysis. The creepy field by which cops and spies collect and analyze social media data to understand how web users feel about stuff, sentiment analysis is one of the sketchier use-cases for the little chatbot to yet emerge.
Social Links, which was previously kicked off Meta’s platforms for alleged surveillance of users, showed off its unconventional use of ChatGPT at a security conference in Paris this week. The company was able to weaponize the chatbot’s ability for text summarization and analysis to troll through large chunks of data, digesting it quickly. — Read More
AI Hallucinations
Sam Altman, Mira Murati, Emmett Shear — 3 CEOs in 3 Days
Amazon will host free ‘AI Ready’ courses in an effort to boost the AI talent pool
OpenAI may grab all the headlines, but Amazon has been quietly toiling on AI across all its divisions and even using AI-powered robots in its warehouses. Now, in a bid to expand the AI talent pool, the company is launching a free program called “AI Ready,” with the aim of providing generative AI training to two million people globally by 2025.
Consisting of eight free courses, the classes will be available through Amazon’s learning website and offered to non-Amazon employees as well. They’ll teach people AI skills including the generative AI technology that powers ChatGPT and other language models. — Read More
Microsoft hires former OpenAI CEO Sam Altman
Microsoft is hiring former OpenAI CEO Sam Altman and co-founder Greg Brockman.
Altman was fired from OpenAI on Friday, after the board said it “no longer has confidence in his ability to continue leading OpenAI.” After a weekend of negotiations to potentially bring Altman back to OpenAI, Microsoft CEO Satya Nadella announced that both Sam Altman and Greg Brockman will be joining to lead Microsoft’s new advanced AI research team. — Read More
Rick's Cafe AI 