A 39-year-old entrepreneur named Rocky Yu founded AGI House, the baller hacker crib for young AI entrepreneurs, in a $68 million mansion in Hillsborough, California. Talking shop and working on their AI startups is the highlight instead of chillin’ in the massive mansion. — Read More
Daily Archives: October 2, 2023
Wanted: High Performers for the Last Job You’ll Ever Have
Legend has it that in the winter of 1913, explorer Ernest Shackleton put out an ad for sailors to join him on an expedition to Antarctica:
Men wanted for hazardous journey. Low wages, bitter cold, long hours of complete darkness. Safe return doubtful. Honour and recognition in event of success.
Today, in September of 2023, I’d like to propose a similarly adventurous job ad for the age of AI:
Talented engineers, designers, and copywriters wanted for a new agency. All work will be recorded, labeled, and organized for AI training. Your role will be progressively phased out. Salary paid and profits shared to you indefinitely. Failure likely. In the event of success: the last job you’ll ever have—or need. — Read More
The synthetic social network is coming
Today, let’s consider the implications of a truly profound week in the development of artificial intelligence and discuss whether we may be witnessing the rise of a new era in the consumer internet.
… You can imagine the next steps here. A bot that gets to know your quirks; remembers your life history; offers you coaching or tutoring or therapy; entertains you in whichever way you prefer. A synthetic companion not unlike the real people you encounter during the day, only smarter, more patient, more empathetic, more available. — Read More
Effective Long-Context Scaling of Foundation Models
We present a series of long-context LLMs that support effective context windows of up to 32,768 tokens. Our model series are built through continual pretraining from LLAMA 2 with longer training sequences and on a dataset where long texts are upsampled. We perform extensive evaluation on language modeling, synthetic context probing tasks, and a wide range of research benchmarks. On research benchmarks, our models achieve consistent improvements on most regular tasks and significant improvements on long-context tasks over LLAMA 2. Notably, with a cost-effective instruction tuning procedure that does not require human-annotated long instruction data, the 70B variant can already surpass gpt-3.5-turbo-16k’s overall performance on a suite of long-context tasks. Alongside these results, we provide an in-depth analysis on the individual components of our method. We delve into LLAMA’s position encodings and discuss its limitation in modeling long dependencies. We also examine the impact of various design choices in the pretraining process, including the data mix and the training curriculum of sequence lengths – our ablation experiments suggest that having abundant long texts in the pretrain dataset is not the key to achieving strong performance, and we empirically verify that long context continual pretraining is more efficient and similarly effective compared to pretraining from scratch with long sequences. — Read More
#nlpThe state of AI in 2023: Generative AI’s breakout year
The latest annual McKinsey Global Survey on the current state of AI confirms the explosive growth of generative AI (gen AI) tools. Less than a year after many of these tools debuted, one-third of our survey respondents say their organizations are using gen AI regularly in at least one business function. Amid recent advances, AI has risen from a topic relegated to tech employees to a focus of company leaders: nearly one-quarter of surveyed C-suite executives say they are personally using gen AI tools for work, and more than one-quarter of respondents from companies using AI say gen AI is already on their boards’ agendas. What’s more, 40 percent of respondents say their organizations will increase their investment in AI overall because of advances in gen AI. The findings show that these are still early days for managing gen AI–related risks, with less than half of respondents saying their organizations are mitigating even the risk they consider most relevant: inaccuracy. — Read More
Rick's Cafe AI 