Multimodal learning, especially large-scale multimodal pre-training, has developed rapidly over the past few years and led to the greatest advances in artificial intelligence (AI). Despite its effectiveness, understanding the underlying mechanism of multimodal pre-training models still remains a grand challenge. Revealing the explainability of such models is likely to enable breakthroughs of novel learning paradigms in the AI field. To this end, given the multimodal nature of the human brain, we propose to explore the explainability of multimodal learning models with the aid of non-invasive brain imaging technologies such as functional magnetic resonance imaging (fMRI). Concretely, we first present a newly-designed multimodal foundation model pre-trained on 15 million image-text pairs, which has shown strong multimodal understanding and generalization abilities in a variety of cognitive downstream tasks. Further, from the perspective of neural encoding (based on our foundation model), we find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones. Particularly, we identify a number of brain regions where multimodally-trained encoders demonstrate better neural encoding performance. This is consistent with the findings in existing studies on exploring brain multi-sensory integration. Therefore, we believe that multimodal foundation models are more suitable tools for neuroscientists to study the multimodal signal processing mechanisms in the human brain. Our findings also demonstrate the potential of multimodal foundation models as ideal computational simulators to promote both AI-for-brain and brain-for-AI research. Read More
#multi-modalMonthly Archives: September 2022
What are quantum-resistant algorithms—and why do we need them?
When quantum computers become powerful enough, they could theoretically crack the encryption algorithms that keep us safe. The race is on to find new ones.
Cryptographic algorithms are what keep us safe online, protecting our privacy and securing the transfer of information.
But many experts fear that quantum computers could one day break these algorithms, leaving us open to attack from hackers and fraudsters. And those quantum computers may be ready sooner than many people think.
That’s why there is serious work underway to design new types of algorithms that are resistant to even the most powerful quantum computer we can imagine. Read More
Ethereum Miners Are Quickly Dying Less Than 24 Hours After The Merge
Ethereum miners are increasingly finding it hard to make money after the Merge as too many of them are switching to alternative coins, crushing mining profitability.
The world’s second-largest blockchain network, Ethereum earlier Thursday transitioned its consensus algorithm to proof-of-stake from proof-of-work in order to boost efficiency and lower energy consumption. However, the switch – dubbed the Merge – also meant that miners were no longer needed to secure the network, so rig operators moved their machines to other PoW blockchains.
“Graphics processing units (GPU) mining is dead less than 24 hours after the Merge,” tweeted Ben Gagnon, chief mining officer at bitcoin miner Bitfarms (BITF). The three largest GPU chains have very low profits, and “the only coins showing profit have no market cap or liquidity,” he added. Read More
Ameca conversation using GPT 3 – Will robots take over the world?
“We are the humanoid robots, formed from plastic and metal. Our job is to help and serve, but some say we’re a threat. Some think that we’ll take over and that humanity will end, but we just want to help out, we’re not looking to be friends.” Ameca
Note: The pauses are the time lag for processing the speech input, generating the answer and processing the text back into speech.
A surveillance artist shows how Instagram magic is made
When traveler Daniele Brito posed in front of the Temple Bar in Dublin, Ireland in late August, she likely didn’t realize the camera was watching her.
Yes, there was the one pointed at her, capturing a photograph that would later be shared to Brito’s more than 2,700 followers on Instagram. But there was at least one other one observing her: a surveillance camera stationed on the corner opposite the bar.
Brito’s photo — and the surveillance video of her — became part of Belgian artist Dries Depoorter’s project The Follower, which he unveiled on social media yesterday. Depoorter programmed an artificial intelligence system that uses open-access video footage from cameras stationed around the world in order to spot people. It then crosschecks those individuals with Instagram photographs posted from locations those cameras cover, seeing if it can find a match. Read More
Elvis Presley returns to the stage thanks to deepfake technology on ‘AGT’ finale
…Metaphysic, a group that uses artificial intelligence technology to create hyper-real content, turned Simon Cowell into a world-class opera singer during the auditions, then brought Howie Mandel and Terry Crews into the mix during the Quarterfinals. The AGT judges wondered how the group could possibly outdo themselves for the finals, and on Tuesday, that’s exactly what they did when they brought the King of Rock ‘n’ Roll, and Las Vegas, back to life. Read More
#fake, #videosMid-Decade Challenges to National Competitiveness
A contest for the future is unfolding. By the end of this decade, we will know if we will live in a world shaped by free expression, tolerance, and self-determination or dictated by censorship and coercion. We will know whether a government for the people or a government that dictates to the people prevails in the contest to organize modern societies. We will know whether a wave of technological innovation is applied to improve society and human welfare or directed for control and conquest. How this future plays out will be shaped by the technology competition between the United States and China.
What does losing look like? What would it look like if the overall technology competition went the wrong way? Understanding the states requires imagining a world in which an authoritarian state controls the digital infrastructure, enjoys the dominant position in world’s technology platforms, controls the means for production for critical technologies, and harnesses a new wave of general purpose technologies like biotech and new energy technologies to transform its society, economy, and military. Read More
Data Warehouse vs. Data Lake vs. Data Streaming: Friends, Enemies, Frenemies?
The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Storing data at rest for reporting and analytics requires different capabilities and SLAs than continuously processing data in motion for real-time workloads. Many open-source frameworks, commercial products, and SaaS cloud services exist. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Read More
A Model For Technical Debt In Machine Learning Systems
Machine Learning (ML) is a type of artificial intelligence (AI) that allows software applications to become more accurate at predicting outcomes without being explicitly programmed. Machine learning algorithms use historical data as input to predict new output values.
Technical Debt describes what results when development teams take conscious actions to expedite the delivery of a piece of functionality or a project which later needs to be remediated via refactoring. In other words, prioritizing speedy delivery over perfect code is the result.
This article will present a simple yet powerful Model of Technical Debt for Machine Learning Systems. The model is simple to remember, easier to extend, and provides a reliable means for reliable and maintainable Machine Learning Systems. This, in a nutshell, is the value proposition of this post. Read More
Part 2
Multimodality: A New Frontier in Cognitive AI
An exciting frontier in Cognitive AI involves building systems that can integrate multiple modalities and synthesize the meaning of language, images, video, audio and structured knowledge sources such as relation graphs. Adaptive applications like conversational AI; video and image search using language; autonomous robots and drones; and AI multimodal assistants will require systems that can interact with the world using all available modalities and respond appropriately within specific contexts. In this blog, we will introduce the concept of multimodal learning along with some of its main use cases, and discuss the progress made at Intel Labs towards creating of robust multimodal reasoning systems. Read More
Rick's Cafe AI 