There has been great progress towards adapting large language models (LLMs) to accommodate multimodal inputs for tasks including image captioning, visual question answering (VQA), and open vocabulary recognition. Despite such achievements, current state-of-the-art visual language models (VLMs) perform inadequately on visual information seeking datasets, such as Infoseek and OK-VQA, where external knowledge is required to answer the questions. — Read More
Tag Archives: Frameworks
Introducing Google’s Secure AI Framework
The potential of AI, especially generative AI, is immense. However, in the pursuit of progress within these new frontiers of innovation, there needs to be clear industry security standards for building and deploying this technology in a responsible manner. That’s why today we are excited to introduce the Secure AI Framework (SAIF), a conceptual framework for secure AI systems.
SAIF is inspired by the security best practices — like reviewing, testing and controlling the supply chain — that we’ve applied to software development, while incorporating our understanding of security mega-trends and risks specific to AI systems. — Read More
Causal AI — Enabling Data-Driven Decisions
Understand how Causal AI frameworks and algorithms support decision making tasks like estimating the impact of interventions, counterfactual reasoning and repurposing previously gained knowledge on other domains.
AI and Machine Learning solutions have made rapid strides in the last decade and they are being increasingly relied upon to generate predictions based on historical data. However they fall short of expectations when it comes to augmenting human decisions on tasks where there is a need to understand the actual causes behind an outcome, quantifying the impact of different interventions on final outcomes and making policy decisions, perform what if analysis and reasoning for scenarios which have not occurred etc.
…While generation of model predictions and explaining key features influencing the outcomes is helpful, it does not allow making decisions.
To facilitate decision regarding the right interventions needed to reduce attrition, we need answers to below questions :
- What is the impact on final outcomes if the firm decides to make an intervention and organize regular quarterly training for its staff?
- How can we compare the impact of different competing interventions, say organizing quarterly trainings with that of arranging regular senior leadership connect?
Read More #frameworks
Avalanche: an End-to-End Library for Continual Learning
Avalanche is an End-to-End Continual Learning Library based on PyTorch, born within ContinualAI with the unique goal of providing a shared and collaborative open-source (MIT licensed) codebase for fast prototyping, training and reproducible evaluation of continual learning algorithms.
Avalanche can help Continual Learning researchers and practitioners in several ways:
- Write less code, prototype faster & reduce errors
- Improve reproducibility
- Improve modularity and reusability
- Increase code efficiency, scalability & portability
- Augment impact and usability of your research products
Facebook details self-supervised AI that can segment images and videos
Facebook today announced that it developed an algorithm in collaboration with Inria called DINO that enables the training of transformers, a type of machine learning model, without labeled training data. The company claims it sets a new state-of-the-art among unlabeled data training methods and leads to a model that can discover and segment objects in an image or video without a specific objective.
Segmenting objects is used in tasks ranging from swapping out the background of a video chat to teaching robots that navigate through a factory. But it’s considered among the hardest challenges in computer vision because it requires an AI to understand what’s in an image. Read More
PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models
We present PyTorch Geometric Temporal a deep learning frame-work combining state-of-the-art machine learning algorithms for neural spatiotemporal signal processing. The main goal of the library is to make temporal geometric deep learning available for researchers and machine learning practitioners in a unified easy-to-use framework. PyTorch Geometric Temporal was created with foundations on existing libraries in the PyTorch ecosystem, stream-lined neural network layer definitions, temporal snapshot generators for batching, and integrated benchmark datasets. These features are illustrated with a tutorial-like case study. Experiments demonstrate the predictive performance of the models implemented in the library on real world problems such as epidemiological forecasting, ridehail demand prediction and web-traffic management.Our sensitivity analysis of runtime shows that the framework can potentially operate on web-scale datasets with rich temporal features and spatial structure. Read More
Microsoft details the latest developments in machine learning at GTC 21
With the rapid pace of change taking place in AI and machine learning technology, it’s no surprise Microsoft had its usual strong presence at this year’s Nvidia GTC event.
Representatives of the company shared their latest machine learning innovations in multiple sessions, covering inferencing at scale, a new capability to train machine learning models across hybrid environments, and the debut of the new PyTorch Profiler that will help data scientists be more efficient when they’re analyzing and troubleshooting ML performance issues.
In all three cases, Microsoft has paired its own technologies, like Azure, with open source tools and NVIDIA’s GPU hardware and technologies to create these powerful new innovations. Read More
Google’s Model Search automatically optimizes and identifies AI models
Google today announced the release of Model Search, an open source platform designed to help researchers develop machine learning models efficiently and automatically. Instead of focusing on a specific domain, Google says that Model Search is domain-agnostic, making it capable of finding a model architecture that fits a dataset and problem while minimizing coding time and compute resources. Read More
Build Your First Image Classifier With Convolutional Neural Network (CNN)
A Beginners Guide to CNN with TensorFlow
Convolutional Neural Network (CNN) is a type of deep neural network primarily used in image classification and computer vision applications. This article will guide you through creating your own image classification model by implementing CNN using the TensorFlow package in Python. Read More
CrypTen – A Research Tool for Secure and Privacy – Preserving Machine Learning in Pytorch
Facebook’s Pytorch had created a huge buzz in the market when it was released five years ago. Now, it is not only the most preferred frameworks for Machine Learning and Deep Learning models but also one of the most powerful tools in research to develop new libraries and frameworks(like Huggingface, Fast.ai, etc). One of the most captivating libraries released by Facebook’s AI Research Lab(FAIR) is CrypTen – a tool for secure computation in ML. CrypTen is an open-source Python framework, built on Pytorch, to provide secure and privacy-preserving machine learning.
Crypten serves Secure Multiparty Computation as its secured computing backend and lessens the gap between ML researchers/developers and cryptography by facilitating Pytorch API’s to perform encryption techniques. Read More
Rick's Cafe AI 