The key to managing a mountain of data and disruptive technologies may lie in establishing a center of competency.
Although many organizations are using artificial intelligence (AI) and machine language (ML) tools as core enablers in their data analytics projects, and AI spending worldwide continues to rise, the hard truth is that most data science projects are doomed to fail.
There are several reasons for these failures, ranging from the inherent complexity of AI/ML initiatives and the persistent lack of skilled talent to challenges that exist in data security, governance, and data integration. These issues are collectively referred to as concerns for” data readiness,” according to an IDC global survey of more than 2,000 IT and line-of-business decision-makers, all of whom are involved in some level of AI use or development. Read More
Tag Archives: Data Science
Never invest your time in learning complex things.
The data scientist hype train has come to a grinding halt . It has been a joy ride for me for I was one of the people who got hooked into data science as soon as it came out. Math, engineering and the ability to predict stuff was very attractive indeed for a self-professed geek . I couldn’t resist and soon I was devouring one book after the other. I started with Springer Publications (Max Kuhn) , Tevor Hastie, a lot of Orielly books and followed it up with Statistics and Math courses until I had the math and the techniques (Linear/Logistic Regression, SVM,Random Forests, Decision Trees and few 20 others) down pat. Sounds great right, not quite.
Then came the Deep Learning revolution. I was first exposed to it thanks to Jeremy Howard who in my opinion still runs the best damn Deep learning course on the internet. He explains vision, NLP and even structured data machine learning. The guy is literally able to translate gobbledygook for the masses ( Me :-)) Plug: https://www.fast.ai/ . Read More
How to Become Data Scientist – A Complete Roadmap
Markov models and Markov chains explained in real life: probabilistic workout routine
Markov defined a way to represent real-world stochastic systems and processes that encode dependencies and reach a steady-state over time.
Andrei Markov didn’t agree with Pavel Nebrasov, when he said independence between variables was necessary for the Weak Law of Large Numbers to be applied.
The Weak Law of Large Numbers states something like this:
When you collect independent samples, as the number of samples gets bigger, the mean of those samples converges to the true mean of the population.
But Markov believed independence was not a necessary condition for the mean to converge. So he set out to define how the average of the outcomes from a process involving dependent random variables could converge over time. Read More
The Role of Surrogate Models in the Development of Digital Twins of Dynamic Systems
Digital twin technology has significant promise, relevance and potential of widespread applicability in various industrial sectors such as aerospace, infrastructure and automotive. However, the adoption of this technology has been slower due to the lack of clarity for specific applications. A discrete damped dynamic system is used in this paper to explore the concept of a digital twin. As digital twins are also expected to exploit data and computational methods, there is a compelling case for the use of surrogate models in this context. Motivated by this synergy, we have explored the possibility of using surrogate models within the digital twin technology. In particular, the use of Gaussian process (GP) emulator within the digital twin technology is explored. GP has the inherent capability of addressing noisy and sparse data and hence, makes a compelling case to be used within the digital twin framework. Cases involving stiffness variation and mass variation are considered, individually and jointly along with different levels of noise and sparsity in data. Our numerical simulation results clearly demonstrate that surrogate models such as GP emulators have the potential to be an effective tool for the development of digital twins. Aspects related to data quality and sampling rate are analysed. Key concepts introduced in this paper are summarised and ideas for urgent future research needs are proposed. Read More
AutoML will not replace your data science profession
Many people who are already data scientists or new to the field of data science are looking at an answer to the question “Will AutoML (Automated Machine Learning) replace data scientists?” Asking a question like this is very reasonable because Automation has already been introduced to Machine Learning and it plays a key role in the modern world. In addition to that, people who want to become data scientists are thinking about ways to secure a spot in the job market for a long period of time.
AutoML will NOT replace your data science profession. It’s just here to make things easier for you, such as assisting you in boring repetitive tasks, saving your valuable time, assisting you in code maintenance and consistency, etc!
Let’s walk through the steps of a machine learning process to find out why. Read More
A data science book for kids!
The team at Domino Data Lab has cracked open their crayon box to write and illustrate the world’s first-ever children’s book about data science. Introduce your kids to Florence the Data Scientist and Her Magical Bookmobile! Read More
9 Comprehensive Cheat Sheets For Data Science
Sometimes we need a short and to-the-point resource.
In this age of technology, if you ever need to find information about any topic — tech-related or not — you can head to Google, and you will find thousands of materials, articles, books, and videos about that topic. Although this easy access to information had allowed many people worldwide to learn new skills, start a new career and explore topics from the comfort of their home, sometimes the massive amount of information can be overwhelming.
When you look for something and end up with so much information, it can get frustrating and confusing because you don’t know where to start, and at the beginning, it is difficult to see the big picture. Situations like this have lead to the appearance of cheat sheets.
Cheat sheets are an amazing resource for shortcut information about a certain topic. Often, cheat sheets are useful in many ways, but mainly initially, so you can grasp the main concepts and build stones of the topic you’re searching for. In case you want to refresh your memory and go through a straightforward reminder of the topic’s basics. Read More
A Chat with Andrew on MLOps: From Model-centric to Data-centric AI
A ‘Glut’ of Innovation Spotted in Data Science and ML Platforms
These are heady days in data science and machine learning (DSML) according to Gartner, which identified a “glut” of innovation occurring in the market for DSML platforms. From established companies chasing AutoML or model governance to startups focusing on MLops or explainable AI, a plethora of vendors are simultaneously moving in all directions with their products as they seek to differentiate themselves amid a very diverse audience.
“The DSML market is simultaneously more vibrant and messier than ever,” a gaggle of Gartner analysts led by Peter Krensky wrote in the Magic Quadrant for DSML Platforms, which was published earlier this month. “The definitions and parameters of data science and data scientists continue to evolve, and the market is dramatically different from how it was in 2014, when we published the first Magic Quadrant on it.”
The 2021 Magic Quadrant for DSML is heavily represented by companies to the right of the axis, which anybody who’s familiar with Gartner’s quadrant-based assessment method knows represents the “completeness of vision.” No fewer than 13 of the 20 vendors to make the quadrant’s cut landed on the right side, which indicates active innovation. Read More
