Data Scientists, Not AutoML, Driving Model Development
source: Shutterstock
As machine learning technology matures and moves into the mainstream, analysts are attempting to figure out who within organizations is building machine learning models used in production and what tools and methodologies they are using.
It turns out that those furthest down the learning curve have well established data science teams building machine learning models for production. Meanwhile, much ballyhooed AutoML services from Google and other cloud vendors are so far gaining little traction among enterprises, according to a recent survey of the state of machine learning adoption by O’Reilly Media.
About half of survey respondents said their data science teams build machine learning models, indicating that those companies were further along in development and have adopted processes used for software development to build machine learning models. About three-quarters characterized their models as “sophisticated” while two-thirds described their data science teams as “early adopters” of machine learning technology.
Traditional product development teams accounted for about one-quarter of machine learning model builders while just 3 percent of respondents said they were using cloud machine learning services like AutoML or other cloud APIs. “Not many respondents belonged to organizations that rely on AutoML services offered by cloud providers,” the O’Reilly survey noted.
Part of the reason is cultural. For example, companies with extensive experience in building machine learning models have built up data science teams that evaluate models for bias, “fairness” and privacy protection. For example, 43 percent of respondents said they account for “privacy-by-design” rules established under the EUs General Data Protection Regulation.
That, the researchers note, “means more companies will add privacy to their machine learning checklist,” adding that new data privacy regulations “coincide with the rise of tools and methods for privacy-preserving analytics and machine learning.”
O’Reilly said about half of the more than 11,400 respondents to its machine learning adoption survey were based in North America. While North American companies accounted for the most advanced stages of machine learning development (18 percent characterized their development as “sophisticated”), 42 percent of Chinese and other “South Asia” companies consider themselves “early adopters” of machine learning technology.
Taken together, data science teams that have built production machine learning models have also blazed a trial for neophytes, providing not only a set of best practices and tools but also metrics for evaluating the success of machine learning models in deployment, the researchers noted.
