Google launches AI Hub in alpha and Kubeflow Pipelines, a machine learning workflow

Google Cloud today announced the launch of Kubeflow Pipelines to foster collaboration within businesses and further democratize access to artificial intelligence. Kubeflow Pipelines is available for free and is being open-sourced.

Google Cloud senior director of product management Rajen Sheth said he agrees with estimates that there are only a few thousand machine learning engineers in the world with the ability to take deep learning from concept to production, but there are millions of data scientists and tens of millions of developers.

Kubeflow Pipelines was designed to deal with that gap, empowering more data scientists and developers and helping businesses overcome the obstacles to becoming AI-first companies.

“One of the biggest problems we’re seeing right now is companies are now trying to build up teams of data scientists, but it’s such a scarce resource that unless that’s utilized well, it starts to get wasted,” Sheth said. “One observation we’ve seen is that in probably over 60 percent of cases, models are never deployed to production right now. So we’re building a number of things to hopefully help cure that.”

Pipelines is a composable layer, so different parts of the machine learning journey can be snapped together like Legos, Sheth said.

This approach allows different members of a team to do things like label data, convert that data into features, and validate data. It can also come in handy for testing several iterations and replacing a model or approach if a better one is found.

“They can just swap in the new model, keep the rest of the pipeline in place, and then see: ‘Does that new model help the output significantly?’ So it enables … rapid experimentation in a much better way,” he said. “What we are doing with Pipelines, it can start to involve developers, it can start to involve business analysts, it can start to involve end users such that they can become part of this team that can build a Pipeline.”

Kubeflow is an open source project from Google released earlier this year for machine learning with Kubernetes containers. Using Kubernetes will allow businesses to be flexible and avoid having to commit entirely to training AI models with on-premise data and frameworks or training models in the cloud.

Kubeflow Pipelines is partly based on and utilizes libraries from TensorFlow Extended (TFX), which was used internally at Google to build machine learning components and then allow developers on various internal teams to utilize that work and put it into production.

Also launching today in alpha is AI Hub, which builds on top of machine learning module TensorFlow Hub, made available earlier this year. AI Hub is designed to be a one-stop shop for people interested in training or deploying AI models.

In addition to providing training, AI Hub will be populated with resources from Google, such as popular TensorFlow embeddings and content from Kaggle, a community of more than 2 million data scientists.

In time, Google wants AI Hub to become a place for popular models generated by the larger ecosystem.

“We eventually want AI Hub to be a place where third parties can also share information and turn it more into a marketplace over time,” Sheth said. “What we’re finding is that community could actually solve the problems of many of our customers.”

AI Hub will initially be made available to roughly 100 business partners.

Like Kubeflow Pipelines, AI Hub also aims to educate workforces to tear down barriers between teams within companies so they can make the work of developers, data scientists, and ML engineers more valuable.

AI literacy is a concept discussed last month at VB Summit with senior executives from Google and Google Cloud, among others.

“I think  the real big challenge is that in order to become AI first, everybody needs to have literacy in AI, and that’s everything from a product manager thinking about the problem through to a developer through to a data scientist through to the production teams. And once you have that, you can start to incorporate AI into almost any business problem, and that’s where we are now,” Sheth said.

“Almost every product at Google is now using AI in interesting ways, and we’re realizing it can solve more and more problems for us, and we’re hoping that this can then help foster that culture within other companies, too.”

Source: VentureBeat
To Read Our Daily News Updates, Please Visit Inventiva Or Subscribe Our Newsletter & Push.