IBM Extends Jupyter Notebooks for AI Development
via Shutterstock
The gateway is designed to ease the scaling of enterprise workloads. IBM said Elyra (pronounced, el-EYE-rah) would ease workload development. Elyra “aims to help data scientists, machine learning engineers and AI developers through the model development lifecycle complexities,” the company added in a blog post announcing the open source release.
The visual editor for building AI pipelines is designed to ease the conversion of multiple notebooks into simpler workflows. The user interface also allows the submission of single notebook as a batch job.
Among the Jupyter extensions is the ability to “decompose” different AI development tasks into different notebooks, allowing each to use different frameworks such as TensorFlow for deep learning models. The editor can then be used to build notebook-based AI pipelines.
Multiple notebooks can then be converted into batch jobs or workflows.
IBM recently added JupyterLab to its Cloud Pak and Watson Studio platforms.
The stable release in March of a related Kubernetes-based automation tool called Kubeflow includes a Jupyter Notebooks controller. Elyra currently supports the Kubeflow Pipelines runtime.
Along with Kubeflow and Kubeflow Pipelines, IBM said it is also contributing to PyTorch, Spark, TensorFlow and other machine learning model development tools.
Jupyter users welcomed the open source notebook extensions. “As much as AI tools rely on vast amounts of data and computational resources, the human in the loop remains the critical element for both asking the right questions and making decisions responsibly,” said Fernando Pérez, co-founder and co-director of Project Jupyter.
This article originally appeared in sister publication Datanami.
