From development to production in a snap with Ploomber

Interactive development tools such as Jupyter are prevalent among data scientists because they provide an environment to perform data transformations and get immediate visual feedback. However, when deploying a project, it must be refactored using a production-friendly environment like Airflow or Argo; this causes data scientists to move code back and forth between their notebooks and these production tools. Furthermore, data scientists have to spend time learning an unfamiliar framework and writing pipeline code, which severely delays the deployment process.

Inspired by Ruby on Rails convention over configuration philosophy, Ploomber solves this problem by providing:

1. A workflow orchestrator that automatically infers task execution order using static analysis.
2. A sensible layout to bootstrap projects.
3. A development environment integrated with Jupyter.
4. Capabilities to export to production systems (Airflow and Argo) without code changes.

This talk introduces Ploomber, an open-source workflow orchestrator to seamlessly develop and deploy data products. Ploomber is already used in production at a few companies and has been in development for over a year.