Evolving Workflow Orchestration

Evolving Workflow Orchestration // MLOps Podcast #291 with Alex Milowski, Entrepreneur and Computer Scientist.

// Abstract
There seems to be a shift from workflow languages to code - mostly annotation pythons - happening and getting us. It is a symptom of how complex workflow orchestration has gotten. Is it a dominant trend or will we cycle back to "DAG specifications"? At Stitchfix, we had our own DSL that "compiled" into airflow DAGs and at MicroByre, we used a external workflow langauge. Both had a batch task executor on K8s but at MicroByre, we had human and robot in the loop workflows.

// Bio
Dr. Milowski is a serial entrepreneur and computer scientist with experience in a variety of data and machine learning technologies. He holds a PhD in Informatics (Computer Science) from the University of Edinburgh, where he researched large-scale computation over scientific data. Over the years, he's spent many years working on various aspects of workflow orchestration in industry, standardization, and in research.

// MLOps Swag/Merch
shop.mlops.community/

// Related Links
Website: www.milowski.com/

-------------- ✌️Connect With Us ✌️ ------------
Join our slack community: go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: mlops.community/

Connect with Demetrios on LinkedIn: www.linkedin.com/in/dpbrinkm/
Connect with Alex on LinkedIn: www.linkedin.com/in/alexmilowski/