I am trying to stitch together multiple packages and tools from multiple languages (R, python, C etc.) in a single analysis workflow.
Is there any standard way to do it? Preferably (but not necessarily) in python.
Topic methods
Category Data Science
Luigi is an open source python package by Spotify that does exactly what you described:
Luigi is a Python (2.7, 3.3, 3.4, 3.5) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.
Its philosophy is similar to GNU Make, letting you define tasks and their dependencies.
There is also Apache Airflow (originally from Airbnb), another python solution:
Airflow is a platform to programmatically author, schedule and monitor workflows. Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies.
You may found a complete comparative table here.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.