GalaxyCloudRunner: job bursting for Galaxy¶
GalaxyCloudRunner enables bursting of user jobs to remote compute resources for the Galaxy application. It provides several dynamic job rules that can be plugged into Galaxy, enabling Galaxy to submit jobs to remote compute nodes.
How it works¶
The GalaxyCloudRunner provides a library of rules that can be plugged into
Galaxy through its configuration via
job_conf.xml. Once configured, Galaxy
jobs can be automatically routed to a Galaxy remote job runner, called
Pulsar, on nodes running on the cloud. Adding a new node is a simple matter
of visiting the CloudLaunch site and launching a new worker node on your
desired cloud. The GalaxyCloudRunner will discover what Pulsar nodes are
available by querying the CloudLaunch API.
Getting started with the GalaxyCloudRunner is a simple process.
- First, install
galaxycloudrunnerlibrary into your Galaxy’s virtual environment
- Configure Galaxy to use GalaxyCloudRunner job destination rules
- Launch as many worker nodes as you need through CloudLaunch
- Submit jobs as usual
- Configuring Galaxy
- Obtaining a CloudLaunch API key
- Adding new Pulsar nodes
- Job configuration for Galaxy 19.01 or higher
- Job configuration for Galaxy versions lower than 19.01
- Additional Configuration and Limitations