Usage documentation¶
This page describes some sample use cases for CloudMan API and provides
examples for these API calls.
In addition to this page, there are functional examples of complete scripts in
docs/examples
directory of the BioBlend source code repository.
Setting up custom cloud properties¶
CloudMan supports Amazon, OpenStack, OpenNebula, and Eucalyptus based clouds and
BioBlend can be used to programatically manipulate CloudMan on any of those
clouds. Once launched, the API calls to CloudMan are the same irrespective of
the cloud. In order to launch an instance on a given cloud, cloud properties
need to be provided to CloudManLauncher
. If cloud properties are not specified,
CloudManLauncher
will default to Amazon cloud properties.
If we want to use a different cloud provider, we need to specify additional cloud
properties when creating an instance of the CloudManLauncher
class. For
example, if we wanted to create a connection to NeCTAR, Australia’s national
research cloud, we would use the following properties:
from bioblend.util import Bunch
nectar = Bunch(
name='NeCTAR',
cloud_type='openstack',
bucket_default='cloudman-os',
region_name='NeCTAR',
region_endpoint='nova.rc.nectar.org.au',
ec2_port=8773,
ec2_conn_path='/services/Cloud',
cidr_range='115.146.92.0/22',
is_secure=True,
s3_host='swift.rc.nectar.org.au',
s3_port=8888,
s3_conn_path='/')
Note
These properties are cloud-specific and need to be obtained from a given cloud provider.
Launching a new cluster instance¶
In order to launch a CloudMan cluster on a chosen cloud, we do the following (continuing from the previous example):
from bioblend.cloudman import CloudManConfig
from bioblend.cloudman import CloudManInstance
cmc = CloudManConfig('<your AWS access key', 'your AWS secret key', 'Cluster name',
'ami-<ID>', 'm1.medium', 'choose_a_password_here', nectar)
cmi = CloudManInstance.launch_instance(cmc)
Note
If you already have an existing instance of CloudMan, just create an
instance of the CloudManInstance
object directly by calling its
constructor and connecting to it (the password you provide must match
the password you provided as part of user data when launching this
instance). For example:
cmi = CloudManInstance('http://115.146.92.174', 'your_UD_password')
We now have a CloudManInstance
object that allows us to manage created
CloudMan instance via the API. Once launched, it will take a few minutes for the
instance to boot and CloudMan start. To check on the status of the machine,
(repeatedly) run the following command:
>>> cmi.get_machine_status()
{'error': '',
'instance_state': 'pending',
'placement': '',
'public_ip': ''}
>>> cmi.get_machine_status()
{'error': '',
'instance_state': 'running',
'placement': 'melbourne-qh2',
'public_ip': '115.146.86.29'}
Once the instance is ready, although it may still take a few moments for CloudMan to start, it is possible to start interacting with the application.
Note
The CloudManInstance
object (e.g., cmi
) is a local representation
of the actual CloudMan instance. As a result, the local object can get
out of sync with the remote instance. To update the state of the local
object, call the update
method on the cmi
object:
>>> cmi.update()
Manipulating an existing cluster¶
Having a reference to a CloudManInstance
object, we can manage it via the
available CloudManInstance API:
>>> cmi.initialized
False
>>> cmi.initialize('SGE')
>>> cmi.get_status()
{'all_fs': [],
'app_status': 'yellow',
'autoscaling': {'as_max': 'N/A',
'as_min': 'N/A',
'use_autoscaling': False},
'cluster_status': 'STARTING',
'data_status': 'green',
'disk_usage': {'pct': '0%', 'total': '0', 'used': '0'},
'dns': '#',
'instance_status': {'available': '0', 'idle': '0', 'requested': '0'},
'snapshot': {'progress': 'None', 'status': 'None'}}
>>> cmi.get_cluster_size()
1
>>> cmi.get_nodes()
[{'id': 'i-00006016',
'instance_type': 'm1.medium',
'ld': '0.0 0.025 0.065',
'public_ip': '115.146.86.29',
'time_in_state': '2268'}]
>>> cmi.add_nodes(2)
{'all_fs': [],
'app_status': 'green',
'autoscaling': {'as_max': 'N/A',
'as_min': 'N/A',
'use_autoscaling': False},
'cluster_status': 'READY',
'data_status': 'green',
'disk_usage': {'pct': '0%', 'total': '0', 'used': '0'},
'dns': '#',
'instance_status': {'available': '0', 'idle': '0', 'requested': '2'},
'snapshot': {'progress': 'None', 'status': 'None'}}
>>> cmi.get_cluster_size()
3