backend_clients Package

backend_client

class tvb.core.services.backend_clients.backend_client.BackendClient

Bases: builtins.object

Interface for a backend client that runs operations asynchronously on a specific environment

static execute(operation_id, user_name_label, adapter_instance)

Start operation asynchronously

static stop_operation(operation_id)

Stop the thread for a given operation id

cluster_scheduler_client

class tvb.core.services.backend_clients.cluster_scheduler_client.ClusterSchedulerClient

Bases: tvb.core.services.backend_clients.backend_client.BackendClient

Simple class, to mimic the same behavior we are expecting from StandAloneClient, but firing behind the cluster job scheduling process..

static execute(operation_id, user_name_label, adapter_instance)

Call the correct system command to submit a job to the cluster.

static stop_operation(operation_id)

Stop a thread for a given operation id

hpc_scheduler_client

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCJobStatus

Bases: enum.Enum

An enumeration.

FAILED = <HPCJobStatus.FAILED: 'FAILED'>

QUEUED = <HPCJobStatus.QUEUED: 'QUEUED'>

READY = <HPCJobStatus.READY: 'READY'>

STAGINGIN = <HPCJobStatus.STAGINGIN: 'STAGINGIN'>

STAGINGOUT = <HPCJobStatus.STAGINGOUT: 'STAGINGOUT'>

SUCCESSFUL = <HPCJobStatus.SUCCESSFUL: 'SUCCESSFUL'>

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCOperationThread(operation_id, *args, **kwargs)

Bases: threading.Thread

stop()

stopped()

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCSchedulerClient

Bases: tvb.core.services.backend_clients.backend_client.BackendClient

Simple class, to mimic the same behavior we are expecting from StandAloneClient, but firing the operation on an HPC node. Define TVB_BIN_ENV_KEY and CSCS_LOGIN_TOKEN_ENV_KEY as environment variables before running on HPC.

CONTAINER_INPUT_FOLDER = '/home/tvb_user/.data'

CSCS_DATA_FOLDER = 'data'

CSCS_LOGIN_TOKEN_ENV_KEY = 'CSCS_LOGIN_TOKEN'

CSCS_PROJECT = 'CSCS_PROJECT'

HOME_FOLDER_MOUNT = '/HOME_FOLDER'

OUTPUT_FOLDER = 'output'

TVB_BIN_ENV_KEY = 'TVB_BIN'

static compute_available_disk_space(operation)

static execute(operation_id, user_name_label, adapter_instance)

Call the correct system command to submit a job to HPC.

static stage_out_to_operation_folder(working_dir, operation, simulator_gid)

static stop_operation(operation_id)

storage_interface = <tvb.storage.storage_interface.StorageInterface object at 0x7f00885a37d0>

update_datatype_groups()

static update_db_with_results(operation, sim_h5_filenames, metric_operation, metric_h5_filename)

Generate corresponding Index entities for the resulted H5 files and insert them in DB.

tvb.core.services.backend_clients.hpc_scheduler_client.get_op_thread(op_id)

standalone_client

class tvb.core.services.backend_clients.standalone_client.OperationExecutor(op_id)

Bases: threading.Thread

Thread in charge for starting an operation, used both on cluster and with stand-alone installations.

run()

Get the required data from the operation queue and launch the operation.

static stop_pid(pid)

Stop a process specified by PID. :returns: True when specified Process was stopped in here, False in case of exception(e.g. process stopped in advance).

stopped()

Check if current thread was marked for stop.

class tvb.core.services.backend_clients.standalone_client.StandAloneClient

Bases: tvb.core.services.backend_clients.backend_client.BackendClient

Instead of communicating with a back-end cluster, fire locally a new thread.

static execute(operation_id, user_name_label, adapter_instance)

Start asynchronous operation locally

static process_queued_operations()

static start_operation(operation_id)

static stop_operation(operation_id)

Stop a thread for a given operation id

static stop_operation_process(operation_id, notify_pods=False)