Ballista Scheduler Metrics¶
Prometheus¶
Built with default features, the ballista scheduler will automatically collect and expose a standard set of prometheus metrics. The metrics currently collected automatically include:
job_exec_time_seconds - Histogram of successful job execution time in seconds
planning_time_ms - Histogram of job planning time in milliseconds
failed - Counter of failed jobs
job_failed_total - Counter of failed jobs
job_cancelled_total - Counter of cancelled jobs
job_completed_total - Counter of completed jobs
job_submitted_total - Counter of submitted jobs
pending_task_queue_size - Number of pending tasks
NOTE Currently the histogram buckets for the above metrics are set to reasonable defaults. If the defaults are not
appropriate for a given use case, the only workaround is to implement a customer SchedulerMetricsCollector. In the future
the buckets should be made configurable.
The metrics are then exported through the scheduler REST API at GET /api/metrics. It should be sufficient to ingest metrics
into an existing metrics system by point your chosen prometheus exporter at that endpoint.