Fairness | 木叶吟

ASTRAEA: Fairness Is More Than Counting GPUs

Sun, 17 May 2026 13:00:00 +0800

Fairness sounds simple until a GPU cluster starts running real deep learning workloads.

In a shared research or production cluster, different tenants submit jobs with very different shapes. Some jobs need one GPU for a quick debugging run. Others need many GPUs and run for days. A scheduler that only optimizes utilization may let long jobs dominate the cluster. A scheduler that aggressively favors short jobs may make large training jobs wait forever. Both users can reasonably say the system is unfair.

ASTRAEA was built around this problem: how should a multi-tenant GPU cluster enforce fairness without wasting expensive accelerators?

TL;DR. ASTRAEA makes fairness measurable in the unit the cluster actually spends: long-term GPU-time. It then uses that signal at both tenant and job levels, so fairness accounts for how many GPUs a job occupies and how long it occupies them.

Why Existing Fairness Breaks

Traditional cluster schedulers often think in terms of instantaneous resource fairness. If two users share a cluster, each should receive a fair share of resources at the current moment. This works well for many big-data workloads, where tasks are easier to split, migrate, and rebalance.

Deep learning training is less flexible. Jobs usually require gang scheduling: all requested GPUs must be allocated together. Communication-heavy jobs are sensitive to GPU topology. Preemption is also costly because model state must be checkpointed, moved, and restored. If a scheduler tries to enforce fairness by frequently reshuffling GPUs, it can destroy the performance it was meant to protect.

Another approach is finish-time fairness, where the scheduler asks whether a job would finish no later than it would in a private fair-share cluster. That is useful, but incomplete. It focuses on time and can miss the spatial side of fairness: a job that asks for more GPUs consumes more cluster capacity per unit time. Treating a 1-GPU job and an 8-GPU job only through finish time can create incentives to overclaim resources.

ASTRAEA’s core idea is to measure what the cluster is actually spending: GPU-time.

Key observation. In GPU clusters, fairness has both a spatial dimension and a temporal dimension. Counting only GPUs ignores time; counting only finish time ignores how much cluster capacity a job consumed.

Long-Term GPU-Time Fairness

ASTRAEA introduces Long-Term GPU-Time Fairness, or LTGF. Instead of asking only “how many GPUs does a tenant have right now?” or “when will this job finish?”, LTGF asks how much GPU service a tenant or job has received over a period of time compared with how much it deserves.

This captures both dimensions of allocation:

temporal impact: how long the job runs;
spatial impact: how many GPUs it occupies while running.

At the tenant level, LTGF distributes GPU-time according to tenant weights, such as budget or quota. At the job level, it distributes GPU-time fairly among concurrent jobs inside a tenant. This two-level view is important because a fair cluster should protect both the organization sharing contract and the individual jobs waiting inside each tenant’s queue.

The metric also avoids relying on fragile remaining-time prediction. In real clusters, users cancel jobs, jobs fail, and training throughput changes with placement. ASTRAEA can evaluate fairness from past allocation history, then use that signal to decide who should receive service next.

Fairness becomes easier to reason about once the scheduler measures service in GPU-time instead of only instantaneous allocation or completion time.

How ASTRAEA Schedules

ASTRAEA uses a two-phase scheduling algorithm.

First, it selects the tenant with the lowest tenant-level fairness index. In plain language: the scheduler finds the tenant that has received the least GPU-time relative to what it should have received. If that tenant has pending jobs and the cluster can place one of them, ASTRAEA grants resources to it.

Second, ASTRAEA selects a job within that tenant using the job-level fairness index. This keeps one tenant’s internal queue from becoming unfair even when the tenant as a whole is being treated fairly. Job-level policies can still incorporate practical priorities, but they are constrained by the fairness signal.

The scheduler is lease-based. Instead of preempting whenever fairness changes, ASTRAEA gives a running job a lease term. At lease boundaries, the scheduler can rearrange execution order to repair fairness. This is a practical compromise: short leases improve fairness response, but too-short leases increase preemption overhead and hurt job completion time. ASTRAEA chooses a lease length that balances those forces for deep learning training.

Scheduling layer	Fairness signal	Scheduling decision
Tenant level	Tenant-level LTGF index	Pick the tenant that has received the least service relative to its share
Job level	Job-level LTGF index	Pick a job inside that tenant without making the tenant’s queue unfair
Lease boundary	Updated allocation history	Repair fairness while avoiding constant preemption

What It Buys

ASTRAEA was evaluated with large-scale simulations on real GPU cluster traces, including SenseTime’s Venus trace and Microsoft’s Philly trace. The paper reports that ASTRAEA improves tenant-level fairness by up to 9.42x and job-level fairness by up to 10.3x compared with state-of-the-art schedulers, without sacrificing average job completion time.

What ASTRAEA buys.

It measures fairness in long-term GPU-time, combining space and time.
It improves tenant-level fairness by up to 9.42x.
It improves job-level fairness by up to 10.3x without sacrificing average job completion time.

The important lesson is that fairness in GPU clusters is not just a policy preference. It is a measurement problem. If the metric ignores GPU count, users can overclaim. If it ignores time, long-running jobs can be starved. If it ignores tenants, the cluster violates sharing agreements. If it ignores jobs, individual users still experience unfairness.

ASTRAEA’s contribution is to make fairness measurable in the unit that matters most for deep learning clusters: long-term GPU-time.

Paper: ASTRAEA: A Fair Deep Learning Scheduler for Multi-tenant GPU Clusters
Code: Astraea Artifacts

ASTRAEA：GPU 集群里的公平，不只是分到几张卡

Sun, 17 May 2026 13:00:00 +0800

公平性听起来很简单，直到一个 GPU 集群真的开始承载各种深度学习任务。

在共享的科研或生产集群里，不同租户提交的任务形态差异很大。有些任务只需要一张 GPU，跑几分钟做调试；有些训练需要很多张 GPU，并且连续运行好几天。只追求利用率的调度器，可能让长任务长期占据集群；过度偏向短任务的调度器，又可能让大型训练一直排队。两类用户都可以很合理地说：这个系统不公平。

ASTRAEA 关注的正是这个问题：多租户 GPU 集群如何在不浪费昂贵加速器的前提下，把公平性真正落到调度决策里？

TL;DR. ASTRAEA 把公平性度量放到集群真正消耗的单位上：长期 GPU-time。它同时在租户层和任务层使用这个信号，让公平性既考虑任务占了多少 GPU，也考虑占用了多久。

为什么已有公平性会失效

传统集群调度器经常从“瞬时资源公平”的角度思考问题。比如两个用户共享一个集群，那么在某个时刻，每个人都应该拿到自己的公平份额。对很多大数据任务来说，这个思路很自然，因为任务更容易切分、迁移和重新平衡。

深度学习训练没有这么灵活。训练任务通常需要 gang scheduling：请求的所有 GPU 必须同时分配到位。通信密集型任务对 GPU 拓扑很敏感。抢占也很贵，因为模型状态和优化器状态需要 checkpoint、移动和恢复。如果调度器为了追求公平而频繁重排 GPU，反而可能破坏原本想保护的性能。

另一类方法是 finish-time fairness，也就是判断一个任务是否不晚于它在私有 fair-share cluster 里的完成时间。这个目标有用，但不完整。它强调时间，却容易忽略公平性的空间维度：一个请求更多 GPU 的任务，在单位时间内消耗了更多集群容量。只用完成时间看待 1-GPU 任务和 8-GPU 任务，可能鼓励用户多报资源。

ASTRAEA 的核心想法是直接度量集群真正付出的东西：GPU-time。

关键观察。 GPU 集群中的公平性同时有空间维度和时间维度。只数 GPU 会忽略时间，只看完成时间又会忽略一个任务消耗了多少集群容量。

长期 GPU-Time 公平性

ASTRAEA 提出了 Long-Term GPU-Time Fairness，简称 LTGF。它不只问“这个租户现在有几张 GPU”，也不只问“这个任务什么时候完成”，而是问：在一段时间内，某个租户或任务实际获得了多少 GPU service，相比它应得的份额是否公平。

这个指标同时捕捉了分配的两个维度：

时间维度：任务运行了多久；
空间维度：任务运行时占用了多少 GPU。

在租户层面，LTGF 按照预算或 quota 这类权重分配 GPU-time。在任务层面，它在同一个租户内的并发任务之间公平分配 GPU-time。这个两级视角很重要，因为公平集群既要保护组织层面的共享契约，也要照顾每个租户队列里正在等待的具体任务。

这个指标还避免了对剩余时间预测的过度依赖。真实集群里，用户会取消任务，任务会失败，训练吞吐也会随 placement 变化。ASTRAEA 可以根据历史分配记录评估公平性，再用这个信号决定下一步该服务谁。

当调度器用 GPU-time 而不是瞬时 allocation 或完成时间来衡量 service 时，公平性才更容易被推理和修复。

ASTRAEA 如何调度

ASTRAEA 使用两阶段调度算法。

第一阶段，它选择 tenant-level fairness index 最低的租户。直白地说，调度器会找到那个相对自己应得份额而言，过去获得 GPU-time 最少的租户。如果这个租户有等待中的任务，并且集群可以放下其中某个任务，ASTRAEA 就会把资源分给它。

第二阶段，ASTRAEA 在该租户内部用 job-level fairness index 选择具体任务。这样即使一个租户整体上被公平对待，它内部的任务队列也不会变得很不公平。任务级策略仍然可以加入实际优先级，但需要受到公平性信号约束。

ASTRAEA 采用 lease-based 调度。它不会在公平性发生变化时立刻抢占，而是给运行中的任务一个租约周期。在租约边界，调度器可以重新安排执行顺序来修复公平性。这是一个务实折中：短租约能更快响应公平性变化，但太短又会增加抢占开销，拉长任务完成时间。ASTRAEA 会为深度学习训练选择一个平衡这些因素的租约长度。

调度层次	公平性信号	调度决策
租户层	Tenant-level LTGF index	选择相对自身份额获得 service 最少的租户
任务层	Job-level LTGF index	在该租户内部选择任务，避免队列内部不公平
租约边界	更新后的 allocation history	在避免频繁抢占的同时修复公平性

它带来了什么

ASTRAEA 在真实 GPU 集群 trace 上做了大规模仿真评估，包括 SenseTime 的 Venus trace 和 Microsoft 的 Philly trace。论文报告称，相比已有先进调度器，ASTRAEA 将租户级公平性最多提升 9.42x，将任务级公平性最多提升 10.3x，同时没有牺牲平均任务完成时间。

ASTRAEA 换来了什么。

用 long-term GPU-time 衡量公平性，同时覆盖空间和时间。
将 tenant-level fairness 最多提升 9.42x。
将 job-level fairness 最多提升 10.3x，同时不牺牲平均任务完成时间。

这里重要的经验是：GPU 集群里的公平性不只是政策偏好，它首先是一个测量问题。如果指标忽略 GPU 数量，用户就可能 overclaim；如果指标忽略时间，长任务就可能被饿死；如果指标忽略租户，集群就会违反共享契约；如果指标忽略任务，个体用户仍然会感受到不公平。

ASTRAEA 的贡献，是把深度学习集群中最关键的资源服务单位“长期 GPU-time”变成了可测量、可调度的公平性目标。

Paper: ASTRAEA: A Fair Deep Learning Scheduler for Multi-tenant GPU Clusters
Code: Astraea Artifacts