MESOS-3896: Add accounting for reservation slack in the allocator
原文链接 http://www.k8s.tips/tech/2015/12/16/MESOS-3896_design_doc/
注:以下为加速网络访问所做的原文缓存,经过重新格式化,可能存在格式方面的问题,或偶有遗漏信息,请以原文为准。
Description:
The target of this JIRA is to offer the allocation slack resources to the framework. Here're the key points of this design:
- It's different with oversubscription: oversubscription "dynamic adjust resources", make cluster has more resources (revocable); but Optimistic Offer lends reserved resources to other framework (did not increase the total resources i cluster).
- For allocation slack resources, it should NOT included in Quota.
- After Quota,
remainClusterResource.contains
should NOT account ALLOCATION_SLACK; for example, if 40 CPU in remainClusterResources, in one slave, 30 CPU is "unreserved + reserved" and 20 CPU is ALLOCATION_SLACK, master should offer those resources to the framework. - In DRF, should ALLOCATION_SLACK account into the sorter?
- Currently, allocator assign all ALLOCATION_SLACK resources to the first framework; it should also be re-visit in MESOS-3765 (Make offer size adjustable (granularity)).
- I'm also thinkging to add a sorter for allocation slack after DRF; but it seems overkilled.
JIRAs of Design
MESOS-XXX: Optimsistic accounter
class HierarchicalAllocatorProcess
{
struct Slave
{
...
struct Optimistic
{
Resources total; // The total allocation slack resources
Resources allocated; // The allocated allocation slack resources
};
Optimistic optimistic;
};
}
MESOS-4146: flatten & allocationSlack for Optimistic Offer
class Resources
{
// Returns a Resources object with the same amount of each resource
// type as these Resources, but with all Resource objects marked as
// the specified `RevocableInfo::Type`; the other attribute is not
// affected.
Resources flatten(Resource::RevocableInfo::Type type);
// Return a Resources object that:
// - if role is given, the resources did not include role's reserved
// resources.
// - the resources's revocable type is `ALLOCATION_SLACK`
// - the role of resources is set to "*"
Resources allocationSlack(Option<string> role = None());
}
MESOS-XXX: Allocate the allocation_slack resources to framework
void HierarchicalAllocatorProcess::allocate(
const hashset<SlaveID>& slaveIds_)
{
foreach slave; foreach role; foreach framework
{
Resource optimistic;
if (framework.revocable) {
Resources total = slaves[slaveId].optimistic.total.allocationSlack(role);
optimistic = total - slaves[slaveId].optimistic.allocated;
}
...
offerable[frameworkId][slaveId] += resources + optimistic;
...
slaves[slaveId].optimistic.allocated += optimistic;
}
}
Here's some consideration about ALLOCATION_SLACK
:
- 'Old' resources (available/total) did not include ALLOCATION_SLACK
- After
Quota
,remainingClusterResources.contains
should not check ALLOCATION_SLACK; if there no enough resources, master can still offer ALLOCATION_SALCK resources. - In sorter, it'll not include ALLOCATION_SLACK; as those resources are borrowed from other role/framework
- If either normal resources or ALLOCATION_SLACK resources are allocable/!filtered, it can be offered to framework
- Currently, allocator will assign all ALLOCATION_SALCK resources in slave to one framework
MESOS-XXX: Update ALLOCATION_SLACK for dynamic reservation (updateAllocation)
void HierarchicalAllocatorProcess::updateAllocation(
const FrameworkID& frameworkId,
const SlaveID& slaveId,
const vector<Offer::Operation>& operations)
{
...
Try<Resources> updatedOptimistic =
slaves[slaveId].optimistic.total.apply(operations);
CHECK_SOME(updatedTotal);
slaves[slaveId].optimistic.total =
updatedOptimistic.get().stateless().reserved().flatten(ALLOCATION_SLACK);
...
}
MESOS-XXX: Add ALLOCATION_SLACK when slaver register/re-register (addSlave)
void HierarchicalAllocatorProcess::addSlave(
const SlaveID& slaveId,
const SlaveInfo& slaveInfo,
const Option<Unavailability>& unavailability,
const Resources& total,
const hashmap<FrameworkID, Resources>& used)
{
...
slaves[slaveId].optimistic.total =
total.stateless().reserved().flatten(ALLOCATION_SLACK);
...
}
No need to handle removeSlave
, it'll all related info from slaves
including optimistic
.
MESOS-XXX: return resources to allocator (recoverResources)
void HierarchicalAllocatorProcess::recoverResources(
const FrameworkID& frameworkId,
const SlaveID& slaveId,
const Resources& resources,
const Option<Filters>& filters)
{
if (slaves.contains(slaveId))
{
...
slaves[slaveId].optimistic.allocated -= resources.allocationSlack();
...
}
}