Introduction

Purpose

Currently in CloudStack, the admin can limit the resources managed by CloudStack by setting limits for accounts and domains. But these existing limits are based on resource count, that limits the user on the basis of no. of VMs, no. of Volumes, no. of snapshots etc. and not of the basis of storage space, CPU, memory etc.

This feature will add new (CPU, Memory, Primary storage, Secondary storage, network rate etc.) resource types to the existing pool of resources. CS admin would be able to set the limits i.e. CPU, memory, storage etc. for accounts/domains through API or through CloudStack UI. This is functional specifications of feature CLOUDSTACK-713.

Current Scenario

  • updateResourceLimit API has the following resource types:** Instance - Number of instances a user can create.
    • Public IPs - Number of public IP addresses a user can own.
    • Volume - Number of disk volumes a user can create.
    • Snapshot - Number of snapshots a user can create.
    • Template - Number of templates that a user can register/create. 
    • Network - Number of guest networks that a user can create.
    • VPC - Number of VPCs that a user can create.

References

Feature Specifications

  • Requirements

    • Update Resource Limit: Root Admin would be able to limit the following resources for domain/accounts, domain admin would be able to limit the follwoing resources only for sub-domains/accounts under his own domain or all sub-domain.
      • CPUs
      • Memory
      • Primary (shared) storage (Volumes)
      • Secondary storage  (Snapshots, Templates & ISOs)
      • Network bandwidth rate (in mbps)
    • Usage: Limits can be imposed on Accounts, Domains and Projects.
    • List Resource Limit: Any User/RootAdmin/Domain Admin can list Resources
    • Resource Limit Logs: Ensure proper logs are maintained into vmops.log and api.log 

  • Non-requirements:

  • Use Cases

    Use-Case 1: ‘Custom VM’ model for an employee in an Enterprise (Limit Only) 
    IT Admin would like to allow users to create custom VMs based on need. For example, one large VM or many small VMs etc. However the admin would like to restrict the maximum resources a employee can consume by setting limits 
    Use-Case 2: ‘Custom VM’ model for a researcher in a University (Reservation & Limit) 
    Cloud Admin wants to reserve certain resources to the researcher and allow him to create VMs of choice. Based on the research needs the VM configuration could be different. The researcher is guaranteed the assigned resources, but at the same time he is not allowed to exceed the resources he is assigned.

  • Test guidelines

    • Functional test

  • Configuration

    • global configuration parameters:
      • max.account.cpus - Maximum number of CPU cores that can be used for an account. Default value for this is 40.
      • max.account.memory (in MB) - Maximum Memory that can be used for an account. Default value for this is 40960.
      • max.account.primary.storage (in GB) - Maximum primary storage space that can be used for an account. Default value for this is 20*10.
      • max.account.secondary.storage (in GB) - Maximum secondary storage space that can be used for an account. Default value for this is 20*20.
      • max.account.network.rate(in mbps) - Maximum network rate that can be used for an account. Default value for this is 200.
      • max.project.cpus - Maximum number of CPU cores that can be used for an account. Default value for this is 40.
      • max.project.memory (in MB) - Maximum Memory that can be used for an account. Default value for this is 40960.
      • max.project.primary.storage (in GB) - Maximum primary storage space that can be used for an account. Default value for this is 20*10.
      • max.project.secondary.storage(in GB) - Maximum secondary storage space that can be used for an account. Default value for this is 20*20.
      • max.project.network.rate(in mbps) - Maximum network rate that can be used for an account. Default value for this is 200.

  • User permission

    • Normal users would have privilege to list resource limits (listResourceLimits).
    • Root-Admin would have privilege to list and update resource limits.
    • Domain-Admin would have privilege to list resources and update resource limits that are assigned to the domain.

Use cases

Domain D1 , Account A1 and  Project P1

Point to remember: When we refer Primary/Seconday storage space, it means the stated size of the volume and not the physical size i.e actual consumed size on disk (in case of thin provisioning).

1. Number of CPUs:

    1. Root/Domain-Admin update CPU to 10 for domain D1. Users belonging to Domain D1 should not be able to use more than that.
    2. Root/Domain-Admin update CPU to 10 for Account A1 in Domain D1. Users belonging to Account A1 should not be able to use more than that. Also, they should not cross CPU limit defined for D1 domain.
    3. Limits imposed on Domain D1 should be imposed on sub-domain SD1 as well.
    4. Root/Domain-Admin update CPU to 10 for Project P1 having account A1 . Users belonging to Project P1 should not be able to use more CPUs than the defined limit.

2. Amount of Memory:

    1. Root/Domain-Admin update Memory to 1GB for domain D1. Users belonging to Domain D1 should not be able to use more than that.
    2. Root/Domain-Admin update Memory to 1GB for Account A1 in Domain D1. Users belonging to Account A1 should not be able to use more than that. Also, they should not cross Memory limit defined for D1 domain.
    3. Limits imposed on Domain D1 should be imposed on sub-domain SD1 as well.
    4. Root/Domain-Admin update Memory to 1GB for Project P1 having account A1. Users belonging to Project P1 should not be able to use more Memory than the defined limit.

3. Amount of Primary Storage:

    1. Root/Domain-Admin update primary storage to 5GB for domain D1. Users belonging to Domain D1 should not be able to use more than that.
    2. Root/Domain-Admin update primary storage to 5GB for Account A1 in Domain D1. Users belonging to Account A1 should not be able to use more than that. Also, they should not cross primary storage limit defined for D1 domain.
    3. Limits imposed on Domain D1 should be imposed on sub-domain SD1 as well.
    4. Root/Domain-Admin update primary storage to 5GB for Project P1 having account A1. Users belonging to Project P1 should not be able to use more primary storage than the defined limit.

4. Amount of Secondary Storage:

    1. Root/Domain-Admin update secondary storage to 5GB for domain D1. Users belonging to Domain D1 should not be able to use more than that.
    2. Root/Domain-Admin update secondary storage to 5GB for Account A1 in Domain D1. Users belonging to Account A1 should not be able to use more than that. Also, they should not cross secondary storage limit defined for D1 domain.
    3. Limits imposed on Domain D1 should be imposed on sub-domain SD1 as well.
    4. Root/Domain-Admin update secondary storage to 5GB for Project P1 having account A1. Users belonging to Project P1 should not be able to use moresecondary storage than the defined limit.

5. Network bandwidth rate:

  1.  
    1. Root/Domain-Admin update network bandwidth rate to 200mbps for domain D1. Users belonging to Domain D1 should not be able to use more than that.
    2. Root/Domain-Admin update network bandwidth rate to 200mbps for Account A1 in Domain D1. Users belonging to Account A1 should not be able to use more than that. Also, they should not cross network bandwidth rate limit defined for D1 domain.
    3. Limits imposed on Domain D1 should be imposed on sub-domain SD1 as well.
    4. Root/Domain-Admin update network bandwidth rate to 200mbps for Project P1 having account A1. Users belonging to Project P1 should not be able to use more network bandwidth rate than the defined limit.

6. If admin reduces the resource limit for an account and set it to less then resources currently consumed by that account, under this scenario, CS would not destroy the existing VMs/templates/volumes etc. using those resources, it only impose limits if user under that account tries to execute some new operation.  For e.g., in case of VM, the existing behavior of CS for the following commands are:

  1.  
    1. migrateVirtualMachine: User under that account will be able to migrate the running VM into any other host without facing any limit issue.
    2. recoverVirtualMachine: If user destroy one of the running VMs and later if he tries to recover it, CS will not allow this operation.

7. For any resource type, if a domain has limit X, sub-domain/accounts under that domain can have there own limits, but at any point of time the sum of resource allocated to sub-domain/accounts under the domain should never exceed the value X.
For e.g., if a domain has CPU limit of 40 and sub-domain D1 and account A1 can have limits of 30 each, but at any point of time the resource allocated to D1 and A1 should not exceed the limit 40.

8. If any operation needs to pass through two of more resource limit check, then the lower of 2 limits will be enforced, For e.g. if an account has VM limit of 10 and CPU limit of 20 and user under that account requests 5 VMs of 4 CPUs each, after this user can deploy 5 more VMs(because VM limit is 10) but user has exausted his CPU limit and cannot deploy any more instance.

Architecture and Design description

Existing API Changes:

1. updateResourceLimit

ApiName Request Parameters Response Parameters Available to regular user
updateResourceLimit
  • resourcetype (admin can pass new resource type to update CPU, Memory, primary storage,secondary storage and network rate)
  • account
  • domainid
  • max
  • projectid
  • account
  • domain
  • domainid
  • max
  • project
  • projectid
  • resourcetype
 No

In the above mentioned API, admin can pass the newly added resource types to set the limits.

2. updateResourceCount

ApiName Request Parameters Response Parameters Available to regular user
updateResourceCount
  • domainid
  • account
  • projectid
  • resourcetype (admin can pass new resource type to update CPU, Memory, primary storage, secondary storage *and network rate*)
  • account
  • domain
  • domainid
  • project
  • projectid
  • resourcecount
  • resourcetype
 No

In the above mentioned API, admin can pass the newly added resource types to get the resource count.

3. listResourceLimits 

  This API is available to regular users and this will list the resource limits including the new resource types.

Example:

http://localhost:8080/client/api?command=updateResourceLimit&resourcetype=9&max=1024&domainid=2&apikey=pFZ6ZdlN-TmlBQWJRhncYUAUFCOGk3aCbbLGRAqtSG3KnbYnTEHXvh1MP-Y5801JAn-aFPODB-7vl1P8DiPQ1A&signature=6he37Flw1zRwNDe0yejyHk6oXcE%3d

DB Changes:

No DB Changes.

Changes to Existing Files:

  • Add new resource types in Resource.java .
  • Add new resource type in description field of resource_types in UpdateResourceCountCmd.java and in UpdateResourceLimitCmd.java .
  • Add new resource type in description field of resource_types in* ResourceCountResponse.java and in ResourceLimitResponse.java .*
  • Add new methods in* ResourceLimitManagerImpl.java* to get resourceCount for newly added resource types*.*
  • Add new method in UserVmDaoImpl.java to get a list of serviceOfferingIds by passing a vmId.
  • Add new calls in UserVmManagerImpl.java and BareMetalVmManagerImpl.java to check the limits for an account and to increment or decrement resource count for an account.
  • (i will keep updating the changes as implementation progresses).

Design Decisions: TBD

  • The Limit imposed (per account, per  Domain, per Project) should not conflict  
  • What should be the maximum limit for CPU, Memory, Primary Storage, Secondary Storage, Network rate

UI flow

  • _On Project Dashboard: Resource page - add resource options as following:

_ Limit Resources to domains and accounts_IT

  • On Domain and Account Info page add the option of additional resource proposed in this spec