Managing Storage Costs with File Storage Service Quotas

We are excited to announce that the Oracle Cloud Infrastructure (OCI) File Storage Service now supports user, group and file system level quotas empowering you as the file system administrator to manage end-user consumption and cost. This enhancement enables you to:

• Set the file system level quota to help ensure the total usage doesn't exceed the set quota
• Set the default user and group quota on the file system
• Have granular control at the individual user/group level

Customers use OCI File Storage as a shared file system where end-users store Files, AI/ML training data, application binaries, backups, and logs. The file system can automatically scale up to exabytes to meet usage demand. Quotas enable you as the administrator, to manage the capacity consumed by individual users or groups in a file system. You can also use Quotas to manage the size of the file system. Quotas empower you to control costs. It enables you to set guardrails and reduce cost spikes by controlling how much data each user stores, and prevent unnecessary and/or excessive storage consumption. Quotas also help with capacity planning. After quotas are enabled, you can monitor usage and adjust the quotas as needs change. With the introduction of Quotas, you can anticipate storage needs and take appropriate actions to manage costs for your organization.

1.1. How Quotas work

You as the file system administrator (IAM user with manage permission) can set default quota for the total capacity of the file system. You can further limit the amount of storage a user or group (Operating system uid and gid) can consume with individual quotas that override the default. For flexible experience, you have the option to set soft or hard limits on quotas. You can set both soft and hard limits or just soft or hard limits only. A hard quota means the file storage consumption cannot exceed the limit. Once the hard quota limit is reached, users can only read and delete files. On the other hand, a soft quota warns the user and allows the storage quota to be exceeded. You can take action by managing the quota limits or file system capacity to avoid hitting the hard quota limit. For operational convenience, we enable you with the usage report with quota violations for visibility. As a best practice, we recommend setting threshold for soft quota limit at 20% less than that of hard quota limit to manage your storage consumption.

Along with Quotas, you can use our metrics to monitor users who have exceeded soft and hard quotas. 

• userSoftQuotaViolations : Number of violations of user soft quota.
• userHardQuotaViolations : Number of violations of user hard quota.

You can specify quota with the minimum limit of 10GB and with 1 GB granularity. The quota limit only applies to the storage of the live file system. Storage that is not part of the live file system such as data captured within a snapshot is not counted against quota limits. You can access the usage report with the violations for monitoring.

You can use API/SDK/CLI/Console/Terraform to perform below operations on Quotas:

• Create quota rules at file system level or group level or user level. You can modify specific quota rules or delete them.
• You can view(list) the usage for each user in the system and associated quota limits.  Usage is available only when quota is enabled.
• You can disable all quotas associated with FS using DisableQuotaRules Api.
• When hard quota limit is reached, write operations will fail.
  • Admin (or tenancy owner) can ask user to reduce usage or change the limits of the specific user or group.
  • If admin takes no action and usage does not go down write requests will keep failing.
• A notification or alarm can be set up based on public metrics such that when soft quota limit is reached, a notification will be sent and write operations will continue.

Enabling File system level quota rule:

Adding a quota rule at user level:

Quota usage view showing both user level and the file system level:

Quota limit getting applied during write operation based on set limit:

Note: Quota feature counts only data usage, so creating an empty file or directory will not be blocked after hard limit is reached.

Soft limit violators:

Disaster Recovery

• You have to explicitly enable the Quotas on target region file system for enforcement. 
• Quota enforcement does not happen on the target unless the target is promoted to a live file system. 

FSS provides cross region replication for data protection. The announcement blog for OCI File Storage Replication explains how to use the OCI FSS native snapshots and replication for your disaster recovery. All quota related metadata is stored and transferred from source region to target region as part of replication system snapshot. The quota rules, like all other file system metadata and data, is transferred asynchronously on a snapshot-by-snapshot basis to the target side. 

Snapshots and Clones

• You cannot set quotas on snapshots.
• A cloned file system inherits the quota rules from the snapshot it was cloned from and not from the live production file system. The cloned file system will have quotas disabled by default and will have to be enabled by you explicitly. 

1.2. Try it for yourself 

We want you to experience these new features available with OCI File System. Interested in trying File Storage? Sign up for a free trial.

We value your feedback as we continue to make our service the best in the industry. Contact us to share your thoughts on how we can continue to improve or if you want more details about any topic. More feature updates are on the horizon for our cloud storage platform.

For more information, see the following resources:

• OCI File Storage Service
• OCI File System Replication
• File Storage snapshots
• File Storage clones