Thursday Nov 20, 2008

Administrators in the Fishworks Appliance Kit

As discussed in other blog entries, many of the problems we solved in building the Sun Storage 7000 series were not specific to the NAS market. Moreover, the appliance kit framework we built solves many problems associated with an appliance in any domain -- that is, in addition to providing new innovative features, the appliance kit exposes and clarifies existing Solaris abstractions. In that vein, we needed a system to manage which users function as administrators of the appliance. We wanted the capability to pull users from an existing directory service, such as NIS or LDAP. Moreover, administrators need their own properties which were specific to our framework, such as whether they required a session annotation, or were a kiosk user. Finally, administrators have several user preferences which customize their environment to their liking, such as their initial login screen or character locale.

The 7000 series functions as a NIS and LDAP client1 and queries those directory services for information about users. These directory services consolidate the management of users in that single place, and allow a user's password and other properties (UID, home directory, etc.) to remain consistent throughout an entire organization. Adding a administrator from a directory service allows that user to login to the 7000 series and administer the appliance. Attributes and passwords for these directory users are stored in a directory service, and the appliance authenticates those administrators through a directory service. Administrators can also be added locally, in which case passwords for those users are stored locally on the appliance. These local users must be added to each appliance separately.

Best Practices

When our team was discussing how users should administer the appliance, we decided that the best practice was to add all administrators from a directory service and forego adding any local users. We established this best practice for the following reasons:

  • Simplified management of users. If an employee changes his password, that change will take effect on all machines.
  • No UID conflicts. Because local users are given UIDs starting from 2000000000 in the order in which they're added, two local users on different appliances can have different UIDs, or even worse, the same UID can represent two different users. Even though these local users should not be used to access data, local users are no different than any other user on a Solaris system -- they can be used in establishing permissions on the root directory of a filesystem. If a particular user requires the same UID across several machines, shouldn't that user be stored in a directory service?

  • No orphaned accounts. When an employee leaves a company, only the user account in the directory service must be deleted. The administrative accounts added to the appliance will not function if that user cannot be resolved in the directory service. These users are called degraded users, and while they should be cleaned up as part of normal housekeeping, they pose no security risk since they cannot login.

We've had some beta customers request the ability to prevent any local users on an appliance and restrict all administrators (other than root, of course) to be stored in a directory service. We'll work on this feature for the next software update. The appliance kit also has a rich set of fine-grained access controls for administrators which closely resembles RBAC in its design -- I'll cover this feature in a later entry.

[1] The 7000 series can also join an Active Directory domain and communicate with a domain controller for the purposes of authenticating CIFS clients. However, support for pulling administrators from Active Directory is not currently supported and will be coming a future software update.

Monday Nov 10, 2008

Session Annotations

As I described in my earlier blog entry on kiosk users, our team has been in contact with evaluation and beta customers to understand the problems they face as storage administrators. These administrators are responsible for the security of the storage infrastructure, and must understand how and why changes are made to the storage configuration. Many IT departments use a centralized ticket management system to manage open issues and requests. When a storage consumer reports a problem and requests some configuration change, that user is given a unique identifier to track the issue in this database. The ticket database contains all the details of the request, like the person requesting the change, the date and time of the request, which administrator will make the change, and the current status of the change.

One of our larger beta customers requested the ability to tag the audit log with a user-defined string which annotates that particular session. Certain users, upon logging in, must provide a session annotation:

Then, that annotation is saved with every audit record generated in that session. Users who are required to indicate this annotation may change their annotations in the middle of the session, but can never provide an empty session annotation:

Customers who have a centralized ticket management system can have their administrators provide the ticket identifier upon logging in. With those annotations, the auditors who examine the audit log gain a richer understanding of those actions, as each audit record can be correlated with the information in the database.

Kiosk Users: Observability for Everyone

While the Fishworks team was developing our new storage server, we enlisted the help of several dozen experts in the enterprise storage field to help us define useful and necessary features. In addition to meeting the basic needs of storage administrators, we also wanted to provide additional features which administrators would find convenient. Across many enterprise domains, we continually heard one common complaint: storage servers were being blamed for performance problems, when the real performance problem lay somewhere else in the IT infrastructure.

At large companies, the storage infrastructure is consolidated into a single team of administrators who manage storage for the entire company. This team of administrators has service agreements which stipulate certain capacity, performance, and uptime requirements for other groups, and they must address problems if those service levels are not maintained. Storage administrators complained that they often spent time debugging performance problems with a particular group's application, only to find the storage was performing exactly as prescribed. Storage administrators could not gain any insight into a particular server's operation; instead, they wasted hours proving to application developers that the storage was not the source of the problem.

Our revolutionary analytics interface helps storage administrators understand how the box is performing. The dashboard page provides a summary view of key metrics, including a weather metaphor for each statistic. While this information is useful to storage administrators, it can also be useful to application developers who are actually using the storage. With this idea in mind, we created a new kind of administrator: a kiosk user. Kiosk users are created using the normal user dialog:

Storage administrators can add the application developers as kiosk users, or create a kiosk account for an entire team. By setting the screen to which kiosk users are restricted, administrators control the level of access application developers have to analytics data. For example, a kiosk user may be able to only view the dashboard, but not more specific worksheets. Likewise, a different kiosk user may be restricted to a worksheet for a particular project instead of having access to analytics data across all projects.

We anticipate that our customers will find this kiosk feature most useful for granting access to the dashboard and analytics data, though there are many other useful scenarios:

  • Clients who have connectivity problems can use the services pages to check the state of system and data services.
  • By viewing the audit logs, auditors can understand who accessed the appliance what changes those users have made.
  • Network administrators can inspect the networking and routing configuration to understand and troubleshoot any problems.

One important note about kiosk users: even though they are restricted to viewing a certain screen, a malicious Javascript client can still make XML-RPC calls. A kiosk user cannot navigate to any other screen in the UI; however, that user will be able to see the results from raw XML-RPC calls which are not associated with that screen. Because of the appliance's fine-grained access control, a kiosk user with no roles or authorizations will not be able to change any configuration. Do not make the mistake of assuming a kiosk user will not be able to view the current shares, users, or other configuration parameters even if their kiosk screen would not normally allow them to access that data.

I hope our customers are able to find many and varied uses for kiosk users; please share your experiences with them in the comments.


The blog of Bill Pijewski, a member of the Fishworks engineering team.


« August 2016