Virtual Identity Dialogue

James McGovern recently wrote a question on whether using changelog was sign of good or bad architecture in response to another post on Sun's issues integrating their provisioning product with ADAM.

And James wanted to know - could OVD help?

First - I was surprised that Sun had issues - I figured this was a solved problem in provisioning maybe instead of Sun you should check out OIM :). Second, ADAM is really designed to be an "end-point" (hence no changelog) not a source, thus I would always question an architecture that used ADAM as a source of data.

But to more specifically answer James' question..

Whenever I hear about ADAM being deployed - I always question why is it being deployed - so see if there is a better way.

For example if the reason ADAM is being deployed is because you have data that exists in another database but you need it to be LDAP accessible, then that is clearly a benefit of deploying OVD. OVD could make that DB data look like LDAP without copying it into another storage system that needs to be made highly available, backed up and secured. OVD does all of that with a smaller footprint by leveraging all of the work you already have done in the existing system.

Or it could be that you need to make your existing AD user data look like InetOrgPerson (instead of AD's proprietary user schema). OVD can on the fly make AD look like InetOrgPerson without needing to bring up ADAM.

However, another use case could be is that you have a user population that needs to be stored in a directory but can't be deployed in AD. For example this could be external customers, partners or vendors. While ADAM definitely could be used there - OID could also be an option.

Another slight twist to this use case is where you are deploying an application such as a portal,web  access management or Unix authentication that needs to store data in user's entry but you can't extend the AD schema. OID allows you to store this data in OID while leaving the password stored in AD. OVD is also an option as long as the data can be stored somewhere (benefit of OVD is that possibly - depending upon the type of data, could be stored in a non-Oracle EE database).

And an additional benefit of OID is that it provides default support for synchronization to LDAP or databases via Directory Integration Platform (DIP) and exposes a standards-based changelog so your provisioning tools can easily integrate with it.

While you might be asking "Mark, why are you pushing OID if I have ADAM" - my answer would be two reasons.
First reason is that if you need to be notified of changes in the data being stored, ADAM is probably not the best option. It's not designed for that purpose.
Second reason is that with over 13,000 deployments of OID world-wide, there's a reasonable chance you may have OID (or need it to help deploy another Oracle application like Oracle Portal) and thus this will help increase the value of that solution.

In Summary:

• OVD is useful in avoiding unnecessary synchronization by leveraging existing data
  
• OID has unknown potential as a way to store extended AD attributes that otherwise would increase time needed to deploy new applications.

Today is a day it catch up on some blogging.

James McGovern posted a few questions on our new operating system security product - aka Oracle Authentication Services for Operating Systems or OA4OS.

First quote - " On one level, this feels like a good story, but on another it feels like a long-term trap."
It's just a good story :). There is no trap. Unlike competing solutions - we don't use any proprietary hooks or changes to the Unix /Linux systems. We are using all standard - based interfaces like PAM, NSS and SUDO.

Thus it would be possible to move to another directory solution.

Second quote - "First, if you are running Solaris, this you can setup NIS domains to aid in this problem."
NIS has been out of favor for a while. It has now been officially deprecated. And for the kicker - it is not SOX compliant.

Thus many customer's we've talked to about OAS4OS are specifically looking for how to replace NIS. This is one of the features we offer.

Third quote - "Consider that if you are a shop running Active Directory, Microsoft
provides Active Directory Services for Unix where by you can have Unix
servers and daemons participate as if they are native to the Windows
domain. This simplifies administration significantly, cheap to rollout
and even cheaper over the lifetime. There are of course some features
missing, which Microsoft will be addressing in upcoming releases."

Yes - Microsoft does offer this. However, it has many limitations that in many organizations will not be solvable. For starters - you must extend AD schema - in many organizations this is not allowed by corporate policy. Second, by storing this data this can add severe impact to your AD replication which affect desktop login (which is why this is not allowed by many corporate policies). Third - it does not auto-generate UID & GUID numbers (we do :)). Fourth - they do not have any system to allow you to address use case of where you have same username but different uid/gid numbers on different hosts (hello OVD)).  These are all features that AD lacks and some (such as schema change) will never be avoided.

Final quote- "You can also consider third party software such as Vintela and Centrify
which also provide deeper Unix/Linux integration to Active Directory.
Anyway, I humbly predict that the open source community will realize
that this type of integration should be in the box and not something
add-on and therefore will address within the next six months."

To my knowledge Vintela and Centrify require proprietary components and/or extensions to AD. Also they don't provide any mechanism to manage SUDO policies in your directory.  And I would also point out that this if our first release (if he can mention MSFT updating AD as being OK, I can use it hear for us too :)) so we are going to be adding in additional functionality in the future.

Jackson Shaw responded to my post on OS security.

I have not yet updated any of test AD servers to R2 and thus wasn't aware that AD automatically was updating the schema to support NIS. I am familiar with the RFC but this is a repeated concern I do hear from customers - "we can't extend our AD schema". I literally hear this on a daily basis but this could be because we are one of the few options out there that can help customers who can't extend their AD schema.

I also learned something new - that others are supporting SUDO policies too. :) I didn't mean for my post to be obvious FUD on this feature. I'm still coming up to speed on the competitive landscape for OAS4OS myself (other members of the PM team primarily running that show while I have been concentrating on other areas that I can't yet publicly discuss - but since I am the more outbound focused PM on directory services team - I am talking more about it now).

In short - let me just say where I think organizations may get the most out of OAS4OS:

1  - Simplified installation of secure (e.g. SSL/TLS setup which apparently is pretty nasty manually) PAM and related configuration
2 - Migration tools from local/NIS to LDAP
3 - Support for managing user UNIX footprint outside of AD but keep password in AD (for those who can't or don't want to extend AD schema)
4  - Increase utilization of an existing OID deployment (for example if you have Oracle Portal - you have to have OID - thus could potentially get an increased ROI by using OAS4OS)
5 - The OAS4OS  is currently provided as a feature of the Oracle Directory Services or Oracle Identity & Access Management license(s)


.

So James has responded back to me on ADAM.

"I would love to know your thinking on why a database itself can't be
exposed via the LDAP protocol? For example, what would prevent
Microsoft from allowing SQL Server
to be accessed by not only the TDS protocol but also LDAP? The code
required to make this happen, wouldn't be too difficult for a talented
team to get right. Of course, if you have less talented individuals
then performance or other factors may emerge. Anyway, by putting it
into the DB itself, wouldn't it help customers avoid purchasing yet
another product with yet another support model?"

The point isn't to just use DB for LDAP. The purpose is that you have existing databases (which could be Oracle, SQL Server, DB2, MySQL, etc) that contain existing data you wish to integrate into an identity infrastructure. OVD allows you to quickly and easily do this. And while it's technically not hard to write the back-end code to that - it does take skill and effort to make it easy to configure and support. Additionally this doesn't count for other features we have such as renaming attributes on the fly, translating namespaces and of course linking data together (e.g. user your AD username and password, but link to data stored in HR for a complete real-time picture of your identity data).

"Relational databases have always had the notion of a view and it was built-in the database without requiring an extra product. So, isn't this another scenario where Microsoft or other LDAP vendors should be thinking about views for LDAP?"

OVD does views today.. I can't speak to whether other vendors should or shouldn't do that.

"OK, so what would be the real reason why the AD schema can't be
extended? Microsoft provides all the right tools. Is it stupid thinking
in large enterprises?"

I don't disagree with the fact that Microsoft allows you to extend schema. I think there are two primary reasons why organizations don't want to extend AD schema. First is that there is not currently anyway to remove added elements. Second is the more data you add into AD, the more impact it has on AD replication. This in turn can effect desktop login performance. And thus organizations try to keep AD lean. I'm not saying you can't or shouldn't extend AD schema - I'm just pointing out that in many organizations - this is not allowed.

OK, I guess I am now officially confused. I wasn't aware that the
concept of changelog was in any LDAP standard. If ADAM doesn't
implement changelog, does this make ADAM non-LDAP compliant? I do know
that the use of changelog isn't just limited to provisioning tools and
many fugly ECM products also leverage (used in the negative sense) this functionality.

It's not standard in terms of any RFC - but it is standard in the fact that every other storage-based LDAP server provides one. I wouldn't say ADAM is not LDAP compliant. Lack of changelog does make it harder to integrate with systems who wish to replicate ADAM data. While you can use other attributes (for example modifytimestamp) the problem is that without changelog, you must reconcile the data to figure out what has changed. A changelog gives you specifically what has changed.

I guess I would ask Mark, to instead of questioning the usage of ADAM, help others outside of Oracle write better enterprise applications that bind natively to LDAP and don't require anything fugly like syncronization...

As I have expressed before - we are already doing this. OVD is one option to help with this because developers can just learn to interact with a single consistent interface instead of having to deal with different back-ends. It will be fundamental component of Oracle's "identity dial-tone" (aka Identity As A Service) concept.

We are also helping to drive a new API via Project Liberty - the Identity Attribute Service API - which is one of the Identity Governance Framework components which we hope will make it easy to use identity information on demand as developers interact with databases similar to frameworks like JDO or ADO.


 

One benefit of blogs is that you can get educated fast by your own mis-informed comments :).

Since I linked to Jackson Shaw's post, I wanted to share a quick clarification I got from a friend I have who turns out to work for Centrify and reads my blog.

"Centrify does not require any schema extensions on AD in order to integrate a non-Windows system into AD, see our FAQ http://www.centrify.com/directcontrol/faq.asp#schemaextensions.
DirectControl was designed to integrate seamlessly into Unix by
supporting the established UNIX standards you mention (�PAM, NSS and
SUDO�), as well as standards such as Kerberos and RFC 2307 assuming a
customer is using Windows 2003 R2 or installs the Microsoft R2 schema
(customers will trust Microsoft and install their schema, just not any
3rd party schema extensions). However, DirectControl can install and
operate perfectly even without schema modifications of any kind."

So to clarify all of the products in this space that I am aware of (OAS4OS, Vintella and Centrify) don't require schema extensions to AD.

Thus we'll all have to come up with different FUD to throw at each other :).

I will admit it - This post got under my skin. The summary of this post is "You can't use APEX or JDeveloper without Oracle and thus they are not really free because you must have an Oracle licensed product". And then says these can't claim to be "free" unless they work with non-Oracle technologies.

I want to take a moment to explain what "free" means here and correct some misconceptions.

For those new to these technologies - Applications Express (APEX) is an application framework that lets you build Web applications with just some PL/SQL, SQL (plus some Javscript in certain cases) and the Oracle database. It is free to use but it does require Oracle database to run (while it is part of Oracle XE - most people use it with EE or SE in production). Prior to 11g, it was a separate install but in 11g it is a default feature.

JDeveloper is Oracle's Java IDE. It is highly optimized to be used for developing with Oracle's Java related products in particular ADF (our core UI framework), Web Center (an application development framework that lets you add portal concepts (such as personalization and content from multiple sources on a single screen) to any Web applications and SOA Suite (and SOA is much more design intensive than it is code intensive thus the need for a good design tool).

Since APEX is really a way to simplify taking your existing PL/SQL knowledge and use it to build Web applications - this is clearly something that would only work against Oracle DB. While there maybe some other databases out there that might run some PL/SQL - let's just be truthful to each other an admit that people who want APEX want to run that against Oracle DB. If you are running another DB - you're more likely to go reach for another tool - probably any tool that doesn't say "ORACLE".

That being said - PHP/Ruby/Perl/.NET all run well against Oracle DB :).

JDeveloper on the other hand is a different context. While it is not open-source it truly is free to use. And while it does let you edit any Java code - frankly if you are not using any of the Oracle related Java stack - I'd probably run Eclipse (and Oracle is also active in Eclipse project). It's not that JDev wouldn't work - but you'd probably have an eaiser time with non-Oracle stack.

On the other hand - if you want to build Web-applications running against an Oracle DB - there really isn't any faster way that I know than ADF. And with the new stuff coming in 11g ADF - you will be able to do many AJAX functions without touching Javascript.

And ADF itself is based on a standard (JSF) and is donated to the Apache MyFaces project.

Finally - you probably are not looking at the Web Center and SOA Suite components unless you planning to use those products..

So in summary - You are probably only interested in APEX or Jdeveloper because you are investing in Oracle technologies (DB and/or Middleware). If you are investing in these technologies - then these tools exist to help you more rapidly develop against this stack but they are not the only tools you can use.

And there is also a rich external market out there - both commercial and open-source tool/frameworks that let you develop against Oracle too. And I hear a rumor or two that there are even other databases :).



 

James McGovern responded back with some more questions based on mine and Shaw's feedback:

Here are the questions and my responses:

1. Do you think the open source community will quickly step up and put the
equivalent of Vintela into Linux? Are there vendors that should help
this along?

There are already tools to do this in open-source - in fact we just leveraged the tools that are there for OAS4OS. Fundamentally OAS4OS is designed to maximize investment(s) in current technology. The "magic sauce" in the current release is that we helped simplify the configuration of a secure setup.


2. What would you think if there was a way for PAM to talk with an XACML PDP?

I don't think this would make much sense. PAM only deals with authentication. It would make more sense for something like SUDO to leverage XACML.

3. Would you find it interesting if you could log into a Solaris server by using your OpenID or CardSpace?

Sure it would be interesting. However, those of us who find changing how we login to our host systems interesting - did this already in the 90s :). And if you wanted strong auth or SSO - you implement(ed) Kerberos. What is driving many customers to look for a Unix integration like OAAS4OS is compliance requirements. Their company needs to comply with Sarbanes-Oxley (SOX) or similar rules and locally managed passwords for priviliged accounts don't meet those rules (in particular for audit). NIS doesn't meet the requirements either. Managing the data via LDAP does.

Frankly I am a bit surprised James has never mentioned meeting compliance in any of his posts. Surely the challenge of meeting regulatory compliance in a financial company like his employer must be a major challenge? How does he solve these things....

4. Not all applications are PAM-enabled. I think FTP is one of them. So, how should it work in the world?

PAM is purely a back-end system that is abstracted from the application. If the application makes a call to the UNIX (ok, probably POSIX) API login function - Unix/Linux will call the proper PAM module(s). I am sure there is a FTP server out there that can make use of PAM.

That being said - almost nobody uses authenticated FTP for file transfer. Instead Secure Shell (SSH) Copy (SCP) is used because it's much more secure. And SSH can use PAM.


5.
Isn't the concept of logging into a server somewhat dated? Shouldn't
the notion of domain be escalated within Linux community?

Has the enterprise architect forgotten his roots? :)  Has he not talked to his actual system admins?

While it is true that most non-system administrator accounts no longer login directly to machines - system administrators must still routinely be able to login to systems (regardless of OS).

Unix systems (in particular Trusted Solaris and SE Linux) have much more stricter/fine grained controls they can employ in their security models within the system than mere "domains".

And the whole concept of domains in Windows - is lifted from Unix to begin with. So I'm not really sure what James is trying to ask here.

If the question is "why are they entering passwords - can't they use Kerberos" - the answer is yes of course. However, that just isn't something many organizations have wanted to implement even though Microsoft AD practically made Kerberos universal (something the Unix world was never able to do even with a several year headstart).

A regular question I get is "does OVD support SQL". And I'm sure that most of the other directory vendors get this too.

This is because either their own development experience and/or their staff developer experience has been heavily tied to SQL. This does not mean though that their developers know SQL - most often their tools/frameworks know SQL.

Here is why directory services don't support SQL directly. The reason is that SQL is not a protocol - it's a query language designed to be used by relational databases. Every relational database must provide its own client-server protocol to enable database clients to connect to the database. This is why abstraction API (JDBC, ODBC, Perl-DBI) were created. They allowed developers to do standard database operations (connect to database, do some SQL, process results, close connection) without needing to learn every single database API out there.

And since most applications use objects and not SQL itself there are
even higher level frameworks like Toplink and Hibernate that abstract
this even further.

LDAP on the other hand - specified both a client/server protocol as well as its own data access language (plus even standard object/record types - aka standard LDAP schema).

To be candid - of course an LDAP server could offer a database protocol and SQL interface to its sources. However, most LDAP implementations did not use relational databases as their storage, thus it wasn't practical. While OID does use Oracle database for storage - it's an optimized schema for LDAP not designed for general purpose relational access.

This doesn't mean there have not been attempts to help solve this quandary. One option for Java developers is to use something like JDBC-LDAP. JDBC-LDAP is a JDBC driver that converts SQL calls into LDAP. We created it at OctetString and then donated it OpenLDAP a few years ago.

While it sounds like a good idea (querying LDAP via JDBC-LDAP) - the fact is that it doesn't usually work well in real-life.

There are a few common reasons for this:

• Clients still must be rather directory centric and thus benefit more from object abstraction using native APIs like JNDI or in certain situations a custom web service (yes, OVD can speak SOAP (and DSML and even JSON) as well as LDAP)
  
• Most applications that want to use LDAP want it for authentication & authorization - which frameworks like JAAS, Oracle Access Manager (or CA Siteminder or Tivoli ), Apache modules externalize from applications
  
• The database may provide a native mechanism to retrieve data from an LDAP server (Oracle Database supports DBMS_LDAP which can be used to query data from LDAP). Thus you can use your standard database tools to connect to Oracle and show LDAP in your database (useful for Virtual Private Databases or building a white pages service in APEX)
• Future of identity access lies in identity services and thus will be commonly accessed by developers similar to the way they interact with databases through abstraction frameworks like Java Persistence Architecture (TopLink, Hibernate).
  

• OVD supports multiple client-access protocols currently those are LDAP, HTTP, DSML (1&2) and SOAP.
• SQL is just a query language and you need much more for a database application to use data than just a query language
• There are options like JDBC-LDAP or DBMS_LDAP to investigate to see if they can meet tactical problems
• While LDAP is the core identity protocol (and likely to remain that way) - the future is through Identity Services and following standards like Identity Governance Framework will prepare developers for tomorrow.

James McGovern has more questions. 

That being said - here are the questions and my answers.

1. There is a lot of talk about leveraging a virtualization layer but no discussion on when it is a better strategy to simply copy data. Most directories I have ran across aren't that big and most will fit into memory.

Answer: Virtual directories are NOT in memory copies of existing directories.

Identity Virtualization means that when clients make a request, the virtual directory figures out how to retrieve and package the data from the proper sources and then present it in proper format to the client.

Virtualization is not intended to replace replication (for example your core master sources must be replicated for high availability which you need to do regardless ).

What virtualization allows you to avoid is unnecessary synchronization of data and abstracts your applications from the sources of identity data. This gives you more flexibility and allows you to deploy applications in less time because you don't have to worry about continuing to build identity silos.

2. Mark Wilcox provided an example of virtualizing Microsoft ADAM that
was technically sound but didn't talk about the economic aspects. He
didn't mention what OVD costs but I have to assume it is pricier than
an ADAM instance. Does it make sense to spend say $30K to virtualize
something that costs say $3K?

Answer: I don't cover pricing because there is a lot of negotiation that goes into it.

Anyway ADAM is a meta-directory solution and as with any meta-directory solution - the license cost is the smallest part of the total cost.

Meta-directory means you must synchronize data, backup that data, secure that data, make it highly available by replicating that data, monitor that all of that is working and assign someone to manage that data.

I shouldn't have to remind you - that you're already doing that with your existing sources - why would anyone want to redo all of that work? Virtualization allows you to increase your return on existing investments (ROEI) because you can leverage what you already have done in new ways.

Even if the license was free (I mean you could do what ADAM does with OpenLDAP or Fedora directory and they are free) - you can see that license cost is not the true total cost.

And oh - if it turns out you deploy a new application that can use that same data but needs it in different format (e.g. they want different attribute names, different structure OR require different access rules that maybe ADAM can't support) - you have to setup that entire infrastructure all over again.

Compare that to identity virtualization which you just run as a lightweight service that transforms your data on demand. And since it can do discrete application views (Regardless if that data is stored in AD, a database, a Web Service or anything you can connect Java to) thus a single service can appear to be many. Finally because it's stateless it's much easier to manage and make highly available.

3. Enterprise Applications such as Documentum
use a syncronization paradigm for group structures and at some level
this approach is fugly. Directories such as Active Directory have the
ability to create dynamic group structures based on specifying
attributes. How should products consume dynamic group structures?
Additionally, what will cause Documentum, Alfresco and other products
that are still doing LDAP syncronization in a legacy fashion to
modernize?

Answer: I don't disagree that synchronizing groups is fugly :).

Dynamic Groups are a very nice way of simplifying the management of groups in a directory.

However, as mentioned applications can have a hard time dealing with dynamic groups.

This is why in OVD we ship with a plug-in that converts Netscape-style dynamic groups into looking & acting like static groups. And if you are using OID - it will answer membership test queries against its dynamic groups just as if it was a static group.

In terms of when will this architecture change - the same way it always has. Either they will change their product to meet this need or competing products emerge that do this and the market decides that's a compelling feature and customers move to this new architecture.

But the primary reason why this has happened is because developer's have had to be identity management experts as well as experts in their business domain.

This is why at Oracle all of our Fusion applications will leverage our Identity Services as a platform service instead of writing their own IdM code (plus any application written on Fusion Middleware will be able to take advantage of this as well). And it's why we're helping to make it easier on developers to use Identity as a platform service via standard efforts like the IGF Identity Attribute Services API.

4. If directory enabled products are still doing syncronization instead
of binding at runtime, this would lead me to believe that the community
at large needs to define best practices in creating directory enabled
enterprise applications. Who is in the best position to lead this
effort?

Answer: Instead of trying to just focus on how to best do LDAP it would be better for community to converge around something like IGF.

IGF is helping define the standards and a simpler Identity Attribute access API for developers so that they can begin to access identity in the same way they use DNS or Database.

They make the call and don't really worry about how the data is accessed or stored.

And it is protocol agnostic so that it could be LDAP today or some type of Web Service in the future.

Because IGF defines a standard way to represent the Identity object(s) needed for any application via the CARML specification that itself is also abstracted from its storage it will make it easier to map domain specific identity to an enterprise source(s) very similar to how applications are able to port their application specific data to different databases. I would also encourage everyone to keep tabs on Clayton Donley's thoughts on this.

5. Without exploring the whole legacy conversation, shouldn't it be
considered a best practice for modern applications to not even be aware
of LDAP as a protocol? If an application instead interacted with an
STS, shouldn't it pick up the attributes at runtime vs having
hard-coded binding to a directory interactions?

Answer: For current applications LDAP is the only standard and LDAP is going to be around for a long-time. Obviously in the future new standards may emerge. STS may or may not be one of them But that is the goal of IGF Identity Attributes API - applications won't need to worry about that. The Identity Attribute Service provider will handle that for them.

6. Taking this one step further, if you have XACML and you are writing
a PEP, why would your application ever need to know about LDAP?

Answer: Correct (thous if you write your application using JAAS standard today - you don't need to worry much about LDAP anyway) - if you do XACML you will be focusing on XACML. However, the information the PDP needs that your PEP talks to likely will be pulling data from at least one directory. So a PEP probably won't need to know LDAP, but the PDP will. But considering it's been over a decade since LDAP was created and applications are still just now adopting it, I would not be planning on XACML happening any faster.

7. Oracle has a wonderful product known as OctetString which allows a
LDAP directory service to work with JDBC but this is on the client.
This begs the question of whether products such as Sun One Directory
Server, OpenLDAP, Microsoft Active Directory and others should instead
figure out how to allow a SQL client to connect to the directory and
support it natively. What prevents vendors from going down this route?

Answer:  I'm glad James gave OctetString nice praise but OctetString was the name of our company that created the virtual directory before we were acquired in November 2006. The name of the product is Oracle Virtual Directory.

The SQL question is common enough, that I will answer it in a separate post to make it easier to reference in the future.

Oracle Global Support has published a Metalink article on how to use CamStudio for making screencasts to send to support. This is a great way to help report problems that are visual in nature. It also can speed up our response because we don't need to do an OWC or waste time trying to figure out what steps were done.

This process is called Demo It To Oracle or DITO.

Thanks to Laurent Schneider for pointing it out to me.