LPS:Grouper/Canc
Java System Sun Identity Manager (Sun IdM)
There is deployed Sun Java System Identity Manager (IdM) of at University of West Bohemia. Groups and memberships shall be managed by Grouper.
Data synchronization
There are several synchronization methods in Sun IdM. The two most important are Reconciliation and Active Sync.
Reconciliation: it's used for basic comparing of the data in identity management and on target resource. Synchronization goes through all identities, so most usual way is using it just for examination missing identities and performing basic responses (create, delete, unlink, ...). Checking attributes changes are possible, but quite exhaustive.
Active Sync: it's important for on-line updates. There is expected to update only identities changed on the source resource, so this synchronization type is usually used for transfer changed attributes too. Using this method requires support on the source resource - possibility to select all identities changed from the last processed synchronization.
Support for groups
IdM has some basic support for managing groups on several types of the target resources. For example directory based (LDAP, Active Directory) and unix-like resources have manageable group members via multivalue attribute of the user identity. Group memberships on resource are managed by updating this attribute.
Groups itself and their metadata are represented as special type of resource objects on target resources in Sun IdM. Creating, updating and deleting groups itself is not supported in Sun IdM as is, but it's possible by own custom workflows. More simplistic approach could treat groups the same way, as user identities, using two separated resources - one for users and other for groups. We chose the more difficult and more interesting way - using custom workflows and treat groups as special identities.
University of West Bohemia environment
The distributed computing environment at the university has central administration. All passwords are stored in Kerberos system, authorization in web applications (WebAuth, SSO) is using groups in LDAP or static lists.
The systems with groups at University of West Bohemia are:
- LDAP "rfc2307" - unix accounts and groups, used for autorization too
- LDAP "portal" - identities for portal, less number of groups, different schemas
- AFS - pts groups and users, updated only several selected groups
- Windows Active Directory - domain control for Windows world
LDAP directory
Two duplicated balanced OpenLDAP servers. There are two branches ("rfc2307" and "portal"), in both there are identities with reverse information about memberships too (group list for each account).
Unix group numbers
In addition we have two group types in main LDAP directory branch:
- posixGroup - with gid number
- pleiadesGroup - without gid number
Sun IdM resource adapter was extended to support our additional schema pleiadesGroup.
Custom workflow can change the schema when needed, by recreating the group.
Grouper deployment with Sun IdM
At University of West Bohemia we keep dynamic groups up to date via our import tools, synchronizing data in Grouper directly from source agendas.
Our goal on the Sun IdM side was the unidirectional way, from Grouper to Sun IdM, where Sun IdM will take care of all various target resources at university. So there relations:
- grouper "funnel" - checking out from source systems and importing to Grouper
- grouper -> IdM - read-only interface of the Grouper for Identity Mangement
- IdM -> end systems
1. Grouper funnel
We have several information systems from which we need to get information about members of these groups:
- teachers, staff, students,...
- courses students and teachers
- lessons students and teachers
- departments students, teachers, ...
- ...
Standard GrouperLoader doesn't fit for us because some systems (for example AFS groups) doesn't have database interface. Also we want fast tool which will return groups and their members in few seconds and doesn't make significant load to source information systems.
So we develop simple application which provides generic database interface and interface for developing custom classes.
You can download GrouperFunnel.zip which contains:
- sample config file
- binary distribution with required libraries
- Java sources
- shell script
Every source is fetched in separate thread. Result is stored in simple csv file. We plan to change it to more verbose XML format.
Currently we have config file with twelve sources. With optimized sql and custom classes completes fetching of all data in several seconds. With our custom provisioning we are able to accomplish near realtime distribution of changes in central systems to Grouper and consequently in Sun IdM.
Second stage of the funnel is a feeding of the gathered data to the Grouper. We implement custom importer for it, most of its logic is described in the following chapter.
2. Grouper interface
As Grouper interface for Sun IdM we chose database views.
We need some way of logging changes in groups and their memberships. It is important for integration of our Sun IdM installation with Grouper. Grouper 1.4 doesn't offer this feature (hooks may do the job) so we develop set of triggers and views which provides this functionality. We are using Oracle and Postgres, but final solution is based on Postgres (best open-source database). There is no fancy logic so it should be possible convert these triggers and use the in other RDBMS (Oracle, MSSQL, ...).
Info about changes in groups and memberships are stored in table grouper_changes which has following columns:
- operation - specify what kind of operation was made: insert, delete
- item_type - specify type of entity: group or membership
- item_id - id of entity: group_id or member_id
- item_name - name of group or member
- parent_item_id - id of parent group or stem
- parent_item_name - name of parent group or stem
- timestamp
SQL scripts
- grouper_changes.sql - sql script for table grouper_changes
- record_attribute_changes.sql - function which monitors changes in table grouper_attributes and logs info about changes in groups
- record_membership_changes.sql - function which monitors changes in table grouper_memberships and logs info about changes in memberships
- triggers.sql - sql script for triggers
For Grouper IdM resource adapter we have two views:
- grouper_idm_groups.sql - list of groups
- grouper_idm_memberships.sql - list of members
Interesting is the using of access control from Grouper here. In those views we have only groups with assigned 'IDM' access right. This way the Grouper could be used for different systems management independently on identity management.
Synchronize groups between Grouper and Sun IdM
When we will take into account removing groups too, managing users AND groups may lead to race conditions. For Sun IdM we needed to avoid to assign users to non-existing groups on target resources. Solution we chose was the adjusted order of the updates:
- process all changed (or created) existing groups
- process all users with changed memberships
- process all groups for delete
As interface for Sun IdM on the Grouper side we used databases views: one for groups and one for group memberships. Views are periodically polled by Active Sync, using timestamps in the views to get last changed identities.
On Sun IdM side we used one single source resource adapter to get both - users and groups:
| column | value | |
|---|---|---|
| users | groups | |
| accountId | login name | group internal identifier ('group.'STEM:GROUP_NAME) |
| objectClass | 'user' | 'group' |
| grpgroups | membership group names (multivalue) | group name |
| grpenabled | membership group states (multivalue) | group state |
| grpchanged | time of the membership group changes (multivalue) | time of the group metadata change |
| grpgid | - | group unix id number |
| grpstem | - | group stem in Grouper |
As seen from the table: removing groups are not deleted from the views. Just marked as disabled, with the proper timestamp in grpchanged.
Sun IdM deployment
Different groups should be propagated to the different systems. In simpler cases it's possible to classify groups to stems and via regular expressions control which one group to which targed system shall be propagated.
We have two regular expressions per target system to specify desired groups: selective (hits), suppresive (excetions). See also ZCU Rules:Group Regex.
Configuration files
- Resources/grouper.xml - configuration of the resource adapter
- Resources/actions/grouper-*.xml - resource actions in Java (using JDBC API)
- ZCU ActiveSync Grouper Handle Group.xml - group synchronization custom workflow
- other objects (ZCU Rules.xml, ...) - helper functions, configurations, ...
Realization
Basic concept:
- Grouper: "funnel" - regular checkout from source systems (Stag - study agenda, CRO - person registry), list of logins in LDAP directory
- Sun IdM: updating end systems (LDAPs, Active Directory, AFS, ...)
Original Grouper
We already had the Grouper at university before this grant. Loading data was very inefficient, with duration times about hours (see C6, S2 in the figure). End systems was updated via formless scripts from cron (see G1, G2 in the figure).
Important bindings in the figure:
- C6, S2 - conceptually OK, but very heavy and slow updates
- G1, G2 - need to be removed and replaced by Identity Management
New optimized Grouper
The result of this grant is more efficient quicker updates. Grouper "funnel" was drastically speed up, with data transfer in a minute. Information goes from Grouper through only one channel to Identity Management, that keep end systems up to date by unified way (see I4, I5 bindings in the figure). It's more easy to add new end systems with groups (planned is to extend I2 - AFS, and I13 - Active Directory).
Important bindings in new environment:
- G1, G2 - optimized Grouper "funnel"
- I4, I5 (planned I13, I14) - already existing bindings, groups and memberships update though these
Practical problems
Initial set-up
There was some practical problems during initial synchronization. Joining identities with Grouper inside Sun IdM took some time, and not joined identities has no groups in IdM point of view. So they may not be propagated to target systems.
There was running some other updates at the time... But we found the problem quite soon and solved it without stopping the time consuming initial synchronization.
Big transaction logs
There are several bigger groups (on UWB it's staff, students, and users). In OpenLDAP changing one membership in group means replace whole multivalue attribute by the new one. Sun IdM is configured to change memberships by updating identities one-by-one. This lead to very big transaction log on LDAP server.
There is space for further optimization though. We have groups in Grouper synchronized regularly each two hours. It's possible to keep members in groups (memberUid and uniqueMember attributes) independently from group lists in identities (memberof attribute). In ActiveSync synchronization process we could update big multivalue member attribute in group in advance just once, each two hours.
Summary
- optimized Grouper in production
- synchronization through central Identity Management in production
- simpler custom user interface for managing groups
- contributed to Internet2: https://spaces.internet2.edu/display/GrouperWG/University+of+West+Bohemia
