Virtual List View functionality in ADAM and Outlook

Discussion in 'Active Directory' started by Matt Totty (LMCO/USAF), Nov 1, 2004.

  1. I have an ADAM instance with 800-900K objects.

    It serves as the LDAP repository for all of our connected MIIS servers.
    What we have found is that it is very difficult at best to perform any type
    of bulk operation on an MIIS server with a million objects. It's a management
    nightmare.

    We are primarily using MIIS as a GAL synchronization mechanism and plan to
    use it for workflow provisioning type scenarios in the future.

    To alleviate the GAL issue and a host of others, I thought it would be
    interesting to experiment connecting directly from a client to an ADAM
    instance with all of the objects.

    To make this solution work, I need a smiliar look and feel as users
    currently get with their AD/Exchange GAL implementations.

    I have noticed that the virtual list view fucntionality doesn not work
    properly when connecting to ADAM. Every time I connect I get an unavailable
    critical extension error.

    Are there any plans to integrate the outlook client closer with ADAM and to
    include the VLV fucntionality to support this number of users?

    There are third party tools and directories that we can also play with to
    get this solution to work- and we have proven one already. But things would
    be much simpler if they ran "out of the box". Any opinions or direction is
    appreciated.

    Thanks
     
    Matt Totty (LMCO/USAF), Nov 1, 2004
    #1
    1. Advertisements

  2. There's a hotfix that enables VLV functionality in ADAM. Get 838474 or a
    later hotfix (e.g. 840901). Note you will need to create a new index
    (subtree-ized index on cn attribute).

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 1, 2004
    #2
    1. Advertisements

  3. Do you have to stop and restart ADAM for the index to take effect?
    Is there a way to monitor the indexing or does it just crunch along in the
    background?
     
    Matt Totty (LMCO/USAF), Nov 1, 2004
    #3
  4. How do you create the (subtree-ized index)? Should that be an available
    option in the ADAM schema or do you have to do this manually?
     
    Matt Totty (LMCO/USAF), Nov 1, 2004
    #4
  5. The indexing is done in the background. When the index is built, an event is
    logged.

    To create the index, you need to set the 6th bit (64) of searchFlags.
    SchemaMgmt snapin does not know about this bit, so you'll have to use LDP or
    ADSIEdit or script it.

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 2, 2004
    #5
  6. Matt Totty (LMCO/USAF)

    Lee Flight Guest

    What does this kind of index do? I had thought that the 2nd bit
    was for VLV functionality...

    Thanks
    Lee Flight
     
    Lee Flight, Nov 2, 2004
    #6
  7. The second bit (contanerized index) is useful for one-level VLV searches.
    Basically, it is the index on (parentID + attributeValue). This index is
    useless for subtree searches, the ones that Outlook does for generic LDAP
    sources.

    The new index (subtree-ized index) is on (ancestorID + attributeValue). Note
    that ancestorID has multiple values, one for each ancestor of the given
    node. This index allows to run subtree searches by fixing the ancestorID to
    the ID of the search base, and then using the corresponding index range for
    VLV window positioning.

    If there's no index that can be used for VLV, then we perform the complete
    search and dump the results into a temp table, then use it for window
    positioning. However, if the search is too large (more than 10000 entries by
    default -- defined by an ldap policy), then we fail with unavailable
    extension error.

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 2, 2004
    #7
  8. Thank you - this is outstanding information.

    The last question I have now before I get the patch and begin testing is this.

    Is there a way to create ABV type views within ADAM? It may be a stretch,
    but this is the final piece of the puzzle before I lay out my proposal. The
    end user would like to be able to drill down by AF base level.Thanks guys.
     
    Matt Totty (LMCO/USAF), Nov 3, 2004
    #8
  9. Hmm. ADAM base schema does not include showInAddressBook attribute, which
    means all special logic that exists in AD for this attribute will not work
    (even if you import it). So, you will have to implement your own solution
    based on regular LDAP queries (including VLV, if you need it).

    If you define your scenario in more detail, we might be able to generate
    some ideas.

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 3, 2004
    #9
  10. We need to be able to present the end user with a drop down view of Air Force
    bases using outlook/ADAM. Currently- the user achieves this via an Exchange
    address book view based on SMTP address in their respective Active
    Directories. If I can somehow do the same thing using Outlook/ADAM, then the
    Air Force can effectivly offload thier Global Address List to ADAM. There are
    multiple reasons why we would like to do this which I can describe in more
    detail if you wish. But the main point I am trying to make is that we need to
    achieve a similar end user experience with the end user using ADAM as he
    would by using the AD/Exchange ABV. If we can even come close that would be
    outstanding.
     
    Matt Totty (LMCO/USAF), Nov 3, 2004
    #10
  11. Ah, you are using Outlook. Then your choices are somewhat limited.

    First of all, you can only have a single "address book" entry per LDAP
    service in the dropdown. You can certainly point it at ADAM, and it will
    work just fine (after you apply the QFE and create a subtree-ized index for
    CN).

    I guess you can configure multiple LDAP services, pointing to the same ADAM
    instance, each with a different search base -- this will give you an
    illusion of multiple address books. I am not sure if you can control the
    name of the service that appears in the drop-down. The other limitation is
    that you can not have one user belong to multiple "address books", because
    you can not have it in two different containers at the same time.

    However, if you just need to support a single GAL, then you can certainly
    push it into ADAM. Take a look at adamsync (currently in beta), that will
    help you to bring data from AD into ADAM and keep it in sync. Alternatively,
    you can use MIIS to do the syncing. IIFP (MIIS-lite) is free -- it will work
    for AD-ADAM synchronization.

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 3, 2004
    #11
  12. Matt Totty (LMCO/USAF)

    Lee Flight Guest

    Following from Dmitri, you can change the name of the service that
    appears in the drop-down. I currently have Outlook 2003 running on
    WinXP with three LDAP address books LDAP1,LDAP2 and LDAP3
    in the drop-down, respectively pointing to:

    an ou in an application partition in an ADAM instance

    a container in a different application partition in the same ADAM instance

    an ou in an application partition in a different ADAM instance, this one
    is ADAMsync'ed from AD

    all instances are on the same WinXP box

    A couple of other things to think about are:

    what objects you will have in the address list, contacts will be fine
    but you will have problems, I think, if you need to handle Distribution
    Lists that might exist in your Exchange GAL

    what credentials that you will be using to bind to the ADAM instance,
    I'm presuming that the outloook clients are domain based. I just wondering
    if there is a way of using permissions for the binding user to return the
    appropriate "view" (if you are using restricted views in that way).


    Lee Flight
     
    Lee Flight, Nov 3, 2004
    #12
  13. Thank you gentlemen.

    Dmitri- we do currently use MIIS to bring all of our entries into ADAM from
    multiple AD's. I am trying to OFFLOAD the GAL to ADAM so that each instance
    of MIIS (15 severs curently) does not have to worry about crunching through 1
    million entries. We currently do a push and pull- and I want to do pulls
    only- to ADAM and have the end user hit ADAM for GAL purposes. In our case it
    greatly reduces a lot of overhead. Overhead from an MIIS point of view and
    also from an AD point of view. Today a million objects- tomorrow millions
    more as the Air Force will have a need to integrate other services- NAVY,
    Army, into their Global. I see this coming and am trying to come up with a
    solution that is going to keep us from failing. Have you ever synchronized a
    million entries in MIIS? ;)

    Lee- we only pull contact information- we exclude distribution lists.

    So something to consider from a technical standpoint- I do not want to even
    go off on a cost and development tanget- is to somehow integrate features
    into Outlook/ADAM that will enable Address Book View functionality. Is this
    something that is just too wild to consider? It seems on a high level that
    since ADAM and AD are closely married this would be something that is at
    least achievable from a technical standpoint. I am not of the school that you
    would even want to have a view by base since I can type in a last name and
    have the entries appear. But this is the military and they demand it.

    As far as LDAP instances go- We have close to 108 bases- so that is not
    going to be a solution.

    For the tactical untis ( people fighting the wars) in the middle east and
    elsewhere- they are going to love the fact that they can access the GAL via
    ADAM.
     
    Matt Totty (LMCO/USAF), Nov 4, 2004
    #13
  14. Thank you gentlemen.

    Dmitri- we do currently use MIIS to bring all of our entries into ADAM from
    multiple AD's. I am trying to OFFLOAD the GAL to ADAM so that each instance
    of MIIS (15 severs curently) does not have to worry about crunching through 1
    million entries. We currently do a push and pull- and I want to do pulls
    only- to ADAM and have the end user hit ADAM for GAL purposes. In our case it
    greatly reduces a lot of overhead. Overhead from an MIIS point of view and
    also from an AD point of view. Today a million objects- tomorrow millions
    more as the Air Force will have a need to integrate other services- NAVY,
    Army, into their Global. I see this coming and am trying to come up with a
    solution that is going to keep us from failing. Have you ever synchronized a
    million entries in MIIS? ;)

    Lee- we only pull contact information- we exclude distribution lists.

    So something to consider from a technical standpoint- I do not want to even
    go off on a cost and development tanget- is to somehow integrate features
    into Outlook/ADAM that will enable Address Book View functionality. Is this
    something that is just too wild to consider? It seems on a high level that
    since ADAM and AD are closely married this would be something that is at
    least achievable from a technical standpoint. I am not of the school that you
    would even want to have a view by base since I can type in a last name and
    have the entries appear. But this is the military and they demand it.

    As far as LDAP instances go- We have close to 108 bases- so that is not
    going to be a solution.

    For the tactical untis ( people fighting the wars) in the middle east and
    elsewhere- they are going to love the fact that they can access the GAL via
    ADAM.
     
    Matt Totty (LMCO/USAF), Nov 4, 2004
    #14
  15. If people from each base live in a separate OU, then you can get away with
    setting up multiple LDAP sources, each with a different search base, like
    Lee described. Plus you can have one global list that includes everybody.

    If you have people that need to appear in multiple address books, then we
    might be able to craft something up for you, using showInAB attribute.
    However, it will also require changes to Outlook, because it is currently
    sends a generic filter (&(cn=*)(mail=*)) for generic LDAP sources. You'd
    need it to pass a different filter, similar to the one it uses for AD,
    involving (showInAB=AB-name) clause.



    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm
     
    Dmitri Gavrilov [MSFT], Nov 4, 2004
    #15
  16. Matt Totty (LMCO/USAF)

    Lee Flight Guest

    Re: Outlook changes.

    I think a QFE for Outlook (looks like EMABLT32.DLL on my machine)
    that allowed a filter to be specified, defaulting to (&(cn=*)(mail=*)),
    in addition to the search base would be a valuable enhancement.

    Lee Flight

     
    Lee Flight, Nov 4, 2004
    #16
  17. Matt Totty (LMCO/USAF)

    Lee Flight Guest

    Matt,
    if you are seeking feedback/comparison on your MIIS performance
    experiences you might want to try on the MMSUG forum on yahoo
    groups if you have not already done so. All the MIIS experts seem
    to be there.

    Lee Flight
     
    Lee Flight, Nov 4, 2004
    #17
  18. Yeah, I know the guy who owns this stuff. But he's in the Office team, too
    far away in the hierarchy :)
    Valuable suggestion, surely, but no promises.

    The general problem with patching Outlook is that they have too many
    versions, and it affects too many clients. In most cases, it is easier to
    just patch the server (and this is exactly what we did for the recent VLV
    fix for AD).

    Unfortunately, in this case, I am afraid it is impossible to avoid patching
    outlook. I am just hoping Matt can get away with different OUs.

    --
    Dmitri Gavrilov
    SDE, Active Directory Core

    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at
    http://www.microsoft.com/info/cpyright.htm

     
    Dmitri Gavrilov [MSFT], Nov 4, 2004
    #18
  19. Matt Totty (LMCO/USAF)

    Lee Flight Guest

    Talking of distribuing to clients, another problem here is how
    to distribute the LDAP sources to the Outlook clients. A preliminary
    dig around indicates to me that they are stored in the user's Outlook
    profile, i.e. in HKCU so that will be fun to try and distribute :(

    There seems to be an interesting challenge here to build a pure LDAP
    analog of the ABV render by Outlook as derived from the Address List
    object and purportedSearch attribute of the Exchange extensions to the
    AD schema. Maybe this says the project needs a bespoke address book
    application that can integrate with Outlook...

    Lee Flight

     
    Lee Flight, Nov 5, 2004
    #19
  20. Matt Totty (LMCO/USAF)

    Al Mulnick Guest

    Maybe I missed something here. Isn't the end solution to get something
    similar to what the old 5.5 site boundaries gave in the GAL or what is
    achieved with ABV's? Ultimately so users can find people in a certain and
    familiar fashion?

    I wonder if there isn't a more creative idea in Exchange that can be done
    vs. duplicating data to multiple data stores and worrying about
    configuration and versions of so many Outlook clients?

    For example, the problem at a high level is that they want to be able to
    present useful information to the people that consume it. Those people want
    to, for some reason see the users on a particular base right? Can I ask for
    further information on that? What's the reason they would want to know the
    users on a particular base in the GAL? I would *think* that users in a
    particular unit vs. an entire base would be of interest in the GAL and
    that's likely done with groups vs. views I would imagine.

    I know I'll hate myself for this, but for identification of the users on a
    particular base, why didn't you use the search by attribute function of
    Outlook for users that need to find all users on a base? What functional
    advantage do they get by using the ABV instead? I'm trying to understand
    the problem before making suggestions about the ADAM idea or any others.

    The reason to check out the ADAM ideas are to scale to the millions without
    using ABV's correct?

    Al

     
    Al Mulnick, Nov 5, 2004
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.