Restoring the Provisioning DB in TMS seems to be overridden when Re-Enableing TMS Agent Data Replication on the VCS

[Chris Swinney]

Hi All,

There was a posting in a thread recently about the  restoration of Provisioing information in TMS (actually it was in the  Ask the Expert thread on TMS and Exchange Replication), but now this  thread is locked.(https://supportforums.cisco.com/thread/2156753)

Andrii  left with the following question that was left unanswered.  Interestingly, I see the same issue (TMS 13.2.2, VCS x7.1, Legacy Agent)

-----------------------------------------------------------------------

Hi Tim.

I've made some testing and have another question.

First of all I made a TMS Agent backup and than I deleted a user.

Than I disabled replication on VCS Control TMS Agent;

Than I cleared the list of Replicating agents.

Than I complited the restore TMS-Agent.

After that I saw the deleted user in the Provisioning Directory. So the user was restored.

But when I enabled replication on VCS Control TMS agent the user disappeared again.

Is it correct behavior?

Andrii

-----------------------------------------------------------------------

Surely  ther must be a way to break the replication partners, restore a  provisioing DB, the re-create the partnership without the Provisioing DB  from being overwritten again?

Andrii, did you manage to work out what was going on?

Cheers

Chris

[mahkrish]

Hi Chris, The successfully completed TMSagent replication with VCS-C should have restored the deleted user on VCS as well.

There have been defects open in the past about TMSagent replication issue and it's good to upgrade to TMSPE.

Here’s a link to white paper:

http://www.cisco.com/en/US/docs/telepresence/infrastructure/tmspe/release_note/Cisco_TMSPE_migration_white_paper.pdf

BR,

Mahesh Adithiyha

[Chris Swinney]

Hey Mahesh,

TMSPE is on the cards and indeed in need to finish reading that paper. We have a complex VCS infrustructure depoloyment and my first concern is if TMSPE useing differnt ports to replicate data - I hope not, but from the very quick skim read I have the other day, I didn't see anything obvious.

Chris

[Martin Koch]

I would try to do a (ssh as root to the vcs(s) running the tms-agent) tmsagent_destroy_and_purge_data

before reenabling the replication. (And before that do a backup of the vcs system and the tms agent, just in case)

Be aware that this will kill for example the findme settings / devices which would have to be re-set!

So only handle with care.

An other thing can be that there is something not ok with that account or that it is removed as the

AD sync tells TMS so.

Cant you just re create the one user? :-)

[Chris Swinney]

Hey MArtin,

Thats pretty much what I was about to do. The VCS local find me stuff is minimal and I can easily recreate.

AD is not an issue - these are local stores.

In my case, its not just a user, it a group!

We had issue with a VCS replicating some time ago, and I wonder if it was anything to do with that. Although I have purge the data in this way before, it has been a few months and I can never rememebr the API calls, so that for that.

Chris

[Chris Swinney]

Would you believe it - the same thing happen even after a purge of the OpenDS DB on the VCS. As so as the replication is enabled, the restore group disapears!

[Martin Koch]

Sure I believe it. Not sure if the agent diagnostics are automatically started after reverting a backup.

If its not the synchronization it still can be damaged / inconsistant data which is getting purged out.

If its critical to know whats in the Groups I would look into the backup file, if I remember it right

it should contain ldif information so you should at least get some info out of it.

Also in the short time window which the data seem to exist you could use some ldap tool

to export a list of users. Search in the forum, there was a thread how to  export it to an excel file.

[Chris Swinney]

Hey Martin,

I ran the agent diagnostics on the TMS server when not conencted to any VCS - All ran OK.

When the VCS peered with the TMS, I then ran the agen diagnostic on the VCS via TMS using the Natigator section for the VCS. Again, all ran OK

The particular group contains several users. When TMS is not replicating, and the DB is retored, I can view the directory from TMS, Systems --> Provisioning --> Directory, and navigate all the users with no issues. Whilst these could be re-created, I take it that passwrod would need to be reset.

I'm not sure what further info I could get using an LDAP tool.Whilst I could export them, I still have to recreat them individually don't I?

[timwalk]

Chris,

The process that you have outlined is correct in that, I would have expected that when you restored the TMSAgent database that the deleted user would have been restored.  You are also correct in that when you then re-enable replication to the VCS that the deleted user should have been replicated back to the VCS.  The fact that is was not is an indication to me that when restoring the TMSAgent backup that only the OpenDS database is restored and the tables (TMSAgentDirectoryEntry and TMSAgentDirectoryGroup) within the tmsng database are not over written with the restore data.

This is only a theory as when replicating from TMS to the VCS I am not sure wether these tables have anything to do with it, as my understanding is that these tables are only for displaying data in the Provisioning Directory webpage.

At any rate, if this is a defect, it will most likely not be fixed as the TMSPE is now Cisco's defacto standard for Provisioning, therefore our recommendation to you would be to upgrade to the TMSPE.  Reviewing the currently available TMSPE documentation, it is not outlined exactly what ports are used for replication between the VCS and the TMSPE.  Although, in the TMSPE now the VCS polls the TMSPE for updates.  That poll is over Http or Https depending on how you have configured the TMSPE.

I will open a documentation defect to have the TMSPE deployment guide have that information included.  This is important as in many environments certain ports are locked down so we need to provide an administrator with this information to assist them with deploying the TMSPE. 

[daleritc]

Tim,

The tables your referring to in the tmsng db have nothing to do with replication between the replicating partners in the TMS Agent Legacy model. But as you say, if this is some kind of anomaly within the legacy product, then your correct when you say that it won't be fixed since TMSPE is now the defacto standard for the 'VCS centered' provisioning solution.

As far as TMSPE and the word 'replication', this isn't used anymore since there is no more 'replication' taking place in the new model and ALL communication between the VCS and TMSPE is via HTTP (80) or HTTPS (443), depending on what you choose to use. And yes, when you 'flip the switch' on TMSPE and enable the TMSPE services, the VCS will come to TMSPE and grab the data. Thereafter, the 'polls' you refer to are the services (users, findme, phonebook and devices) you configured when flipping that switch' and the polling intervals are also configurable, except for devices.

So in conclusion, no need for a documentation defect since HTTP and HTTPS are the only ports that need to be opened between TMSPE and the VCS Control.

cheers,

Dale

P.S. BTW, and for everyone's info, any and all port information with regards to ports used on the TMS server (which includes TMSPE since it has to be installed to the TMS server), can be found in the Cisco TMS Install doc.

[daleritc]

Hi Tim and others reading this thread,

This thread got me to thinking about the tables in the  TMS SQL db (tmsng) that Tim mentions above. Therefore, I just want to  clarify with everyone what these tables are all about since they still  exist in the tmsng db as it relates to the new TMSPE solution.

So  in the days of TMS Agent Legacy we had a job (and still do) in TMS that  loaded user data into TMS. It was created specifically to make user  data available to TMSAE (Analytics Extension), as the OpenDS database  that the TMS Agent was using could not be queried from TMSAE.

The job that I’m referring to can be found on the  “Administrative Tools -> Activity Status” page with the title “Mirror  user directory”.  And the specific tables in tmsng are “TmsAgentDirectoryEntry” and “TmsAgentDirectoryGroup”.

Looking at our internal production server running latest  TMS and TMSPE code, this still seems to work fine, and the contents of  TmsAgentDirectoryEntry matches the contents of findme_account.

So, and going back to the original question and slightly retracting on my above post, there could be something here when it comes to the actions taking place and this job. However, and as already stated, if this is a particular issue in TMS Agent Legacy, it won't be fixed.

rgds,

Dale

[Chris Swinney]

Hi All,

Apologies for taking some time to reply.

I am looking at the tables in the TMS SQL DB (tmsng) at this very moment ([TmsAgentDirectoryEntry], and [TmsAgentDirectoryGroup]) via MS SQL Management Studio.

There is NO replication in existence between TMS and any VCS. The groups/users in the Provision Directory are as they should be, however, to get to this state I have had to restore the TMS Agent DB on TMS.

I can see an entry in the [TmsAgentDirectoryGroup] table that refers to the group in question, and in the [TmsAgentDirectoryEntry] table, I can see all the users where the [TmsAgentDirectoryEntry].[TmsAgentDirectoryGroupId] field corresponds to the [TmsAgentDirectoryGroup].[Id] field.

So the entries look correct and the relationship seems intacked.

To be sure, I also checked the Provisioning directory via TMS (TMS --> Systems --> Provisioning --> Directory). I assume this information is drawn from the TMS Agent DB rather than the "tmsng" DB? I'm not sure if there is a way to query the raw TMS Agent DB? Anyhow, all looks good.

Now we re-enable replication between one VCS and TMS (taking backups/snapshots of everything, just in case).

Once the TMS and VCS have peered and replicated, we see the the groups/users in the provisioning directory on TMS have once again disappeared.

However, on checking the entries in the TMS SQL DB (tmsng), we see that the groups and user entries still exist.

So, how does the mirroring work? Does TMS mirror from the TMS Agent DB --> SQL (tmsng)  DB, or the other way round? Does this need to be forced once  replication has been set up (currently runs once a day at around 11am).

Should the entries in the tmsng tables be deleted prior to restoring the TMS Agent DB?

Many thanks

Chris

[daleritc]

So, how does the mirroring work? Does TMS mirror from the TMS Agent  DB --> SQL (tmsng)  DB, or the other way round?

Dale says: It’s a one way operation. We clear our tables in TMS (tmsng), then copy all the data from TMS Agent Legacy/TMSPE. After the job runs, TMS should have an up to date "copy" of the data, there is no data going back to TMS Agent  Legacy/TMSPE.

Does this need to be  forced once  replication has been set up (currently runs once a day at  around 11am).

Dale says: Unfortunately you can't force the Mirror User Directory job. It's in-built job that runs once a day.

Should the entries in the tmsng tables be deleted prior to restoring the TMS Agent DB?

Dale says: You can certainly try that but as discussed earlier, when Mirror User Directory job runs, we clear the tables and then copy all the data from TMS Agent Legacy/TMSPE. Again, no data going back to TMS Agent Legacy/TMSPE.

IMO though, I think the problem exists specifically in the TMS Agent backup your restoring...meaning when you restore a TMS Agent backup, we're not only restoring the data but configuration information as well...meaning there could be something wrt the configuration in this backup that is actually causing this to occur. What would be interesting to have a completely fresh install of TMS and TMS Agent Legacy...meaning fresh TMS Agent db...and then see if you can reproduce, i.e. don't reintroduce this backup you have but create new backups, etc.

cheers,

Dale

[Chris Swinney]

Dale Ritchie wrote:
....
IMO though, I think the problem exists specifically in the TMS Agent backup your restoring...meaning when you restore a TMS Agent backup, we're not only restoring the data but configuration information as well...meaning there could be something wrt the configuration in this backup that is actually causing this to occur. What would be interesting to have a completely fresh install of TMS and TMS Agent Legacy...meaning fresh TMS Agent db...and then see if you can reproduce, i.e. don't reintroduce this backup you have but create new backups, etc.
cheers,
Dale

This is on my list of things to do. I need to upgrade from the 2003 OS to 2008, plus I will be splitting the TMS front end from the SQL backend.

However, time is not kind to me at this moment.

[Chris Swinney]

Just to finish off this thread, we had a Cisco Engineer look at the issue the other day and the response was "oh, I haven't seem that before!". After checking out the Open DS DB on TMS both after the TMS Agent DB was restored, than again after replication with the VCS was enable, they were unable to determine what was going wrong.

The solution? We'll they suggested that we move to TMSPE.

We installed TMSPE after the TMS Agent DB had been restored, and during the install the we migrated the TMS Agent DB setting to the TMSPE DB. This seems to have achieved the desire afect and we again have the VCSs contianing the correct provisioibng info and find me data that can be polled from TMS.

All I need to do now is figure out what good stuff this gives us