Login and contact issues (Windows Live Messenger Team)

Good morning everyone,

First, I would like to apologize for the gap in coverage of the newsgroups.
As some of you undoubtedly know, LeahPear left Microsoft roughly two weeks
ago for new and challenging opportunites and the team has been scrambling to
find a replacement for her. Combine this with the very real issues being
discussed here and...well, we've been focussing on getting these issues
fixed. And, for the most part, they are. Explanations on that to come in a
little bit.

Second, I would like to introduce myself to the group. I'm Steve-C,
LeahPerl's replacement program manager and your voice (that of the customer)
back to the Messenger Team. I'm the guy who will be gathering all the
customer feedback (good and bad) and making sense out of it and driving your
wishes back to the product planning, design and development staff. Despite
what some might think in an understandble moment of frustration, Microsoft
*IS* listening and truly wants to create the most awesome and intuitively
usable messaging product imaginable. I am dedicated to that role but to do
that, we need your help and feedback.

That being said, I'd like to explain the issues we've been having. With a
very few exceptions affecting a tiny percent of our customer base, we have
the problems fixed. These were tough issues to find and coordinate the fixes
for because the Messenger code itself wasn't the source of the sign in and
contact retrieval/update problems the community experienced.

What happened was that a key back end system which provides the
ultra-high-speed storage and retrieval of contact information upgraded their
system late last week. This upgrade coincidentally happened at the same time
as a routine, monthly security key change (done to protect sensitive info
like login and contact info) and a windows update. Despite rumors, the
windows update had no impact on this -- please install those updates! They
are a good thing.

The other mods did affect each other. The mechanism which the Messenger
code exchanges information with the contact info system is called a
"handshake" and it consists of a series of back and forth messages to
exchange credentials, authenticate that a given client is allowed to read
specific information and then to retrieve and verify the retrieval of that
information. The language of this handshake is called a "policy".

What happened was the back end changed their policy! This disconnect
occurred for a period of about three hours late last week. Yes, this breaks
the cardinal rule of "don't tweak or update your stuff between Friday and
Monday" but, alas, it happened.

The policy was reverted shortly after it was discovered and a fix was rolled
out early yesterday and should be in place for all but a very small number of
folks who happened to log in to Messenger during that 3 hour window. For
those folks, removing that policy from the Windows Registry (the specific
steps have been posted numberous time and in numerous places) then restarting
Messenger will fix the problem. We're also altering the Messenger code
itself to detect and appropriately handle such a condition in the future.

Since the Messenger servers and systems were fine, albeit deadlocked trying
to handshake with this back end, that's why those of you who accessed the
Messenger Status sites got an "all systems are running" message.

Ok, so that's the scoop behind all the 8100314, spinning buddies, "can't log
in" and associated problems. What else happened on or about Friday the 13th?

One of the (tens of thousands of) servers in our data centers which stored
contacts also had problems. This didn't affect a lot of customers but if it
did affect you you're undoubtedly aware of that fact. The effect was that
any contacts added within about a 24 hour period were lost. Contacts stored
prior to the time of the outage were fine. Yes, the servers are redundant
but the particular nature of the problem allowed the flaky server to report
back an "all is well" status despite writing bad data to disk. Again, this
outage only affected a very small number of customers whose contacts were
stored on this particular server, one of thousands. Chances are any
individual reading this was mapped elsewhere.

Shortly after customers notified us of the problem, the server was taken
offline and replaced. We're evaluating various mods to the server redundancy
code to better detect and handle any such reoccurrance. As I understand it,
this was a very, very rare condition.

So...thank you for bearing with us during this rather star-crossed weekend.
We apologize for the outage and although it's impossible with a system this
complex and heavily used to say such a thing will *never* reoccur, know that
this particular set of problems won't.

Take care and thank you for using Windows Live and MSN Messenger.

Steve-C
Windows Live Messenger Team

Hi all,

Yes, the above post is more or less redundant with my post from last night.
I didn't see the other one go through.

Donny, I have a copy of your log and am about to take it down to the
development team. Top priority - we WILL get this figured out.

Bear with us please.

Steve

Steve i am happy to report as of 3pm EST (when i returned from an
apointement) i was able to log in.

"SteveC" wrote:

> Hi all,
>
> Yes, the above post is more or less redundant with my post from last night.
> I didn't see the other one go through.
>
> Donny, I have a copy of your log and am about to take it down to the
> development team. Top priority - we WILL get this figured out.
>
> Bear with us please.
>
> Steve

Hello Steve,

I too have the problem with the spinning buddies when using msn live 8.0,
and I've tried the reg fix posted previously by Jonathon. It didn't work.
Now I see you have something else to try.. "Messenger 8.0: delete
HKCU\Software\Microsoft\MSNMessenger\Policies" I see a key named
"CachedPolicy0" should I delete this?

The trouble is.. I don't see such a folder in the registry. I do however
see a folder by the same name when I look in HKEY_LOCAL_MACHINE
Is this maybe the Policies folder I should delete??

Maybe I'm just destined to stick with MSN 7.5?

Thanks in advance,
itsmesheri


"SteveC" wrote:

> Good morning everyone,
>
> First, I would like to apologize for the gap in coverage of the newsgroups.
> As some of you undoubtedly know, LeahPear left Microsoft roughly two weeks
> ago for new and challenging opportunites and the team has been scrambling to
> find a replacement for her. Combine this with the very real issues being
> discussed here and...well, we've been focussing on getting these issues
> fixed. And, for the most part, they are. Explanations on that to come in a
> little bit.
>
> Second, I would like to introduce myself to the group. I'm Steve-C,
> LeahPerl's replacement program manager and your voice (that of the customer)
> back to the Messenger Team. I'm the guy who will be gathering all the
> customer feedback (good and bad) and making sense out of it and driving your
> wishes back to the product planning, design and development staff. Despite
> what some might think in an understandble moment of frustration, Microsoft
> *IS* listening and truly wants to create the most awesome and intuitively
> usable messaging product imaginable. I am dedicated to that role but to do
> that, we need your help and feedback.
>
> That being said, I'd like to explain the issues we've been having. With a
> very few exceptions affecting a tiny percent of our customer base, we have
> the problems fixed. These were tough issues to find and coordinate the fixes
> for because the Messenger code itself wasn't the source of the sign in and
> contact retrieval/update problems the community experienced.
>
> What happened was that a key back end system which provides the
> ultra-high-speed storage and retrieval of contact information upgraded their
> system late last week. This upgrade coincidentally happened at the same time
> as a routine, monthly security key change (done to protect sensitive info
> like login and contact info) and a windows update. Despite rumors, the
> windows update had no impact on this -- please install those updates! They
> are a good thing.
>
> The other mods did affect each other. The mechanism which the Messenger
> code exchanges information with the contact info system is called a
> "handshake" and it consists of a series of back and forth messages to
> exchange credentials, authenticate that a given client is allowed to read
> specific information and then to retrieve and verify the retrieval of that
> information. The language of this handshake is called a "policy".
>
> What happened was the back end changed their policy! This disconnect
> occurred for a period of about three hours late last week. Yes, this breaks
> the cardinal rule of "don't tweak or update your stuff between Friday and
> Monday" but, alas, it happened.
>
> The policy was reverted shortly after it was discovered and a fix was rolled
> out early yesterday and should be in place for all but a very small number of
> folks who happened to log in to Messenger during that 3 hour window. For
> those folks, removing that policy from the Windows Registry (the specific
> steps have been posted numberous time and in numerous places) then restarting
> Messenger will fix the problem. We're also altering the Messenger code
> itself to detect and appropriately handle such a condition in the future.
>
> Since the Messenger servers and systems were fine, albeit deadlocked trying
> to handshake with this back end, that's why those of you who accessed the
> Messenger Status sites got an "all systems are running" message.
>
> Ok, so that's the scoop behind all the 8100314, spinning buddies, "can't log
> in" and associated problems. What else happened on or about Friday the 13th?
>
> One of the (tens of thousands of) servers in our data centers which stored
> contacts also had problems. This didn't affect a lot of customers but if it
> did affect you you're undoubtedly aware of that fact. The effect was that
> any contacts added within about a 24 hour period were lost. Contacts stored
> prior to the time of the outage were fine. Yes, the servers are redundant
> but the particular nature of the problem allowed the flaky server to report
> back an "all is well" status despite writing bad data to disk. Again, this
> outage only affected a very small number of customers whose contacts were
> stored on this particular server, one of thousands. Chances are any
> individual reading this was mapped elsewhere.
>
> Shortly after customers notified us of the problem, the server was taken
> offline and replaced. We're evaluating various mods to the server redundancy
> code to better detect and handle any such reoccurrance. As I understand it,
> this was a very, very rare condition.
>
> So...thank you for bearing with us during this rather star-crossed weekend.
> We apologize for the outage and although it's impossible with a system this
> complex and heavily used to say such a thing will *never* reoccur, know that
> this particular set of problems won't.
>
> Take care and thank you for using Windows Live and MSN Messenger.
>
> Steve-C
> Windows Live Messenger Team

Hi Sheri,

If you've run that .reg file, it would've deleted the Policies key already. So actually,
it's the same fix.

You can try deleting that one in LOCAL_MACHINE and see what happens. Messenger re-creates
all the registry keys it needs when it starts and logs in so it's not really doing any harm
removing anything under the \MSNMessenger hierarchy.

--
Jonathan Kay
Microsoft MVP - Windows Live Messenger/MSN Messenger/Windows Messenger
Associate Expert
http://www.microsoft.com/windowsxp/expertzone/
Messenger Resources - http://messenger.jonathankay.com
All posts unless otherwise specified are (c) 2006 Jonathan Kay.
You *must* contact me for redistribution rights.
--



"itsmesheri" <> wrote in message
news:24B3747C-C7C9-4BAE-9AA6-...
> Hello Steve,
>
> I too have the problem with the spinning buddies when using msn live 8.0,
> and I've tried the reg fix posted previously by Jonathon. It didn't work.
> Now I see you have something else to try.. "Messenger 8.0: delete
> HKCU\Software\Microsoft\MSNMessenger\Policies" I see a key named
> "CachedPolicy0" should I delete this?
>
> The trouble is.. I don't see such a folder in the registry. I do however
> see a folder by the same name when I look in HKEY_LOCAL_MACHINE
> Is this maybe the Policies folder I should delete??
>
> Maybe I'm just destined to stick with MSN 7.5?
>
> Thanks in advance,
> itsmesheri
>
>
> "SteveC" wrote:
>
>> Good morning everyone,
>>
>> First, I would like to apologize for the gap in coverage of the newsgroups.
>> As some of you undoubtedly know, LeahPear left Microsoft roughly two weeks
>> ago for new and challenging opportunites and the team has been scrambling to
>> find a replacement for her. Combine this with the very real issues being
>> discussed here and...well, we've been focussing on getting these issues
>> fixed. And, for the most part, they are. Explanations on that to come in a
>> little bit.
>>
>> Second, I would like to introduce myself to the group. I'm Steve-C,
>> LeahPerl's replacement program manager and your voice (that of the customer)
>> back to the Messenger Team. I'm the guy who will be gathering all the
>> customer feedback (good and bad) and making sense out of it and driving your
>> wishes back to the product planning, design and development staff. Despite
>> what some might think in an understandble moment of frustration, Microsoft
>> *IS* listening and truly wants to create the most awesome and intuitively
>> usable messaging product imaginable. I am dedicated to that role but to do
>> that, we need your help and feedback.
>>
>> That being said, I'd like to explain the issues we've been having. With a
>> very few exceptions affecting a tiny percent of our customer base, we have
>> the problems fixed. These were tough issues to find and coordinate the fixes
>> for because the Messenger code itself wasn't the source of the sign in and
>> contact retrieval/update problems the community experienced.
>>
>> What happened was that a key back end system which provides the
>> ultra-high-speed storage and retrieval of contact information upgraded their
>> system late last week. This upgrade coincidentally happened at the same time
>> as a routine, monthly security key change (done to protect sensitive info
>> like login and contact info) and a windows update. Despite rumors, the
>> windows update had no impact on this -- please install those updates! They
>> are a good thing.
>>
>> The other mods did affect each other. The mechanism which the Messenger
>> code exchanges information with the contact info system is called a
>> "handshake" and it consists of a series of back and forth messages to
>> exchange credentials, authenticate that a given client is allowed to read
>> specific information and then to retrieve and verify the retrieval of that
>> information. The language of this handshake is called a "policy".
>>
>> What happened was the back end changed their policy! This disconnect
>> occurred for a period of about three hours late last week. Yes, this breaks
>> the cardinal rule of "don't tweak or update your stuff between Friday and
>> Monday" but, alas, it happened.
>>
>> The policy was reverted shortly after it was discovered and a fix was rolled
>> out early yesterday and should be in place for all but a very small number of
>> folks who happened to log in to Messenger during that 3 hour window. For
>> those folks, removing that policy from the Windows Registry (the specific
>> steps have been posted numberous time and in numerous places) then restarting
>> Messenger will fix the problem. We're also altering the Messenger code
>> itself to detect and appropriately handle such a condition in the future.
>>
>> Since the Messenger servers and systems were fine, albeit deadlocked trying
>> to handshake with this back end, that's why those of you who accessed the
>> Messenger Status sites got an "all systems are running" message.
>>
>> Ok, so that's the scoop behind all the 8100314, spinning buddies, "can't log
>> in" and associated problems. What else happened on or about Friday the 13th?
>>
>> One of the (tens of thousands of) servers in our data centers which stored
>> contacts also had problems. This didn't affect a lot of customers but if it
>> did affect you you're undoubtedly aware of that fact. The effect was that
>> any contacts added within about a 24 hour period were lost. Contacts stored
>> prior to the time of the outage were fine. Yes, the servers are redundant
>> but the particular nature of the problem allowed the flaky server to report
>> back an "all is well" status despite writing bad data to disk. Again, this
>> outage only affected a very small number of customers whose contacts were
>> stored on this particular server, one of thousands. Chances are any
>> individual reading this was mapped elsewhere.
>>
>> Shortly after customers notified us of the problem, the server was taken
>> offline and replaced. We're evaluating various mods to the server redundancy
>> code to better detect and handle any such reoccurrance. As I understand it,
>> this was a very, very rare condition.
>>
>> So...thank you for bearing with us during this rather star-crossed weekend.
>> We apologize for the outage and although it's impossible with a system this
>> complex and heavily used to say such a thing will *never* reoccur, know that
>> this particular set of problems won't.
>>
>> Take care and thank you for using Windows Live and MSN Messenger.
>>
>> Steve-C
>> Windows Live Messenger Team