Production experience to share

I wanted to post this in case anyone else runs into it - save you loads of
time.
Yesterday we deployed our latest release of reports - someone forgot to move
the pin on the report in source control and yet the updated stored proc for
the report went out. Net result was the updated stored proc was out there
but the report wasn't. The new stored proc was requesting a new parameter
the report couldn't supply. Normally on our dev box we get the error message
back from reporting services rather quickly (~3 seconds). On our test box it
was taking 3 minutes and production was just hanging.
We are using C# to proxy all report requests (so nobody can hit the box
directly) and the error codes being returned to C# were basically saying that
reporting services was abruptly cutting the connection. We received Denial
of Service Attach messages, Tls errors, SSL/Tls trust errors and timeout
errors. Nothing returned about missing parameters - the exception object
doesn't receive detailed info as to why the connection was killed - only that
it was killed. In order to get the details we will have to modify the code
to go into the HTML returned by the WebRequest object.
I don't know why the report was hanging for as long as it was- I went into
the IIS logs to see what was being requested (fortunately we have to use URL
requests in our case) and when picking through the requests I noticed that it
wasn't passing in a required parameter. (The report didn't request it, so it
didn't pass it). When I recreated the request through report manager I
noticed that it wasn't using all of the parameters expected.
In our attempts we upped the MaxReqPerUser property in RSReportServer.config
file from default (20) to 140 as a guess - it helped for about 5 minutes,
then everybody crashed again.
Microsoft - hopefully the new RS release will interrogate the stored proc at
runtime and return results quicker? Hopefully error integration will be
better with C# next year? I doubt it b/c there may not be a good way to pass
errors to the exception object.
I am sharing this b/c I was on the phone with MS premier support ($$)
yesterday for 7 hours and they offered no creative ideas or really anything
of signifigance. Please add this to your knowledge base so the next guy
might avoid 12 hours of headache.
Any comments are welcomed!
John Hennesey
JohnHennesey82:hotmail.comThanks for sharing...I'll have to remember this one...
"JohnHennesey" <JohnHennesey@.discussions.microsoft.com> wrote in message
news:C285281B-4230-4114-8B3A-E303CB2FADD9@.microsoft.com...
>I wanted to post this in case anyone else runs into it - save you loads of
> time.
> Yesterday we deployed our latest release of reports - someone forgot to
> move
> the pin on the report in source control and yet the updated stored proc
> for
> the report went out. Net result was the updated stored proc was out there
> but the report wasn't. The new stored proc was requesting a new
> parameter
> the report couldn't supply. Normally on our dev box we get the error
> message
> back from reporting services rather quickly (~3 seconds). On our test box
> it
> was taking 3 minutes and production was just hanging.
> We are using C# to proxy all report requests (so nobody can hit the box
> directly) and the error codes being returned to C# were basically saying
> that
> reporting services was abruptly cutting the connection. We received
> Denial
> of Service Attach messages, Tls errors, SSL/Tls trust errors and timeout
> errors. Nothing returned about missing parameters - the exception object
> doesn't receive detailed info as to why the connection was killed - only
> that
> it was killed. In order to get the details we will have to modify the
> code
> to go into the HTML returned by the WebRequest object.
> I don't know why the report was hanging for as long as it was- I went into
> the IIS logs to see what was being requested (fortunately we have to use
> URL
> requests in our case) and when picking through the requests I noticed that
> it
> wasn't passing in a required parameter. (The report didn't request it, so
> it
> didn't pass it). When I recreated the request through report manager I
> noticed that it wasn't using all of the parameters expected.
> In our attempts we upped the MaxReqPerUser property in
> RSReportServer.config
> file from default (20) to 140 as a guess - it helped for about 5 minutes,
> then everybody crashed again.
> Microsoft - hopefully the new RS release will interrogate the stored proc
> at
> runtime and return results quicker? Hopefully error integration will be
> better with C# next year? I doubt it b/c there may not be a good way to
> pass
> errors to the exception object.
> I am sharing this b/c I was on the phone with MS premier support ($$)
> yesterday for 7 hours and they offered no creative ideas or really
> anything
> of signifigance. Please add this to your knowledge base so the next guy
> might avoid 12 hours of headache.
> Any comments are welcomed!
> John Hennesey
> JohnHennesey82:hotmail.com