View Notification Server constipation

[Steve]

The VNS server appears to be taking hours and days to process our xml files.
We are getting so congestted that users cannot see their assignments.

Here is what we do know:
* The appearance of the XML file is not a true representation of what is
being processed - the xmls may be deleted a whjile after completion of the
publish
* we've told the admins/pm to not publish resglobal, master projects, or
cube build for the time being.
* we have over 700 xmls built up spanning over 30 hours.
* The VNS server is at about 155M of memory and has as much as 3M page
faults per minute. we have over 315M faults since last week.
* only one CPU shows activity- as expected for a single threaded app.

How can we get more insight on what is actually going on in the service and
if any tuning is available?

Thanks

 

[Gary L. Chefetz [MVP]]

Steve:

We often see usage habits as causative factors when we hear about symptoms
like this. Are your users enabling a combination of autosave and publish on
every save in their client applications? Is the volume of XML drops
unexpectedly high? We couldn't know this unless you described your system
metrics in depth.

Are you seeing errors in the application event log? Are certain projects
taking inordinately long to process? Are your projects extremely large?
Could file corruption be part of the cause? You can move all of your xml
backup to a different folder and examine them. You can drop them in one by
one and watch what happens.

Does this give you some ideas on how to start investigating?

 

[Steve]

Thanks for your ideas - I think we've tried all of these.

We have instructed users to minimize their publishing while changing plans.
This helps. But with over 500 plans, weekly time approvals and updates, we
have backlogs of up to 24 hours. We also need to build our cube manually to
avoid resource contention.

There are no errors in the application event log, and only a few in the
system log when we restart the service (7031, 7032, 7035, 7046).

We have been moving xml files to another folder and feeding them 1 at a
time, but of course PMs start adding to the VIEWDROP in the course of the
day. And with over 500 project updates on Monday's that means dedicating a
resource to excessive manual work.

Also, if VNS has begun processing files removed from the folder, doesn't the
MSP_VIEWS_SERVICE_DELETE_PENDING table fill up with stranded files no longer
being processed?

Finally, do you know how the VNS table function
(MSP_VIEWS_SERVICE_DELETE_PENDING, MSP_VIEWS_SERVICE_NEW_DROPS,
MSP_VIEWS_SERVICE_PUBLISH_STATUS)? It would be nice to know what the status
values mean, and if we can get more insight on how to monitor or even affect
the effectiveness of this part of PS2003.

Thanks in advance,

Steve Feldman

 

[Gary L. Chefetz [MVP]]

Steve:

Can you give us a rough idea of your architecture? What is the connectivity
between your VNS server and your SQL box? Is your SQL box dedicated to PS?
Are you running only one VNS instance?

 

[Steve]

Gary,

We have the following dedicated servers: 2 PS2003 application servers, 1
VNS server, 1 server for Analysis Services. we share a SQL server which
houses the Project Server db (all tables), and a separate shared SQL server
for all WSS data. The other dbs on the first SQL server is seldom used.

As far as I know all servers reside on the same backbone.

what else can I tell you?

thanks, steve