Load testing Project Server

[Siva]

My organization is currently evaluating Project Server-2003. The
process in the organization for introduction of any new technology
requires the results from performance / load tests as one of the key
inputs to the selection decision. [Ofcourse, there are functional and
other considerations as well].

Test team used a load testing tool. They are noticing considerable
memory leaks.

Has anyone used any load testing tool against Project Server? What is
your experience?

Thanks.

Siva

 

[Reid McTaggart]

Project has always had memory leaks. 2K3 is not as bad as earlier
versions... or maybe the OS handles them better.

You might ask MS about its load tests; they've done a lot. A guy I know was
involved on one and said they successfully took it up to something like
65,000 users. Don't ask me to get the results though. He's a friendly
competitor now, but a competitor nonetheless, and I wouldn't impose on him.

 

[Siva]

Thanks for the quick reply.

In fact, Microsoft has responded that the memory leak is due to the
test tool and they say they will provide their lab results. The issue
is that the selection decision would be easier with an in-houe
administered tests.

We are planning to use 5 server config.
2 Front-end servers [WSS & PS] [Load-balanced]
2 View / Session servers [Load-balanced]
1 SQL server

Do you think, it will be good enough for upto 500 users?

The concern is that when the functional testing was done with 50 users
[about 100 projects in PS], it took an average of 10 - 15 secs to open
up Project Center. Is it typical?

Siva

 

[Reid McTaggart]

10-15 seconds sounds about typical for that many projects.

Sizing is a tough thing to nail down, since there are so many factors that
have to be estimated. One of the biggies to keep in mind is your
organization's process for updating, and its implications for peak load. If
everyone must update their timesheets and projects by, say, Friday at 5PM,
then you can forget about average loads and focus on the inevitable "rush
hour" traffic.

In a laid-back organization with 500 users and 100 projects, a single server
could work just fine. But, if everyone is going to jump in at once, then
your five-box, load-balanced scheme is a must.

Have you already gone through the envisioning, needs assessment, and design
processes, so you know how the system will be used and which features will be
enabled?

You've probably already gone through Microsoft's publication "Project Server
2003 Configuration Planning Guide," which has some useful scenarios.

--
Reid McTaggart
(e-mail address removed)
Alegient, Inc., Houston
Project Server Experts
Microsoft Certified Partner

 

[Brian K - Project MVP]

Thanks for the quick reply.

In fact, Microsoft has responded that the memory leak is due to the
test tool and they say they will provide their lab results. The issue
is that the selection decision would be easier with an in-houe
administered tests.

We are planning to use 5 server config.
2 Front-end servers [WSS & PS] [Load-balanced]
2 View / Session servers [Load-balanced]
1 SQL server

Do you think, it will be good enough for upto 500 users?

The concern is that when the functional testing was done with 50 users
[about 100 projects in PS], it took an average of 10 - 15 secs to open
up Project Center. Is it typical?

Siva

There must be a problem with the test tool you were using or with the
network. I took part in some load testing at the microsoft test labs
with MANY more projects than that and MANY more users than that with
only some slight differences in configuration and we were never
anywhere neat 15 seconds to open the Project Center. I have set up
several deployments with just a single web server and single SQL server
that served way more than 500 users.

Talk with MS and check out their numbers. I witnessed several big load
tests in person and we never had performance as slow as what you were
seeing.

 

[Siva]

That's really reassuring to know your experience with one server. We
are talking to MS.

Siva

 

[Siva]

We have done a preliminary needs assessment. The objective of our
current POC evaluation is to assess and select the product. A
full-blown needs assessment & process definition are planned as part of
the next phase. [Pilot rollout].

Siva

 

[Brian K - Project MVP]

We have done a preliminary needs assessment. The objective of our
current POC evaluation is to assess and select the product. A
full-blown needs assessment & process definition are planned as part
of the next phase. [Pilot rollout].

Siva

Starting small and then rolling out to a small pilot group is also a
great way to build your own experiences about performance in your
environment and with your processes (as reid talked about). Then add
another group and see where problems are coming up and then add servers
that best address the issues you are facing. It also does a great job
of keeping the initial rollout costs low and spreading the cost of
servers out over time instead of spending $100,000 on servers up front
for a 50 person pilot group! LOL (dont laugh. I have seen it!)

 

[Siva]

Does anyone have seen any performance baselines / test results,
conducted by any independent organization for Project Server? Thanks.

 

[Brian K - Project MVP]

Does anyone have seen any performance baselines / test results,
conducted by any independent organization for Project Server? Thanks.

To my knowledge nobody outside of MS has done any serious perf testing
on Project Server. Sorry

 

[Napone]

This message thread is very interesting. I'm experience slowness with users
across a T1 wan. Project Professional 2003 takes about 2-3 minutes to open
or save. On a local network it's about 10-20 seconds. The wan issue may
point to SQL how the requests are being handled. From talking to my network
guy, any application using 3-tier architecture with sql will be slow over the
wan.

To test the performance, I use the Microsoft utility Smoke Test which is
bunch of scripts creating projects and tasks. It can run clients from local
or wan testing.

One possible solution to the wan performance issue is to setup a local
terminal server with Project Professional 2003 installed. So all
transactions are handled through lan and use wan for screen refresh. I don't
know the legality of the licensing issue but I'm investigating this.

-Napone

 

[Siva]

I can definitely use your help. As I said in the first post, the
organization won't accept any technology without validating performance
using the load testing tool. So, it has reached an impasse.
Participating in a MS lab tests is not an option. The testing has to be
done in-house. Even with a single user execution of a script using ACT
uses up a lot of memory. The memory is released 20 - 30 minutes after
the execution.

Thanks.

 

[Michael Samadi]

WAN access usually can represent an issue for client users, espcially in
cases where your WAN connection has a latency in excess of 30ms irrespective
of bandwidth.

Terminal service for those users is the best option.