Restore of custom fields Proj Server 2007

[marie18]

Hi everyone,

In our environment we have about 20 custom fields. I tried rebuilding the
reporting db by performing a backup/restore of these custom fields. The
backup jobs went ok, three restore jobs were also ok, but the "Reporting
Database Refresh" job is in status "Waiting to be Processed (Sleeping)" for 3
hours now. This job is in queue position 2. Isn't this quite a long time for
this job to be sleeping? As I read that I shouldn't do anyhting to a sleeping
job - I haven't but still I wonder if there is anything I can do make the job
execute either successfully or not.

Any tips will be very much appreciated.

Maria

 

[Gerard Wojcik]

Check this thread

http://www.microsoft.com/communitie...2f-a2cf-2f8223222801&cat=&lang=&cr=&sloc=&p=1

 

[Barbara - Austria]

Hi Marie,

it's not too long after three hours. I had it once with 4 hours. Add job
status success to see that here is still happining a lot. There are some
jobs started. Most of them are starting a lot of others which are quite
fast. However, it still needs some time to get finished.

Good luck!
Barbara

 

[marie18]

Hi Gerard,

thanks for looking into this. As this morning my job is still with status "
Waiting to be Processed (Sleeping) ", 0% complete and Queue Position = 2 I
guess I should take some actions. I am trying to find out which is job with
Queue Position = 1 as it may be the one that is blocking the queue. I do not
see it in the "Manage Queue" section no matter what filters I apply.

Any other ways to check how to find the job which is first in the queue?

Kind regards,

Maria

 

[marie18]

Well a little update. I managed to find the job with position 1 in the queue
- is was with status - "Failed and Blocking" with date 1.1.2010 so I
cancelled it. This way my sleeping job whent to position 1 in the queue but
since then has been still Waiting to be Processed (Sleeping) with progress
0%. SHould I restart the queue service and perform the restore custom field
operation again.

Thanks

 

[Barbara - Austria]

Hi Marie,

to perform a restore a second (in my case even a third time because of my
impatience) was what MS Support told me when I had this issue.

However I suggest to have a look if there is really not happening anything
in the background by looking at the queue including 'completion state'
'success'. This jobs are really fast, but there are hundreds of them. I
never saw them in the queue, only when they had been finished successfully
after changing completion state filter.

I'll keep my fingers crossed! Good luck!
Barbara

 

[marie18]

Hi Barbara,

Thank you for the support. Could you help me with the steps to follow. Are
they:

1.Restart the queue service
2. Perform the backup of custom fields
3. Perform the restore

or

1.Cancel all currently waiting tasks
2. Perform restore of the custom fields
Not sure what sequence is correct and I don't want to mess the data.

Thank you

Maria

 

[Barbara - Austria]

Hi Maria,

I hoped that this post included: it's done ;-)

First of all: don't cancel waiting jobs!
How many jobs are waiting? What kind of jobs are waiting? When went the last
job through sucessfully (how long ago? 3 days, 1 hour, 2 min, ...) ?

Let me know, I will try to help!
Regards
Barbara

 

[marie18]

Hi Barbara,

here is a list of the "stuck" jobs:

Position = 1, Reporting Database Refresh, Status = "Waiting to be Processed
(Sleeping)", Queue Entry time= "20.4.2010 г. 15:11 ", 0 % complete
Position = 2, Reporting (Resource Sync), Status = "Waiting to be Processed
(Sleeping)", Queue Entry time= "20.4.2010 г. 15:22", 0 % complete
Position = 3, Reporting (Resource Sync), Status = "Waiting to be
Processed", Queue Entry time= "20.4.2010 г. 01:00", 0 % complete
Position = 4, Reporting (Resource Sync), Status = "Waiting to be
Processed", Queue Entry time= "20.4.2010 г. 15:27", 0 % complete
Position = 5, Reporting (Resources Capacity Range Sync), Status = "Waiting
to be Processed", Queue Entry time= "21.4.2010 г. 15:27", 0 % complete

Kind regards,

Maria

 

[Barbara - Austria]

Hi Maria,

OK, you started your DB refresh yesterday afternoon. I think you have sth
like AD sync scheduled at night (1:00)? Could you please have another look
at your queue when the last job was finished successfully (completion state
= Success)?

 

[marie18]

Hi Barbara,

the last successful job completed on:21.4.2010 г. 10:46 .

Kind regards,

Maria

 

[Barbara - Austria]

Hi Maria,

so really nothing happens currently. Rebuild for reporting other than
resources went also through without any issues. Could you please verify that
by creating a project without resources and publishing. Does report job for
that project finish without errors? If that is the case, go to Server
Settings - Cube - Configuration and deselect all resource fields (resource
and assignment entity) from dimensions. Give the system a few minutes and
check if your changes finished without errors and got your refresh jobs
working (by checking successful jobs - they themselves will stay sleeping
for some more time).

If that didn't help, restart restore.

Regards
Barbara

Go to cube configuration and delete

 

[marie18]

Well it looks like I have to go for the restart . One last question -
should I cancel the waiting jobs first before the restart?

Maria

 

[Barbara - Austria]

To be honest, I am not sure. When I had this issue, I cancelled it. But I
can't tell you if that was the reason to do it for a third time ;-).
However, it worked fine.

Let me know how you move on!

 

[marie18]

Realy thank you Barbara.

Ok. I cancelled the jobs, restarted the service, restored the custom filelds
again. Now I get status for the "Reporting Database Refresh" job "Failed but
not blocking correlation"

And here are the errors from this failure:

<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="85b7409e-bc0d-4444-a484-331f35001f2f" QueueMessageBody="A RDS refresh
message in the Epm area of the reporting database attempted to execute while
another refresh was active in the Epm area." Error="RDB area: Epm, error
mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="500c9154-d286-469f-affd-8d85184a3eb8" QueueMessageBody="Unhandled EPM
stage when executing RepopulateEpmArea: PurgeDatabase" Error="RDB area: Epm,
error mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="0021a276-6c1e-4dff-abc4-5c2c632f2564" QueueMessageBody="Unhandled EPM
stage when executing RepopulateEpmArea: PurgeDatabase" Error="RDB area: Epm,
error mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="9748d0cd-1e7b-462d-b371-5e326177570b" QueueMessageBody="Unhandled EPM
stage when executing RepopulateEpmArea: PurgeDatabase" Error="RDB area: Epm,
error mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="03804eed-edd0-4391-84d8-4d832d7283c7" QueueMessageBody="Unhandled EPM
stage when executing RepopulateEpmArea: PurgeDatabase" Error="RDB area: Epm,
error mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="1326f3c2-2935-4793-9d4a-a78b33e1a512" QueueMessageBody="Unhandled EPM
stage when executing RepopulateEpmArea: PurgeDatabase" Error="RDB area: Epm,
error mode: ContinueOnErrors, lock RDB on errors: False, refresh sleep time:
00:05:00" />

Any advice what to do next?

Maria

 

[Barbara - Austria]

A third time (I wrote, that I needed to do after cancelling). But give the
system some time before.

Seems that the background jobs had still been active. Are there any other
activities on your Project Server? Did any jobs went through without errors
in the meantime?

Just found that I didn't send .... Sorry!

 

[marie18]

Hi, yes other jobs went just fine. I really don't understand why a cancelled
job is still executing after the restart of the queue serice...

I gave it a try a again now the job "Reporting Database Refresh" is again in
state: "Waiting to be Processed (Sleeping)"... I am planning to wait some
hours hopefully it will refresh this time. If not I am planning to uninstall
the server .

 

[Barbara - Austria]

Hi Maria,

ok, it is started! That's great - there will be no need to re-install, only
some patience. "Waiting to be processed" is quite irritating at this stage,
I know. Currently a lot of other jobs are started, one after another and the
sleeping one is waiting untill all of them are finished. These other jobs
are quite small and really fast, so you will hardly find them in the queue
while they are processed. I could only see them looking at successful
completed jobs - and there are hundreds. Give them their time, have a
coffee, do something different, go home, ..... Depending on number of
projects, resources, enterprise fields, .... it can take some hours.

I cross my fingers and look forward to a positive answer (tomorrow, since I
have to leave now)

Regards
Barbara

 

[marie18]

Hi Barbara,

thank you for the support ). Well I did leave the job to do its work and
this time I got "Failed But Not Blocking Correlation" with the following
error:

"<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="d0764ef2-14fe-42e1-8aea-7133b1612530" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="49795c1e-4bd5-4391-b5f3-c6c5f989c4cc" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="239d39bd-f493-4191-83b9-e9b4212b1e98" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="9f23f14d-0d7e-447b-a146-f6163d6e4542" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="5e356dfa-76f0-43fc-833c-c789340fd859" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
<error id="24023" name="ReportingRDBRefreshMessageFailed"
uid="960ff928-58b1-454e-8cbd-8b95781144be" QueueMessageBody="One of the
stages of the Refresh operation failed" Error="RDB area: Epm, error mode:
ContinueOnErrors, lock RDB on errors: False, refresh sleep time: 00:05:00" />
"
So now I am gonna give it another try

Maria

 

[Barbara - Austria]

Good morning,

seems you are sitting in a time zone next to me. Good luck!

Regards
Barbara
P.s.: Can you find something, why it failed in eventlog of ULS?