17,000 pages getting too long for stabilty?

[Ed]

I'm starting to get a text file that I need to open in Word and run macros
on. The file I got today was 17,291 pages, and it will only get longer as
time goes on. Is there a limit where Word gets unstable? This is
text-only - no tables, fields, formatting, etc. I'm using Word 2000.

Ed

 

[eDoc]

Not sure, but most likely you'll hit one soon If you do, chances are
you'll lose your data. Make sure you keep a few backups around.

Can you split it in to two, create a new doc and put two external references
in it that link doc1 and doc 2?

Just a thought.

eDoc
www.edocshare.com

 

[Ed]

That's a good thought to keep in mind. I'll probably do that. But I don't
want to set this to run and come back to find a crash because the doc itself
was too long. If it's not a problem, I won't worry.

Ed

 

[Rob Schneider]

I think you should at least monitor/think about this. Don't assume it's
not a problem and not "worry".

I don't know that it "won't" work, but it surely is large and is pushing
limits. It also depends on the quality of the macros (and their
underlying programming to ensure no memory leaks and all that) ... which
of course depends on the version of Word you are using.

You may wish to establish contact with the folks who developed these
macros and find out if they tested them for documents this large.

 

[Ed]

No problem there - I'm the one "developing" the macros!

The text file is a database report, with all of its files strung together in
one long document. It's now my job to separate this into individual docs.
Thanks to a previous code by Jay Freedman, I know (barely!) how to select
and copy to a new doc and save. Jay's code won't work with this new report,
though. So here I am ... 8>\

BTW, what's a memory leak?

Ed

 

[Rob Schneider]

Good luck.

Memory Leak: A bug/flaw in a computer program. It's when a program
fails to cleaup it's allocated memory properly. It keeps grabbing more
and more, and eventually fails. I've seen macros on older versions of
Word fail due to repeated running on large files with symptoms of memory
leaks.

Since these are indeed output from a database, I would presume the are
not in Word DOC format and in TEXT files. That's good. Simpler. If it
were me writing this program on such a large file, I'd avoid using Word
entirely. Use something better fit for purpose, where big files are "no
big deal" and at the same time get hugh performance improvement. That's
just my humble opinion though. If all you have is Word macro technology
.... use it.

 

[eDoc]

Good Point Rob!

Import it in to an app better suited for the task. Word *will* cause you a
lot of grief on that file eventually.

eDoc
www.edocshare.com

 

[Ed]

Thanks for all your input, Rob. I've just begun to learn VBA in Word and
Excel as my first forays into any type of programming. What program would
you recommend as better to handle this file? (Yes, it is .TXT as it comes
out, although I saved it as a Word .txt to avoid the file conversion dialog
box every time I opened it. I wanted it in plain text, rather than
Unicode - it looked more like what I wanted.)

Ed

 

[Rob Schneider]

Well ... now we're starting to get off topic here ... I'd probably pick
Python to do processing of text like this. Perl would be a choice of
many, but I personally find it harder to remember how to make it work.
Both are freely available for download languages and available for
Windows platform and used by many people and in production. Both enable
use of "regular expressions" which are invaluable for processing text.
Both have excellent books available in the bookstores for learning. The
ones by O'Reilly are the best, IMHO. O'Reilly's book on Regular
Expressions really helped me for a recent project. Visit the web sites,
and take a look at O'Reilly's books.

 

[Jay Freedman]

Hi, Ed,

Seeing that my name got embedded in this thread...

I agree that Word isn't the best tool for this job. I don't *know* that Word
will crash on it, and the theoretical limit is 32MB of text per document,
but I'm sure you can get both faster and more stable code in other
languages. In addition to Python and Perl, you might look at PHP (see
http://us3.php.net/manual/en/intro-whatcando.php).

 

[Rob Schneider]

Should have mentioned ...

Python: www.python.org
Perl: www.perl.org

 

[Robin Clay]

The text file is a database report, with all of its files

strung together in one long document. It's now my job
to separate this into individual docs.

PMFJI, but....

Surely, all you need is a VBA SubRoutine
that will do that for you -

Open a new document file
Open the *.txt file
DO
Read in the data line by line
IF it reaches a "Trigger",
save the document under a specified name,
and open a new document file
ELSE
Paste the line into a WORD document
END IF
LOOP until EoF
Save current document file
Close open files
Exit

This topic should really be in the "Programming"
newsgroup....

 

[Bob S]

I agree that Word isn't the best tool for this job. I don't *know* that Word
will crash on it, and the theoretical limit is 32MB of text per document,

I would not claim that Word is a good choice here, but as a point of
interest the 32MB limit seems to be a DOC file limit not a Word limit.
If have opened, manipulated, and saved an 80MB TXT file (from a
database) using Word 2002. The only oddities I noticed were that line
and page counts ran out of gas, and there were a few oddities about
scrolling. Not a comprehensive test to be sure, but interesting.

Turn off background pagination and virus checking add-ins to improve
performance from "ridiculously poor" to "tolerable if you are
desperate".

Bob S