Improving File Writing speed

[FCS]

I would like to improve the speed of my code. It generates an output file
for the results in CSV format which may be very large (10-200MB). Each line
of the output file contains a record which is calculated and written
sequentially using WRITE #.
I would like to improve the speed of this process by improving the speed of
the File Output which seems to be the main bottleneck.
I thought about "writing to memory" and then dumping results in big chunks
at regular intervals could improve the speed but I am not sure how to
implement it to respect the CSV formatting that I am getting using the WRITE
# command.

Any suggestions, ideas?

 

[Tim Williams]

Care to share some code and/or timings? You don't say how long the process
takes right now, so it's difficult to imagine how much improvement could be
made.

I seem to recall comparing file creation speeds between either building up
the file as a huge string in memory and writing it in one shot, versus
writing out each line directly to the file. Writing line-by-line was
faster. Note if you want to stry building up a large string before writing
it out then you should avoid repeated concatenations and instead use a
"string builder" class to handle it.
Eg:
http://www.vbaccelerator.com/home/Vb/Code/Techniques/StringBuilder/article.asp

It's not so difficult to create a CSV-formatted line: just separate your
fields with a comma and wrap any content containing a literal comma in
quotes.

Tim