Using System.IO.MemoryStream to handle relatively large zip archives

I’m fairly new to System.IO streams, and are therefore not entirely sure when and how I should use the different streams.

Let me explain my use-case:

Currently, I have a MS SQL database with a FileStream installation, in which I store FileName, Byte[] and Section for the files. i.e.

public partial class MyFiles {
    public int Id { get; set; }
    public int Section { get; set; } 
    public string FileName { get; set; }
    public byte[] Data { get; set; }
}

At some point, I want to be able to download all the files that belongs to a specific section. I therefore want to:

• Query the files specific to a section
• Write to a ZipArchive
• Pass the zipped file as a FileContentResult

I have decided to use a MemoryStream to achieve this, because its fast and convenient in the sense that I don’t have to use the filesystem on the server. The implementation looks as follows:

MemoryStream stream;
using (stream = new MemoryStream())
{
    using (var zipArchive = new ZipArchive(stream, ZipArchiveMode.Create))
    {

        foreach (MyFiles file in fetchedFiles)
        {
            var fileEntry = zipArchive.CreateEntry(file.FileName);
            using (var entryStream = fileEntry.Open())
            {
                entryStream.Write(file.Data, 0, file.Data.Length);
            }
        }
    }
}
return new SuccessResult<MemoryStream>(stream);

Everything is working, and I can successfully retreive my zipped files..

However, now I’m starting to doubt this implementation, as this possibly could end up handling files that combined can add up to say, 512MB – 1GB..

The server is really powerful, but obviously I don’t want to burn all the memory in this process.

Am I moving in a wrong direction with MemoryStream and should I ideally consider something else?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

The other thing to consider is you actually have the files in memory twice – once in fetchedFiles & then again in the zip file.

Ideally you need to stream the files from the DB to the output stream, or indeed the zip file.

Rather than get all the files and then add them to the zip, open your zip file and then stream each file into the zip file.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0