What's the best way to implement file compression?

Edit: Clarification after answer: I chose Lucas's answer as the correct answer for this question. DotNetZip, when implemented with the I18N.dlls works like a champ in the Editor and built .exes. My specific implementation details are located in the "Potential Answer: DotNetZip" answer below.*

Edit 1/30/2011: Just a quick note to say that I have been using the DotNetZip solution in 3.x without any issues.

Using the System.IO.Compression.GZipStream classes results in the dreaded DllNotFoundException: MonoPosixHelper exception (http://answers.unity3d.com/questions/3147/dllnotfoundexception-monoposixhelpersystem-io-compression-deflatestream), so until Unity 3.0 comes out and the later version of Mono it ships with supports a fully managed compression scheme, we have to rely on a third party compression library.

There seem to be two libraries that might fit the bill. SharpZipLib (http://www.icsharpcode.net/opensource/sharpziplib/) is a venerable tool for zip compression, but the API is painful to use.

Some have reported that DotNetZip (http://dotnetzip.codeplex.com) works with Unity. I keep receiving an "IBM437 codepage not supported" (or similar) error when trying to work with this library. It is a much better library than SharpZipLib from a usability perspective.

There is a mono compiled version of SharpZipLib that ships with Unity (as discussed in this forum post http://forum.unity3d.com/viewtopic.php?t=10699&highlight=sharpziplib), which seems like a possibility. However, I am hesitant to use an older version of the library.

Should we just wait and hope? Can someone confirm if later versions of Mono will support this out of the box? Or will we have to roll our own in the 3.0 time frame, too?

Nobody said that Unity3 will ship with a mono that has a fully managed GZipStream implementation. In fact, being the guy that is working on this currently, I can tell you that that would be extremely unlikely. So yes, also for the Unity3.0 timeframe, you'll have to rely on a managed implementation.

I went and tried the 3rd party library you are having trouble with: DotNetZip. I downloaded their distribution package, and grabbed Ionic.Zip.dll from it, and placed it into the Assets folder of an empty Unity2.6.1 project.

I then wrote a quick testscript:

using UnityEngine;
using Ionic.Zip;

public class NewBehaviourScript : MonoBehaviour {
    // Use this for initialization
    void Start () {
        using (ZipFile zip = new ZipFile())
        {
             // add this map file into the "images" directory in the zip archive
             zip.AddFile("c:\\mono.patch", "images");
             // add the report into a different directory in the archive
             zip.Save("MyZipFile.zip");
        }
    }
}

(copied almost verbatim from their website). c:\mono.patch is a file that happened to live in my c:\ drive. When I ran this script, it succesfully wrote MyZipFile.zip.

Naturally the .AddFile() won't work from a webplayer, but this library has entrypoints for creating new files in the zipfile from a stream as well.

If this does not work for you, please describe the exact steps you're taking, and the exact error you get.

Have you tried looking at 7Zip? Has a completely managed C# library that I know works in Unity. It gives you compress/uncompress to and from memory streams/byte arrays, in LZMA format, so it's better than zip and actually what Unity uses to compress assets.

http://www.7-zip.org/sdk.html

If you need something compatible with .zip format you might have some trouble.

The Answer: DotNetZip

I know it's bad juju to use an answer to post a comment, but I need to show some detail. Please accept my apologies.

Here are the implementation details that were answered by Lucas and confirmed by x4000:

Using DotNetZip version 1.9, place the Ionic.Zip.reduced.dll in your Assets folder.

Then, from your (Windows) C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\unity folder, copy the I18N*.dll files to your Assets folder. There should be 6. These files will prevent the IBM487 errors.

My methods for reading and writing to data files are:

    /// <summary>
    /// Puts an item in the archive, overwrites if already exists
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="folder"></param>
    /// <param name="fileName"></param>
    /// <param name="item"></param>
    public void AddItemToArchive<T>(string folder, string fileName, ref T item)
    {
        if (!item.GetType().IsSerializable)
            throw new ArgumentException("item must be serializable");

        using (ZipFile zipFile = new ZipFile())
        {
            zipFile.UseUnicodeAsNecessary = true;

            //serialize item to memorystream
            using (MemoryStream m = new MemoryStream())
            {
                IFormatter formatter = new BinaryFormatter();
                formatter.Serialize(m, item);
                m.Position = 0;

                ZipEntry zipEntry = zipFile.AddEntry("entry", m);
                zipEntry.UseUnicodeAsNecessary = true;
                zipEntry.Password = _password;

                zipFile.Save(filePath);
            }
        }
    }

    /// <summary>
    /// Retrieves an item from the archive
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="folder"></param>
    /// <param name="fileName"></param>
    /// <returns></returns>
    public T GetItemFromArchive<T>(string folder, string fileName)
    {
        //get the stream from the archive
        using (MemoryStream m = new MemoryStream())
        {
            using (ZipFile zipFile = new ZipFile(filePath))
            {
                zipFile.UseUnicodeAsNecessary = true;

                ZipEntry e = zipFile["entry"];
                e.UseUnicodeAsNecessary = true;
                e.Password = _password;
                e.Extract(m);
                m.Position = 0;

                //now serialize it back to the correct type
                IFormatter formatter = new BinaryFormatter();
                T item = (T)formatter.Deserialize(m);

                return item;
            }
        }
    }

I hope this helps. ;o)

Potential Answer: SharpZipLib

There has been no progress on this question so I will post my research into this potential answer.

http://www.icsharpcode.net/opensource/sharpziplib/

The current version of SharpZipLib seems to work. Code will compile and run in the Unity editor. But the same problem with DotNetZip arises:

NotSupportedException: CodePage 437 not supported

is thrown when the project is built and running on the mono bits.

Another SharpZipLib/Unity developer banged his head on the wall last year and was able to get the code to work with the Unity version of the SharpZipLib library: http://community.sharpdevelop.net/forums/p/9762/27065.aspx

His bug demo code was still posted, so I grabbed the .dll from it and used it in my code. It WORKS! But, I won't immediately call this an acceptable answer to this question. This is the version of the library that ships with Unity (located here: Unity\Editor\Data\Frameworks\Mono.framework), so whatever bugfixes or improvements may have happened after this version was published, or may happen in the future, are lost to me.

This is the closest answer yet, but we're still not there yet. If others post their experiences with this solution, then I may feel better about calling this the answer.

CodePage 437 / IBM437 is an error thrown by the Text.Encoding end, which is not implemented in mono 1.2.5

What you might give a look at is http://cheeso.members.winisp.net/DotNetZipHelp/html/2418e26f-685a-b5b8-5c79-984d5b8da61a.htm, a documentation page in relation to the DotNetZip lib (Ionic Zip) which talks about the fact. Going from that, the issue is just a setup problem with you not telling it to use UTF-8 so it use IBM437 which will not work

I’ve been trying to extract a zip to a stream. It works great in the editor. The webplayer doesn’t seems to work though. Are there known issues with the webplayer when using the DotNetZip implementation?

Thanks for this post, was very useful. I would like to add that on iOS webrequests are automatically uncompressed when setting the requestheader to “gzip”, so on iOS is no need for an extra uncompression library.

The DotNetZip file also ships with an Ionic.Zlib.dll, dropping this into your project's root asset directory and then replacing any System.IO.Compression references to Ionic.Zlib will get you access to the GZipStream class, if your intention is to use memory streams rather than disk files.

Bit of an update -

Both Ionic.Zlib and Ionic.Gzip work great on Unity 3.5.3 for Window / mac editor. This does not include the

I began to have issues when I compiled for the I18N*.dll assemblies discussed above.

When I compiled to the iPhone, the error mentioned at the top occurred: “IBM437 codepage not supported”

I attempted to bring all the I18N*.dll’s over, and I got the error that I18N.Rare.dll could not be cross compiled with AOT.

I then deleted that file and attempted again - and everything worked great.

So as for Unity 3.5.3 - You could bring in all I18N* files, but do not include the I18N.Rare.dll.

Thanks for this post - it was very helpful.