CSV import not working

We have a setup where we successfully exported CSVs using the CSV extension and sent them to our localization team.

Now, importing the translated CSVs only seems to bring up the translated keys at the top of the list, but their translations do not appear.

This happens when using the Localization Tables window’s “Import” dropdown button, then choosing “Import>CSV” or “Import>CSV(Merge)”. The same happens when using the string table’s vertical ellipsis menu (⋮) menu, then “Import>CSV” or “Import>CSV(Merge)”.

However, importing from the LocalizationSettings’s “Open” button works, but it clears and replaces our entire string tables, which is not viable for our workflow.

Localization package v.1.5.2 (Latest available from package manager)
Unity 2020.3.48f1 LTS

The log displays no warning or errors, only this profiling message:

Importing C:/Users/lazlo/Desktop/EteFrenchLocSource/StringTable_Props.csv
: 0.02s
Mapping Headers: 0s
Reading Contents: 0.01s
Removed missing entries:
: 1.88s
Total time: 1.91s
Finished Importing
UnityEditor.GenericMenu:CatchMenu (object,string[],int)

Here is a sample of our one of our CSV files. To test for the following screenshots, I kept only the very first line:

Key,Id,Speakers,EN,FR-QC,FR-IN
Prop/Acorn,155148364518682624,,Acorn,Gland,

Here is our setup in the LocalizationSettings:

Here is our setup in the StringTableCollection:

Here is the post-import diff for StringTable Shared Data. For reasons I don’t understand, it inserts the translated item at the top and offsets all the indices afterwards:

Here is the post-import diff for the relevant translated StringTable_fr-qc which should show “Gland” under “m_Localized”, but is actually empty:

Here is the code for our custom “Speakers” column CSV column:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using CsvHelper;
using Sirenix.OdinInspector;
using UnityEditor.Localization;
using UnityEditor.Localization.Plugins.CSV.Columns;
using UnityEngine;
using UnityEngine.Localization.Tables;

namespace Impossible.Ete
{
    public sealed class MomentLineSpeakersCsvColumn : CsvColumns
    {
        [SerializeField]
        private string _fieldName = "Speakers";

        public string fieldName => _fieldName;

        private SharedTableData sharedTableData;

        private int fieldIndex;

        public override void ReadBegin(StringTableCollection collection, CsvReader reader)
        {
            sharedTableData = collection.SharedData;
            fieldIndex = reader.GetFieldIndex(fieldName, isTryGet: true);
        }

        public override void ReadRow(SharedTableData.SharedTableEntry keyEntry, CsvReader reader)
        {
            if (sharedTableData == null)
            {
                return;
            }

            var entry = sharedTableData.GetEntry(keyEntry.Id);

            if (entry == null)
            {
                return;
            }

            var metadata = entry.Metadata.GetMetadata<MomentLineSpeakersMetadata>();

            if (metadata == null)
            {
                metadata = new MomentLineSpeakersMetadata();
                entry.Metadata.AddMetadata(metadata);
            }

            if (fieldIndex != -1)
            {
                metadata.speakers = reader.GetField(fieldIndex);
            }
        }

        public override void WriteBegin(StringTableCollection collection, CsvWriter writer)
        {
            writer.WriteField(fieldName);
        }

        public override void WriteRow(SharedTableData.SharedTableEntry keyEntry, IList<StringTableEntry> tableEntries, CsvWriter writer)
        {
            var metadata = keyEntry.Metadata.GetMetadata<MomentLineSpeakersMetadata>();

            if (metadata != null)
            {
                writer.WriteField(metadata.speakers, shouldQuote: true);
            }
            else
            {
                writer.WriteField(string.Empty);
            }
        }
    }
}

Things I tried but didn’t work:

  • Removing the custom Speakers column from the settings window and reimporting
  • Reimporting the StringTable assets from the Project window context menu
  • Restarting Unity
  • Removing the hyphen in the locale header key (e.g. FR-QC => FRQC)

What am I missing?

Are you relying on the CSV Extension configuration? Thats only used from the extension property drawer, we don’t read that when you do Import>CSV, which uses default values which could be the problem if the column names don’t match what is expected.

Thanks for the quick reply Karl.

The CSV is the result of a Unity export, but are you saying the same column names wouldn’t reimport again?

Seeing all our configuration in screenshots, what do you then suggest we change to make it work exactly? As you can see, our locale identifiers do match the CSV extension headers (unless the capitalization makes a difference?)

It’s also unclear to me how that would explain why the import works from the “Open” button, but not from the other two GUI hooks.

If you export through the CSV extension then you will also need to Open it through the extension. The CSV extension lets you configure custom import/export rules but these rules are not used in the other GUI hooks, they just use the default settings. Your MomentLineSpeakersCsvColumn class will not be used in the default settings. I believe it may be the capitalization that’s causing the other issues.
You can add additional menu items to import/export if you want to use the extension through them.

https://docs.unity3d.com/Packages/com.unity.localization@1.5/api/UnityEditor.Localization.LocalizationExportMenuAttribute.html
https://docs.unity3d.com/Packages/com.unity.localization@1.5/api/UnityEditor.Localization.LocalizationImportMenuAttribute.html

See ImportAllExtensions and ExportAllCsvExtensions

Thanks Karl, this nudged me in the right direction.

In order for the CSV import to work, I had to use the exact same column names as Unity output when exporting (without the CSV extension), which were:

Key,Id,Speakers,English(en),Custom(fr-qc),Custom(fr-in)

Two things come to mind from this user experience:

  • It is unintuitive that the other GUI hooks mentioning “CSV” in their label do not honor or prioritize the CSV extension when it is available and confiugred, as nothing really indicates this behaviour for the end user

  • With that in mind, the CSV extension direly lacks a merged import. Professional localization teams often times split the work and output into multiple files, so importing a single CSV is rarely a viable workflow at scale.

On that second point, is there a relatively trivial way, or a code example, on how I could go about implementing a merge import that honors the CSV extension?

Thanks for the feedback, I can see how it’s confusing. One of the reasons we don’t use the extensions is that its possible to have multiple CSV extensions on a collection so we thought it would be confusing and unclear which one to use.
Ill make a note for us to look into improving the UX for this.

Ill also add a task to add an “Open(Merge)” option to the CSV extension editor.

For now, this is possible via a script by passing false to removeMissingEntries in Csv.ImportInto
You could add a custom menu item to the tables window which does the export and import using your CSV extension.

Thanks!

In the case of multiple extensions, I think an acceptable UX would be to label each Import/Export context menu item under the extension’s name (or at least index). If I saw when opening the menu, for example:

Import>CSV (Replace)…
Import>CSV (Merge)…
Import>CSV from extension #0 (Replace)…
Import>CSV from extension #0 (Merge)…

I would have tried/assumed I could use the latter options!
It’s perhaps not the cleanest labels, but at least it exposes the difference to the user.