How do I use Apple's SFSymbols in my app?

Questions & Answers
1

Apple has an API to get buttons on a gamepad: GCControllerElement. Within it, you can get
sfSymbolsName which is an identifier to look up a graphic in Apple’s SF Symbols. That flow would provide the SF Symbol graphic for the button according to the GameCenter button mapping.

How do I get that graphic into Unity?

I’ve figured some details out:

But I’m not sure how to extract the image data and convert it to an array of bytes in a format Unity can understand.

2

You can get an array of bytes from an NSBitmapImageRep. However, there’s some hoops to jump through to get one of those from an NSImage. You’ll also want to resize the image to avoid getting back a 15x15 image (the default if you try using TIFFRepresentation). I get back a 40x38 image on iOS without any resizing.

Some things I haven’t figured out:

  • How to make the image white instead of black
  • Proper memory usage

Here’s the code to convert to array of bytes, pass to C#, and assign that to a RawImage:

// Obj-C

// macOS-specific helper (mark this to not build for any platform)
#import <AppKit/AppKit.h>
NSData* GetSymbolImageAsBytes(NSString* name, int width, int height)
{
    if (@available(macOS 11.0, *))
    {
    }
    else
    {
        // imageWithSystemSymbolName isn't available
        return nil;
    }

    NSImage *image = [NSImage
        imageWithSystemSymbolName: name
         accessibilityDescription: nil
    ];

    if (image == nil) {
        return nil;
    }

    // Resize to desired size.
    // https://stackoverflow.com/a/38442746/79125

    NSSize size = NSMakeSize(width, height);

    // TODO: Should reuse a single NSBitmapImageRep instance instead of
    // repeated alloc. Otherwise we leak memory here.
    NSBitmapImageRep *rep = [[NSBitmapImageRep alloc]
        initWithBitmapDataPlanes:NULL
                      pixelsWide:size.width
                      pixelsHigh:size.height
                   bitsPerSample:8
                 samplesPerPixel:4
                        hasAlpha:YES
                        isPlanar:NO
                  colorSpaceName:NSCalibratedRGBColorSpace
                     bytesPerRow:0
                    bitsPerPixel:0];
    rep.size = size;

    [NSGraphicsContext saveGraphicsState];
    [NSGraphicsContext setCurrentContext:[NSGraphicsContext graphicsContextWithBitmapImageRep:rep]];
    [image drawInRect:NSMakeRect(0, 0, size.width, size.height) fromRect:NSZeroRect operation:NSCompositingOperationCopy fraction:1.0];
    [NSGraphicsContext restoreGraphicsState];
    if (rep == nil)
    {
        return nil;
    }
    // Passing nil gives a warning but seems to work okay.
    return [rep representationUsingType:NSBitmapImageFileTypePNG properties:nil];
}

// iOS-specific helper (mark this to build for iOS)
#import <UIKit/UIKit.h>
NSData* GetSymbolImageAsBytes(NSString* name, int width, int height)
{
        UIImage *image = [UIImage systemImageNamed: name];
        NSData *png_data = UIImagePNGRepresentation(image);
        // TODO: Rescale to input size.
        return png_data;
}

#import <Foundation/Foundation.h>
#import <GameController/GameController.h>
int GetGamepadButtonImage(int** dataPtr, int width, int height)
{
    *dataPtr = nil;

    GCController *gc = [GCController current];

    NSString *name = gc.extendedGamepad.buttonY.sfSymbolsName;
    // Name is something like
    //~ name = @"triangle.circle";

    NSData* png_data = GetSymbolImageAsBytes(name, width, height);
    if (png_data == nil) {
        return -1;
    }
    
    UInt8 *png_bytes = (UInt8 *)png_data.bytes;
    *dataPtr = (int*)png_bytes;
    return (int)png_data.length;
}



// C#

#ifdef UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX
    // You need to compile the above objc code (minus iOS) into a
    // "Gamepad.bundle". (I used cmake.) Be sure your .bundle is only
    // included on Editor+Standalone.
    [DllImport("Gamepad", CallingConvention = CallingConvention.Cdecl)]
#else
    // Putting above objc code (minus macOS) in Assets/Plugins/iOS/Gamepad
    // auto builds it into this dll.
    [DllImport("__Internal")]
#endif
private static extern int GetGamepadButtonImage(out IntPtr data, int width, int height);


static void ReplaceTexture()
{
    int width = 45;
    int height = 45;
    var num_bytes = GetGamepadButtonImage(out IntPtr unsafe_ptr, width, height);
    Debug.Log($"received {num_bytes} bytes of unmanaged data. {width}x{height}={width*height}.");

    if (num_bytes <= 0)
    {
        Debug.LogWarning($"Received no data: {num_bytes}. Aborting...");
        return;
    }

    var image_bytes = new byte[num_bytes];
    Marshal.Copy(unsafe_ptr, image_bytes, 0, num_bytes);
    // Trying to free like this crashes:
    //~ Marshal.FreeHGlobal(unsafe_ptr);

    var canvas = GameObject.Find("Canvas");
    var im = canvas.GetComponentInChildren<UnityEngine.UI.RawImage>();
    var tex = new Texture2D(width, height);
    ImageConversion.LoadImage(tex, image_bytes);
    im.texture = tex;
}

Be sure to use the PluginInspector to set the .bundle and .m files to build only on the correct platforms!