OK, I solved this, but my solution feels super janky, and I hope that a better solution is implemented in a future version of New Input System.
First, I have a static MainGameInputHandler class. In that class, I define an Action enum that includes every one of my actions. Then I have this code:
public static InputDevice currentDevice { get; private set; } = new Gamepad();
private static int _currentControl = 0;
private static MainGameInputActions _inputActions = new MainGameInputActions();
public static InputAction InputForAction(Action action)
{
InputAction a = _inputActions.MainGame.Get().FindAction(action.ToString());
if(a?.activeControl?.device != null)
{
currentDevice = a.activeControl.device;
if(currentDevice is Keyboard)
{
_currentControl = 1;
}
else if(!(currentDevice is Mouse))
{
_currentControl = 0;
}
}
if(a == null)
{
Debug.LogWarning("can't find action for " + action.ToString());
}
return a;
}
public static InputControl ControlForAction(Action action)
{
return InputForAction(action)?.controls[_currentControl];
}
with this I can get the current input for an action with InputForAction, and I can get the current specific control assigned to an action with ControlForAction (note here that in my InputActions asset, the names of the actions are identical to the stringified entries of the Action enum, and the controls[0] schema is gamepad, and the controls[1] schema is KBM).
With me so far? Great!
Next, everywhere I want to show the control icon for an action, I have a ControlIconBehavior, which has a MainGameInputHandler.Action as a public member, and does this:
void Update()
{
InputControl activeControl = MainGameInputHandler.ControlForAction(action);
if(_control != activeControl)
{
_control = activeControl;
SetControls(_control);
}
}
public void SetControls(InputControl control)
{
if(control.device.name == "Keyboard" && action == MainGameInputHandler.Action.Move)
{
// special cheat to set kbm move icon to WASD
icon.sprite = controlIconRoster.kbmSprites.wasdSprite;
label.text = "";
}
else
{
(icon.sprite, label.text) = controlIconRoster.IconAndLabelForControl(control);
}
}
As you can see, for “keyboard move” it just directly grabs a “WASD” sprite, and for anything else it asks my ControlIconRoster for the proper sprite (and label if the proper sprite is a keyboard key).
Now onward to ControlIconRoster!
“Rosters” are scriptable objects I’ve been using for a long time now (since before there were even scriptable objects! I just used gameobjects that never got instantiated!) as sort of a static storage for Unity objects.
The ControlIconRoster contains a whole bunch of sprite references. It has references to KBM sprites (which are used when input is in KBM mode), and the code to fetch those looks like this:
if(control.device.name.Contains("Keyboard"))
{
string label = control.name.Capitalize();
if(label.Length > 1)
{
label = label.Replace("Left", "L.").Replace("Right", "R.");
sprite = kbmSprites.wideKeySprite;
}
else
{
sprite = kbmSprites.keySprite;
}
return (sprite, label);
}
else if(control.device.name.Contains("Mouse"))
{
if(control.name.Contains("leftButton"))
{
return (kbmSprites.mouseLSprite, "");
}
else if(control.name.Contains("rightButton"))
{
return (kbmSprites.mouseRSprite, "");
}
else
{
return (kbmSprites.mouseSprite, "");
}
}
(keep in mind here that control is a UnityEngine.InputSystem.InputControl)
Hopefully that’s fairly self-explanatory.
Then the roster ALSO has “sets” of icons for different kinds of controllers; each control set has a name, and a list of icon sprites, and I currently have sets for “XInput”, “XboxOne”, and “DualShock” (i.e. PS4). The code to decide which of those to use based on what controller is plugged in looks like this:
ControlSet set = controlSets[0];
string devicename = control.device.name.Replace("XboxGamepad", "XInput");
foreach(ControlSet c in controlSets)
{
if(devicename.Contains(c.name))
{
set = c;
}
}
foreach(ControlSet.ControlIcon i in set.icons)
{
if(control.parent.name.Contains("dpad"))
{
if(i.input.ToString() == "dpad_" + control.name)
{
sprite = i.sprite;
}
}
else if(control.name.Contains(i.input.ToString()))
{
sprite = i.sprite;
}
}
if(sprite == null)
{
Debug.LogWarning("couldn't find match for: " + control.path);
}
return (sprite, "");
so simple! 
Incidentally, I’m using these controller / KBM icons: FREE Keyboard and controllers prompts pack | OpenGameArt.org
I hope that helps out others who need to implement a similar “control icons magically change to reflect whatever input device you touched last” system in their own games, and I hope that this is all made obsolete by a much more straightforward feature integrated into a future release of New Input System. 
