AlexRibard said " we added support for BackgroundMattingv2. It will be present in 2.3.0 It does require you exporting the model with fixed input size. But otherwise than that it works on the base model"
i am using barracuda 2.3.1
but i got still error when i import Model
the message is “Asset import failed, “Assets/Onnx/onnx_mobilenetv2_hd.onnx” > OnnxImportException: Unknown type ScatterElements encountered while parsing layer 878.”
Hi!
Cross posting the answer from the github issue
https://github.com/Unity-Technologies/barracuda-release/issues/136
In order to be able to use BackgroundMattingV2 models, you’ll need to export the .pth to onnx with fixed input size
Here is the code to achieve that:
// https://github.com/PeterL1n/BackgroundMattingV2
// Download the github repo.
// Download the pth models here https://drive.google.com/drive/folders/1cbetlrKREitIgjnIikG1HdM4x72FtgBh
// Execute this script:
import torch
from model import MattingBase, MattingRefine
for model_backbone in ['resnet101', 'resnet50', 'mobilenetv2']:
model = MattingBase(model_backbone)
model.load_state_dict(torch.load('pytorch_' + model_backbone + '.pth', map_location='cpu'), strict=False)
src = torch.randn(2, 3, 224, 224)
bgr = torch.randn(2, 3, 224, 224)
# Export ONNX
input_names=['src', 'bgr']
output_names = ['pha', 'fgr', 'err', 'hid']
torch.onnx.export(
model=model,
args=(src, bgr),
f='MattingBase_' + model_backbone + '.onnx',
verbose=True,
opset_version=12,
do_constant_folding=True,
input_names=input_names,
output_names=output_names)
Let me know if that works for you
i have to using args.model_type == ‘mattingrefine’: to get best qulity of mask
but barracuda 2.3.1 version also does not support ScatterElements by today
i am waitting…
Ok, I checked the model with that option, indeed with static input shapes it generates a ScatterElements
I’ll create a task to support MattingRefine.
But for now I’d suggest using MattingBase as that one works
New
MattingBase is of such a very very poor quality that i can’t use as attached PIC.
It’s much better to use Keijiro’s NNCam than MattingBase
(GitHub - keijiro/NNCam: Virtual background with Unity Barracuda)
Thanks 
import torch
from model import MattingBase, MattingRefine
for model_backbone in ['resnet101', 'resnet50', 'mobilenetv2']:
#model = MattingBase(model_backbone) #this is not a good
model = MattingRefine(
backbone=model_backbone,
backbone_scale=0.25,
refine_mode='sampling', #default='sampling', choices=['full', 'sampling', 'thresholding'])
refine_sample_pixels=80000, #default=80_000
refine_threshold=0.1,
refine_kernel_size=3,
refine_patch_crop_method='roi_align', #choices=['unfold', 'roi_align', 'gather']
refine_patch_replace_method='scatter_element') #choices=['scatter_nd', 'scatter_element'])
model.load_state_dict(torch.load('pytorch_' + model_backbone + '.pth', map_location='cuda'), strict=False)
.........
.........
