ONNX conversion of BLIP Captioning

Unity Engine
,
1

I’d like to convert BLIP Captioning to ONNX format so that I can use it with Unity Sentis. Here’s where I’m at:
-I’m using the Exporting a Model from PyTorch to ONNX method
-Using GitHub - salesforce/BLIP: PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation repo
-Using https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth as model
-My python file to do the conversion looks like this:

import torch
from models.blip import blip_decoder 

image_size = 384
model_path = 'model_base_capfilt_large.pth' 
model = blip_decoder(pretrained=model_path, image_size=image_size, vit='base')

model.eval()

# Create a dummy input image tensor
dummy_image = torch.randn(1, 3, 384, 384)

# Convert the model to ONNX format
output_onnx_file = 'blip_model.onnx'
torch.onnx.export(model, dummy_image, output_onnx_file, export_params=True, verbose=True, input_names=['input'], output_names=['output'])

Errors out with:


  File "C:\Users\indon\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\jit\_trace.py", line 138, in forward
    graph, out = torch._C._create_graph_by_tracing(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: 0 INTERNAL ASSERT FAILED at "..\\torch\\csrc\\jit\\ir\\alias_analysis.cpp":621, please report a bug to PyTorch. We don't have an op for aten::full but it isn't a special case.  Argument types: int[], bool, NoneType, NoneType, Device, bool, 

Candidates:
	aten::full.names(int[] size, Scalar fill_value, *, str[]? names, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
	aten::full(SymInt[] size, Scalar fill_value, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
	aten::full.names_out(int[] size, Scalar fill_value, *, str[]? names, Tensor(a!) out) -> Tensor(a!)
	aten::full.out(SymInt[] size, Scalar fill_value, *, Tensor(a!) out) -> Tensor(a!)
1 Like