Hello

Create a simple torchscript model:

./code/hello/ex1.py
 1#!/usr/bin/env python3
 2
 3import torch
 4
 5
 6class Foo(torch.nn.Module):
 7    def forward(self, x):
 8        return torch.nn.functional.relu(x)
 9
10
11f = Foo()
12x = torch.rand(2)
13m = torch.jit.trace(f, x)
14m.save("ex1.pt")
15print(m.graph)

MNNConvert -f TORCH --modelFile ./ex1.pt --MNNModel ex1.mnn --bizCode MNN

It prints:

Start to Convert Other Model Format To MNN Model...
Start to Optimize the MNN Net...
inputTensors : [ x.1, ]
outputTensors: [ 2, ]
Converted Success!

We can use:

netron ex1.mnn --port 6006

to view it:

Serving 'ex1.mnn' at http://localhost:6006

To convert the model to json, use:

MNNDump2Json ./ex1.mnn ./ex1.json

which generates:

./code/hello/ex1.json
 1{ "bizCode": "MNN", "extraInfo": 
 2{ "version": "2.1.1" }
 3, "oplists": 
 4[ 
 5{ "main_type": "Input", "main": 
 6{ "dtype": "DT_FLOAT", "dformat": "NCHW" }
 7, "name": "x.1", "outputIndexes": 
 8[ 0 ]
 9, "type": "Input", "defaultDimentionFormat": "NHWC" }
10, 
11{ "inputIndexes": 
12[ 0 ]
13, "main_type": "Relu", "main": 
14{ "slope": 0.0 }
15, "name": "2", "outputIndexes": 
16[ 1 ]
17, "type": "ReLU", "defaultDimentionFormat": "NCHW" }
18 ]
19, "outputName": 
20[ "2" ]
21, "preferForwardType": "CPU", "sourceType": "TORCH", "tensorName": 
22[ "x.1", "2" ]
23, "tensorNumber": 0, "usage": "INFERENCE", "mnn_uuid": "d24b26b1-14c8-42cb-9ce1-2a9b867714e9" }

To convert ex1.json back to a .mnn file, use:

MNNRevert2Buffer ex1.json ex11.mnn

To show the information of the model:

MNNConvert --framework MNN --modelFile ./ex1.mnn --info

It prints:

Model default dimensionFormat is NCHW
Model Inputs:
[ x.1 ]: dimensionFormat: NCHW, size: [ ], type is float
Model Outputs:
[ 2 ]
Model Version: 2.1.1

To run it with MNN in Python:

./code/hello/test-ex1-mnn.py
 1#!/usr/bin/env python3
 2
 3import numpy as np
 4import MNN
 5import torch
 6
 7
 8def main():
 9    interpreter = MNN.Interpreter("ex1.mnn")
10
11    config = {}
12    config["precision"] = "low"  # low, high, normal
13    config["backend"] = "CPU"
14    #  config["thread"] = 1
15    session = interpreter.createSession(config)
16
17    #  a = torch.tensor([1, -2.5, 3.2], dtype=torch.float32)
18    #  b = a.numpy()
19
20    input_tensor = interpreter.getSessionInput(session)
21    interpreter.resizeTensor(input_tensor, (3,))
22    interpreter.resizeSession(session)
23    print("input_tensor.getShape()", input_tensor.getShape())
24
25    input_data = MNN.expr.const(
26        np.array([1, -2.5, 3.2], dtype=np.float32),
27        (3,),
28        MNN.expr.NCHW,
29        MNN.expr.float,
30    )
31    input_tensor.copyFrom(MNN.Tensor(input_data))
32    print("input_tensor.getNumpyData()", input_tensor.getNumpyData())
33
34    interpreter.runSession(session)
35
36    output_tensor = interpreter.getSessionOutput(session)
37    print("output_tensor.getNumpyData()", output_tensor.getNumpyData())
38
39    output_data = MNN.Tensor(
40        output_tensor.getShape(), MNN.Halide_Type_Float, MNN.Tensor_DimensionType_Caffe
41    )
42    output_tensor.copyToHostTensor(output_data)
43    print("output_data.getNumpyData()", output_data.getNumpyData())
44
45
46if __name__ == "__main__":
47    main()

It prints:

input_tensor.getShape() (3,)
input_tensor.getNumpyData() [ 1.  -2.5  3.2]
output_tensor.getNumpyData() [1.  0.  3.2]
output_data.getNumpyData() [1.  0.  3.2]