Running keras model in golang

原文链接

为什么要running在golang上

• Current infrastructure is already running Kubernetes / Docker containers and Golang makes the binaries extremely small and efficient 运行比较小
• Web frameworks for Go are much faster than the Python ones golang的web性能高
• The team aren’t necessarily data scientists working in Python and work in Go 没必要切换语言？
• Pushing data internally using GRPC for faster communication between micro services

Binary Classification in Keras

# Use TF to save the graph model instead of Keras save model to load it in Golang
builder = tf.saved_model.builder.SavedModelBuilder("myModel")  
# Tag the model, required for Go
builder.add_meta_graph_and_variables(sess, ["myTag"])  
builder.save()  
sess.close()  

采用saveModel的方式

loading and running the model in Go

package main

import (  
    "fmt"

    tf "github.com/tensorflow/tensorflow/tensorflow/go"
)

func main() {  
    // replace myModel and myTag with the appropriate exported names in the chestrays-keras-binary-classification.ipynb
    model, err := tf.LoadSavedModel("myModel", []string{"myTag"}, nil)

    if err != nil {
        fmt.Printf("Error loading saved model: %s\n", err.Error())
        return
    }

    defer model.Session.Close()

    tensor, _ := tf.NewTensor([1][250][250][3]float32{})

    result, err := model.Session.Run(
        map[tf.Output]*tf.Tensor{
            model.Graph.Operation("inputLayer_input").Output(0): tensor, // Replace this with your input layer name
        },
        []tf.Output{
            model.Graph.Operation("inferenceLayer/Sigmoid").Output(0), // Replace this with your output layer name
        },
        nil,
    )

    if err != nil {
        fmt.Printf("Error running the session with input, err: %s\n", err.Error())
        return
    }

    fmt.Printf("Result value: %v \n", result[0].Value())

}

The tensor we input is in the shape [batch size][width][height][channels].

相同版本的python代码

%%time
from keras.preprocessing import image  
from keras.models import load_model  
import numpy as np  
model = load_model("model.h5")  
img = np.zeros((1,250,250,3))  
x = np.vstack([img]) # just append to this if we have more than one image.  
classes = model.predict_classes(x)  
print(classes)

可以尝试比较一下相关的时间问题？

Performance

Recall the model was:

3x3x32 Convolutional Layer
3x3x32 Convolutional Layer
2x2 Max Pool Layer
64 Node Fully Connected Layer with Dropout
1 Sigmoid output Layer

For Python:

• CPU: - ~2.72s to warm up and run one inference and ~0.049s for each inference after
• GPU: - ~3.52s to warm up and run one inference and ~0.009s for each inference after
• Saved Model Size (HDF5) 242MB

For Go:

• CPU: - ~0.255s to warm up and run one inference and ~0.045s for each inference after
• GPU: - N/A
• Saved Model Size(Protobuf binaries) 236MB

use Go to serve up your models in prod

感觉在k8s上运行更好一些？

在于model训练之后，基本稳定后的predict的操作