Using a tensorflow model as a loss function

I am trying to use an empirical metric as a loss function to train a Tensorflow model. Calculating the metric function is slow, but I can train a regression neural network to accurately and quickly predict the metric score after it is trained. Is there a straightforward way (or tutorial?) to use a trained Tensorflow or scikit-learn model as a custom loss function for a Tensorflow model?

Edit: I have found this StackOverflow entry as a starting point. I will try it out and report back.