Month: December 2020

I am currently studying Industrial Engineering, but I’ve been
learning DL and ML on my own. My goal is to obtain a job related
with ML or AI. I don’t come from a traditional Computer Science
background, do you think this certificate will add value to my
CV?

Hey, this is a question I had that I answered myself after some
research. I can’t find a flair more applicable than ‘Question’ so I
will just answer it myself haha.

I was trying to use tf.keras.preprocessing.text.Tokenizer to
train a model for a language task. I wanted my model to include
certain punctuation in it’s output, like exclamation points and
commas and whatnot, and I wasn’t sure how to do this.

I figured that since the default value for filters in the
Tokenizer constructor is:

filters='!"#$%&()*+,-./:;<=>?@[\]^_`{|}~tn' 

then I would just have to remove the punctuation that I want to
be recognized. I then spent a few hours training my model.

DON’T DO THIS. It will not treat the
punctuation as separate tokens, but rather your vocabulary will be
filled with examples such as ‘man’ vs ‘man.’ vs ‘man,’, etc. These
will all be separate tokens.

Instead, you should preprocess all of your sentences to include
spaces between any punctuation that you want. This is how I did
it:

def separate_punctuation(s, filters=',.()?'): new_s = '' for char in s: if char in filters: new_s += ' ' + char + ' ' else: new_s += char return new_s.strip() 

This way ‘Hello neighbor, how are you?’ will become ‘Hello
neighbor , how are you ?’. Thus, all punctuation will only take up
one element of your vocabulary and your model will generalize much,
much better.

Hope this saves someone else’s time.

I was trying to make a prediction from a loaded tensorflow
model. Though I’m not sure if it’s correct how I previously saved
it, specifically I have doubts about code inside serving_input_fn()
function (MAX_SEQ_LENGTH=128):

def serving_input_fn(): feature_spec = { "input_ids" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "input_mask" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "segment_ids" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "label_ids" : tf.FixedLenFeature([], tf.int64) } serialized_tf_example = tf.placeholder(dtype=tf.string,shape=[None],name='input_example_tensor') receiver_tensors = {'example': serialized_tf_example} features = tf.parse_example(serialized_tf_example, feature_spec) return tf.estimator.export.ServingInputReceiver(features, receiver_tensors) estimator.export_saved_model('gs://bucket/trained_model, serving_input_receiver_fn=serving_input_fn) 

When I try to predict from loaded model:

from tensorflow.contrib import predictor predict_fn = predictor.from_saved_model(LOAD_PATH) input_features_test = convert_examples_to_features( test_examples,label_list, MAX_SEQ_LENGTH, tokenizer) predictions = predict_fn({'example':input_features_test[0]}) 

it returns this error:

ValueError: Cannot feed value of shape () for Tensor
‘input_example_tensor:0’, which has shape ‘(?,)’

How should I change serving_input_fn() method?

If you want to reproduce it: github_repo (you
should download variables from
here and put it in trained_model/1608370941/ folder)

This is the tutorial I followed to fine tune BERT model on
google cloud TPU.

I’m using Colab environment to fine-tune a BERT Model (for
reference this is the
Notebook_with_Cloud_TPU_Sentence_Classification_Tasks.ipynb)).
How can I export fine tuned model (it’s a TPUEstimator object) to
HDF5 format? I need to use the trained model locally on CPU.

I came across this issue in my own projects, and found the
issue
linked here on the TensorFlow github, but I feel like it isn’t
getting much traction for the potential severity of the
problem.

Basically there was a non-release push to TF between 1.14 and
1.15 that broke some functionality for the
tf.image.per_image_standarization routine when used on unsigned
integer inputs. The majority of information content in images ends
up getting lost because of the naïve type conversions done in
per_image_standardization after 1.14. This isn’t addressed in
documentation, and is pretty clearly a major change in behavior
befitting a major release, but was introduced before a major
release, likely pointing to an untested edge case.

I’m concerned that the issue isn’t getting much traction but
could potentially impact labs all over the place. The simple
solution is to convert your unsigned int images to float before
calling per_image_standardization, but that isn’t obvious from any
of the documentation, and used to be handled naturally by the
method.

Thoughts?

Edit: formatting.