What is the role of TimeDistributed layer in Keras?

In keras – while building a sequential model – usually the second dimension (one after sample dimension) – is related to a time dimension. This means that if for example, your data is 5-dim with (sample, time, width, length, channel) you could apply a convolutional layer using TimeDistributed (which is applicable to 4-dim with (sample, … Read more

Why must a nonlinear activation function be used in a backpropagation neural network? [closed]

The purpose of the activation function is to introduce non-linearity into the network in turn, this allows you to model a response variable (aka target variable, class label, or score) that varies non-linearly with its explanatory variables non-linear means that the output cannot be reproduced from a linear combination of the inputs (which is not … Read more

How do I load a caffe model and convert to a numpy array?

Here’s a nice function that converts a caffe net to a python list of dictionaries, so you can pickle it and read it anyway you want: import caffe def shai_net_to_py_readable(prototxt_filename, caffemodel_filename): net = caffe.Net(prototxt_filename, caffemodel_filename, caffe.TEST) # read the net + weights pynet_ = [] for li in xrange(len(net.layers)): # for each layer in the … Read more

Keras custom loss function: Accessing current input pattern

You can wrap the loss function as a inner function and pass your input tensor to it (as commonly done when passing additional arguments to the loss function). def custom_loss_wrapper(input_tensor): def custom_loss(y_true, y_pred): return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor) return custom_loss input_tensor = Input(shape=(10,)) hidden = Dense(100, activation=’relu’)(input_tensor) out = Dense(1, activation=’sigmoid’)(hidden) model = Model(input_tensor, out) … Read more

Keras Text Preprocessing – Saving Tokenizer object to file for scoring

The most common way is to use either pickle or joblib. Here you have an example on how to use pickle in order to save Tokenizer: import pickle # saving with open(‘tokenizer.pickle’, ‘wb’) as handle: pickle.dump(tokenizer, handle, protocol=pickle.HIGHEST_PROTOCOL) # loading with open(‘tokenizer.pickle’, ‘rb’) as handle: tokenizer = pickle.load(handle)

Pytorch – RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed

The problem is from my training loop: it doesn’t detach or repackage the hidden state in between batches? If so, then loss.backward() is trying to back-propagate all the way through to the start of time, which works for the first batch but not for the second because the graph for the first batch has been … Read more