You could try to use local features like SIFT here: http://en.wikipedia.org/wiki/Scale-invariant_feature_transform It should work because logo shape is usually constant, so extracted features shall match well. The workflow will be like this: Detect corners (e.g. Harris corner detector) – for Nike logo they are two sharp ends. Compute descriptors (like SIFT – 128D integer vector) … Read more
A training step is one gradient update. In one step batch_size examples are processed. An epoch consists of one full cycle through the training data. This is usually many steps. As an example, if you have 2,000 images and use a batch size of 10 an epoch consists of: 2,000 images / (10 images / … Read more
So: One-to-one: you could use a Dense layer as you are not processing sequences: model.add(Dense(output_size, input_shape=input_shape)) One-to-many: this option is not supported well as chaining models is not very easy in Keras, so the following version is the easiest one: model.add(RepeatVector(number_of_times, input_shape=input_shape)) model.add(LSTM(output_size, return_sequences=True)) Many-to-one: actually, your code snippet is (almost) an example of this … Read more
Breaking symmetry is essential here, and not for the reason of performance. Imagine first 2 layers of multilayer perceptron (input and hidden layers): During forward propagation each unit in hidden layer gets signal: That is, each hidden unit gets sum of inputs multiplied by the corresponding weight. Now imagine that you initialize all weights to … Read more
I apologise for sounding silly as I am starting out! Do not; this is a subtle issue of great importance, which is usually (and regrettably) omitted in tutorials and introductory expositions. Unfortunately, it is not as simple as taking the square root of the inverse-transformed MSE, but it is not that complicated either; essentially what … Read more
There are two competing concerns: with less training data, your parameter estimates have greater variance. With less testing data, your performance statistic will have greater variance. Broadly speaking you should be concerned with dividing data such that neither variance is too high, which is more to do with the absolute number of instances in each … Read more
I want to explain with picture from C3D. In a nutshell, convolutional direction & output shape is important! ↑↑↑↑↑ 1D Convolutions – Basic ↑↑↑↑↑ just 1-direction (time-axis) to calculate conv input = [W], filter = [k], output = [W] ex) input = [1,1,1,1,1], filter = [0.25,0.5,0.25], output = [1,1,1,1,1] output-shape is 1D array example) graph … Read more
At first glance you can see this log section divided into two: [Forward] and [Backward]. Recall that neural network training is done via forward-backward propagation: A training example (batch) is fed to the net and a forward pass outputs the current prediction. Based on this prediction a loss is computed. The loss is then derived, … Read more
It’s explained well here. If the input variables are combined linearly, as in an MLP [multilayer perceptron], then it is rarely strictly necessary to standardize the inputs, at least in theory. The reason is that any rescaling of an input vector can be effectively undone by changing the corresponding weights and biases, leaving you with … Read more
I came across this phenomenon several times. Here are my observations: Gradient blow up Reason: large gradients throw the learning process off-track. What you should expect: Looking at the runtime log, you should look at the loss values per-iteration. You’ll notice that the loss starts to grow significantly from iteration to iteration, eventually the loss … Read more