How dose dropout affect Weights and Bias?
I applied dropout in my network , and it worked , but i can't interpret dropout effects on weight and bias, to be more specific , i can't interpret why appling droput and not applying dropout have a great effect on mean, std, min, max of weights and biases
What i know is that dropout is regularization, which is applied to the units of network and turns the units off randomly.
Network model
input(1x784) > hidden layer(784x128) > hidden layer(128x256) > output(256x10)
The graph below shows the min, max ,mean ,std value of applying dropout and not applying dropout.
There is a lot of difference between applying droppout and not doing it.
but i don't know why there's a difference
Could you explain why there is such a difference?
See also questions close to this topic

I am stuck with splitting the below code to get it in the expected format
def transitions_as_dict(transitions_as_list): transitions = {} for e in transitions_as_list: a,b=e.split(':') transitions[a]=b return transitions transitions_as_dict(['q0,0:q1', 'q1,1:q0'])

How to count date differences based on ID with pandas?
Hello everyone,
Below is an example of the dataframe i have. For each Serial Number there are some dates.There aren't rules for number of dates for each serial. Date count for each serial >= 1. Date is of dtype datetime[64]
Serial Number Date ABC 15032017 ABC 21042017 ABC 05052018 ABC 09062018 CBB 03042016 CBB 07092017 BDC 03012018 AAA 05052017 AAA 08092018
Dates are sorted per Serial number.
How can i count the dates' differences (in days) for each serial ?
My desired output is :
Serial Date Differences ABC 15032017 Nan ABC 21042017 37 ABC 05052018 379 ABC 09062018 35 CBB 03042016 Nan CBB 07092017 522 BDC 03012018 Nan AAA 05052017 Nan AAA 08092018 491
I've tried many approaches without success, any help would be highly appreciated.

Pull graphs from power bi using Python
I'm new to Python. Came over to the Dark side from JavaScript. Im trying to make a program that will allow me to pull graph images from Power Bi's website and place them into a power point presentation. I havent been able to find any information about Power bi's API keys either. There are a ton of tutorials on how to use python to import data to Power Bi. But none showing how to export data from Power Bi.
Does anyone know of any libs that would make this possible? Maybe web scrapping would work?
Thanks

Bayesian neural network in tensorflowprobability
I am new to tensorflow and I am trying to set of a bayesian neural network with dense flipoutlayers. My code looks as follows:
def train_BNN(training_data, training_labels, test_data, test_labels, layers, epochs): bayesian_nn = Sequential() nbr_samples, dim = training_data.shape training_labels = training_labels.reshape((nbr_samples, 1)) training_data = tf.convert_to_tensor(training_data, tf.float32) training_labels = tf.convert_to_tensor(training_labels, tf.float32) for i in range(0, len(layers)): bayesian_nn.add(tfp.layers.DenseFlipout(layers[i], activation='relu')) bayesian_nn.add(tfp.layers.DenseFlipout(1, activation='sigmoid')) # Define loss function logits = bayesian_nn(training_data) neg_log_likelihood = tf.losses.sigmoid_cross_entropy(training_labels, logits=logits) kl = sum(bayesian_nn.losses) loss = neg_log_likelihood + kl # Define optimizer train_op = tf.train.AdamOptimizer().minimize(loss) bayesian_nn.compile(train_op, loss=loss, metrics=['accuracy'])
I get the following error when trying to compile the network:
TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
I don't know what's causing this error... please help :)

How to use Keras to get multiple possible outputs for the same input in a LSTM?
I try to get a output vector with keras there more than one value cannot be null.
For example I have sentences which are 50 words long. Each sentence can be about multiple classes. In total I have 5 classes (one, two, three, four, five).
Now a sentence can be just about class "one" but another could also be about "two" and "five".
Until now I just know how to build two kinds of models. If I have a output of true or false I can use a one shaped output layer with a binary cross entropy loss function. If just one class could be possible I could onehot encode my output, have a five shaped output layer and use the categorical cross entropy function.
Is there a way to have multiple class output? The best would even be if my model predicts a chance for each class and I can say if the chance is higher than x% you found a class. So if I have a sentence with the output [0,0,1,0,1] and my model predicts [0.12,0.15,0.93,0.11,0.79] and I program everything above 80% is a 1, The prediciton would be correct for 4/5 classes for the sentence (or its false in total because one is false predicted). Is there a way to program a model like this?

Keras vs Matlab
I am using Keras (with TensorFlow as backend) to predict experimental data, which has some scatter around a mean function y = f(x). In the first step, the function has only one quantity as input. I obtain very bad results and simplified the problem. Now, I predict a simple function with a somewhat similar trend: y = x^3 – 2.5*x^2 + 1.5 x
I run trainings and predictions for in Keras and compare them to results from Matlab using the Neural Fitting App. The network architectures are very similar. • 1 hidden layer – 10 neurons – tanh as activation function • 1 output layer
For Matlab, I use the 'trainbr' as train function. With this, it takes only seconds until the prediction is virtually perfect (c.f. first figure attached).
For Keras, after 200 epochs and around 2 minutes of runtime, the function looks clumsy (c.f. second figure attached). With more Epochs (10.000100.000) the results from Keras are getting better but are still far worse than the Matlab predictions. Also, this takes hours to compute instead of a few seconds in Matlab.
Could someone shed some light on why Matlab is creating results so much better and faster?
Please find the code snippets of our problem below (both matlab and python/keras).
I’m looking forward to get some hints. Thank you very much.
Best regards, Paul
Matlab % create function x = 0:0.01:2; y = x.^3  3*x.^2 + 2*x; % create test and training data sets (the same for this simple function) x_tr = x; y_tr = y; x_te = x; y_te = y; % setup of shallow network trainFcn = 'trainbr'; hiddenLayerSize = 10; net = fitnet(hiddenLayerSize,trainFcn); net.performFcn = 'mse'; [net,tr] = train(net,x_tr,y_tr); % create prediction y_prediction_te = net(x_te); % plot results hold on plot(x_tr, y_te,'x') plot(x_tr, y_prediction_te,'') hold off Python/Keras import matplotlib.pyplot as plt import numpy as np from keras.models import Sequential from keras.layers import Dense, Activation from sklearn.model_selection import train_test_split x = np.arange(0,2,0.001) y = np.arange(0,2,0.001) for i in range(1,2000): y[i] = (x[i]**3 )3*(x[i]**2)+2*x[i] #keras code x_train , x_test , y_train , y_test = train_test_split(x , y , test_size = 0.5 , shuffle = True) model = Sequential() model.add(Dense(10, input_dim=1, activation='tanh')) model.add(Dense(1,)) model.compile(loss='mean_squared_error', optimizer='adam') model.fit(x_train, y_train, epochs=200, batch_size=1, verbose=1) y_pred = model.predict(x_test) # plot the results plt.plot(x_test, y_test, '.k') plt.plot(x_test, y_pred, '.r')

Keras batch_size problem when using model.fit in custom layers
I have a custom layer which reshapes the input tensor, perform some dot products with kernels and returns back a tensor with the same number of dimensions . The input to my network is images, say of size 61x80. When the number of train images is a multiple of batch_size , model.fit works fine. eg total train images= 2700, batch_size=10. But when the total train images =2701, batch_size() it doesnt work, throwing an error something like this
Epoch 1/5 2520/2701 [==========================>...]  ETA: 0s  loss: 2.7465  acc: 0.2516Traceback (most recent call last):
File "", line 5, in history = model.fit(x_train, y_train,batch_size=10, epochs=5)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/keras/engine/training.py", line 1039, in fit validation_steps=validation_steps)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/keras/engine/training_arrays.py", line 199, in fit_loop outs = f(ins_batch)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/keras/backend/tensorflow_backend.py", line 2715, in call return self._call(inputs)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/keras/backend/tensorflow_backend.py", line 2675, in _call fetched = self._callable_fn(*array_vals)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/tensorflow/python/client/session.py", line 1439, in call run_metadata_ptr)
File "/home/eee/anaconda3/lib/python3.6/sitepackages/tensorflow/python/framework/errors_impl.py", line 528, in exit c_api.TF_GetCode(self.status.status))
InvalidArgumentError: Input to reshape is a tensor with 4880 values, but the requested shape has 48800 [[{{node my_layer_3/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_2_0_0, my_layer_3/stack)]]
please help how to get around this problem.

Real time hand pose estimation
Now I'm researching about real time hand pose estimation But there are so many tech, but I don't know what tech can run on real time.
https://github.com/xinghaochen/awesomehandposeestimation
I searched about this repo. Please let me know about this!

Vowpal Wabbit hash collision works better than L1 as regularization
I have VW classification model which i wanted to inspect for number of features and number of hash collisions. I trained it and test it on different datasets. The datasets conatins more than 400k features, so with 18bit VW space, it is possible to save only 260k without collisions.
Then, to inspect it, i created two readable models: one with parameter readable_model, to get number of all hashes, second one with parameter invert_hash to get number of all features, even those which were in hash collisions. There were 425k features, and 208k hashes (not 260k, because i used some filtering with keep argument, and as i understand it, vw saves to hash table also ignored namespaces). Then i measured quality of model, with ROC_AUC, MCC, and Average Precision. Results were 0.903, 0.564, 0.591.
As you can see, hash collision is huge, so i decided to increase bit space with b argument value set to 24. Then i trained model again. Now, there were 425k features, and 425k hashes = no collisions. However, results on same metrics, were worse (0.902,0554,0.587).
For now everything looks like there was big overfitting with 24bit space, while 18bit prevented model against overfitting better  that would be good explanation why results were better on test dataset with 18bit model.
But then i decided to reduce number of features on 24bit model with L1 regularization. I was playing with it for a while, and when i got model which had 208k hashes and 208k features i was very suprised, that its results were still worse, than results of 18bit model with same number of hashes. It was 0.901,0.584,0.552.
This makes me to believe, that random hash collisions, even for huge portion of features, works like better regularizer, than L1 regularization. How is this possible?

Cross validation for HDTweedie
I have used cva.glmnet for a Poisson response to find the optimal values of alpha and lambda. Since, glmnet does not allow to penalise Gamma Response, I started using HDtweedie (p=2). However, I am not able to iterate over both alpha and lambda to find optimal values just as cva.glmnet does. Is there any equivalent function for HDtweedie that can help me achieve this?

Elastic net regression
I have just started using the GLM Net package to build Penalised GLM Models. I was just experimenting with different values of alpha and lambda and their effect on Parameters.
I was just trying to develop an intuition on how should the parameters look like if I use lambda = 1 and alpha =0, basically maximum penalty on a ridge regression.
Can anyone help?

How to get the activation values of the Dropout layer in the last minibatch in Keras
I have been trying to make autoencoder with custom loss function where the loss function depends on the values masked and not masked by the dropout layer differently. So I am trying to access the values in the dropout layer in the last iteration. Here is the code sample 
import keras.backend as K from keras import optimizers batchSize = 30 a = 1 b = 1 def lossFunc(model,a,b): def mse3(y_true, y_pred): temp=y_true>0 y_pred0=y_pred*tf.cast(temp,tf.float32) y_inter = model.get_layer("intermediate").output y_inter_filter = tf.math.not_equal(y_inter, tf.zeros_like(y_inter)) y_pred1 = y_pred*tf.cast(y_inter_filter,tf.float32) y_true1 = y_true*tf.cast(y_inter_filter,tf.float32) y_inter_filter2 = tf.math.logical_xor(temp, y_inter_filter) y_pred2 = y_pred*tf.cast(y_inter_filter2,tf.float32) y_true2 = y_true*tf.cast(y_inter_filter2,tf.float32) return a*K.sum(K.square(y_pred1  y_true1), axis=1) #+ b*K.sum(K.square(y_pred2  y_true2), axis=1) return mse3 # def lossFunc(model,a,b): # def mse3(y_true, y_pred): # temp=y_true>0 # y_pred=y_pred*tf.cast(temp,tf.float32) # return K.sum(K.square(y_pred  y_true), axis=1) # return mse3 sgd = optimizers.SGD(lr=0.01, decay=1e6, momentum=0.9, nesterov=True) model.compile(optimizer = sgd, loss = lossFunc(model,a,b)) model.fit(x_input, x_output, epochs = 100, batch_size = batchSize)
I have gone through the source code of Dropout layer. But, I am not sure what are the values returned by
model.get_layer("intermediate").output
. But I am sure It is not something I am looking for as the loss values are different than expected. Any direction will be much appreciated. Thank You.Architecture 
from keras.layers import Input, Dense, Concatenate, Lambda, Dropout from keras.models import Model import tensorflow as tf from keras.regularizers import l2 group = [] dropout_rate = 0.5 reg = .001 inputTensor = Input(shape = (n_criterias*itemCount,)) for i in range(itemCount): group.append(Dense(1, kernel_regularizer=l2(reg), activation='tanh')(Lambda(lambda x: x[:,n_criterias*i:n_criterias*(i+1)], output_shape=((n_criterias,)))(inputTensor))) intermediate = Dropout(rate=dropout_rate, name='intermediate')(Concatenate()(group)) layer1 = 200 layer2 = 100 x = Dense(layer1, kernel_regularizer=l2(reg), activation='tanh')(intermediate) coding = Dense(layer2, kernel_regularizer=l2(reg), activation='tanh', name='coding')(x) x = Dense(layer1, kernel_regularizer=l2(reg), activation='tanh')(coding) outputTensor = Dense(itemCount, kernel_regularizer=l2(reg), activation='tanh')(x) model = Model(inputs = inputTensor, outputs = outputTensor)

Multiple dropout layers in class_head and box_head for fast MCDropout
I am using Faster RCNN Resnet 101 of Tensorflow's API.
I added multiple dropout layers to class_head and box_head prediction methods for a faster MC Dropout sampling. For instance, I want to receive 300*50 detections if I add 50 dropout layers. Then I will compute variance of output.
An example from box_head.py after my change is below.
When I start evaluation I receive an error. I think the model can't understand that I am sending a batch of 50 outputs, but considers it as one output since I am using the same image. The problem is with decoding the boxes using anchors in second stage before NMS.
I wonder if there is a way to overcome this problem. I really need to speed up the inference with a trick like this one.
# New method def multi_dropout(self, features, keep_prob=0.5, is_training=True, n_samples=10): output = [] for dropout_index in range(n_samples): output.append(slim.dropout( features, keep_prob=keep_prob, is_training=is_training)) stacked = tf.stack(output) return stacked def predict(self, features, num_predictions_per_location=1): """Predicts boxes. Args: features: A float tensor of shape [batch_size, height, width, channels] containing features for a batch of images. num_predictions_per_location: Int containing number of predictions per location. Returns: box_encodings: A float tensor of shape [batch_size, 1, num_classes, code_size] representing the location of the objects. Raises: ValueError: If num_predictions_per_location is not 1. """ if num_predictions_per_location != 1: raise ValueError('Only num_predictions_per_location=1 is supported') spatial_averaged_roi_pooled_features = tf.reduce_mean( features, [1, 2], keep_dims=True, name='AvgPool') flattened_roi_pooled_features = slim.flatten( spatial_averaged_roi_pooled_features) if self._use_dropout: # New controls if self._is_training: flattened_roi_pooled_features = slim.dropout( flattened_roi_pooled_features, keep_prob=self._dropout_keep_prob, is_training=self._is_training) else: flattened_roi_pooled_features = self.multi_dropout(features=flattened_roi_pooled_features, keep_prob=self._dropout_keep_prob, is_training=self._is_training, n_samples=50) number_of_boxes = 1 if not self._share_box_across_classes: number_of_boxes = self._num_classes with slim.arg_scope(self._fc_hyperparams_fn()): box_encodings = slim.fully_connected( flattened_roi_pooled_features, number_of_boxes * self._box_code_size, activation_fn=None, scope='BoxEncodingPredictor') box_encodings = tf.reshape(box_encodings, [1, 1, number_of_boxes, self._box_code_size]) return box_encodings
ERROR:
Traceback (most recent call last): File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/framework/ops.py", line 1576, in _create_c_op c_op = c_api.TF_FinishOperation(op_desc) tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 45000 and 900 for 'SecondStagePostprocessor/Decode/mul' (op: 'Mul') with input shapes: [45000], [900]. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./legacy/eval.py", line 141, in <module> tf.app.run() File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/util/deprecation.py", line 272, in new_func return func(*args, **kwargs) File "./legacy/eval.py", line 138, in main graph_hook_fn=graph_rewriter_fn) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/legacy/evaluator.py", line 187, in evaluate ignore_groundtruth=eval_config.ignore_groundtruth) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/legacy/evaluator.py", line 79, in _extract_predictions_and_losses detections = model.postprocess(prediction_dict, true_image_shapes) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 1175, in postprocess mask_predictions=mask_predictions) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 1595, in _postprocess_box_classifier refined_box_encodings_batch, proposal_boxes) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 1665, in _batch_decode_boxes tiled_anchors_boxlist) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/core/box_coder.py", line 86, in decode return self._decode(rel_codes, anchors) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/models/research/object_detection/box_coders/faster_rcnn_box_coder.py", line 111, in _decode w = tf.exp(tw) * wa File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/ops/math_ops.py", line 850, in binary_op_wrapper return func(x, y, name=name) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/ops/math_ops.py", line 1094, in _mul_dispatch return gen_math_ops.mul(x, y, name=name) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/ops/gen_math_ops.py", line 4936, in mul "Mul", x=x, y=y, name=name) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/util/deprecation.py", line 454, in new_func return func(*args, **kwargs) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/framework/ops.py", line 3155, in create_op op_def=op_def) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/framework/ops.py", line 1731, in __init__ control_input_ops) File "/home/a21/.virtualenvs/tf/lib/python3.5/sitepackages/tensorflow/python/framework/ops.py", line 1579, in _create_c_op raise ValueError(str(e)) ValueError: Dimensions must be equal, but are 45000 and 900 for 'SecondStagePostprocessor/Decode/mul' (op: 'Mul') with input shapes: [45000], [900].
I also tried sending the output to box decoder one by one, but again received an error
ValueError: Shape must be rank 3 but is rank 4 for 'BatchMultiClassNonMaxSuppression/map/while/Slice' (op: 'Slice') with input shapes: [1,1,1,4], [3], [3].