Why does this error pop up while working with Deep Q learning?
I have been working with Deep Q Learning on Windows 10 Machine. I have version 0.4.1 of pytorch with NVIDA graphics card.
def select_action(self, state):
probs = F.softmax(self.model(Variable(state, volatile = True))*7)
action = probs.multinomial()
return action.data[0,0]
From this section of the code, I keep getting this error:
TypeError: multinomial() missing 1 required positional arguments: "num_samples"
If any other information is needed, It will be very quickly provided.
1 answer

Based on the documentation you didn't specify the
num_samples
ofmultinomial
function to draw your multinomial distribution.torch.multinomial(input, num_samples, replacement=False, out=None)
Returns a tensor where each row contains num_samples indices sampled from the multinomial probability distribution located in the corresponding row of tensor input.
Change the code as below:
def select_action(self, state): probs = F.softmax(self.model(Variable(state, volatile = True))*7) action = probs.multinomial(1) # 1 is the number of samples to draw return action.data[0,0]
See also questions close to this topic

How to randomly call a function within a given time interval ?  Python
I am creating a simulator which tries to simulate an office Door Sensor. The office is usually open from 8.00am to 6.00pm. The function would give me timestamps when the door was opened and when the door was closed ( in Timestamps ). The door takes 10 seconds to close.
So I want to create a function which I would randomly call between 8.00am to 6.00pm. That function would give me timestamps of when the door was opened and closed.
Simulator runs a loop per second , and I would want to call this function randomly between 8.00am to 6.00pm.
I'd appreciate if you can suggest me a different approach.
Thanks.

How to fix inconsistent return statement in python?
I am new to python and i have this project I am working on a small project with two functions where the first returns the index of the first time a difference is spotted in a string. The next function does that but in a list of strings. Now, due to my being an amateur, i have used an excessive amount of if and else statements which resulted in too many return statements especially in the second function, and i get the error [R1710: inconsistentreturnstatements]. How do i fix it and can anybody give me clear examples to better pieces of code? Sorry for the question being so long.
IDENTICAL = 1 def singleline_diff(line1, line2): """ Inputs: line1  first single line string line2  second single line string Output: Returns the index where the first difference between line1 and line2 occurs. Returns IDENTICAL if the two lines are the same. """ len1 = len(line1) len2 = len(line2) minimum_length = min(len1, len2) if len1 != len2: if minimum_length == 0: return 0 for idx in range(minimum_length): if line1[idx] == line2[idx]: pass else: return idx return idx + 1 for idx in range(len1): if line1[idx] == line2[idx]: pass else: return idx return IDENTICAL def multiline_diff(lines1, lines2): """ Inputs: lines1  list of single line strings lines2  list of single line strings Output: Returns a tuple containing the line number (starting from 0) and the index in that line where the first difference between lines1 and lines2 occurs. Returns (IDENTICAL, IDENTICAL) if the two lists are the same. """ line_no = singleline_diff(lines1, lines2) len_lines1, len_lines2 = len(lines1), len(lines2) if len_lines1 == len_lines2: if (len_lines1 or len_lines2) == 0: if len_lines1 == len_lines2: return (IDENTICAL, IDENTICAL) else: idx = singleline_diff(lines1[line_no], lines2[line_no]) return (line_no, idx) else: idx = singleline_diff(lines1[line_no], lines2[line_no]) if line_no == IDENTICAL: return (IDENTICAL, IDENTICAL) elif line_no != IDENTICAL: return (line_no, idx) else: return (line_no, 0)

how to set sqlite3 select delimeter?
How can I change sqlite3
SELECT
output delimiter in python? I have lot's of created databases on lot's of PC's, so changing delimiter for each pc from sqlite cli isn't possible. Expected format istext1text2text3
. Thx. 
Uber Ludwig Toolbox: export generated model to Google platform ML Engine
In Uber Ludwig toolbox the training from the scratch produced following 10 files:
 checkpoint
 model_hyperparameters.json
 model_weights.data00000of00001
 model_weights.index
 model_weights.meta
 model_weights_progress.data00000of00001
 model_weights_progress.index
 model_weights_progress.meta
 train_set_metadata.json
 training_progress.p
I would like to export them (or only the needed ones) into one file saved_model.pb to be able to load it to my ML Engine Google platform.
I tried the script
freeze_graph.py
but it requires as an input a .pb file which I do not have. (I have only these 10 files mentioned before). How to proceed to get this one saved_model.pb file? I saw several threads but none was working for me. 
Why is the learning rate already very small (1e05) while the model convergences too fast?
I am training a video prediction model.
According to the loss plots, the model convergences very fast while the final loss is not small enough and the generation is not good.
Actually, I have test the
lr=1e04
andlr=1e05
, the loss plots drop down a little more slowly, but it's still not ideal. But I thinklr=1e05
should be small enough, isn't it?How should I fix my model or the hyper parameters?

mxnet cpu memory leak when running inference on model
I'm running into a memory leak when performing inference on an mxnet model (i.e. converting an image buffer to tensor and running one forward pass through the model).
A minimal reproducable example is below:
import mxnet from gluoncv import model_zoo from gluoncv.data.transforms.presets import ssd model = model_zoo.get_model('ssd_512_resnet50_v1_coco') model.initialize() for _ in range(100000): # note: an example imgbuf string is too long to post # see gist or use requests etc to obtain imgbuf = ndarray = mxnet.image.imdecode(imgbuf, to_rgb=1) tensor, orig = ssd.transform_test(ndarray, 512) labels, confidences, bboxs = model.forward(tensor)
The result is a linear increase of RSS memory (from 700MB up to 10GB+).
The problem persists with other pretrained models and with a custom model that I am trying to use. And using garbage collectors does not show any increase in objects.
This gist has the full code snippet including an example imgbuf.
Environment info:
python 2.7.15
gcc 4.2.1
mxnetmkl 1.3.1
gluoncv 0.3.0

Advice for building AI for Phase 10 (card game)
I'm an experienced developer, just not in AI or neural networks.
Looking to build an AI for the card game Phase 10. For each hand, each player's object is to complete and lay down the current phase, and then rid their hand of remaining cards by discarding them on laiddown Phases.
What are my options for building this game and AI? I'm open to any framework and AI libs. Would be cool to see some github projects using these framework or language. I've seen some similar questions, but they've seen pretty outdated.
Not a requirement, but would be awesome to be supported via web and thrown up into AWS.

Keras will work with one approach to tokenize words, but not another
I have a dataframe with the following data:
gender gender_count name vectorized indexised ends_a 18948 1 308 macel [1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] [12, 0, 2, 4, 11, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan] 0
I use two approaches to tokenize words:
number of letters in
az
dictionary:char_dict = [(key,value) for key, value in enumerate(string.ascii_lowercase)] def vectorize(text, dictionary=char_dict): vector = np.zeros(len(dictionary)) for i, word in dictionary: num = 0 for w in text: if w == word: num += 1 if num: vector[i] = num return vector
index of letter position in
az
dictionary:def indexise(name): #longest word len: 15 name = name.lower() word = [string.ascii_lowercase.index(x) for x in name] for x in range(15len(word)): word.append(np.nan) return word
As suggested by @hobbs below, if I change the empty letters in this approach from
np.nan
to say2
, the output is not zero, but very close to zero whist first approach with the same model generates 0.72 accuracy (20x better):Logloss (cost function): training (min: 5.469, max: 5.777, cur: 5.470) validation (min: 7.109, max: 7.452, cur: 7.445) Accuracy: training (min: 0.023, max: 0.119, cur: 0.025) validation (min: 0.027, max: 0.137, cur: 0.031)
I use the same model in both approaches, the only things that's different is the
input_dim
.model = Sequential() model.add(Dense(32, input_dim=26, activation='relu')) model.add(Dense(1, activation='relu')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.fit(X1, Y1, epochs=50, batch_size=1000, validation_split=.2, callbacks=[PlotLossesKeras()], verbose=False)
What I don't understand is why the first approach "works"  the model will train and validate and the second approach is always zero for validation/training and loss?
What makes such a big difference?

Convert different periods into same format
What is the best way to convert different periods like 2 months, 65 days, 5 years, 4 weeks, 5 months 4 days, 2000 days into one consistent period using
Scala
orJava
orAI
technologies?So that they all can be compared and I can find the one with a longest period of time  2000 days in the above case.

Does pytorch do eager pruning of its computational graph?
This is a very simple example:
import torch x = torch.tensor([1., 2., 3., 4., 5.], requires_grad=True) y = torch.tensor([2., 2., 2., 2., 2.], requires_grad=True) z = torch.tensor([1., 1., 0., 0., 0.], requires_grad=True) s = torch.sum(x * y * z) s.backward() print(x.grad)
This will print,
tensor([2., 2., 0., 0., 0.]),
since, of course, ds/dx is zero for the entries where z is zero.
My question is: Is pytorch smart and stop the computations when it reaches a zero? Or does in fact do the calculation "
2*5
", only to later do "10 * 0 = 0
"?In this simple example it doesn't make a big difference, but in the (bigger) problem I am looking at, this will make a difference.
Thank you for any input.

loss.backward() of pytorch 1.0 is slow on Tesla V100
I run a UNet 3D network on both Titan XP and V100, and find that V100 is much slower than Titan XP. The reason is
loss.backward()
is taking 5 seconds/epoch in V100 compared to 1 second/epoch in Titan XP. Can anyone give some clues? Thanks.Our environment:
 Anaconda environment.
Python 3.6.6
Pytorch 1.0.1
.CUDA 10.0
.CUDNN 7.4.2
.NVIDIA driver 410.79
.

NotImplementedError: Cannot transform _Boolean constraints Cannot transform _Boolean constraints when running svi.step() in pyro (pyroppl)
I'm trying to implement a bayesian network in pyro (pyroppl). Here's my model code:
import torch,pyro from pyro.infer import SVI, Trace_ELBO from torch.distributions.constraints import unit_interval,boolean,interval import pyro.contrib.autoguide as ag import pyro.optim as opt from pyro.distributions import Bernoulli def x_model(): pT = pyro.param("pT", torch.tensor(.1), constraint=unit_interval) pF = pyro.param("pF", torch.tensor(.0001), constraint=unit_interval) pS = pyro.param("pS", torch.tensor([.1,.9]), constraint=unit_interval) pA = pyro.param("pA", torch.tensor([[.0001,.85],[.99,.5]]),constraint=unit_interval) pL = pyro.param("pL", torch.tensor([.001,.88]),constraint=unit_interval) pR = pyro.param("pR", torch.tensor([.01,.75]), constraint=unit_interval) T = pyro.sample("T", Bernoulli(0.1)) F = pyro.sample("F", Bernoulli(.0001)) S = pyro.sample("S", 1) #S=TRUE A = pyro.sample("A", Bernoulli(pA[T.long(),F.long()])) L = pyro.sample("L", Bernoulli(pL[A.long()])) R = pyro.sample("R", 1) #R=TRUE # Inference for P(AS,R) svi = SVI(p_model, \ guide=ag.AutoDiagonalNormal(p_model), \ optim=opt.Adam({'lr': .001}), \ loss =Trace_ELBO()) for step in range(2000): svi.step() print('.', end='')
During the inference phase, I get this error:
KeyError Traceback (most recent call last) /usr/local/lib/python3.6/distpackages/torch/distributions/constraint_registry.py in __call__(self,constraint) 138 try: > 139 factory = self._registry[type(constraint)] 140 except KeyError: KeyError: <class 'torch.distributions.constraints._Boolean'>
I tried changing my constraints in the params to interval or removing the constraint totally to no avail. This is running in a google colab notebook.