Pandas stacked bar plotting with different shapes
I'm currently experimenting with pandas and matplotlib. I have created a Pandas dataframe which stores cards from a trading card game (Magic The Gathering). Each card has a specific coloridentity
, stored in a column (for example green, red, etc). To play cards you have to pay their costs in the game which can be simply said from 0 to X (called cmc
and also stored in a column). What I now want to do is to make a stacked bar plot where I can see how many entries per each cost exist. And I want to do that for all colors and stack them above.
My thoughts so far:
#get all unique values of colors
unique_values = df['coloridentity'].unique()
#Create two dictionaries. One for the number of entries per cost and one
# to store the different costs for each color
color_dict_values = {}
color_dict_index = {}
for u in unique_values:
temp_df = df['cmc'].loc[df['coloridentity'] == u].value_counts()
color_dict_values[u] = np.array(temp_df)
color_dict_index[u] = temp_df.index.to_numpy()
width = 0.4
p1 = plt.bar(color_dict_index['G'], color_dict_values['G'], width, color='g')
p2 = plt.bar(color_dict_index['R'], color_dict_values['R'], width,
bottom=color_dict_values['G'], color='r')
plt.show()
So but this gives me an error because the line where I say that the bottom of the second plot shall be the values of different plot have different numpy shapes.
Does anyone know a solution? I thought of adding 0 values and also filling up the index somehow, but I don't know if this is the best solution, and if yes how the best way would be to solve it.
See also questions close to this topic

How to train a multiclass image classifier with Keras
I was following the 2 class image classification tutorial here and wanted to convert it into a multiclass classifier.
I am trying to train a model to predict the brand of a watch from 17 classes. My accuracy after 50 epochs is only 21.88% so I'm not really sure where I am going wrong or if I am even doing this right.
Here is my code:All the images are in their own separate folders under the /data or /valid folders.
Ex: ../watch finder/data/armani
Ex2: ../watch finder/data/gucciimport numpy as np import matplotlib.pyplot as plt import os import cv2 from keras.utils import np_utils from keras.preprocessing.image import ImageDataGenerator from keras.models import Sequential from keras.layers import Conv2D, MaxPooling2D from keras.layers import Activation, Dropout, Flatten, Dense from keras import backend as K import keras.optimizers img_width, img_height = 210, 210 train_data_dir = 'C:/Users/Adrian/Desktop/watch finder/data' validation_data_dir = 'C:/Users/Adrian/Desktop/watch finder/valid' nb_train_samples = 4761 nb_validation_samples = 612 epochs = 50 batch_size = 16 model = Sequential() model.add(Conv2D(32, (3, 3), input_shape=input_shape)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(32, (3, 3))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(64, (3, 3))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Flatten()) model.add(Dense(64)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(17)) model.add(Activation('softmax')) model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy']) train_datagen = ImageDataGenerator( rescale=1. / 255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True) test_datagen = ImageDataGenerator(rescale=1. / 255) train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical') validation_generator = test_datagen.flow_from_directory( validation_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical') model.fit_generator( train_generator, samples_per_epoch=nb_train_samples // batch_size, epochs=epochs, validation_data=validation_generator, validation_steps=nb_validation_samples // batch_size)
This is my first epoch:
Epoch 1/50 18/18 [==============================]  8s 422ms/step  loss: 4.1104  accuracy: 0.0833  val_loss: 2.8369  val_accuracy: 0.0592
And this is my 50th/last epoch:
Epoch 50/50 18/18 [==============================]  7s 404ms/step  loss: 2.4840  accuracy: 0.2188  val_loss: 3.0823  val_accuracy: 0.1795
I am fairly certain I am doing something wrong here but I am really new to deep learning so I'm not sure what that something is. All the help is appreciated.
Also, each brand of watch has between 300400 images and each image size is the same at 210x210.

Find all possible combinations of even or odd numbers and perform mathematical operations on the result
I am trying to find all possible combinations from taking one integer from each of three lists. The combinations must contain either all even or all odd integers.
I would then like to find the sum of the squares of the integers in each combination.
Finally I would like to create a dictionary that uses this value as the key and the combination it used as its stored value.
E.g. for the combination (1,3,1): the key would be the integer 11 (from the sum (1^2) + (3^2) + (1^2)) and the value stored would be (1,3,1)
My code so far:
lists = [[0,1,2,3,4,5,6],[0,1,2,3,4,5,6],[0,1,2,3,4,5,6]] combos = np.array([list(i) for i in np.array(np.meshgrid(*values)).T.reshape(1,len(values))])
This takes three lists that each contain the integers 06
and uses an element from each list to create a combination of three integers
All possible combinations result: [0,0,0], [0,1,0], [0,2,0], [1,0,0] ... [6,6,6]

Is it possible to import a variable without all other data?
If i want to import a variable from another file , lets say file name is 'USA' and i want to import only the 'population' variable.
I do it by
From USA import population
All is good, all is working. But it also imports everything else from the 'USA' file, which means that if i had print Function variables it will print them too at the file i imported the single variable to..
Is it possible to import a variable without importing all the functions, or at least hide the print functions somehow?

How to apply value_counts to all elments of a DataFrame?
It is easy to apply value_counts to a Series:
In [1]: import pandas as pd In [2]: a = pd.DataFrame([[2,3],[2,2],[3,2],[2,1]]) In [3]: a Out[3]: 0 1 0 2 3 1 2 2 2 3 2 3 2 1 In [4]: a[0].value_counts() Out[4]: 2 3 3 1 Name: 0, dtype: int64
I need something like
In [5]: a.value_counts() Out[5]: 2 5 3 2 1 1 dtype: int64
However,
a.value_counts()
returns'DataFrame' object has no attribute 'value_counts'
.How could I apply value_counts to the element of a DataFrame?

Does it make sense to replace nan values by 99999?
how does it make sense to replace
nan
values in the dataframe by the value99999
? I found it here, example 3: https://www.geeksforgeeks.org/pythonpandasdataframereplace/df.replace(to_replace = np.nan, value =99999)
Maybe
99999
should simply representinfinite
, but which intention could be behind this operation? Any ideas, guesses? :/I'd appreciate any advise!

Python datetime problem converting time format
I would like to convert the following time format which is located in a panda dataframe column
100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400
I would like to transform the previous time format into a standard time format of HH:MM as follow
01:00 02:00 03:00 ... 15:00 16:00 ... 22:00 23:00 00:00
How can I do it in python?
Thank you in advance

Python Generating 3D sphere in numpy
So currently I've made a numpy 3D array of zeros, but I am trying to create a bunch of different spheres that can be represented by a numpy array. However, I am stuck on how to actually do that. How would I be able to create a numpy array representing a sphere if I had a specified center and a specified radius?

Is there an efficient way to find averages of boxes within a numpy array?
Explanation
I have a bunch of 450x450 images, and I am looking at each color channel individually. So what I have is a 1D numpy array of length 450*450*numImages. I want to take a box of some size, say 3x3 or 5x5 around each pixel and get the average of those pixel values. So by the end I will still have a 450*450*numImages array, it will just now hold those averages. Imagine something like this where we have one 14x14 image and we're just looking at one of the color channels and taking a 3x3 box average:
Current Implementation
iWidth=450 def boxAverage(imageArr, box_scale=1): avgs=[] for i in range(len(imageArr)): avg=0 avgCount=0 for j in range(box_scale,box_scale): for k in range(box_scale*iWidth,box_scale*iWidth,iWidth): try: temp = imageArr[i+k+j] except IndexError: temp = 1 if temp != 1: avg+=temp avgCount+=1 avg/=avgCount avgs.append(avg)
When box_scale = 1, it is a 3x3 box that I am taking the average of and when box_scale=2, it is 5x5, etc.
Problem
This does not scale very well and as I work with more and more images, this gets very slow. So I was wondering if there is a good way using something like numpy to do this in a way that can be done more efficiently. I appreciate any suggestions you might have!

Numpy sampling from a 2d numpy array of probabilities
I have a 2d array of probabilities,
p
that sum to 1.0:p = np.array([[0.05, 0.05, 0.1 , 0. ], [0.4 , 0.1 , 0.1 , 0.2 ]])
I want to have a function that samples an index from the 2d array
p
based on the probabilities. For example, I should return[1,0]
with a probability of0.4
or[0,2]
with a probability of0.1
.If I call the function
sample(p)
which takes the 2d probability array p. Then with the 2dp
array as I've defined above I could expect indices anywhere from[0,0]
to[1,3]
but not including[0,3]
as that has a probability of0
of appearing:sample(p) >> [1,0] sample(p) >> [0,2] sample(p) >> [1,0] sample(p) >> [1,3]
Normally I would use
numpy.random.choice
however it only accepts 1D array probabilities. Is there a way that I can extend this to work for 2d probability arrays of any size . 
Turning JSON to a Data Frame in Python?
I'm trying to transform JSON into a Dataframe. I'm trying to get daily information about stocks and turning it into a dataframe so I can transform it into a graph in order to make analysis.
"Meta Data": { "1. Information": "Daily Prices (open, high, low, close) and Volumes", "2. Symbol": "^BVSP", "3. Last Refreshed": "20200403", "4. Output Size": "Compact", "5. Time Zone": "US/Eastern" }, "Time Series (Daily)": { "20200403": { "1. open": "72241.0000", "2. high": "72241.0000", "3. low": "67802.0000", "4. close": "69538.0000", "5. volume": "10411300" }, "20200402": { "1. open": "70969.0000", "2. high": "73861.0000", "3. low": "70957.0000", "4. close": "72253.0000", "5. volume": "10540200" }, "20200401": { "1. open": "73011.0000", "2. high": "73011.0000", "3. low": "69569.0000", "4. close": "70967.0000", "5. volume": "10093500" }
This is a portion of the JSON file I'm getting from the API. Currently my code looks like this:
import matplotlib.pyplot as plt import pandas as pd import numpy as np response_ibov = requests.get("https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol=^BVSP&apikey=XXX") url = "https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol=^BVSP&apikey=XXX" r = requests.get(url, headers={'open': 'close'}) data_ibov = pd.DataFrame(r.json())
I wanted to organize this like Column Headers = Open, High, Low, Close, Volume Rows = Dates
Does anyone know how to?
Thanks in advance!

Visual Studio Code, interactive matplotlib plots with Jupyter widgets  Python?
I do not know why but when I run jupyter through VS code, the
%matplotlib widget
does not display anything. When I plot a graph, it only shows a line like :Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …
This does not happen when I run Jupyter Lab / notebook in the browser. The line magic works perfectly and I get a plot which I can pan, zoom etc. Does anyone have any idea about how to fix this? It's the only thing which is stopping me to switch to VS Code fully.
VS Code
MacOS 10.15 Catalina
Python 3.7.7 
Python animation strange behavior
I am doing an animation of a launched projectile and running into some odd behavior that I don't understand. I am plotting the animation of a point, representing the object, and also animating the path so that the trajectory shows up behind the object. However, when I do this the way I think I am supposed to do it, the point is shown one step ahead of the trajectory and the trajectory ends up one point shy of finishing. I can work around this by increasing the index number of the trajectory, but then it seems like the index should be out of bounds at the end. I am really confused and could use some help understanding what's going on.
I am working in a Jupyter notebook and have provided a minimally working example below. I am currently just using 10 points in my linspace command and have slowed down the animation a lot so you can see what's happening. If I use the command
line1.set_data(x[0:frames+1], y[0:frames+1])
along withpoint1.set_data(x[frames], y[frames])
in the animate function, then everything looks fine. But that seems like it shouldn't work!What am I missing?
%matplotlib notebook import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation # Global constants g = 9.8 # gravitational field strength in m/s^2 v0 = 40.0 # initial speed in m/s theta = 30.0 # Launch angle in degrees # determine trajectory theta_rad = np.pi*theta/180 t = np.linspace(0, 2*v0*np.sin(theta_rad)/g, 10) x = v0*np.cos(theta_rad)*t y = v0*t*np.sin(theta_rad)  g*t**2/2 # Plot the results fig1 = plt.figure(figsize=(6,4)) ax1 = fig1.add_subplot(111) line1, = ax1.plot(x[0:1], y[0:1], 'b', label='no drag') point1, = ax1.plot(x[0], y[0], 'bo', ms=3) ax1.set_xlim(0, 170) ax1.set_ylim(0, 50) plt.show() # Animation update function def animate(frames): line1.set_data(x[0:frames], y[0:frames]) point1.set_data(x[frames], y[frames]) return ani = animation.FuncAnimation(fig1, animate, frames=len(t), interval=1000, blit=True, repeat=False) plt.show()