Can I run Processing.org Python sketch from a Jupyter Notebook and pass data?
I was looking for a way to run "Processing in Python" scripts out of Jupyter notebooks so I installed calysto_processing. Then I discovered it wants to speak Javascript but I wanted to stay in Python. I want to launch and feed a sketch so the user can change the POV as the data loads, changes, etc.
import numpy as np
import processing as pg
aData = np.load("myDataFile",...)
sketch = pg.launch("mySketch")
while True:
sketch.update(aData.getNextBatch())
See also questions close to this topic

How does sklearn GridsearchCV in combination with pipline work?
I've some trouble understanding how the GridsearchCV really works, in combination with a self defined transformator.
What I want to achieve? I want to implement a Transformer/Estimator, which allows switching between some methods depending on a parameter, because I want to include these different methods inside the gridsearch.
Example: I have a self defined Transformer called Scaler(), which either chooses MinMaxScaler or StandardScaler. (just for simplicity)
class Scaling(): def __init__(self, **params): self.method=None self.params = {} print("INITIATING CLASS") def fit(self, X, y=None): return self def transform(self, X): print("TRANSFORMING", X) if self.method == "minMax": self.scaler = MinMaxScaler(feature_range=self.params["feature_range"]) elif self.method == "std": self.scaler = StandardScaler() return self.scaler.fit_transform(X) def get_params(self, **params): return {**StandardScaler().get_params(), **MinMaxScaler().get_params(), **{"method":""} } def set_params(self, **params): print("SETTING PARAMETER") self.method = params["method"] self.params = params
This is my example data:
data = np.array([1,2,3,4,5,6,7,8,9,10]).reshape(1,1) y = [2,3,4,5,6,7,8,9,10,11]
My pipline:
p = Pipeline([('scaler', Scaling()), ('model', LinearRegression())])
My paramgrid and Gridsearch
hyperparams = { 'scaler__feature_range' : [(0,1), (100,10)], 'scaler__method':["minMax"] } clf = GridSearchCV(p,hyperparams, cv=2) clf.fit(data, y)
It actually works, but I'm really confused about the printing logs:
INITIATING CLASS INITIATING CLASS INITIATING CLASS SETTING PARAMETER TRANSFORMING [[ 6][ 7][ 8][ 9][10]] TRANSFORMING [[1][2][3][4][5]] TRANSFORMING [[ 6][ 7][ 8][ 9][10]] INITIATING CLASS SETTING PARAMETER TRANSFORMING [[1][2][3][4][5]] TRANSFORMING [[ 6][ 7][ 8][ 9][10]] TRANSFORMING [[1][2][3][4][5]] INITIATING CLASS SETTING PARAMETER TRANSFORMING [[ 6][ 7][ 8][ 9][10]] TRANSFORMING [[1][2][3][4][5]] TRANSFORMING [[ 6][ 7][ 8][ 9][10]] INITIATING CLASS SETTING PARAMETER TRANSFORMING [[1][2][3][4][5]] TRANSFORMING [[ 6][ 7][ 8][ 9][10]] TRANSFORMING [[1][2][3][4][5]] INITIATING CLASS SETTING PARAMETER TRANSFORMING [[ 1][ 2 [ 3][ 4][ 5][ 6][ 7][ 8][ 9][10]]
I have set cv=2. I would expect it like this. 1. Instatiate all the Transformers 2. set the parameters according the the Gridsearch 3. pass the trainfold through the pipline 4. pass the testfold through the pipline 5. repeat So I would have expect 8 calls to the tranformer method because we need one for the train and one for the test fold. Because of cv=2 we do this 2 times and because we are defining two different values for feature_range inside the paramgrid, we have to multiply it with 2, therefore 8. Whats wrong?
But why are there so many calls of my Scaling class? How can this order of logs be explained? Why is the fully sequence at the end transformed?
Thanks for helping!

How to turn multiple rows into multiple headers headers in pandas dataframe
So I have a pandas dataFrame and I would like to turn two rows into multiple headers. so from
1 A  A  B  B 2 C  D  C  D 3 cat dogmousegoose
to
A  B C  D  C  D 1 cat dogmousegoose
I found
df.columns = df.iloc[0]
to work for 1 row but i want multiple headers from the first and second row Thanks in advance!

VTK rescale to visible data range
I for some reason can not manage to find a reasonable way to rescale the plot color to the current camera setting through the python api. Is there any way where i can just get a vtkDoubleArray of the currently viewed data from which i can call GetRange() to update my DataMapper? A simplified version of my current code looks something like this:
import vtk file_name = 'foo.vtk' field_name = "bar" reader = vtk.vtkUnstructuredGridReader() reader.SetFileName(file_name) reader.ReadAllScalarsOn() reader.ReadAllVectorsOn() reader.Update() output = reader.GetOutput() numvals = 1024 ctf = vtk.vtkColorTransferFunction() ctf.SetColorSpaceToRGB() ctf.AddRGBPoint(1, 1, 1, 0) ctf.AddRGBPoint(0, 0, 0, 0) ctf.AddRGBPoint(0.293069/1.72393, 0, 0, 1) ctf.AddRGBPoint(0.586138/1.72393, 0, 1, 1) ctf.AddRGBPoint(0.861967/1.72393, 0, 1, 0) ctf.AddRGBPoint(1.155040/1.72393, 1, 1, 0) ctf.AddRGBPoint(1.448100/1.72393, 1, 0, 0) ctf.AddRGBPoint(1.723930/1.72393, 0.87843, 0, 1) lut = vtk.vtkLookupTable() lut.SetNumberOfTableValues(numvals) lut.Build() for i in range(0,numvals): rgb = list(ctf.GetColor(float(i)/numvals))+[1] lut.SetTableValue(i,rgb) mapper = vtk.vtkDataSetMapper() mapper.SetInputData(output) mapper.ScalarVisibilityOn() mapper.SetColorModeToMapScalars() mapper.SetLookupTable(lut) mapper.SetScalarModeToUsePointFieldData() mapper.SelectColorArray(field_name) mapper.SetScalarRange(output.GetPointData().GetArray(field_name).GetRange()) actor = vtk.vtkActor() actor.SetMapper(mapper) camera = vtk.vtkCamera() camera.SetPosition(0.5, 0.3, 3); camera.SetFocalPoint(0.5, 0.3, 0); renderer = vtk.vtkRenderer() renderer.AddActor(actor) renderer.SetBackground(1, 1, 1) renderer.SetActiveCamera(camera) render_window = vtk.vtkRenderWindow() render_window.AddRenderer(renderer) render_window.SetSize(800,800) render_window.Render() windowToImageFilter = vtk.vtkWindowToImageFilter() windowToImageFilter.SetInput(render_window) #windowToImageFilter.SetInputBufferTypeToRGBA() windowToImageFilter.ReadFrontBufferOff() windowToImageFilter.Update() writer = vtk.vtkPNGWriter() writer.SetFileName("screenshot.png"); writer.SetInputConnection(windowToImageFilter.GetOutputPort()); writer.Write();

Can't find "Syntax error on tokens, delete these tokens"
I am writing an implementation of Dijkstra's Algorithm for a project using Processing, and I keep getting the error "Syntax error on tokens, delete these tokens", without any indication of which file the error is in, or where in the file it is, and I can't for the life of me find where the issue is.
Any help would be appreciated.
LargeGraph.pde:
ArrayList<Vertex> vertices; ArrayList<Edge> edges; void setup() { size(1000, 1000); vertices = new ArrayList<Vertex>(); edges = new ArrayList<Edge>(); randomSeed(22); fillVertices(); fillEdges(); } void draw() { background(0); stroke(255); for (Vertex v : vertices) { if (vertices.indexOf(v) == 7) { fill(255, 0, 0); } circle((v.x + 1) * 19, (v.y + 1) * 19, 10); fill(255); } for (Edge e : edges) { line((e.v1.x + 1) * 19, (e.v1.y + 1) * 19, (e.v2.x + 1) * 19, (e.v2.y + 1) * 19); } calculateSPT(vertices.get(0)); println(getShortestPathTo(vertices.get(7))); } class Vertex implements Comparable<Vertex> { int x, y, id, csf; double minDistance; Vertex previous; Vertex (int num, int xpos, int ypos) { minDistance = 999999999999999999999999999999.; id = num; x = xpos; y = ypos; csf = 0; } ArrayList<Edge> adjacencies() { ArrayList<Edge> adjacencies = new ArrayList<Edge>(); for (Edge e : edges) { if (e.v1.equals(this)) { adjacencies.add(e); } } return adjacencies; } @Override public int compareTo(Vertex other) { if (this.csf > other.csf) { return 1; } else if (this.csf == other.csf) { return 0; } else { return 1; } } } class Edge { Vertex v1; Vertex v2; int weight; Edge(Vertex v, Vertex v0, int w) { v1 = v; v2 = v0; weight = w; } } void fillEdges() { // for each vertex pick a random number 25 and generate an edge // to a random vertex with a weight based on its distance in space // times a random factor for (Vertex v : vertices) { int numEdges = 1; //round(random(2, 5)); //for (int i = 0; i < numEdges; i++) { Vertex vertex = vertices.get(round(random(0, 9))); if (vertex.equals(v)) { vertex = vertices.get(round(random(0, 9))); } int weight = int(dist(v.x, v.y, vertex.x, vertex.y) * random(0, 2)); edges.add(new Edge(v, vertex, weight)); //} } } void fillVertices() { for (int i = 0; i < 10; i++) { int x = round(random(0, 50)); int y = round(random(0, 50)); vertices.add(new Vertex(i, x, y)); } }
Dijkstra.pde:
import java.util.Collections; import java.util.PriorityQueue<E>; void calculateSPT(Vertex start) { start.minDistance = 0.; PriorityQueue<Vertex> vertexQueue = new PriorityQueue<Vertex>(); vertexQueue.add(start); while (!vertexQueue.isEmpty()) { Vertex u = vertexQueue.poll(); for (Edge e : u.adjacencies()) { Vertex v = e.v2; int weight = e.weight; double costSoFar = u.minDistance + weight; if (costSoFar < v.minDistance) { vertexQueue.remove(v); v.minDistance = costSoFar; v.previous = u; vertexQueue.add(v); } } } } ArrayList<Vertex> getShortestPathTo(Vertex target) { ArrayList<Vertex> path = new ArrayList<Vertex>(); for (Vertex v = target; v != null; v = v.previous) { path.add(v); } Collections.reverse(path); return path; }

Processing FFT BackgroundNoise?
I was doing an "liveaudiovisualizer" in Processing. Everything is working fine except of this:
Even if I don't play any music/sound in the background my program shows that there is sound. I don't know why this happens..
Here's my code:import ddf.minim.analysis.*; import ddf.minim.*; Minim minim; AudioInput in; int buffer_size = 512; float sample_rate = 44100; FFT fft; void setup() { size(480, 600); minim = new Minim(this); in = minim.getLineIn(Minim.MONO,buffer_size,sample_rate); fft = new FFT(in.bufferSize(), in.sampleRate()); fft.window(FFT.HAMMING); } void draw() { background(0); stroke(255); fft.forward(in.mix); for(int i = 0; i < fft.specSize(); i++) { ellipse(i, 300, 1, fft.getBand(i) * 100); } float current = fft.getBand(0) * 100; stroke(255, 0, 0); rect(200, 300, 20, current); }
Edit: By now I played around a bit and found out, that I'm analyzing the output and the input, but i just want the output.. how can I do that?

Gif and Processing 3
I've imported my project from processing 3 and I'm facing a problems with loading a gif inside a PFragment, P stand for processing.
I've tried glide but it's asking for an imageview which i can't provide. I've tried to use the findviewbyid with the created fragment but it's trough me an nullpointer exception, and also with the created frame layout which gave me the following error : cannot cast framelayout to imageview. Here's my code of mainActivity The sketch load perfectly so I wont upload it. the part which is not functioning is the glide part.
public class MainActivity extends AppCompatActivity { private PApplet sketch; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); FrameLayout frame = new FrameLayout(this); frame.setId(CompatUtils.getUniqueViewId()); setContentView(frame, new ViewGroup.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT)); sketch = new MySketchfromProcessing(this); PFragment fragment = new PFragment(sketch); fragment.setView(frame, this); ImageView imageView = findViewById(?????); Glide.with(this).load("trophy.gif").into(imageView); }

Error while saving file locally in Jupyter notebooks
I keep getting this error while saving a file locally while working on my laptop. Any way to save this.
Unexpected error while saving file: Untitled%%20Folder/knn.ipynb [Errno 2] No such file or directory: '/home/ieshaan/Desktop/Python/Untitled%%20Folder/knn.ipynb'
This is Ubuntu 18.04, and jupyter version 4.4.0.

Poliastro plot not showing up in Jupyter
Just started learning poliastro and it's already not working. Tried plotting their example code here https://github.com/poliastro/poliastro:
%matplotlib inline from poliastro.examples import molniya molniya.plot()
but nothing came out :( Doesn't work with
plt.show()
either 
Add multiple xml together and then convert to csv
I have 300,000 xml files I need to put together as one big xml file. I then need to convert that to a csv for data analyzing. Any suggestions?

Add multiple values in a single cell of an array
I am trying to fit all data in an array where for any given user id, I should be getting all the Bookids he has purchased. users_list contains data of all users(user ids). purchase_history contains data of all books bought. So for a given user id, there might be many books bought. So, I want to write a program that maps all the books bought by a UserID with the userid.
books_purchased=[] books_purchased_buffer=[] users=[] users=users_list['UserID'].values for i in range(5): books_purchased_buffer=purchase_history.loc[purchase_history['UserID']==users[i]] books_purchased=books_purchased_buffer['BookID'] books_purchased.values
Out:
array([117564, 105481, 169328, 167437, 154892, 114400, 114845, 121906, 132243, 109177, 165478, 117578], dtype=int64)
In the output, I want only 5 values since I have run the loop for 5 times. I dont mind having values separated by ';' to distinguish various books ids for a userid. Ideal output: Ideal output is array([117564 105481 169328, 167437, 154892, 114400 114845 121906, 132243 109177 165478 117578], dtype=int64)
PS: I am new to Python and data science

How to optimize my logistic regression model?
My model is something like Y = XB where Y is binary and X is a matrix with 200 columns (and 400,000) records. My success rate overall is 12%, ie, 12% of my Y vector is 1 and 88% is showing 0.
I am trying to use a logistic regression model to predict the 1s, all of the assumptions of a logistic model are met (the X table is not correlated, there is no duplicated record and there is linearity of independent variables and log odds).
I divided the data between test and training and the model from training set is getting it right 90% of the time, which for me is a low number. I suspect I need to apply a non linear transformation to X to increase the relationship between my log odds and X).
What else can I do to increase the relationship between them and therefore the accuracy of the model?

Predict Model Reciprocal (1/x) function from Statsmodels
I have been wracking my brain at this for 2 days now and cannot seem to find any prediction model that is based off the reciprocal function. What I am trying to do is base a prediction model off the data from my pandas Dataframe. The standard ones I have tried (ex: Linear Regression) have come back with very poor results, so I graphed the data and saw that it is because the data has a (1/x) relationship, instead of being linearly correlated.
#Create a prediction model: result = smf.ols(data = temp2, formula = "n_guns_involved ~ injured_killed").fit() result.summary()
#Bad results, let's see what reciprocal function is like result2 = smf.ols(data = temp2, formula = "n_guns_involved ~ I(1/injured_killed)").fit() result2.summary()
ik_temp = pd.DataFrame({"injured_killed": np.arange(0,20.0,0.01)}) ik_temp['Linear_prediction'] = result.predict(ik_temp) ik_temp['Reciprocal_prediction'] = result2.predict(ik_temp) plt.scatter(temp2['n_guns_involved'], temp2['injured_killed']) plt.plot(ik_temp['injured_killed'], ik_temp['Linear_prediction'], color = 'red') plt.plot(ik_temp['injured_killed'], ik_temp['Reciprocal_prediction'], color = 'green')
For reference, the purple line above is the Reciprocal function (1/x) that I just plotted onto the same graph  as you can see it needs to be adjusted to fit the data and I unfortunately have not been able to do so. The red line is the "Linear" line and the "Green" line is my attempt at the reciprocal function (via result2 in the snippet above).
Please also note that I tried to use X**(1), but of course it failed.
Any and all help is greatly appreciated  Thank you!