matplotlib image to base64 without saving

2021-02-22 22:35 connor449 imported from Stackoverflow

I am creating a wordcloud image with this code.

       wordcloud = WordCloud(
            background_color='white',
            random_state=30,
            width=1500,
            height=1200,
            font_path = font_path,
            prefer_horizontal=1)
        wordcloud.generate_from_frequencies(frequencies=d)

I show the image with matplotlib like this:

        plt.imshow(wordcloud)
        plt.axis('off')
        plt.show()

I am using this as part of a web app. I want to convert this image to base64 and store as a string as a value in a dictionary key for a specific instance. I see a lot of posts about how to convert images to base64 but it looks like they involve saving the figure locally before encoding. How do I do this without saving anywhere so I can just go from image to string?

This code looks kind of like what I want.

import base64
from PIL import Image
from io import BytesIO

with open("image.jpg", "rb") as image_file:
    data = base64.b64encode(image_file.read())

im = Image.open(BytesIO(base64.b64decode(data)))
im.save('image1.png', 'PNG')

If I just did this, would this accomplish my task?

data = base64.b64encode(wordcloud)

1 answer

answered 2021-02-22 23:18 Reti43

If I just did this, would this accomplish my task?

data = base64.b64encode(wordcloud)

No. You need to "save" the image, get that bytestream, and encode that to base64. You don't have to save the image to an actual file; you can actually use a buffer.
```
w = WordCloud().generate('Test')
buffer = io.BytesIO()
w.to_image().save(buffer, 'png')
b64 = base64.b64encode(buffer.getvalue())
```
And to convert that back and display the image
```
img = Image.open(io.BytesIO(base64.b64decode(b64)))
plt.imshow(img)
plt.show()
```

See also questions close to this topic

How to join between different elements of two Pyspark dataframes

I have two dataframes named df1 and df2, the content of data dataframe is as follows.

df1:

line_item_usage_account_id  line_item_unblended_cost    name 
100000000001                12.05                       account1
200000000001                52                          account2
300000000003                12.03                       account3

df2:

accountname     accountproviderid   clustername     app_pmo     app_costcenter      line_item_unblended_cost
account1        100000000001        cluster1        111111      11111111            12.05
account2        200000000001        cluster2        222222      22222222            52

I need the IDs of df1.line_item_usage_account_id that are not in df2.accountproviderid to be added in the join, something like this:

accountname     accountproviderid   clustername     app_pmo     app_costcenter      line_item_unblended_cost
account1        100000000001        cluster1        111111      11111111            12.05
account2        200000000001        cluster2        222222      22222222            52
account3        300000000003        NA              NA          NA                  12.03

the id "300000000003" from df1.line_item_usage_account_id is not found in df2.accountproviderid, so it was added in the new dataframe.

Any idea how to achieve this? I appreciate any help.

Discord bot codes not working (python). How can I fix it?

I recently started coding and decided a good, fun way to learn would be to make a Discord bot. So I did some coding, learnt from a Youtube video and it was all going well but now none of my codes are working and the bot is basically useless. Could someone please look at this code and tell me what's wro

import discord
import os
import requests
import json
import random

client = discord.Client()

@client.event
async def on_ready():
    print('We have logged in as {0.user}'.format(client))

@client.event
async def on_message(message):
    if message.author == client.user:
        return

    if message.content.startswith('Hello'):
        await message.channel.send('Hi!')

client.run(os.getenv('TOKEN'))


sad_words = ["sad","alone","scared","depressed","SAD","Sad"]

starter_encouragements = [
  "You got this. I know you do 😁",
  "I know it's hard, but you'll get through this :)",
]

def get_quote():
  response = requests.get("https://zenquotes.io/api/random")
  json_data = json.loads(response.text)
  quote = json_data[0]['q'] + " -" + json_data[0]['a']
  return(quote)

@client.event
async def on_ready():
  print('We have logged in as {0.user}'.format(client))

@client.event
async def on_message(message):
  if message.author == client.user:
    return

  msg = message.content

  if message.content.startswith('inspire'):
    quote = get_quote()
    await message.channel.send(quote)

  if any(word in msg for word in sad_words):
    await message.channel.send(random.choice(starter_encouragements))


client.run(os.getenv('TOKEN'))

Thanks so much

Why do I get a syntax error python for print statment
I'm getting a syntax error for the following code:
```
counter = 0
i = 0  

print(f"{i}^{counter}")
```
Rearranging the order of multiple boxplots on matplotlib
I would like to rearrange the boxplots such that the Network number starts at 1 and ends at 17. My dataset is already arranged in ascending order, so I'm not sure how to go about doing this.

I would've added a picture, but the site won't let me upload photos. Essentially imagine 17 boxplots next to each other, but instead of starting at Network 1 and ending at Network 17, the networks are jumbled up like 3, 17, 4, 10, 1, 2, etc.

This is my code:
```
lunch_df.boxplot(column=['free_lunch_percentage'], by='network_number',figsize=(15, 7))
```

Extrapolating Data from best fit plot with python

I have created the following scatter plot and linear fit. What code do I write in order to find a temperature value corresponding to a pressure value of 20 mmHg?

import numpy as np
import matplotlib.pyplot as plt 

x=[65,75,85,95,105]
y=[-20,17,42,94,127]

plt.scatter(x,y,color="red")
plt.title("Temperature vs Pressure")
plt.xlabel("Pressure (mmHg)")
plt.ylabel("Temperature (C)")

linear_model=np.polyfit(x,y,1)
linear_model_fn=np.poly1d(linear_model)
x_s=np.arange(0,110)
plt.plot(x_s,linear_model_fn(x_s),color="Blue")

plt.show()

Python To Visualize An Equation

I need someone who can program a python code that visualizes this equation solution in the picture below:

with D=0.21 , σ=2 , t=0

and get a result like that one:
Convert a base64 string encoded image to a tiff format in javascript
I have a base64 image that I need to convert into a .tiff format. Is there a (pure) javascript way to do it please ?

NB : I saw base64 encoding was just a compress of the original file, I did this to recover the original file but now have to transform it to tiff
```
function urltoFile(url, filename, mimeType){
    mimeType = mimeType || (url.match(/^data:([^;]+);/)||'')[1];
    return (fetch(url)
        .then(function(res){return res.arrayBuffer();})
        .then(function(buf){return new File([buf], filename, {type:mimeType});})
    );
}
```
ERROR in ./node_modules/file-base64/index.js Module not found: Error: Can't resolve 'fs'

I'm using vue js and laravel mix. I dl the file-base64 npm package.

And when i compile, i get this error =>

ERROR in ./node_modules/file-base64/index.js Module not found: Error: Can't resolve 'fs'

in the file-base-64 it import var fs = require('fs');

But i have access the fs.d.ts file. It feels the when it compile, it don't have a direct access.

Do someone got the same issue?
confusions about how images are transferred via HTTP and handled in JavaScript
I learned that the body of a response of an HTTP message can be an arbitrary sequence of bytes so it doesn't need to be text/string. I guess that means when we need to transfer any binary files e.g. images / audio files over HTTP and we don't necessarily need to encode them into text/string using methods like base64, instead, we can just transfer them directly (of course with some compression e.g. gzip)

My first question is, I have not encountered any situations where I need to get actual images or other files via an API call e.g. a REST API call, instead normally we just get the URL to where the image is hosted and we use that URL to display the image in the web page and that seems to be more efficient to me. So I wonder in what circumstances we would have to transfer the actual image file over HTTP?

My second question is, if we are transferring the actual binary data over HTTP, I was looking at this and this and I found that there seems to be two ways to handle the response: 1. read the response as blob by response.blob() 2. read the response as arraybuffer by response.arrayBuffer(). I don't know what the differences are here. What determines whether or not we need to use blob or arraybuffer? Please refer to the below code snippet.
```
fetch(request, options).then((response) => {
  response.arrayBuffer().then((buffer) => {
  });
});
// 🤔 differences?
fetch(request, options)..then(response => response.blob())
  .then(images => {
})
```
My last question is, after we get the binary data either via response.blob() or response.arrayBuffer(), it seems like we still need to convert the binary data into string using window.btoa i.e. base64 encoding and turn that into URL and then assign it to a img element's src attribute. I guess I don't understand why we would still have to convert the binary data into URL when we already have the data. Is that the only way to display an image in JavaScript, i.e. using src attribute on img tag? Also, according this this and this , I found there seems to be 2 ways to get the URL to a file: 1. converting the buffer to string using btoa and then use data: URL with that string 2. Use URL.createObjectURL directly. Again, I wonder what the differences are here about these two different methods.

Loading LSA sklearn vector

I trained an LSA model with sklearn, this model was saved with pickle.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import TruncatedSVD
from sklearn.pipeline import Pipeline
import numpy as np
import os.path
from nltk.tokenize import RegexpTokenizer
from nltk.corpus import stopwords
import pickle


def load_data(path,file_name):
    """
    Input  : path and file_name
    Purpose: loading text file
    Output : list of paragraphs/documents and
             title(initial 100 words considred as title of document)
    """
    documents_list = []
    titles=[]
    with open( os.path.join(path, file_name) ,"r") as fin:
        for line in fin.readlines():
            text = line.strip()
            documents_list.append(text)
    print("Total Number of Documents:",len(documents_list))
    titles.append( text[0:min(len(text),100)] )
    return documents_list,titles

document_list,titles=load_data("","a-choose")
#clean_text=preprocess_data(document_list)


# raw documents to tf-idf matrix: 

vectorizer = TfidfVectorizer(stop_words='english', 
                             use_idf=True, 
                             smooth_idf=True)

# SVD to reduce dimensionality: 

svd_model = TruncatedSVD(n_components=4,
                         algorithm='randomized',
                         n_iter=10)

# pipeline of tf-idf + SVD, fit to and applied to documents:

svd_transformer = Pipeline([('tfidf', vectorizer), 
                            ('svd', svd_model)])

svd_matrix = svd_transformer.fit_transform(document_list)

# svd_matrix can later be used to compare documents, compare words, or compare queries with documents
sentence=["football"]
sentence2=["match"]

query=svd_transformer.transform(sentence2)

query_vector = svd_transformer.transform(sentence)

#print(query_vector)
#print(query)


with open("lsa_model.bin","wb") as f:
    pickle.dump(svd_matrix, f)

As a second step, I use another program that load this model, wich will compare word vectors. The problem I am not able to load these vectors, my code is below

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import TruncatedSVD
from sklearn.pipeline import Pipeline
import numpy as np
import numpy as np
from gensim.models import KeyedVectors
import codecs
import pickle

model = pickle.load(open('lsa_model.bin', 'rb'))
query="best"

query_vector = model.transform(query)

print(query_vector)

This generate an error

query_vector = model.transform(query) AttributeError: 'numpy.ndarray' object has no attribute 'transform'

Load, Store and delete config parameters from file using pickle

I am trying to persist the state of my app and so far have found the pickle library.

I found out how to set and get config parameters into a dictionary from When using Python classes as program configuration structures (which includes inherited class attributes), a good way to save/restore?

I have managed to get it to save to an external config file but i don't think I'm doing it right.

I have manages to detect a state change for a checkbox and dropdown widget but still need to work out how to do it for an entry widget.

I haven't managed to delete a config paraameter yet

here is a cut down version to demo:

Config.py

# https://stackoverflow.com/questions/50613665/when-using-python-classes-as-program-configuration-structures-which-includes-in

import pickle

class Config(dict):
    __getattr__ = dict.__getitem__
    __setattr__ = dict.__setitem__
    # Commented out because i do not know how to call it
    # __delattr__ = dict.__delitem__

    def __getstate__(self):
        return self

    def __setstate__(self, state):
        self.update(state)
   
    # commented out as im not sure i need this part
    # def copy(self, **extra_params):
    #     return Config(**self, **extra_params)

App.py

import tkinter as tk
import pickle
from Config import Config

class App(tk.Tk):

    def __init__(self):
        tk.Tk.__init__(self)

        # Init config
        self.config = Config()

        # Load config from file
        self.filename = 'config'  

        infile = open(self.filename, 'rb')
        self.config = pickle.load(infile)
        infile.close()     
         
        # initalise variables from config
        self.cbVar = tk.BooleanVar()
        if "MY_PARAM" in self.config:
            self.cbVar.set(self.config.MY_PARAM)

        self.entryVar = tk.StringVar()
        if "MY_PARAM1" in self.config:
            self.entryVar.set(self.config.MY_PARAM1)
        
        self.omVar = tk.StringVar()
        if "MY_PARAM2" in self.config:
            self.omVar.set(self.config.MY_PARAM2)
        else:
            self.omVar.set("one")

        # Build config page
        cb = tk.Checkbutton(self, text = "Param", variable=self.cbVar, command=self.saveConfig)
        cb.pack()

        # Need to detect changes in this
        entry = tk.Entry(self, textvariable=self.entryVar)
        entry.pack()

        om = tk.OptionMenu(self, self.omVar, "one", "two", "three", command=self.saveConfig)
        om.pack()

    # dummy added as option menu sends 2 args
    def saveConfig(self, dummy=None):
        self.config.MY_PARAM = self.cbVar.get()
        self.config.MY_PARAM1 = self.entryVar.get()
        self.config.MY_PARAM2 = self.omVar.get()
        
        outfile = open(self.filename, 'wb')

        pickle.dump(self.config, outfile)
        outfile.close()        

if __name__ == "__main__":
    app=App()
    app.mainloop()

I would appreciate it if you could help me with these questions

Am I using the config class correctly?
How do I delete a key from the config?
How do I best load from and save to an external file for persisting - preferably I would like the Config class to do this itself?

I'not able to correctly store a ml gridsearch in database with pickle
I'm trying to store a machine learning gridsearch model with pickle into a database using python. I correctly convert the model in pickle, connect to db but when I try to insert the model in the table i just created it produces:

"TypeError: not all arguments converted during string formatting".

Any suggestion would greatly be appreciated (following the code):
```
pickledList = pickle.dumps(estimator_rand_for)
connection = pymysql.connect(xxxxx,yyyyy,zzzz,jjjj)
cursor = db.cursor()
cursor.execute("""create table if not exists tab1 ( pkl longtext)""")
cursor.execute("""insert into tab1 (pkl) VALUE (?)""", [pickledList] )
```
thank you!
Wordcloud for only emojis
I have a text full of emojis.
```
import matplotlib.pyplot as plt
from wordcloud import WordCloud
wordcloud = WordCloud().generate(text)
plt.imshow(wordcloud)
plt.axis("off")
plt.show()
```
when i try to get a wordcloud for those emojis like the code above, it returns an error like this:
```
ValueError: We need at least 1 word to plot a word cloud, got 0.
```
I think the wordcloud library is unable to read emojis. I want to get an output like this(with emojis of course):

Anyone knows how to solve this problem?

Wordcloud is not generating an image

#what is the most frequent words?

data['document_content']=data['document_content'].apply(lambda x:"".join(x.lower() for x in x.split()))
data['document_content']=data['document_content'].str.replace('[^\w\s]',' ')
data['document_content']=data['document_content'].str.replace('\d',' ')

#stopwords
import nltk
nltk.download('stopwords')
from nltk.corpus import stopwords
sw=stopwords.words('english')
data['document_content']=data['document_content'].apply(lambda x: " ".join(x for x in x.split() if x not in sw))

#lemma
from textblob import Word
nltk.download('wordnet')
data['document_content']=data['document_content'].apply(lambda x:" ".join([Word(word).lemmatize() for word in x.split()]))

#punctuation
data['document_content']=data['document_content'].str.replace("rt"," ")

data['document_content'].head()

from os import path
from PIL import Image
from wordcloud import WordCloud,STOPWORDS,ImageColorGenerator
import matplotlib.pyplot as plt

text="".join(i for i in data.document_content)

wc=WordCloud(background_color="black").generate(str(data.document_content))
plt.imshow(wc, interpolation="bilinear")
plt.axis("off")
plt.tight_layout(pad=1)
plt.show()

but it does not return the image but only "Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"

Not sure how I should change the code could you help me with this ?

Wordcloud and list of list

I am trying to create wordclouds with the following data:

my list= [[],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     ['EMF'],
     ['body'],
     [],
     [],
     [],
     [],
     ['water', 'juice'],
     ['What', 'are', 'u', 'doing'],
     [],
     [],
     [],
     [],
     [],
     [],
     [],
     ['EVENT'],
     ['christmas'],
     [],
     ['shalala'],
     ['happy'],
     []]

Generally I do

import numpy as np
import pandas as pd
from PIL import Image
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator

import matplotlib.pyplot as plt

corpus = " ".join(x for x in my_list)
df_wordcloud = WordCloud(background_color='white',max_font_size = 50).generate(corpus)

plt.imshow(df_wordcloud, interpolation='bilinear')
plt.axis("off")
plt.show()

but this time I got the error:

TypeError: sequence item 0: expected str instance, list found

Do you know how to create a wordcloud from a list of lists?

Related questions

matplotlib image to base64 without saving

1 answer

See also questions close to this topic