Problem switching from Binary to PositiveInteger domain
Just started with Pyomo and was taking some samples from this knapsack problem tutorial: https://pyomocontribsimplemodel.readthedocs.io/en/latest/knapsack.html
However for my optimization problem I'd like to be able to utilize the PositiveInteger domain. I tried editing the code above as follows:
v = {'hammer':8, 'wrench':3, 'screwdriver':6, 'towel':11}
w = {'hammer':5, 'wrench':7, 'screwdriver':4, 'towel':3}
limit = 14
items = list(sorted(v.keys()))
# Create model
m = ConcreteModel()
# Initialization **ADDED BY ME**
m.x_init = Set(initialize=['hammer','wrench','screwdriver','towel'])
lb = {'hammer':0,'wrench':0,'screwdriver':0,'towel':0}
ub = {'hammer':2,'wrench':2,'screwdriver':2,'towel':2}
# **ADDED BY ME**
def fb(model,i):
return (lb[i],ub[i])
# Variables
m.x = Var(m.x_init,domain=PositiveIntegers,bounds=fb)
## Changed from
# m.x = Var(items, domain=Binary)
# Objective
m.value = Objective(expr=sum(v[i]*m.x[i] for i in items))
# Constraint
m.weight = Constraint(expr=sum(w[i]*m.x[i] for i in items) <= limit)
# Optimize
solver = SolverFactory('glpk')
status = solver.solve(m)
# Print the status of the solved LP
print("Status = %s" % status.solver.termination_condition)
# Print the value of the variables at the optimum
for i in items:
print("%s = %f" % (m.x[i], value(m.x[i])))
# Print the value of the objective
print("Objective = %f" % value(m.value))
And I am getting the error "No value for uninitialized NumericValue object x[hammer]". I'm really stumped. If the line m.x_init = Set(initialize()) isn't creating a set/unordered list of initialized variables what is it doing? Or is it just not in the form of x['index']?
See also questions close to this topic

Got stuck with re.findall function for regular expressions
I have a string which looks something like
text = "customer: Anna Smith; payment: 123; date: 12022020; customer: Jack; payment: 10.3; date: 20032020"
Now I want to turn it into a list of tuples (which later I can use to create a dictionary):
[('customer', 'Anna Smith'), ('payment', '123'), ('date', '12022020'), ('customer', 'Jack'), ('payment', '10.3'), ('date', '20032020')]
I tried to use re.findall for this purpose in the following way:
re.findall(u'(\w+): (.+?);', text)
Of course it doesn't capture the last pair of a key and a value, because of the semicolon in the regular expression. I think that I need an ifelse if operation here: if the parser encounters a semicolon, then it extracts the words, else it checks for a regular expression for EOL (
\Z
). Please, help 
NameError: name 'all_df' is not defined help please
#!/usr/bin/env python3 # * coding: utf8 * """ Created on Mon Aug 14 2017 Modified on Wed Aug 16 2017 Author: Yanfei Wu Get the past 500 S&P 500 stocks data """ from bs4 import BeautifulSoup import requests from datetime import datetime import pandas as pd import pandas_datareader.data as web def get_ticker_and_sector(url='https://en.wikipedia.org/wiki/List_of_S%26P_500_companies'): """ get the s&p 500 stocks from Wikipedia: https://en.wikipedia.org/wiki/List_of_S%26P_500_companies  return: a dictionary with ticker names as keys and sectors as values """ r = requests.get(url) data = r.text soup = BeautifulSoup(data, 'lxml') # we only want to parse the first table of this wikipedia page table = soup.find('table') sp500 = {} # loop over the rows and get ticker symbol and sector name for tr in table.find_all('tr')[1:]: tds = tr.find_all('td') ticker = tds[0].text sector = tds[3].text sp500[ticker] = sector return sp500 def get_stock_data(ticker, start_date, end_date): """ get stock data from google with stock ticker, start and end dates """ data = web.DataReader(ticker, 'google', start_date, end_date) return data if __name__ == '__main__': """ get the stock data from the past 5 years """ # end_date = datetime.now() end_date = datetime(2017, 8, 14) start_date = datetime(end_date.year  5, end_date.month , end_date.day) sp500 = get_ticker_and_sector() sp500['SPY'] = 'SPY' # also include SPY as reference print('Total number of tickers (including SPY): {}'.format(len(sp500))) bad_tickers =[] for i, (ticker, sector) in enumerate(sp500.items()): try: stock_df = get_stock_data(ticker, start_date, end_date) stock_df['Name'] = ticker stock_df['Sector'] = sector if stock_df.shape[0] == 0: bad_tickers.append(ticker) #output_name = ticker + '_data.csv' #stock_df.to_csv(output_name) if i == 0: all_df = stock_df else: all_df = all_df.append(stock_df) except: bad_tickers.append(ticker) print(bad_tickers) all_df.to_csv('./data/all_sp500_data_2.csv') """ Write failed queries to a text file """ if len(bad_tickers) > 0: with open('./data/failed_queries_2.txt','w') as outfile: for ticker in bad_tickers: outfile.write(ticker+'\n')

GUI interface with python objects and methods
I am looking for a library or some other framework that would allow me to build a light GUI interface for exploring python object methods. The goal behind this is to allow a user to, in a pointandclick fashion, explore and build, without having to either flip back and forth to documentation, or constantly run the
dir
command to see what methods are available.To take a simple example, if I create a list:
x=[0]*5
and I now want to know what I can do with it, I can either go to the python documentation for lists, or run
dir
. Have some sort of GUI that shows the different methods, given the object could be convenient.ADDENDUM
I apologize that this question is vague; I am not quite sure how to put this question.

Modifying Executable python file issue
I have a python project that just need to modify a label.
This is the structure of the project.
If the user click on
GeneraXml.exe
the application will run, but I need to modify a lable. I opened theGeneraXml.spec
file inNotePad++
and based on information, it seems that it reads themain.py
file# * mode: python * block_cipher = None added_files = [ ( 'header.gif', '.' ), ( 'ko.gif', '.' ), ( 'ok.gif', '.' ) ] a = Analysis(['main.py'], pathex=[':\\Users\\UUUU\\Desktop\\New'], binaries=[], datas=added_files, hiddenimports=[], hookspath=[], runtime_hooks=[], excludes=[], win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher, noarchive=False) pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) exe = EXE(pyz, a.scripts, a.binaries, a.zipfiles, a.datas, [], name='GeneraXML', debug=False, bootloader_ignore_signals=False, strip=False, upx=True, runtime_tmpdir=None, console=False )
I opened the
main.py
file withNotePad++
and changed the label... then I tried to recreate again exe file with this command line:pyinstaller onefile main.py
But when I open the new exe file I see just a black window.Could you please help me to understand what is the issue? how can I modify a lable of
.py
file and create executable file? 
How do I make the image generation and hashing more efficient?
I am using code to create a random image and then I check the hash value of this image and compare it to a known hash value. Here is my code:
import os import time import hashlib import numpy from io import BytesIO from PIL import Image def create_image(width = 1920, height = 1080, num_of_images = 100): width = int(width) height = int(height) num_of_images = int(num_of_images) current = time.strftime("%Y%m%d%H%M%S") os.mkdir(current) i=0 for n in range(num_of_images): filename = '{0}/{0}_{1:03d}.jpg'.format(current, n) rgb_array = numpy.random.rand(height,width,3) * 255 image = Image.fromarray(rgb_array.astype('uint8')).convert('RGB') image.save(filename) imagee = open(filename, "rb").read() hashnum = hashlib.md5(imagee).hexdigest() if(hashnum=="B3D740C2F83F7EE120FD16EAED266B43"): image.save(filename) print(filename) else: os.remove(filename) i+=1 print("Done with ", i, end='\r') def main(args): create_image(width = args[0], height = args[1], num_of_images = args[2]) return 0 if __name__ == '__main__': import sys status = main(sys.argv[1:]) sys.exit(status)
So far this program is going through about 1k 17 width 14 height images per second and I was wondering if this could be more efficient. I thought about getting rid of the whole save image and then delete, but I am not really sure how to do that. Also sorry if this post has errors, English is not my first language.

python3: regex, finding specific numbers in a string
I'm trying to find the numbers 2, 3, 4, 255, 100, 101, 200 in the string below and put them into a list, but I cannot figure it out.
<form method="POST"> Node:<br><input type="text" name="node"><br> Command(on, off):<br><input type="text" name="command"><br> <input type="submit" value="Submit"><br> {2: {'name': 'Garden', 'state': 0, 'temperature': '+18.0°C'}, 3: {'name': 'Conservatory', 'state': 0, 'temperature': '+20.5°C'}, 4: {'name': 'Lounge Lamp', 'state': 0, 'temperature': ''}, 255: {'name': 'Test Node', 'state': 0, 'temperature': '+18.0°C'}}<br> {100: {'name': 'Greenhouse', 'state': 0, 'temperature': ''}, 101: {'name': 'Garden', 'state': 0, 'temperature': ''}}<br> {200: {'name': 'Front Room', 'backLight': 0, 'setTemperature': '', 'calibration': [], 'heating': 0, 'chTimer': 0, 'temperature': '', 'pressure': '', 'humidity': '', 'tempSelection': ''}} </form>
Thanks

Pyomo: coupling variables
I am try to decide on an optimal battery size for an electric bus, where the battery size needs enough capacity for the whole day but should be as light possible. I want to be able to define a set of batteries that can be chosen from. Is it possible in Pyomo to define a Variable that can only be integer values (> yes) and let this variable define other variables?
I am getting this value error:
ValueError: Error retrieving component battery_power_outs[battery_choice]: The component has not been constructed.
Is there a clean way to solve my problem? What concept in Pyomo should I be using?
I would be very grateful for any hints/avenues I should be following!
from pyomo.environ import * m = AbstractModel() # ====== Parameters ====== # m.timesteps = Param() # amount of timesteps > 1440 (each minute of day) m.amount_battery_choices = Param() # 4 choices # m.battery_choices = Set([1, 2, 3, 4]) m.time = RangeSet(1, m.timesteps) m.battery_choices = RangeSet(1, m.amount_battery_choices) m.battery_consumption = Param(m.time) # consumption of driving fixed in [kWh/min/t] m.electricity_price = Param(m.time) m.depot_charging_allowed = Param(m.time) # zero if bus is at depot, one if bus on route m.vehicle_weight = Param() # in [kg] m.battery_weight_factor = Param() # [t/kWh] m.battery_capacities = Param(m.battery_choices) # in [kWh] m.battery_power_outs = Param(m.battery_choices) # in [kW] per minute m.battery_power_ins = Param(m.battery_choices) # in [kW] per minute m.battery_costs = Param(m.battery_choices) # in [$] m.upper_SOC_bound = Param() # between 0 and 1 m.lower_SOC_bound = Param() # between 0 and 1 # ====== Variables ====== # # choose a battery (14) m.battery_choice = Var(domain=PositiveIntegers, bounds=(1, 4)) # for each minute of day. Charging/discharging of battery: keep in max power out and max power in m.battery_flow = Var(m.time, bounds=((1)*m.battery_power_outs[m.battery_choice], m.battery_power_ins[m.battery_choice])) # for each minute of day. Charge level of battery: keep between 1585% of full capacity m.battery_SOC = Var(m.time, bounds=(m.lower_SOC_bound*m.battery_capacities[m.battery_choice], m.upper_SOC_bound*m.battery_capacities[m.battery_choice])) # for each minute of day. Charge flowing to battery while bus in depot m.depot_charging = Var(m.time, bounds=(0, m.battery_power_ins[m.battery_choice])) # ====== Constraints ====== # # start battery at 85% capacity, Otherwise battery is SOC charge at previous timestep + battery flow def maintain_SOC_bounds(m, t): if t == 1: return m.battery_SOC[t] == m.upper_SOC_bound*m.battery_capacities[m.battery_choice] # fix battery to be at max capacity allowed at beginning else: return m.battery_SOC[t] == m.battery_SOC[t1] + m.battery_flow[t] m.SOC_bounds = Constraint(m.time, rule=maintain_SOC_bounds) # recharge battery at final time to same level as start time def fix_final_SOC(m): return m.battery_SOC[m.timesteps] == m.upper_SOC_bound*m.battery_capacities[m.battery_choice] m.final_SOC = Constraint(rule=fix_final_SOC) # depot charging is zero when bus is not at depot, otherwise must be in charge limit of battery def fix_depot_charging(m, t): if m.depot_charging_allowed[t] == 1: return m.depot_charging[t] == 0 else: return m.depot_charging[t] <= m.battery_power_ins[m.battery_choice] m.depot_charging_rule = Constraint(m.time, rule=fix_depot_charging) # battery consumption fixed while bus is on route (as function of weight which is function of battery size) # otherwise if bus not in depot (depot_charging_allowed=1) battery flow is zero # otherwise battery flow is same value as depot_charging # else (shouldn't catch?) ??? def fix_battery_consumption(m, t): if m.battery_consumption[t] != 0: return m.battery_flow[t] == m.battery_consumption[t]*(m.vehicle_weight + m.battery_capacities[m.battery_choice]*m.battery_weight_factor) elif m.depot_charging_allowed[t] == 1: return m.battery_flow[t] == 0 elif m.depot_charging_allowed[t] == 0: return m.battery_flow[t] == m.depot_charging[t] else: return 0 <= m.battery_flow[t] <= m.battery_power_ins[m.battery_choice] m.battery_flow_rule = Constraint(m.time, rule=fix_battery_consumption) # ====== Objective ====== # # cost of battery + cost of electricity to charge back to full (different prices for different hours) def optimiseCost(m): overall_cost = m.battery_costs[m.battery_choice] for t in m.time: overall_cost = overall_cost + m.depot_charging[t]*m.electricity_price[t] return overall_cost m.obj_cost = Objective(rule=optimiseCost, sense=minimize)
EDIT: Approach this by coupling Variables using Constraints. E.g if Variable
battery_capacity == 50
then constrainbattery_power_out == 50

Calculation of standard deviation in pyomo if variance is avaiable
I have constructed a pyomo code in which I calculate the weighted mean of 3 vectors as
def meanX_definition(model, i): return m.meanX[i] ==(m.x1[i]+4*m.x2[i]+m.x3[i])/6 m.meanX_const = Constraint(m.N, rule = meanX_definition)
And then, I have calculated the variance as
def varX_definition(model, i): return m.varX[i] ==((m.x1[i]m.meanX[i])**2+ 4*(m.x2[i]m.meanX[i])**2+(m.x3[i]m.meanX[i])**2)/6 m.varX_const = Constraint(m.N, rule = varX_definition)
Since my model is a stochastic one, I want to put restrictions as x1, x2 and x3 have to be localidez between the mean and +2 standard deviations.
For that, I have attempted to calculate the standard deviation as
def stdX_definition(model, i): return m.desvestX[i] == sqrt(m.varX[i]) m.stdX_const = Constraint(m.N, rule = stdX_definition)
and
def stdX_definition(model, i): return m.desvestX[i] == m.varX[i]**(0.5) m.stdX_const = Constraint(m.N_notinitial_notfinal, rule = stdX_definition)
I have also tried to define the standard deviation at all points but the first and the last in case the error was the standard deviation of 0 at [0] and [n], since the start and endpoints are the same for all 3 vectors.
Unfortunately, an error occurs
ApplicationError: Solver (ipopt) did not exit normally
. Can someone please tell me how to fix it so I can get the standard deviation? 
Pyomo constraints and using pandas
I am using Pyomo to optimize a cashflow matching problem using bonds. I also want to have a detailed constraint that does looks at the cashflows I am expecting to get from my portfolio versus fixed requirements and conduct a number of calculations on the differences:
 Calculate difference (wanted minus expected to receive or "in  out" in the picture)
 Calculate the accumulation of these differences to the last end point using accumulation factors (multiply difference with accumulation factors  which are stored as a model.AccumFactors)
 Sum these yearonyear accumulation factors (cumsum(axis=1))
 Find the minimum
[Excel description of process][1]
Now Panda commands don't work in this situation. Is there anything I can do to fix this? Alternative approaches?