How to get the pixel location of an image from cv2 matchTemplate result
In python I am importing mss to capture my screen, numpy to process the screen grab, cv2 to matchTemplate an image from the screen grab, and pyagutogui to control my mouse.
Below is code that takes a screenshot and if there is a match to an image with >85% accuracy it draws a bounding box around it.
This works fine. However, instead of showing the screen grab, I want to just click on the location of the matchTemplate on my screen. And in the code below I believe I am telling my mouse to move to a location in my numpy array??
My Question: How do I convert image location in my cv2.matchTemplate to image location on my computer screen?
import cv2
import numpy as np
import pyautogui as pg
import mss
sct = mss.mss()
screen_grab_dimensions = {
'left': 0,
'top': 0,
'width': 750,
'height': 500
}
img = cv2.imread('img.png')
w = img.shape[1]
h = img.shape[0]
scr = np.array(sct.grab(screen_grab_dimensions))
# Cut off alpha
scr_remove = scr[:,:,:3]
result = cv2.matchTemplate(scr_remove, img, cv2.TM_CCOEFF_NORMED)
_, max_val, _, max_loc = cv2.minMaxLoc(result)
print(f"Max Val: {max_val} Max Loc: {max_loc}")
src = scr.copy()
if max_val > .85:
cv2.rectangle(scr, max_loc, (max_loc[0] + w, max_loc[1] + h), (0, 255, 255), 2)
print(f"Position: {pg.position(max_loc[0], max_loc[1])}")
pg.moveTo(max_loc[1], max_loc[0])
pg.click()
# will delete below
cv2.imshow('Screen Shot', scr)
cv2.waitKey()
cv2.destroyAllWindows()
do you know?
how many words do you know
See also questions close to this topic
-
Python File Tagging System does not retrieve nested dictionaries in dictionary
I am building a file tagging system using Python. The idea is simple. Given a directory of files (and files within subdirectories), I want to filter them out using a filter input and tag those files with a word or a phrase.
If I got the following contents in my current directory:
data/ budget.xls world_building_budget.txt a.txt b.exe hello_world.dat world_builder.spec
and I execute the following command in the shell:
py -3 tag_tool.py -filter=world -tag="World-Building Tool"
My output will be:
These files were tagged with "World-Building Tool": data/ world_building_budget.txt hello_world.dat world_builder.spec
My current output isn't exactly like this but basically, I am converting all files and files within subdirectories into a single dictionary like this:
def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return tree
Right now, my dictionary looks like this:
key:''
.In the following function, I am turning the empty values
''
into empty lists (to hold my tags):def empty_str_to_list(d): for k,v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v)
When I run my entire code, this is my output:
hello_world.dat ['World-Building Tool'] world_builder.spec ['World-Building Tool']
But it does not see
data/world_building_budget.txt
. This is the full dictionary:{'data': {'world_building_budget.txt': []}, 'a.txt': [], 'hello_world.dat': [], 'b.exe': [], 'world_builder.spec': []}
This is my full code:
import os, argparse def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return tree def empty_str_to_list(d): for k, v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v) parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter) parser.add_argument("--filter", action="store", help="keyword to filter files") parser.add_argument("--tag", action="store", help="a tag phrase to attach to a file") parser.add_argument("--get_tagged", action="store", help="retrieve files matching an existing tag") args = parser.parse_args() filter = args.filter tag = args.tag get_tagged = args.get_tagged current_dir = os.getcwd() files_dict = fs_tree_to_dict(current_dir) empty_str_to_list(files_dict) for k, v in files_dict.items(): if filter in k: if v == []: v.append(tag) print(k, v) elif isinstance(v, dict): empty_str_to_list(v) if get_tagged in v: print(k, v)
-
Actaully i am working on a project and in it, it is showing no module name pip_internal plz help me for the same. I am using pycharm(conda interpreter
File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\Scripts\pip.exe\__main__.py", line 4, in <module> File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_internal\__init__.py", line 4, in <module> from pip_internal.utils import _log
I am using pycharm with conda interpreter.
-
Looping the function if the input is not string
I'm new to python (first of all) I have a homework to do a function about checking if an item exists in a dictionary or not.
inventory = {"apple" : 50, "orange" : 50, "pineapple" : 70, "strawberry" : 30} def check_item(): x = input("Enter the fruit's name: ") if not x.isalpha(): print("Error! You need to type the name of the fruit") elif x in inventory: print("Fruit found:", x) print("Inventory available:", inventory[x],"KG") else: print("Fruit not found") check_item()
I want the function to loop again only if the input written is not string. I've tried to type return Under print("Error! You need to type the name of the fruit") but didn't work. Help
-
TypeError: 'float' object cannot be interpreted as an integer on linspace
TypeError Traceback (most recent call last) d:\website\SpeechProcessForMachineLearning-master\SpeechProcessForMachineLearning-master\speech_process.ipynb Cell 15' in <cell line: 1>() -->1 plot_freq(signal, sample_rate) d:\website\SpeechProcessForMachineLearning-master\SpeechProcessForMachineLearning-master\speech_process.ipynb Cell 10' in plot_freq(signal, sample_rate, fft_size) 2 def plot_freq(signal, sample_rate, fft_size=512): 3 xf = np.fft.rfft(signal, fft_size) / fft_size ----> 4 freq = np.linspace(0, sample_rate/2, fft_size/2 + 1) 5 xfp = 20 * np.log10(np.clip(np.abs(xf), 1e-20, 1e100)) 6 plt.figure(figsize=(20, 5)) File <__array_function__ internals>:5, in linspace(*args, **kwargs) File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\numpy\core\function_base.py:120, in linspace(start, stop, num, endpoint, retstep, dtype, axis) 23 @array_function_dispatch(_linspace_dispatcher) 24 def linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, 25 axis=0): 26 """ 27 Return evenly spaced numbers over a specified interval. 28 (...) 118 119 """ --> 120 num = operator.index(num) 121 if num < 0: 122 raise ValueError("Number of samples, %s, must be non-negative." % num) TypeError: 'float' object cannot be interpreted as an integer
What solution about this problem?
-
How to convert a array of string to array of float using
I have a array as shown below, in that I am having a variable called
f
, I need to assign the some value for this variablef
say 2 and convert this into a floating point array.(['0.', '0.', '-6.190649155150273*^15 + 0.7747634892904517*f^2', '2.2098855503598858*^10 + 4.250697125128597*^-7*f^2', '0.', '0.', '0.', 0.0, 0.0, -1427.5184297531378], dtype=object)
I tried with
df.convert_objects(convert_numeric=True)
but it is not working -
Pandas 1.4.2 upgrade casuing FFlask: array(0.78015261) (0d array) is not JSON serializable at the moment
We upgraded to a newer version of python with new pandas, numpy, etc. The versions are now:
- python=3.10.4 - pandas=1.4.2 - numpy=1.22.3
In previous versions, the error never occurred. When I do some debugging, it's not that the
.to_json()
is wrong or fails, it's that thepd.DataFrame([my_result])
doesn't return correctly.The code is this:
// Used in debugging and doesn't return properly dataframe_test = pd.DataFrame([my_result]) // Error gets thrown here return pd.DataFrame([my_result]).to_json()
The
dataframe_test
looks like this when I go to view it in Data Viewer within VS Code (it also keeps processing as the data seems to still be running in VS Code - the bar above the viewer is indicating that its still trying to run/process):The
my_result
variable looks like this upon entering the dataframe:I am not sure what exactly is causing the issue and not sure how to debug in Pandas to see what is happening. Any ideas?
-
How To set the size of image more than default value in OpenCV
What is the default frame size of opencv image streaming frame ?
If I set Width and height of frame more than default frame size what happens?
-
How to read grayscale img from a video with OpenCV?
I read all pictures from my
pic
directory and then convert them each to gray-scale withcanny
edge detections before writing it all to a video.But, when I use my video software to play it, it shows a green background, and I can't read video frames from it. Could someone show me how to solve it?
Sample code
import numpy as np import cv2 as cv import matplotlib.pyplot as plt fourcc = cv.VideoWriter_fourcc(*"I420") out = cv.VideoWriter("t2.avi", fourcc, 1, (640, 480), 0) for pic in glob.glob1("./pic/", "A*"): img = cv.imread(f"./pic/{pic}", -1) edge = cv.Canny(img, 100, 200) edge = cv.resize(edge, (640, 480)) out.write(edge) out.release() # Cant read video frame here: cap = cv.VideoCapture("t2.avi") ret, frame = cap.read() if ret: plt.imshow(frame) else: print("end") cap.release()
-
Is there a way to guarantee a certain number of lines detected with cv2.HoughLines()?
This question is an extension to my previous question asking about how to detect a pool table's corners. I have found the outline of a pool table, and I have managed to apply the Hough transform on the outline. The result of this Hough transform is below:
Unfortunately, the Hough transform returns multiple lines for a single table edge. I want the Hough transform to return four lines, each corresponding to an edge of the table given any image of a pool table. I don't want to tweak the parameters for the Hough transform method manually (because the outline of the pool table might differ for each image of the pool table). Is there any way to guarantee four lines to be generated by
cv2.HoughLines()?
Thanks in advance.
-
The while loop inside the Python for loop is repeated only once
This is a part of the code that checks the notice that appears while working using pyautogui and resumes the work when it disappears.
Capture the guide and make it into an image, check it with pyautogui, and if it exists, move on to the next one.
number = 0 for mark in sub_list: pyautogui.moveTo(1105, 356) pyautogui.click() time.sleep(random.uniform(0.5, 1.2)) pyautogui.moveTo(291, 529) pyautogui.click() time.sleep(random.uniform(0.5, 1.2)) pyautogui.moveTo(190, 207) pyautogui.click() time.sleep(random.uniform(0.5, 1.2)) pyautogui.moveTo(347, 529) pyautogui.click() pyautogui.write(sub_list[number]) print(sub_list[number]) time.sleep(random.uniform(0.5, 1.2)) pyautogui.moveTo(702, 531) pyautogui.click() icon_to_click = "Recycle Bin" r = None while r is None: r = pyautogui.locateOnScreen('rb.png', grayscale=True) print(icon_to_click + ' now loaded') pyautogui.write(''.join(a_record)) time.sleep(random.uniform(0.5, 1.2)) pyautogui.moveTo(210, 616) pyautogui.click() tiu = str(sub_list[number]) + "." + str(main_domain) sub_domain.append(tiu) print(sub_domain[number]) time.sleep(random.uniform(4, 5)) number = number + 1
However, it works fine once, but from the second iteration it can't find the image and stops.
<pre> icon_to_click = "Recycle Bin" r = None while r is None: r = pyautogui.locateOnScreen('rb.png', grayscale=True) print(icon_to_click + ' now loaded') </pre>
Should I change the settings in this part? Help
-
use pyautogui in a certain open program in windows
I would like to use image search in a certain open application, for example, I want it to search the image only in the windows "Calculator" application, how could I do that?
Today pyautogui searches the whole screen, is it possible to limit only one open application?
def main(): try: while True: button7location = pyautogui.locateOnScreen('images/calc7Key.png', region=(0,0,1920, 1080), confidence=.5) print(button7location) except KeyboardInterrupt: print('\nDone.') main()
-
TCP response breaks after 1460bytes
Response from one linux server to another linux server breaks after 1460 bytes , checked on server (MTU is default 1500). but why it is happening to few requests only because all responses are more than 1460 bytes.
-
CoreGraphics.CGWindowListCreateImage() failed
I am on OSX 10.13 and python 3.8
The code works fine when run from terminal but throws error when scheduled as cron
from __future__ import print_function from robot.api import logger from robot.libraries.BuiltIn import BuiltIn from RPA.Desktop import Desktop from RPA.Desktop.keywords import keyword from RPA.Images import Images import easyocr import numpy as np import pandas as pd import pyautogui import pywinauto lib = Images() screenshot = lib.take_screenshot(filename="screnshot")
Here's the error
File "/Users/ishandutta2007/Documents/Projects/LaunchTerminalTabs/flip_ig_firefox.py", line 78, in easyocrclick screenshot = lib.take_screenshot(filename="screnshot") File "/Users/ishandutta2007/.pyenv/versions/3.8.0/lib/python3.8/site-packages/RPA/Images.py", line 167, in take_screenshot image = sct.grab(sct.monitors[0]) File "/Users/ishandutta2007/.pyenv/versions/3.8.0/lib/python3.8/site-packages/mss/base.py", line 88, in grab return self._grab_impl(monitor) File "/Users/ishandutta2007/.pyenv/versions/3.8.0/lib/python3.8/site-packages/mss/darwin.py", line 215, in _grab_impl raise ScreenShotError("CoreGraphics.CGWindowListCreateImage() failed.") mss.exception.ScreenShotError: CoreGraphics.CGWindowListCreateImage() failed.
-
TCP MSS not enforced
I am doing some TCP related experiment between two virtualbox VMs. On the client side, I sent out a TCP syn packet with the MSS option of 1400 bytes. However, it seems that the server (sender) ignored this option and sent out a packet with very large payload, something like 10000+ bytes.Why didn't the MSS option honored by the server? BTW, the server is a Nginx server.
Below this some PCAP showing the problem. First is the SYN packet with MSS = 1400.
Second is the payload sent by the server:
As can be seen that the payload size is 11200.
BTW the MTU on the interface is 1500 bytes.
Thanks.