Django ORM - a model object from using .get or indexing a QuerySet
my_model = MyModel.objects.get(pk=5)
So 'my_model' is not a queryset object, neither would it be if I indexed it from a queryset. Is there something special about a QuerySet other than it is a list of objects from the table(s)?
Also I was wondering, I know that simply creating a QuerySet does not involve a database lookup, but what about getting just one object like in 'my_model'?
A queryset is a "list of objects" in the most basic sense, but it also has a fair number of methods that are not available to a normal list.
So for example you can filter a queryset further to produce another queryset. You can view the SQL generated by the queryset, you can get aggregates from a queryset.
I know that simply creating a QuerySet does not involve a database lookup
Querysets are lazily evaluated. That means they don't hit the database until needed. So in your example "getting one just model" would need to get information from the database.
Other things that will cause the query to actually lookup the database will be anything that actually needs information that is stored in the database - so iterating over the queryset, rendering a value in a template, calling list(my_queryset) or the count() method. There is a section at the top of the link I posted expalining this.
See also questions close to this topic
Creating dynamic reports based on location specific using Django and other frameworks
Please suggest me , if any tools or framework , which can help in building reports table which should be dynamic , user friendly , and location specific , with django , postures as database.
Suggestions will be welcome if data from postgres is returned as json format and render it as a dynamic customized report tables.
django exclude urls using regular expression in list
I'm working on
Middlewarein django version 2.1. It is
LoginMiddleWarewhere i need to exclude some urls if the
requested url exitsit returns
below is the portion of middleware code : middleware.py
def process_view(self, request, view_func, view_args, view_kwargs): exclude_urls = [r'[admin]', r'[login]', r'[register]', r'[demo]'] if(request.path_info in exclude_urls): print('BELOW IS EXCLUDED URL') print(request.path_info) return None else: print('BELOW IS NOT EXCLUDED URL') print(request.path_info)
The above code must identify the url requested by user
request.path_infoand must check exits in
EXAMPLE : if requested url is
https://www.example.com/login?query=1this must identify
loginin that particular url and must return
After reviewing my code here below came with a solution
url_list = [ 'admin/login', '/demo/login' ] requested_path = request.path_info print('requested path :'+requested_path) for i in url_list: r1 = re.findall(r'^'+requested_path, i) print(len(r1)) if len(r1) > 0: return None else: if not request.session.get('user_logged', None): print("Session Empty") return redirect('login') //HERE USER IS GOING INTO CONTINUOUS REDIRECT LOOP HOW CAN I STOP INFINITE REDIRECTS else: print("Session Exits")
THE ABOVE CHANGES IN CODE WORKS IN SOME EXTEND
NOW THE ISSUE IS IT REDIRECTING TO https://www.example.com/demo/loginpage again and again how can i stop this.
What i'm looking to achieve
admin/login and demo/logincurrently are the two urls i need to exclude while check if
session exits in middleware.py
CASE 1 : IF URL EXISTS IN LIST AND SESSION IS EMPTY AND USER HAS
REQUESTED demo/loginurl the i need him to redirect to his url
CASE 2 : IF USER REQUESTED URL IS OTHER THAN URL MENTIONED IN
list url_list. IN THE ELSE I'M CHECKING IF SESSION IS EMPTY THEN I'M REDIRECTING USER TO
demo/loginpage[## BUT USER IS GOING INTO CONTINUOUS REDIRECT LOOP HOW CAN I STOP INFINITE REDIRECTS ##]
THANKS IN ADVANCE
Django Full Text Search Optimization - Postgres
I am trying to create a Full Text Search for an address autocomplete feature, leveraging Django (v2.1) and Postgres (9.5), but the performance is not suitable for an auto-complete at the moment and I don't get the logic behind the performance results I get. For info the table is quite sizeable, with 14 million rows.
from django.db import models from postgres_copy import CopyManager from django.contrib.postgres.indexes import GinIndex class Addresses(models.Model): date_update = models.DateTimeField(auto_now=True, null=True) longitude = models.DecimalField(max_digits=9, decimal_places=6 , null=True) latitude = models.DecimalField(max_digits=9, decimal_places=6 , null=True) number = models.CharField(max_length=16, null=True, default='') street = models.CharField(max_length=60, null=True, default='') unit = models.CharField(max_length=50, null=True, default='') city = models.CharField(max_length=50, null=True, default='') district = models.CharField(max_length=10, null=True, default='') region = models.CharField(max_length=5, null=True, default='') postcode = models.CharField(max_length=5, null=True, default='') addr_id = models.CharField(max_length=20, unique=True) addr_hash = models.CharField(max_length=20, unique=True) objects = CopyManager() class Meta: indexes = [ GinIndex(fields=['number', 'street', 'unit', 'city', 'region', 'postcode'], name='search_idx') ]
I created a little test to check performance based on number of words in the search:
search_vector = SearchVector('number', 'street', 'unit', 'city', 'region', 'postcode') searchtext1 = "north" searchtext2 = "north bondi" searchtext3 = "north bondi blair" searchtext4 = "north bondi blair street 2026" print('Test1: 1 word') start_time = time.time() result = AddressesAustralia.objects.annotate(search=search_vector).filter(search=searchtext1)[:10] #print(len(result)) time_exec = str(timedelta(seconds=time.time() - start_time)) print(time_exec) print(' ') #print(AddressesAustralia.objects.annotate(search=search_vector).explain(verbose=True)) print('Test2: 2 words') start_time = time.time() result = AddressesAustralia.objects.annotate(search=search_vector).filter(search=searchtext2)[:10] #print(len(result)) time_exec = str(timedelta(seconds=time.time() - start_time)) print(time_exec) print(' ') print('Test3: 3 words') start_time = time.time() result = AddressesAustralia.objects.annotate(search=search_vector).filter(search=searchtext3)[:10] #print(len(result)) time_exec = str(timedelta(seconds=time.time() - start_time)) print(time_exec) print(' ') print('Test4: 5 words') start_time = time.time() result = AddressesAustralia.objects.annotate(search=search_vector).filter(search=searchtext4)[:10] #print(len(result)) time_exec = str(timedelta(seconds=time.time() - start_time)) print(time_exec) print(' ')
I get the following results, which seems quite correct:
Test1: 1 word 0:00:00.001841 Test2: 2 words 0:00:00.001422 Test3: 3 words 0:00:00.001574 Test4: 5 words 0:00:00.001360
However if I uncomment the print(len(results)) lines, I get the following results:
Test1: 1 word 10 0:00:00.046392 Test2: 2 words 10 0:00:06.544732 Test3: 3 words 10 0:01:12.367157 Test4: 5 words 10 0:01:17.786596
This is obviously not suitable for an autocomplete feature.
Could someone be able to explain why the execution takes longer when performing an operation on the queryset result? It seems the Database retrieval is always fast but then going through the results takes time, which does not make sense to me as because I'm limiting the results to 10, the queryset returned is always the same size.
Also, although I have created a GIN index, this index does not seem to be used. It seems it has been created correctly:
=# \d public_data_au_addresses Table "public.public_data_au_addresses" Column | Type | Collation | Nullable | Default -------------+--------------------------+-----------+----------+------ --------------------------------------------------------- id | integer | | not null | nextval('public_data_au_addresses_id_seq'::regclass) date_update | timestamp with time zone | | | longitude | numeric(9,6) | | | latitude | numeric(9,6) | | | number | character varying(16) | | | street | character varying(60) | | | unit | character varying(50) | | | city | character varying(50) | | | district | character varying(10) | | | region | character varying(5) | | | postcode | character varying(5) | | | addr_id | character varying(20) | | not null | addr_hash | character varying(20) | | not null | Indexes: "public_data_au_addresses_pkey" PRIMARY KEY, btree (id) "public_data_au_addresses_addr_hash_key" UNIQUE CONSTRAINT, btree (addr_hash) "public_data_au_addresses_addr_id_key" UNIQUE CONSTRAINT, btree (addr_id) "public_data_au_addresses_addr_hash_e8c67a89_like" btree (addr_hash varchar_pattern_ops) "public_data_au_addresses_addr_id_9ee00c76_like" btree (addr_id varchar_pattern_ops) "search_idx" gin (number, street, unit, city, region, postcode)
When I run the explain() method on my query I get that:
Test1: 1 word Limit (cost=0.00..1110.60 rows=10 width=140) -> Seq Scan on public_data_au_addresses (cost=0.00..8081472.41 rows=72767 width=140) Filter: (to_tsvector((((((((((((COALESCE(number, ''::character varying))::text || ' '::text) || (COALESCE(street, ''::character varying))::text) || ' '::text) || (COALESCE(unit, ''::character varying))::text) || ' '::text) || (COALESCE(city, ''::character varying))::text) || ' '::text) || (COALESCE(region, ''::character varying))::text) || ' '::text) || (COALESCE(postcode, ''::character varying))::text)) @@ plainto_tsquery('north'::text))
So it still shows a Sequential scan instead of using an Index scan. Does anyone know how to fix or debug that?
Would a GIN index still be efficient with so many fields to search on anyway?
And lastly, does anyone know how I can improve the code to improve the performance further?
Thank you! Regards
List first entry from reverse foreign key
I have the following Models
class Book(models.Model): title = description = class Chapter(models.Model): book = models.ForeignKey(Book,on_delete=models.CASCADE)
How to list all first chapters by date added from all books? like
I want to have a queryset or a list of Chapter, to display in templates.
Big O Notation: Using multiple forloops within Django template VS multiple queries using a filter in views.py
This is a specific question based on an attempt to increase performance speeds when querying data.
So I was trying to figure out within a Django template itself, how to break a conditional on the first attempt. It seems that it is not possible, and the suggestions were all to use the views.py for the logic instead. This led me to try filtering based on a condition being met.
In my example, I have two scenarios that I'm comparing.
(1) In the first one, I have one query inside my views.py to get all items. Naturally, Item is a schema I have in my models. Anyway, inside the template I want to render, I have the context being passed and have 11 separate forloops all iterating over the same all_items loop. Then based on the condition (ie item.category), the appropriate html is rendered.
Again, what I wanted to do was have 1 loop, and then based on the condition, render to the appropriate places, but only go through that all_items loop once. Unfortunately, I'm not able to break the loop in the template after the condition to not over-render html I don't want on each successive iteration.
So this led me the next scenario:
(2) In my views I created 11 separate queries (eg item.objects.filter(category='1'), item.objects.filter(category='2', etc.). I in turn assign each one of those to 11 separate variables passed through my views as context to render in the template. This also allows me to remove the conditional in the template where I check for the category.
What I'm wondering is, in the latter example, is each query filter the same as running the same forloop for all_items in the template, just done a little differently? Am I saving any time with the latter? It is hard to say just based on my user experience, so any insights would be great. Thank you.
How to get multiple values from joined tables
How can I get all the laws from all the courses that an specific user is subscribed?
class Law(models.Model): name = models.CharField('Nome', max_length=100) slug = models.SlugField('Atalho') description = models.TextField('Description', blank = True, null=True) class Course(models.Model): name = models.CharField('Nome', max_length=100) slug = models.SlugField('Atalho') description = models.TextField('Descrição', blank = True, null=True) laws = models.ManyToManyField(Law, related_name='law_course') class Subscription(models.Model): STATUS_CHOICES=( ( (0, 'Pendente'), (1, 'Approved'), (2, 'Canceled'), ) ) user = models.ForeignKey(settings.AUTH_USER_MODEL,on_delete=models.CASCADE, verbose_name="Usuário", related_name="inscricao") #inscricao é criado no usuario course = models.ForeignKey(Course, on_delete=models.CASCADE, verbose_name="Course", related_name="subscription") status = models.IntegerField( 'Status', choices=STATUS_CHOICES, default = 0, blank=True ) class Meta: unique_together = (('user','course'))
So, the user can be subscribed in one or more courses. Each course has one or many laws.
I have a page that I need to list all the laws of each course that an specific user is subscribed if the subscription is Approved (
status = 1). Example: The user John has subscribed in 2 courses: Course A and Course B.
Course A laws: law1, law2, law3 Course B laws: law1, law20, law30
So, when John is logged in, I need to show the laws of the courses and the name of the course:
law1 - Course A, Course B law2 - Course A law3 - Course A law20 - Course B law30 - Course B
Note that I can't show
law 1twice, but I showed each course that contains this law.
I did my best to be clear. Tks in advance