How to dynamically compose an OR query filter in Django?

From an example you can see a multiple OR query filter:

Article.objects.filter(Q(pk=1) | Q(pk=2) | Q(pk=3))

For example, this results in:

[<Article: Hello>, <Article: Goodbye>, <Article: Hello and goodbye>]

However, I want to create this query filter from a list. How to do that?

e.g. [1, 2, 3] -> Article.objects.filter(Q(pk=1) | Q(pk=2) | Q(pk=3))

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You could chain your queries as follows:

values = [1,2,3]

# Turn list of values into list of Q objects
queries = [Q(pk=value) for value in values]

# Take one Q object from the list
query = queries.pop()

# Or the Q object with the ones remaining in the list
for item in queries:
    query |= item

# Query the model
Article.objects.filter(query)

Method 2

To build more complex queries there is also the option to use built in Q() object’s constants Q.OR and Q.AND together with the add() method like so:

list = [1, 2, 3]
# it gets a bit more complicated if we want to dynamically build
# OR queries with dynamic/unknown db field keys, let's say with a list
# of db fields that can change like the following
# list_with_strings = ['dbfield1', 'dbfield2', 'dbfield3']

# init our q objects variable to use .add() on it
q_objects = Q(id__in=[])

# loop trough the list and create an OR condition for each item
for item in list:
    q_objects.add(Q(pk=item), Q.OR)
    # for our list_with_strings we can do the following
    # q_objects.add(Q(**{item: 1}), Q.OR)

queryset = Article.objects.filter(q_objects)

# sometimes the following is helpful for debugging (returns the SQL statement)
# print queryset.query

Method 3

A shorter way of writing Dave Webb’s answer using python’s reduce function:

# For Python 3 only
from functools import reduce

values = [1,2,3]

# Turn list of values into one big Q objects  
query = reduce(lambda q,value: q|Q(pk=value), values, Q())  

# Query the model  
Article.objects.filter(query)

Method 4

from functools import reduce
from operator import or_
from django.db.models import Q

values = [1, 2, 3]
query = reduce(or_, (Q(pk=x) for x in values))

Method 5

Maybe it’s better to use sql IN statement.

Article.objects.filter(id__in=[1, 2, 3])

See queryset api reference.

If you really need to make queries with dynamic logic, you can do something like this (ugly + not tested):

query = Q(field=1)
for cond in (2, 3):
    query = query | Q(field=cond)
Article.objects.filter(query)

Method 6

See the docs:

>>> Blog.objects.in_bulk([1])
{1: <Blog: Beatles Blog>}
>>> Blog.objects.in_bulk([1, 2])
{1: <Blog: Beatles Blog>, 2: <Blog: Cheddar Talk>}
>>> Blog.objects.in_bulk([])
{}

Note that this method only works for primary key lookups, but that seems to be what you’re trying to do.

So what you want is:

Article.objects.in_bulk([1, 2, 3])

Method 7

Solution which use reduce and or_ operators to filter by multiply fields.

from functools import reduce
from operator import or_
from django.db.models import Q

filters = {'field1': [1, 2], 'field2': ['value', 'other_value']}

qs = Article.objects.filter(
   reduce(or_, (Q(**{f'{k}__in': v}) for k, v in filters.items()))
)

p.s. f is a new format strings literal. It was introduced in python 3.6

Method 8

In case we want to programmatically set what db field we want to query:

import operator
questions = [('question__contains', 'test'), ('question__gt', 23 )]
q_list = [Q(x) for x in questions]
Poll.objects.filter(reduce(operator.or_, q_list))

Method 9

You can use the |= operator to programmatically update a query using Q objects.

Method 10

This one is for dynamic pk list:

pk_list = qs.values_list('pk', flat=True)  # i.e [] or [1, 2, 3]

if len(pk_list) == 0:
    Article.objects.none()

else:
    q = None
    for pk in pk_list:
        if q is None:
            q = Q(pk=pk)
        else:
            q = q | Q(pk=pk)

    Article.objects.filter(q)

Method 11

Another option I wasn’t aware of until recently – QuerySet also overrides the &, |, ~, etc, operators. The other answers that OR Q objects are a better solution to this question, but for the sake of interest/argument, you can do:

id_list = [1, 2, 3]
q = Article.objects.filter(pk=id_list[0])
for i in id_list[1:]:
    q |= Article.objects.filter(pk=i)

str(q.query) will return one query with all the filters in the WHERE clause.

Method 12

For loop:

values = [1, 2, 3]
q = Q(pk__in=[]) # generic "always false" value
for val in values:
    q |= Q(pk=val)
Article.objects.filter(q)

Reduce:

from functools import reduce
from operator import or_

values = [1, 2, 3]
q_objects = [Q(pk=val) for val in values]
q = reduce(or_, q_objects, Q(pk__in=[]))
Article.objects.filter(q)

Both of these are equivalent to Article.objects.filter(pk__in=values)

It’s important to consider what you want when values is empty. Many answers with Q() as a starting value will return everything. Q(pk__in=[]) is a better starting value. It’s an always-failing Q object that’s handled nicely by the optimizer (even for complex equations).

Article.objects.filter(Q(pk__in=[]))  # doesn't hit DB
Article.objects.filter(Q(pk=None))    # hits DB and returns nothing
Article.objects.none()                # doesn't hit DB
Article.objects.filter(Q())           # returns everything

If you want to return everything when values is empty, you should AND with ~Q(pk__in=[]) to ensure that behaviour:

values = []
q = Q()
for val in values:
    q |= Q(pk=val)
Article.objects.filter(q)                     # everything
Article.objects.filter(q | author="Tolkien")  # only Tolkien

q &= ~Q(pk__in=[])
Article.objects.filter(q)                     # everything
Article.objects.filter(q | author="Tolkien")  # everything

It’s important to remember that Q() is nothing, not an always-succeeding Q object. Any operation involving it will just drop it completely.

Method 13

Found solution for dynamical field names:

def search_by_fields(value, queryset, search_in_fields):
    if value:
        value = value.strip()

    if value:
        query = Q()
        for one_field in search_in_fields:
            query |= Q(("{}__icontains".format(one_field), value))

        queryset = queryset.filter(query)

    return queryset

Method 14

easy..
from django.db.models import Q
import you model
args = (Q(visibility=1)|(Q(visibility=0)&Q(user=self.user))) #Tuple
parameters={} #dic
order = ‘create_at’
limit = 10

Models.objects.filter(*args,**parameters).order_by(order)[:limit]

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0