citusdata/django-multitenant

Python/Django support for distributed multi-tenant databases like Postgres+Citus

django-multitenant

Python/Django support for distributed multi-tenant databases like Postgres+Citus

Enables easy scale-out by adding the tenant context to your queries, enabling the database (e.g. Citus) to efficiently route queries to the right database node.

There are architecures for building multi-tenant databases viz. Create one database per tenant, Create one schema per tenant and Have all tenants share the same table(s). This library is based on the 3rd design i.e Have all tenants share the same table(s), it assumes that all the tenant relates models/tables have a tenant_id column for representing a tenant.

The following link talks more about the trade-offs on when and how to choose the right architecture for your multi-tenant database:

https://www.citusdata.com/blog/2016/10/03/designing-your-saas-database-for-high-scalability/

The following blogpost is a good starting point to start to use django-multitenant https://www.citusdata.com/blog/2023/05/09/evolving-django-multitenant-to-build-scalable-saas-apps-on-postgres-and-citus/

Other useful links on multi-tenancy:

1. https://www.citusdata.com/blog/2017/03/09/multi-tenant-sharding-tutorial/
2. https://www.citusdata.com/blog/2017/06/02/scaling-complex-sql-transactions/
3. https://www.youtube.com/watch?v=RKSwjaZKXL0

Installation:

1. pip install --no-cache-dir django_multitenant

Supported Django versions/Pre-requisites.

Python	Django	Citus
3.8 3.9 3.10 3.11	4.2	11 12
3.8 3.9 3.10 3.11	4.1	11 12
3.8 3.9 3.10 3.11	4.0	10 11 12
3.7	3.2	10 11 12

Usage:

In order to use this library you can either use Mixins or have your models inherit from our custom model class.

Changes in Models:

1. In whichever files you want to use the library import it:

   from django_multitenant.fields import *
   from django_multitenant.models import *

2. All models should inherit the TenantModel class. Ex: class Product(TenantModel):

3. Define a static variable named tenant_id and specify the tenant column using this variable.You can define tenant_id in three ways. Any of them is acceptable

   • Using TenantMeta.tenant_field_name variable
   • Using TenantMeta.tenant_id variable
   • Using tenant_id field
   
   

   Warning Using tenant_id field directly in the class is not suggested since it may cause collision if class has a field named with 'tenant'

   
   

4. All foreign keys to TenantModel subclasses should use TenantForeignKey in place of models.ForeignKey

5. A sample model implementing the above 2 steps:

   class Store(TenantModel):
     name =  models.CharField(max_length=50)
     address = models.CharField(max_length=255)
     email = models.CharField(max_length=50)
     class TenantMeta:
       tenant_field_name = "id"
   
   class Product(TenantModel):
     store = models.ForeignKey(Store)
     name = models.CharField(max_length=255)
     description = models.TextField()
     class Meta:
       unique_together = ["id", "store"]
     class TenantMeta:
       tenant_field_name = "store_id"
   class Purchase(TenantModel):
     store = models.ForeignKey(Store)
     product_purchased = TenantForeignKey(Product)
     class TenantMeta:
       tenant_field_name = "store_id"

Changes in Models using mixins:

1. In whichever files you want to use the library import it by just saying

   from django_multitenant.mixins import *

2. All models should use the TenantModelMixin and the django models.Model or your customer Model class Ex: class Product(TenantModelMixin, models.Model):
3. Define a static variable named tenant_id and specify the tenant column using this variable. Ex: tenant_id='store_id'
4. All foreign keys to TenantModel subclasses should use TenantForeignKey in place of models.ForeignKey
5. Referenced table in TenantForeignKey should include a unique key including tenant_id and primary key

   Ex:       
   class Meta:
        unique_together = ["id", "store"]
   

6. A sample model implementing the above 3 steps:

   class ProductManager(TenantManagerMixin, models.Manager):
     pass
   
   class Product(TenantModelMixin, models.Model):
     store = models.ForeignKey(Store)
     tenant_id='store_id'
     name = models.CharField(max_length=255)
     description = models.TextField()
   
     objects = ProductManager()
   
     class Meta:
       unique_together = ["id", "store"]
   
   class PurchaseManager(TenantManagerMixin, models.Manager):
     pass
   
   class Purchase(TenantModelMixin, models.Model):
     store = models.ForeignKey(Store)
     tenant_id='store_id'
     product_purchased = TenantForeignKey(Product)
   
     objects = PurchaseManager()

Automating composite foreign keys at db layer:

1. Creating foreign keys between tenant related models using TenantForeignKey would automate adding tenant_id to reference queries (ex. product.purchases) and join queries (ex. product__name). If you want to ensure to create composite foreign keys (with tenant_id) at the db layer, you should change the database ENGINE in the settings.py to django_multitenant.backends.postgresql.

  'default': {
      'ENGINE': 'django_multitenant.backends.postgresql',
      ......
      ......
      ......
}

Where to Set the Tenant?

1. Write authentication logic using a middleware which also sets/unsets a tenant for each session/request. This way developers need not worry about setting a tenant on a per view basis. Just set it while authentication and the library would ensure the rest (adding tenant_id filters to the queries). A sample implementation of the above is as follows:

       from django_multitenant.utils import set_current_tenant
       
       class MultitenantMiddleware:
           def __init__(self, get_response):
               self.get_response = get_response
   
           def __call__(self, request):
               if request.user and not request.user.is_anonymous:
                   set_current_tenant(request.user.employee.company)
               return self.get_response(request)

   In your settings, you will need to update the MIDDLEWARE setting to include the one you created.

      MIDDLEWARE = [
          # ...
          # existing items
          # ...
          'appname.middleware.MultitenantMiddleware'
      ]

2. Set the tenant using set_current_tenant(t) api in all the views which you want to be scoped based on tenant. This would scope all the django API calls automatically(without specifying explicit filters) to a single tenant. If the current_tenant is not set, then the default/native API without tenant scoping is used.

    def application_function:
      # current_tenant can be stored as a SESSION variable when a user logs in.
      # This should be done by the app
      t = current_tenant
      #set the tenant
      set_current_tenant(t);
      #Django ORM API calls;
      #Command 1;
      #Command 2;
      #Command 3;
      #Command 4;
      #Command 5;

Supported APIs:

1. Most of the APIs under Model.objects.*.
2. Model.save() injects tenant_id for tenant inherited models.

   s=Store.objects.all()[0]
   set_current_tenant(s)
   
   #All the below API calls would add suitable tenant filters.
   #Simple get_queryset()
   Product.objects.get_queryset()
   
   #Simple join
   Purchase.objects.filter(id=1).filter(store__name='The Awesome Store').filter(product__description='All products are awesome')
   
   #Update
   Purchase.objects.filter(id=1).update(id=1)
   
   #Save
   p=Product(8,1,'Awesome Shoe','These shoes are awesome')
   p.save()
   
   #Simple aggregates
   Product.objects.count()
   Product.objects.filter(store__name='The Awesome Store').count()
   
   #Subqueries
   Product.objects.filter(name='Awesome Shoe');
   Purchase.objects.filter(product__in=p);

Credits

This library uses similar logic of setting/getting tenant object as in django-simple-multitenant. We thank the authors for their efforts.

License

Copyright (C) 2023, Citus Data Licensed under the MIT license, see LICENSE file for details.

django-multitenant's Issues

2.0.5 fails to import TenantQuerySet

After installing 2.0.5, my application reports that django-multitenant can't import TenantQuerySet:

Traceback (most recent call last):
  File "/usr/local/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 347, in execute
    django.setup()
  File "/usr/local/lib/python3.6/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.6/dist-packages/django/apps/registry.py", line 112, in populate
    app_config.import_models()
  File "/usr/local/lib/python3.6/dist-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/helm/invitations/models.py", line 25, in <module>
    from .base_invitation import AbstractBaseInvitation
  File "/app/helm/invitations/base_invitation.py", line 6, in <module>
    from .managers import BaseInvitationManager
  File "/app/helm/invitations/managers.py", line 8, in <module>
    from django_multitenant.django_multitenant import TenantManager
  File "/usr/local/lib/python3.6/dist-packages/django_multitenant/django_multitenant.py", line 14, in <module>
    from .models import TenantManager, TenantModel, TenantQuerySet
ImportError: cannot import name 'TenantQuerySet'

Checking models.py in 2.0.5, there's no longer an import for TenantQuerySet:

import logging

from django.db import models

from .mixins import (TenantManagerMixin,
                     TenantModelMixin)

from .utils import (
    set_current_tenant,
    get_current_tenant,
    get_model_by_db_table,
    get_tenant_column
)

logger = logging.getLogger(__name__)



class TenantManager(TenantManagerMixin, models.Manager):
    #Below is the manager related to the above class.
    pass


class TenantModel(TenantModelMixin, models.Model):
    #Abstract model which all the models related to tenant inherit.

    objects = TenantManager()

    class Meta:
        abstract = True

Reverting to 2.0.4 resolves the problem.

Does this library restrict the version of Django?

I added django_multitenant to the requirements.txt on an existing Django 1.11 project. However running pip install -r requirements.txt then uninstalls Django and downgrades it to 1.10:

Installing collected packages: Django, django-multitenant
  Found existing installation: Django 1.11
    Uninstalling Django-1.11:
      Successfully uninstalled Django-1.11
  Running setup.py install for django-multitenant ... done
Successfully installed Django-1.10 django-multitenant-0.3

Not sure if the dependencies of this library caused that to happen, and recording an issue just in case.

Remove all AGPL code (blocker for usage) and use MIT license only

The usage of AGPL viral licensed code in some of this implementation and it viral nature makes it impossible to use django-multitenant in a commercial application.

Please remove all AGPL code that was copied from other projects and provide a clean room equivalent implementation, all 100% under MIT license.

psycopg2.errors.InvalidForeignKey: there is no unique constraint matching given keys for referenced table "users_employer"

I have my tenant model as this

class Company(TenantModel):
    tenant_id = "id"
    name =   models.CharField(max_length=50)
    address =  models.CharField(max_length=255)
    description            = models.TextField(blank=True, null=True)
    email                  = models.CharField(max_length=255, blank=True, null=True) 
    subdomain_prefix       =  models.CharField(max_length=255, blank=True, null=True)
    cell_phone             = models.CharField(max_length=255, blank=True, null=True) 
    active                 = models.BooleanField(default=True)
    land_phone             = models.CharField(max_length=255, blank=True, null=True) 
    country                = models.CharField(max_length=32, choices=COUNTRY_CHOICES, blank=True)
    state                  = models.CharField(max_length=255, blank=True, null=True) 
    city                   = models.CharField(max_length=255, blank=True, null=True) 
    zip_code               = models.CharField(max_length=255, blank=True, null=True) 
    about                  = models.TextField(blank=True, null=True)
    contact_details        = models.TextField(blank=True, null=True)
    latitude               = models.CharField(max_length=512, blank=True, null=True)  
    longitude              = models.CharField(max_length=512, blank=True, null=True)  
    year_established       =  models.DateField( blank=True, null=True)
    total_employees        =  models.CharField(max_length=255, blank=True, null=True) 
    business_type          =  models.CharField(max_length=255, blank=True, null=True)  
    main_products          =  models.CharField(max_length=255, blank=True, null=True) 
    total_annual_revenue   =  models.CharField(max_length=255, blank=True, null=True) 
    url                    =  models.CharField(max_length=255, blank=True, null=True) 
    social_link            =  models.CharField(max_length=255, blank=True, null=True) 

    def __str__(self):
        return self.name

class UserProfile(TenantModel):
    is_admin = models.BooleanField(default=False)
    title = models.CharField(max_length=255, blank=True, null=True) 
    last_name = models.CharField(max_length=255, blank=True, null=True) 
    first_name = models.CharField(max_length=255, blank=True, null=True) 
    other_names = models.CharField(max_length=255, blank=True, null=True) 
    address = models.CharField(max_length=255, blank=True, null=True) 
    tenant_id = 'company_id'

    class Meta:
        abstract = True

class SuperAdmin(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE, blank=True, null=True)

class Action(models.Model):
    action = models.TextField(null=True, blank=True)
    time = models.DateTimeField(auto_now=True)
    by =  models.ForeignKey(SuperAdmin, on_delete=models.CASCADE, blank=True, null=True)

class Employer(UserProfile):
    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name='employerprofile', blank=True, null=True)
    company = models.ForeignKey(Company, on_delete=models.CASCADE, related_name='employercompany', blank=True, null=True, unique=True)
    # departments = models.ManyToManyField('Department', blank=True, related_name="members")
    # class Meta: 
    #     unique_together = ["id", "company"]


class Employee(UserProfile):
    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name='employeeprofile', blank=True, null=True)
    company = models.ForeignKey(Company, on_delete=models.CASCADE, related_name='employeecompany', blank=True, null=True)
    departments = models.ManyToManyField(Department, blank=True)
    employer = TenantForeignKey(Employer,  on_delete=models.PROTECT, blank=True, null=True)
    reg_id = models.CharField(max_length=255, blank=True, null=True) 
    role = TenantForeignKey("company.Role",  on_delete=models.PROTECT, blank=True, null=True)
    leave_day = TenantForeignKey("company.LeaveDays",  on_delete=models.PROTECT, blank=True, null=True)
    identidication_type = models.CharField(max_length=255, blank=True, null=True) 
    passport = models.CharField(max_length=255, blank=True, null=True) 
    identidication = models.CharField(max_length=255, blank=True, null=True) 
    employment_type = models.CharField(max_length=255, blank=True, null=True) 
    known_as = models.CharField(max_length=255, blank=True, null=True) 
    gender = models.CharField(max_length=255, blank=True, null=True) 
    date_of_birth = models.DateField( blank=True, null=True)
    blood_group = models.CharField(max_length=255, blank=True, null=True) 
    genotype = models.CharField(max_length=255, blank=True, null=True) 
    employee_photo = models.FileField(upload_to='assets/',blank=True, null=True) 
    start_date = models.DateField( blank=True, null=True)
    reports_to = models.ForeignKey(User, on_delete=models.CASCADE, related_name='reportperson', blank=True, null=True)
    additional_reports = models.FileField(upload_to='assets/%d',blank=True, null=True) 
    job_role = models.CharField(max_length=255, blank=True, null=True) 
    salary = models.FloatField( blank=True, null=True)
    daily_cost_of_absence = models.FloatField( blank=True, null=True)
    nationality = models.CharField(max_length=255, blank=True, null=True) 

    
    def get_full_name(self):
            return self.user.first_name+" "+self.user.last_name

    def __str__(self):
            return self.user.username

After creating migrations, when i try ti migrate, I get this error
django.db.utils.ProgrammingError: there is no unique constraint matching given keys for referenced table "users_employer"
Kindly look into it for me.

Issue when running migration

I am using Django-2.1 with last release of django-multitenant and I am writing a migration on a model which inherits from TenantModel. But when I run the migration, I get error that

AttributeError: type object 'Shipment' has no attribute 'tenant_id'

The model that I have is:

class Fulfillment(TenantModel):
  ....
  shipment = TenantForeignKey('Shipment', on_delete=models.CASCADE)
  tenant_id = 'store_id'

  class Meta:
       unique_together = ('id', 'store')

But if I set the tenant_id = 'store_id' explicitly in migration, it works fine. I am not sure why is it complaining and what is the right way to fix this. Any help is appreciated.

Calls to TenantModel.delete() cause issues in Citus

Example:

from django.db import models
from django_multitenant import set_current_tenant, TenantModel

class Corporation(models.Model):
    class Meta:
        db_table = 'corporation'
    pass

class Store(TenantModel):
    class Meta:
        db_table = 'store'
    corporation = models.ForeignKey(Corporation)
    isStoreOpen = models.BooleanField()
    tenant_id = 'corporation_id'



corporation = Corporation()
corporation.save()
store = Store(corporation=corporation, isStoreOpen=True)
store.save()

set_current_tenant(corporation)

Store.objects.filter(isStoreOpen=True).delete()

The SQL for the delete statement:

SELECT "store"."id",
       "store"."corporation_id",
       "store"."isStoreOpen"
FROM "store"
WHERE ("store"."corporation_id" = 2
       AND "store"."isStoreOpen" = TRUE);

DELETE
FROM "store"
WHERE "store"."id" IN (2);

ERROR:  multi-task query about to be executed

This one may be tough to fix, since the logic behind crafting DELETE queries seems to be outside of TenantQuerySet's control: https://github.com/django/django/blob/1.11.8/django/db/models/sql/subqueries.py#L48-L81

Error running migrations

The fix for https://github.com/citusdata/django-multitenant/blob/master/django_multitenant/backends/postgresql/base.py#L37 introduced a new issue.

If I make a change to a TenantForeignKey that requires an AlterField operation (e.g. change the related_name, the migration that's created will try and recreate the constraint that already exists and fail with the following error.

django.db.utils.ProgrammingError: constraint "core_order_customer_id_brand_id_c6c43641_fk_core_cust" for relation "core_order" already exists

TenantForeignKey FieldError when `tenant_id` is different

Suppose the following models:

class Account(TenantModel):
  tenant_id = 'id'
  ...

class Project(TenantModel):
  tenant_id = 'account_id'
  ...

class Revenue(TenantModel):
  acc = models.ForeignKey(Account, ...)
  project = TenantForeignKey(Project, ...)
  tenant_id = 'acc_id'
  ...

When accessing revenue.project Django will throw:

django.core.exceptions.FieldError: Cannot resolve keyword 'acc_id' into field. Choices are: ...

Cannot delete instance of model referenced by ManyToManyField

I have the following models:

models.py

class School(TenantModel):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    name = models.CharField(max_length=100)
    tenant_id = 'id'

class User(AbstractBaseUser, PermissionsMixin):
    email = models.EmailField(max_length=255, unique=True)
    objects = UserManager()

    USERNAME_FIELD = 'email'

class UserProfile(TenantModel):
    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='userprofiles')
    school = models.ForeignKey(School, on_delete=models.CASCADE, related_name='members')
    departments = models.ManyToManyField('Department', blank=True, related_name="members")
    is_admin = models.BooleanField(default=False)
   
    tenant_id = 'school_id'

class Department(TenantModel):
    name = models.CharField(max_length=500, unique=True)
    school = models.ForeignKey(School, on_delete=models.CASCADE)

    tenant_id = 'school_id'

Attempting to delete a Department instance throws the error "TypeError: related_objects() takes 3 positional arguments but 4 were given". This happens even if a tenanted through model is used.

Traceback

  File "/Users/alex/Library/Mobile Documents/com~apple~CloudDocs/Documents/bunsen/backend/schools/tests/test_schools_api.py", line 171, in test_delete_department_successful
    response = self.client.delete(
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/test.py", line 317, in delete
    response = super().delete(
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/test.py", line 219, in delete
    return self.generic('DELETE', path, data, content_type, **extra)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/test.py", line 231, in generic
    return super().generic(
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/test/client.py", line 470, in generic
    return self.request(**r)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/test.py", line 283, in request
    return super().request(**kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/test.py", line 235, in request
    request = super().request(**kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/test/client.py", line 709, in request
    self.check_exception(response)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/test/client.py", line 571, in check_exception
    raise exc_value
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/viewsets.py", line 114, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/mixins.py", line 91, in destroy
    self.perform_destroy(instance)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/rest_framework/mixins.py", line 95, in perform_destroy
    instance.delete()
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/db/models/base.py", line 943, in delete
    collector.collect([self], keep_parents=keep_parents)
  File "/Users/alex/.local/share/virtualenvs/backend-xQfT061F/lib/python3.8/site-packages/django/db/models/deletion.py", line 313, in collect
    sub_objs = self.related_objects(related_model, related_fields, batch)
TypeError: related_objects() takes 3 positional arguments but 4 were given```

Objects with same id cannot be stored in database for two different Tenants

Since the id is created automatically by Django's ORM and sets it to be primary_key, even if we have

 class Product(TenantModelMixin, models.Model):
    store = models.ForeignKey(Store)
    tenant_id='store_id'
    name = models.CharField(max_length=255)
    description = models.TextField()

    class Meta(object):
        unique_together = ["id", "store"]

Another store with same id cannot be created, assuming the id is not autoadd. Ideally the primary key should be set as composite one with (id, store_id). I am aware though that composite primary key support is still not there in Django as it breaks a lot of other things like admin where the lookup is primarily based on id only.

It is not clear then in the docs that why having unique_together on id, store is needed, if id is anyways a primary key.

Incorrect guidance in docs

In the docs, the following is suggested:

     def process_response(self, request, response):
             set_current_tenant(None)
             return response

This means that the response has no tenant when returned, incapacitating the project. Presumably this should be removed? It is also now depreciated and would require django.utils.deprecation.MiddlewareMixin

Tenant not filtered in model forms

Suppose I have the following 2 models and a model form

`

class CostCenter(ClientModelMixin, models.Model):
    tenant_id = 'client_id'
    client = models.ForeignKey('accounts.Client', on_delete=models.CASCADE)
    name = models.CharField(max_length=50)
    objects = TenantManager()

class Account(ClientModelMixin, models.Model):
    tenant_id = 'client_id'
    client = models.ForeignKey('accounts.Client', on_delete=models.CASCADE)
    name = models.CharField(max_length=50)
    cost_center = TenantForeignKey(CostCenter, on_delete=models.PROTECT)
    objects = TenantManager()

class CreateForm(forms.ModelForm):
    class Meta:
        model = Account
        fields = ['cost_center']

`

When the model form is rendered it shows all cost centers in the dropdown. I think it happens because the model form is created before the tenant is set and then no tenant filtering is applied.

I can override the init method of every form and set the queryset correctly but this isn't ideal.

How can this be solved?

Can threading.local() and uWSGI cause unexpected behaviour?

I noticed that threading.local() is used to set and retrieve the current tenant.

There's a discussion here mentioning threading.local() being shared across threads in uWSGI.

Could this potentially cause unexpected behaviour when serving a django application that is using django-multitenant through uWSGI?

Tenant not set when using bulk_create

Django doesn't call the save() method when using bulk_create; per the docs.

It looks like the TenantModelMixin therefore doesn't set the current tenant on object creation via this method.

It may be worth adding this, or otherwise referring to the exception in the docs.

Tags in github

Would it be possible to get release tags in GitHub for the corresponding PyPi versions? I've encountered a few issues with particular versions, specifically 2.0.5 and 2.0.6 and it would be easier to work out what's going on if I could see the changes between versions in git.

Cannot plan queries which include both local and distributed relations

When I try to do the migrations as the docs example, I got this error:

django.db.utils.InternalError: cannot plan queries which include both local and distributed relations
CONTEXT:  SQL statement "SELECT fk."account_id" FROM ONLY "public"."others_other" fk LEFT OUTER JOIN ONLY "public"."core_account" pk ON ( pk."id" OPERATOR(pg_catalog.=) fk."account_id") WHERE pk."id" IS NULL AND (fk."account_id" IS NOT NULL)"

Is this problem related with the models being separated on different apps?

My models are:

core/models.py

class Account(models.Model):

   plan = models.CharField(max_length=25)

others/models.py

class Other(models.Model):

    account = models.ForeignKey(Account, on_delete=models.CASCADE)

    class Meta(object):
        unique_together = ["id", "account"]

My Migrations:

core/migrations/0002_distribute_tables.py

    ...
    operations = [
        migrations.RunSQL(
            "SELECT create_distributed_table('core_account','id')"
        ),
    ]

others/migrations/0002_remove_simple_pk.py

    ...

    operations = [
        migrations.RunSQL(
            "ALTER TABLE others_other DROP CONSTRAINT others_other _pkey;",
                "ALTER TABLE others_other ADD CONSTRAINT others_other _pkey PRIMARY KEY (account_id, id)"
        ),
    ]

others/migrations/0003_distribute_tables.py

    ...

    operations = [
        migrations.RunSQL(
            "SELECT create_distributed_table('others_other','account_id')"
        ),
    ]

Document examples for where to add set_current_tenant

Give some examples of where to include calls to set_current_tenant - whether as part of a middleware or as part of the individual views themselves.

cannot create instance for subclass model of TenantModel

python:3.6.6
django:2.2.6

models.py

from django.db import models
from django_multitenant.fields import TenantForeignKey
from django_multitenant.models import TenantModel


class Tenant(TenantModel):
    tenant_id = "id"
    name = models.CharField("tenant name", max_length=100)


class Business(TenantModel):
    tenant = models.ForeignKey(Tenant, blank=True, null=True, on_delete=models.SET_NULL)
    tenant_id = "tenant_id"
    bk_biz_id = models.IntegerField("business ID")
    bk_biz_name = models.CharField("business name", max_length=100)


class Template(TenantModel):
    tenant = models.ForeignKey(Tenant, blank=True, null=True, on_delete=models.SET_NULL)
    business = TenantForeignKey(Business, blank=True, null=True, on_delete=models.SET_NULL)
    name = models.CharField("name", max_length=100)
    created_by = models.CharField("created by", max_length=100)

when I run

In [1]: from home_application.models import *

In [2]: t1 = Tenant.objects.get(id=1)

In [3]: biz1 = Business(tenant=t1, bk_biz_id=2, bk_biz_name="A")

raise error

---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
/Users/page/.pyenv/versions/3.6.6/envs/test_django_multitenant_py3.6.6/lib/python3.6/site-packages/django_multitenant/utils.py in get_tenant_field(model_class_or_instance)
     51     try:
---> 52         return next(field for field in all_fields if field.column == tenant_column)
     53     except StopIteration:

StopIteration: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-3-4a184dd85d2f> in <module>()
----> 1 biz1 = Business(tenant=t1, bk_biz_id=2, bk_biz_name="A")

/Users/page/.pyenv/versions/3.6.6/envs/test_django_multitenant_py3.6.6/lib/python3.6/site-packages/django_multitenant/mixins.py in __init__(self, *args, **kwargs)
     56             UpdateQuery.update_batch = wrap_update_batch(UpdateQuery.update_batch)
     57 
---> 58         super(TenantModelMixin, self).__init__(*args, **kwargs)
     59 
     60     def __setattr__(self, attrname, val):

/Users/page/.pyenv/versions/3.6.6/envs/test_django_multitenant_py3.6.6/lib/python3.6/site-packages/django/db/models/base.py in __init__(self, *args, **kwargs)
    484             else:
    485                 if val is not _DEFERRED:
--> 486                     _setattr(self, field.attname, val)
    487 
    488         if kwargs:

/Users/page/.pyenv/versions/3.6.6/envs/test_django_multitenant_py3.6.6/lib/python3.6/site-packages/django_multitenant/mixins.py in __setattr__(self, attrname, val)
     60     def __setattr__(self, attrname, val):
     61 
---> 62         if (attrname in (self.tenant_field, get_tenant_field(self).name)
     63             and not self._state.adding
     64             and val

/Users/page/.pyenv/versions/3.6.6/envs/test_django_multitenant_py3.6.6/lib/python3.6/site-packages/django_multitenant/utils.py in get_tenant_field(model_class_or_instance)
     53     except StopIteration:
     54         raise ValueError('No field found in {} with column name "{}"'.format(
---> 55                          model_class_or_instance, tenant_column))
     56 
     57 

ValueError: No field found in Business object (None) with column name "1"

Suport for TenantForeignKey as recursive Model

Suport for models like this:

class Task(TenantModel):
tenant_id = 'business_id'
business = models.ForeignKey(Business, on_delete = models.CASCADE)
sub_task = TenantForeignKey(
'self',
on_delete=models.CASCADE,
db_index=False,
blank=True,
null=True
)

Add unit tests to the repo

Forms from django-simple-multitenant

Hi.
I use django form and have to filter my foreign key field by myself and it is not safe and secure. Are you going to make base classes and mixins for forms.ModelForm and maybe some solution for Built-in class-based generic views?

Tenant pk field hardcoded as 'id'

TenantModel provides tenant_id attribute to specify the pk field to be used, or at least, the examples points out to specify tenant_id = 'id' to the Store model, which is used as the Tenant model.
But if I try to specify a different pk column to the Tenant model I got several errors because TenantManager and TenantQuerySet refers to current_tenant.id

This should probably be getattr(current_tenant, current_tenant.tenant_id)

#Below is the manager related to the above class. 
class TenantManager(TenantQuerySet.as_manager().__class__):
    #Injecting tenant_id filters in the get_queryset.
    #Injects tenant_id filter on the current model for all the non-join/join queries. 
    def get_queryset(self):
        current_tenant=get_current_tenant()
        if current_tenant:
            kwargs = { self.model.tenant_id: getattr(current_tenant, current_tenant.tenant_id)}
            return super(TenantManager, self).get_queryset().filter(**kwargs)
        return super(TenantManager, self).get_queryset()

Through model saves are not supported

When using a ManyToManyField with a through model, the tenant_id is not applied to the through model when the parent instance is saved.

Consider the following:

  class Store(TenantModel):
    tenant_id = 'id'
    name =  models.CharField(max_length=50)
    address = models.CharField(max_length=255)
    email = models.CharField(max_length=50)

  class Product(TenantModel):
    store = models.ForeignKey(Store)
    tenant_id='store_id'
    name = models.CharField(max_length=255)
    description = models.TextField()
    class Meta(object):
      unique_together = ["id", "store"]

  class Purchase(TenantModel):
    store = models.ForeignKey(Store)
    tenant_id='store_id'
    product_purchased = models.ManyToManyField(Product, through=Transaction)

 class Transaction(TenantModel):
    store = models.ForeignKey(Store)
    tenant_id='store_id'
    purchase = TenantForeignKey(Purchase)
    product = TenantForeignKey(Product)
    date = models.DateField()

When a Purchase instance is created and/or the product_purchased field is updated, the tenant_id is not set for the Transaction.

This appears to be because the tenant_value is set in the model save function:

    def save(self, *args, **kwargs):
        tenant_value = get_current_tenant_value()
        if not self.pk and tenant_value and not isinstance(tenant_value, list):
            setattr(self, self.tenant_field, tenant_value)

        return super(TenantModelMixin, self).save(*args, **kwargs)

This means it is not called.

Is there any fundamental reason why select_related is not supported

Is it because it is too hard to do with django-multitenant or is it because it is not possible at all? Without select_related how would one optimise the queries?

Support sub-domains?

Does this package support the use of sub-domains or does this need to be implemented separstely or with another package like django-subdomains or django-hosts?

Tenant filter does not work in django-filter ModelChoiceFilter

Problem: sql-clause "table"."tenant_id" = current_tenant_value dissapers from query for django-filter's ModelChoiceFilter when order_by is added to queryset.

Code to reproduce.

models.py

from django_multitenant.fields import TenantForeignKey
from django_multitenant.mixins import TenantManagerMixin, TenantQuerySet
from django_multitenant.models import TenantModel
from django_multitenant.utils import get_current_tenant, set_current_tenant

class CompanyObjectModel(TenantModel):
    """Абстрактный класс для разделяемых по компаниям моделей"""
    company = models.ForeignKey(
        Company,
        default=get_current_tenant,
        on_delete=models.PROTECT
    )

    tenant_id = 'company_id'
    class Meta:
        abstract = True
        unique_together = ["id", "company"]


class EventClass(CompanyObjectModel):
    """
    Описание шаблона мероприятия (Класс вид).
    Например, тренировки в зале бокса у Иванова по средам и пятницам
    """
    name = models.CharField("Наименование", max_length=100)
    location = TenantForeignKey(
        Location,
        on_delete=models.PROTECT,
        verbose_name="Место проведения")
    coach = TenantForeignKey(
        Coach,
        on_delete=models.PROTECT,
        verbose_name="Тренер")
    date_from = models.DateField("Начало тренировок", null=True, blank=True)
    date_to = models.DateField("Окончание тренировок", null=True, blank=True)
    planned_attendance = models.PositiveSmallIntegerField(
        "Плановая посещаемость",
        null=True,
        blank=True,
        validators=[MinValueValidator(0)]
    )

    objects = EventClassManager()


class EventClassManager(TenantManagerMixin, models.Manager):
    def active(self):
        return self.get_queryset().filter(
            Q(date_to__isnull=True) | Q(date_to__gte=date.today())
        )

    def in_range(self, day_start, day_end):
        return self.get_queryset().filter(
            (
                Q(date_from__isnull=False) & Q(date_to__isnull=False) &
                Q(date_from__lte=day_end) & Q(date_to__gte=day_start)
            ) | (
                Q(date_from__isnull=False) & Q(date_to__isnull=True) &
                Q(date_from__lte=day_end)
            ) | (
                Q(date_from__isnull=True) & Q(date_to__isnull=False) &
                Q(date_to__gte=day_start)
            ) | (
                Q(date_from__isnull=True) & Q(date_to__isnull=True)
            )
        )

filters.py

import django_filters
from crm import models

class VisitReportFilterNew(django_filters.FilterSet):

    event_class = django_filters.ModelChoiceFilter(
        label='Группа:',
        field_name='event_class',
        queryset=models.EventClass.objects,
        empty_label=None,
        widget=forms.Select(
            attrs={
                'class': 'selectpicker form-control',
                'title': 'Группа',
            }
        ),
        required=False,
    )

    class Meta:
        model = models.Event
        fields = ('date',)

report.html

        <div class="col-6 col-md-3">
          <div class="form-group select">
            <label for="id_event_class">{{ filter.form.event_class.label }}</label>
            {{ filter.form.event_class }}
          </div>
        </div>

in ModelChoiseFilter queryset=models.EventClass.objects generates valid SQL-query:

DECLARE "_django_curs_140336784169760_1" NO SCROLL
CURSOR WITH HOLD
   FOR SELECT "crm_eventclass"."id",
       "crm_eventclass"."company_id",
       "crm_eventclass"."name",
       "crm_eventclass"."location_id",
       "crm_eventclass"."coach_id",
       "crm_eventclass"."date_from",
       "crm_eventclass"."date_to",
       "crm_eventclass"."planned_attendance"
  FROM "crm_eventclass"
 WHERE "crm_eventclass"."company_id" = 71

But when I try to sort filter values changing queryset in ModelChoiseFilter to queryset=models.EventClass.objects.order_by("name"). The SQL-query is ordered, but tenant filter is missing, it returns all rows from table:

DECLARE "_django_curs_140075094870816_1" NO SCROLL
CURSOR WITH HOLD
   FOR SELECT "crm_eventclass"."id",
       "crm_eventclass"."company_id",
       "crm_eventclass"."name",
       "crm_eventclass"."location_id",
       "crm_eventclass"."coach_id",
       "crm_eventclass"."date_from",
       "crm_eventclass"."date_to",
       "crm_eventclass"."planned_attendance"
  FROM "crm_eventclass"
 ORDER BY "crm_eventclass"."name" ASC

Versions:

Django==2.1.7
django-multitenant==2.0.0
django-filter==2.1.0

No module named 'django_multitenant.db'

Hello everyone ,
I followed the tutorial on : http://docs.citusdata.com/en/v9.1/develop/migration_mt_django.html
And when i arrived on the migration part (distribution) i discovered that the db module misses from the project .
So when you try to import like this:
"from django_multitenant.db import migrations as tenant_migrations"
I get the error:
"ModuleNotFoundError: No module named 'django_multitenant.db'"
I downloaded the tar.gz from version 2.00 to 2.09 and not even one had the 'db' module inside .

I can do it manually but i would love to know what happened .

does this support MySQL??

My project database uses MySQL. I want to switch schemas dynamically. so can it support MySQL???

Attribute Error 'Store' object has no attribute 'tenant_id'

I have cloned master branch to use django_multitenant so as to have django_multitenant.backends.postgresql available as Engine to me. But on doing so as soon as I try to create an object in database, it throws this exception. The exception happens because it is trying to access tenant_id attribute on Tenant model. The documentation does not say that tenant_id needs to be set on the tenant model too (Store model).

Offending line:

 current_tenant_id = getattr(current_tenant, current_tenant.tenant_id, None) 

The variable at this point are:

current_tenant | <Store: MyStore>
queryset | <TenantQuerySet [<UserProfile: UserProfile object (7)>]>
self | <django_multitenant.models.TenantManager object at 0x7f9638dfe940

And UserProfile looks something like this:

class UserProfile(TenantModel):
...
...
store = models.ForeignKey(Store, on_delete=models.CASCADE, null=True)
tenant_id = 'store_id'

class Meta:
    unique_together = ['id', 'store']

Store model:

class Store(models.Model):
name = models.CharField(max_length=64, unique=True)
...

ImportError: No module named django_multitenant

New to Python/Django and only reporting this because I am not sure if there's something wrong with my setup or just how the plugin/package is packaged/distributed.

I have a virtual environment with .. python 2.7 and django 1.11

Installed the package using 'pip install django_multitenant', but ran into an ImportError because of the line "from django_multitenant.django_multitenant import *", which wasn't in my code. I had to refactor the init.py and remove the redundant django_multitenant from the import to fix this.

Is this the right approach? Or is there a distribution available with the fix? I don't want everyone in the team to have to do this.

The create() method does not fill in tenant id

In my app Store is the tenant. Trying to create a product

s = Store.objects.get(id=1)
set_current_tenant(s)
Product.objects.create(name="nice product", description="buy me", quantity = 1)

The result is

django.db.utils.DataError: cannot perform an INSERT with NULL in the partition column

Access to foreign keys does not result in a filter on the tenant_id

The following:

from django.db import models
from django_multitenant import set_current_tenant, TenantModel

class Tenant(models.Model)
	pass


class Bar(TenantModel):
    tenant = models.ForeignKey(Tenant)
    tenant_id = 'tenant'

	
class Foo(TenantModel):
    tenant = models.ForeignKey(Tenant)
    tenant_id = 'tenant'

    bar = models.ForeignKey(Bar)
	
	

tenant = Tenant.objects.create()
bar = Bar.objects.create(tenant=tenant)
foo = Foo.objects.create(tenant=tenant, bar=bar)

set_current_tenant(tenant)

print(Foo.objects.get().bar)

...will output this SQL:

SELECT "foo"."id", "foo"."tenant_id", "foo"."bar_id" FROM "foo" WHERE "foo"."tenant_id" = '3d340b31-cd78-4224-9ac0-00455933c1b8'::uuid
SELECT "bar"."id", "bar"."tenant_id" FROM "bar" WHERE "bar"."id" = 2

The second query there should have an additional AND "bar"."tenant_id" = ... in the WHERE clause

Changing TenantModel to include:

class TenantModel:
  class Meta:
    base_manager_name = 'tenantManager'
  tenantManager = TenantManager()

seems to fix the issue

Trouble getting started with DRF - No filtering, and error message on getattr

Thank you for providing this library for free. I think it is very valuable. I'm struggling to get the library to work for my setup, though.

I created a tenant and a user model, the object I want to filter is a tag model:

class Tenant(TenantModal):
    tenant_id = 'id'
    name = models.CharField(max_length=255)


class CustomUser(AbstractBaseUser, TenantModel):
    tenant_id = "tenant_id"
    uuid = models.UUIDField(
        primary_key=True, default=uuid.uuid4, editable=False)
    tenant = TenantForeignKey('tenants.Tenant', blank=True, null=True, on_delete=models.CASCADE)

    objects = CustomUserManager()

    USERNAME_FIELD = 'email'


class TenantTag(TenantModel):

    tenant_id = "tenant_id"
    uuid = models.UUIDField(primary_key=True, editable=False, default=uuid.uuid4)
    tenant = TenantForeignKey(Tenant, on_delete=models.CASCADE)

In this setup, I've tried to use a normal ForeignKey (as shown in the docs) and a TenantForeignKey (as shown here).
Both resulted in the following 2 problems:

1.) Whenever I want to save an instance (via the admin) I receive an error message:
File "/Users/xxxxxxxx/.virtualenvs/hr-djang/lib/python3.7/site-packages/django_multitenant/mixins.py", line 84, in tenant_value return getattr(self, self.tenant_field, None) TypeError: getattr(): attribute name must be string

2.) I can not get the models to be filtered for tenants in any way. Even though the tenants seem correctly set by the middleware and I'm able to get the tenant by invoking get_current_tenant()no filtering is applied. Sending an authenticated request from a tenant_1_user shows tags belonging to tenant_1 and tenant_2 as well.

I would really appreciate any help! Thanks.

.exclude not fully supported but fails with confusing error message

Example error:

>>> CampaignEmailOptions.objects.exclude(campaign_email_shot__isnull=False)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 250, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 274, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1087, in execute_sql
    sql, params = self.as_sql()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 489, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 355, in as_sql
    return super().as_sql(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 163, in as_sql
    rhs_sql, rhs_params = self.process_rhs(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 346, in process_rhs
    return super().process_rhs(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 220, in process_rhs
    return super().process_rhs(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 92, in process_rhs
    sql, params = compiler.compile(value)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/query.py", line 1018, in as_sql
    return self.get_compiler(connection=connection).as_sql()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 489, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related_lookups.py", line 130, in as_sql
    return super().as_sql(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 163, in as_sql
    rhs_sql, rhs_params = self.process_rhs(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 260, in process_rhs
    return super().process_rhs(compiler, connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/lookups.py", line 92, in process_rhs
    sql, params = compiler.compile(value)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/expressions.py", line 734, in as_sql
    return "%s.%s" % (qn(self.alias), qn(self.target.column)), []
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 396, in quote_name_unless_alias
    r = self.connection.ops.quote_name(name)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/postgresql/operations.py", line 105, in quote_name
    if name.startswith('"') and name.endswith('"'):
AttributeError: 'NoneType' object has no attribute 'startswith'

I've traced this back to https://github.com/citusdata/django-multitenant/blob/master/django_multitenant/fields.py#L72. This code is assuming that both alias and related_alias have a value which isn't true for this kind of exclude query.

This doesn't seem particularly easy to fix but I would suggest at least catching this condition and raising a more useful error:

e.g.

        if not (alias and related_alias):
            raise ValueError(
                """
                Subquery pushdowns are not currently supported by django-multitenant.
                If you're using .exclude() then you may need to split out your subquery manually.

                e.g. Model.objects.exclude(foo__bar=1) could become Model.objects.exclude(id__in=Model.objects.filter(foo__bar=1))
                """
            )

Automate create composite primary keys

One important pre-requisite for migration to citus is for all the models which are inherited from TenantModel, we need to replace the primary key on id column to a composite primary key (tenant_id,id). As we did this for foreign keys in this #16 (specifically in this file), we could do automate dropping single primary keys and creating composite primary keys.

How to create ForeignKey to self?

What is the optimal solution for this model?

from django.db import models


class Business(models.Model):
    business_name=models.CharField(
        max_length=128)

    class Meta:
        db_table='business'
        verbose_name='Business'
        verbose_name_plural='Business'


# This class have a foreign key to self, parent_id to id
class AccountingEntries(models.Model):
    ENTRY_TYPES = (
        ('D', 'Debtor'),
        ('A', 'creditor'),
    )
    parent=models.ForeignKey(
        'self',
        null=True,
        blank=True,
        on_delete=models.CASCADE)
    business=models.ForeignKey(
        Business,
        on_delete=models.CASCADE)
    entry_code=models.CharField(
        null=True,
        blank=True,
        max_length=18)
    entry_name=models.CharField(
        max_length=128)
    entry_type=models.CharField(
        max_length=2,
        choices=ENTRY_TYPES,
        default='D')
    status=models.BooleanField(
        default=True)

    class Meta:
        unique_together = ['id', 'business']
        db_table='accounting_entries'
        verbose_name='Accounting Entries'
        verbose_name_plural='Accounting Entries'

Error with Django 2.1 on Python 3.7

./dev.sh django-admin sqlmigrate core 0001
Starting machinelabsbackend_memcached_1 ... done
Reading config from /volumes/environment/01_environment.json
Reading config from /volumes/environment/02_secrets.json
Traceback (most recent call last):
  File "/usr/local/bin/django-admin", line 11, in <module>
    sys.exit(execute_from_command_line())
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/sqlmigrate.py", line 29, in execute
    return super().execute(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/sqlmigrate.py", line 58, in handle
    sql_statements = executor.collect_sql(plan)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 225, in collect_sql
    state = migration.apply(state, schema_editor, collect_sql=True)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/operations/models.py", line 91, in database_forwards
    schema_editor.create_model(model)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 285, in create_model
    self.deferred_sql.append(self._create_fk_sql(model, field, "_fk_%(to_table)s_%(to_column)s"))
  File "/usr/local/lib/python3.7/site-packages/django_multitenant/backends/postgresql/base.py", line 58, in _create_fk_sql
    "name": self.quote_name(self._create_index_name(model, from_columns, suffix=suffix)),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 877, in _create_index_name
    _, table_name = split_identifier(table_name)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 204, in split_identifier
    namespace, name = identifier.split('"."')
AttributeError: type object 'CoreLTVCohort' has no attribute 'split'

Is recommended practice to get_current_tenant for every view?

The docs state that:

Set the tenant using set_current_tenant(t) api in all the views which you want to be scoped based on tenant.

Does this mean that the recommended approach is to store the current tenant in a cookie (session variable) and then call it for each and every view that is rendered?

When simply setting the variable it appears to persist between views in thread local storage. Is this discouraged? Why otherwise incur the extra overhead of accessing session variables in every view?

support for uuid

Hi, I tried using uuids as ids and it doesn't work because of the save method in TenantMixin. When saving the setattr(self, self.tenant_field, tenant_value) line never gets called because self.pk has value when dealing with uuids different from IntegerField. I created my own abstract class and added a created field it did the trick, but it would be nice to have native support for uuids.

def save(self, *args, **kwargs):
    tenant_value = get_current_tenant_value()
    if not self.pk and tenant_value and not isinstance(tenant_value, list):
        setattr(self, self.tenant_field, tenant_value)

pip package without database backend

When installing the django-multitenant package with pip the backends folder doesn't exist.
django-multitenant version 1.1.0

Unable to import mixins

I am trying to import mixins in my code.

from django_multitenant.mixins import *

I am getting this error ModuleNotFoundError: No module named 'django_multitenant.mixins'.

Python version: 3.6
Library version: 1.1.0

Historical migrations break when models are removed

Due to https://github.com/citusdata/django-multitenant/blob/master/django_multitenant/backends/postgresql/base.py#L79 the migrations are dependent on the current state of the application, not on the state that it was at when the migration was created.

This means that if you remove a model your migrations will no longer be able to run cleanly from start to finish and will instead hit the error from https://github.com/citusdata/django-multitenant/blob/master/django_multitenant/backends/postgresql/base.py#L85

Change documentation

Add the fact that parent tenant model should have tenant_id.

Related to #37

PK(field name 'id') of the tenant model('Store') does not auto-incriment

class Store(TenantModel):
tenant_id = 'id'
name = models.CharField(max_length=50)
address = models.CharField(max_length=255)
email = models.CharField(max_length=50)

class Product(TenantModel):
store = models.ForeignKey(Store)
tenant_id='store_id'
name = models.CharField(max_length=255)
description = models.TextField()

class Purchase(TenantModel):
store = models.ForeignKey(Store)
tenant_id='store_id'
product_purchased = TenantForeignKey(Product)

Dropping foreign keys fails with "cannot execute multiple utility events"

Here is what happens.

When dropping a model, the default migration drops foreign key constraints. And is executed in a single transaction:

SET CONSTRAINTS "my_fk_constraint" IMMEDIATE; ALTER TABLE "my_table" DROP CONSTRAINT "my_fk_constraint"

On the shards are executed

SELECT worker_apply_shard_ddl_command (102259, 'public', 'SET CONSTRAINTS "my_fk_constraint" IMMEDIATE; ALTER TABLE "my_table" DROP CONSTRAINT "my_fk_constraint"')

This fails with the error cannot execute multiple utility events

In the django it comes from
https://github.com/django/django/blob/master/django/db/backends/postgresql/schema.py#L21

In the backend, it would be great to avoid that.

The goal is for customer to be able to use auto-generated migrations to drop models.

Issue with using TenantModel in DjangoRestFramework viewsets

To use TenantModel with Django Rest Framework's ViewSets, the following refactor was necessary:

# before
class MyModelViewSet(GenericViewSet):
  queryset = MyModel.objects.all()

# after
class MyModelViewSet(GenericViewSet):
    def get_queryset(self):
        return MyModel.objects.all()

Duplicate key value integrity error thrown if updating tenant model instance with different tenant set

When updating a tenant model if another current tenant is set then an integrity error is thrown due to a duplicate key value.

For example, consider the following tenant model:

from django_multitenant.mixins import TenantModelMixin

  class Store(TenantModelMixin):
    tenant_id = 'id'
    name =  models.CharField(max_length=50)

Create 2 objects and set the current tenant to the first:

s1 = Store.objects.create(name="store1")
s2 = Store.objects.create(name="store2")
set_current_tenant(s1)

Then attempt to update the second object s2 whilst the current_tenant is still set to s1.

s2.name = 'new name'
s2.save()

This results in:

IntegrityError at /store/settings/1/
duplicate key value violates unique constraint "store_pkey"
DETAIL:  Key (id)=(1) already exists.

Aggregations fail under certain conditions

The following test case demonstrates one of the cases under which this fails. I investigated the cause of this and found it to be https://github.com/django/django/blob/7acef095d73322f45dcceb99afa1a4e50b520479/django/db/models/sql/compiler.py#L167. This alters the group by statement to just be grouped by 'id', but this doesn't work with Citus because we are dealing with composite primary keys and also need to include the tenant id in the group by.

    def test_aggregate(self):
        from .models import *
        projects = self.projects
        managers = self.project_managers
        unset_current_tenant()
        projects_per_manager = ProjectManager.objects.annotate(Count('project_id'))
        list(projects_per_manager)

test

test

2.0.6 fails to add the tenant in create() and subqueries

It seems that since version 2.0.6, the TenantQuerySetMixin has been removed. Now the tenant_id is not set when performing a create().
The subqueries are also not filtered when joining other tables.

See this pull request
More particularly, this file :