Django admin page: Customize dictionary (JSONField) of IDs through multiple Models selects instead of raw text

Question

I have a model in which one of its fields is a postgres.fields.JSONField.

The Json that is going to be stored there is a variable dictionary of IDs referencing other items (possible relations/attributes) in the database.

Allow me to be more specific:

Basically, I'm trying to create a discount system, in which some discounts would apply to certain products. The JSON field contains the constraints to know what products can receive a discount.

For instance:

• If I want to apply a 50% off to all products that fall under the "Beverages" category, and the "Beverages" category has id 5 in the database, the discount record would look like:

  discount_type='percent'
  discount='0.5'
  filter_by={
      'category': [5]
  }
  

• If I wanted to apply a $20 off to all the products in the "Beverages" category AND that are manufactured by, let's say, CocaCola, the filter_by dictionary would look like:

  discount_type='fixed amount'
  discount='20'
  filter_by={
      'category': [5],
      'manufacturer': [2]   # Assuming coca-cola is the Manufacturer 
                            # with id==2 in the 'Manufacturers'
                            # table of the database (NOTE: this is 
                            # needed since CocaCola manufactures
                            # products besides "Beverages")
  }
  

• If I wanted to apply a 25% off to a particular product (let's say to the product whose id is 3) the dictionary would look like:

  discount_type='percent'
  discount='0.25'
  filter_by={
      'id': [3]
  }
  

This idea seems to be flexible enough for my needs, and I'm happy (so far) with it.

---

Now, the problem comes on how to enter these values in the Django admin area for the Discount model.

As expected, the filter_by dictionary renders as as a text field that initially looks like this:

If I want to add fields to it, I need to write the exact JSON of what I want... Which means that if I want to apply a discount to the "Beverages" category, I need to go figure out which ID that category has in the database, and then manually type {"category": [5]}, while being extremely careful when typing the ', the :, make sure that I don't miss a ] or a [...

Thaaaat... well, that is not very helpful...

Since I am only going to be filtering by a few fields (category, manufacturer, product...) which are actually lists of IDs of other elements of the database, I would like to show a big MultiSelect box per thingy I can filter for, so I can see a user friendly list of all the elements I can filter by, select a few, and then, when I click on "Create discount", I would get the filter_by dictionary (I'm still far from worrying about how to generate the dictionary, since I don't even know how to properly render the Admin form).

Something like what Django Admin automatically did for my Products' categories:

That is really, really, nice: One product can belong to several categories. For that, Django renders, side by side, two <select multiple boxes, with the available categories, and the categories that the product already belongs to... I can add/remove categories through the stroke of a mouse... Really, really nice. But Django can do that because it knows that the categories are a ManyToMany relation in the Product model.

class Product(models.Model):
    parent = models.ForeignKey('self', null=True, blank=True)
    manufacturer = models.ForeignKey('Manufacturer')
    categories = models.ManyToManyField('Category',
                                         related_name='products', blank=True)

The problem with the Discount model is that there is no ManyToMany field to category, manufacturer or product. Poor Django doesn't know that a Discount is related to all those things: It only knows there's a Json field.

I would really like to be able to show a bunch of those <select> in the Django Area listing all the possible filters (Category, Manufacturer, ID...) that can be stored in the filter_by dictionary (one entry with the double <select> for Category showing all the available categories in the database, one entry for Manufacturer, showing all the available manufacturers... etcetera). But I really, really don't know how to do that.

I could bore you with a bunch of tries I've done, using Widgets, trying to represent the JSON field through a form, through forms.ModelMultipleChoiceField (which by the way, seems to have been the closest thing to what I want, although still very far)... But I think that is kind of pointless, since nothing came close to what I wanted.

As usual, thank you for reading this huge email and thank you in advance. Any hint will be really appreciated, even just a you should take a look to "this"

Answer 1

So... I appreciate @alfonso.kim's answer, but the idea of creating a whole new Django's model just for "rendering" purposes sounded like a bit of an overkill to me. Please! Don't get me wrong: It might be the "canonical" way of doing it (I've seen that approach recommended many times) and maybe is better than what I did, but I wanted to show how did I solve my particular question:

I took a look at Django's source code, particularly how a ManyToMany relation is shown in the Admin. If you look at my original question above, I wanted to figure out which class did Django use to display the categories while editing one product (that "double column select", to give it a name, which I so much liked). It turns out it is a django.forms.models.ModelMultipleChoiceField, "seasoned" with a hint of a FilteredSelectMultiple widget.

With this information I created a custom admin Form for my Coupon class, manually adding the fields I wanted shown:

class CouponAdminForm(forms.ModelForm):
    brands = forms.ModelMultipleChoiceField(
                            queryset=Brand.objects.all().order_by('name'),
                            required=False,
                            widget=FilteredSelectMultiple("Brands", is_stacked=False))
    categories = forms.ModelMultipleChoiceField(
                            queryset=Category.objects.all().order_by('name'),
                            required=False,
                            widget=FilteredSelectMultiple("Categories", is_stacked=False))
    products = forms.ModelMultipleChoiceField(
                            queryset=Product.objects.all().order_by('name'),
                            required=False,
                            widget=FilteredSelectMultiple("Products", is_stacked=False))

    def __init__(self, *args, **kwargs):
        # ... we'll get back to this __init__ in a second ... 

    class Meta:
        model = Coupon
        exclude = ('filter_by',)  # Exclude because we're gonna build this field manually

And then told the ModelAdmin class for my coupons to use that form instead of the default one:

class CouponsAdmin(admin.ModelAdmin):

    form = CouponAdminForm

# ... #
admin.site.register(Coupon, CouponsAdmin)

Doing this displayed the three Form's manually added fields (brand, categories and products) at the root of the formulary. In other words: This produced three new fields at the same level than the rest of the fields in my Coupon model. However: they were not trully "first class" fields, since they were actually going to determine the contents of one particular field in my Model (the Coupon.filter_by field) which, let's remember, is a dictionary looking more or less like:

filter_by = {
    "brands": [2, 3],
    "categories": [7]
}

In order to make clear for the human using the Admin web page that these three fields weren't "really" first level fields in the Coupon model, I decided to show them grouped.

To do that, I needed to change the CouponsAdmin layout of fields. I didn't want this grouping to affect how other fields of my Coupon model were displayed, even if new fields were later added to the model, so I let every other field of the form untouched (in other words: only apply the special/grouped layout to the brands, categories and products fields in the Form). To my surprise, I wasn't able to do this in the ModelForm class. I had to go to the ModelAdmin instead (I'm really not sure why...):

class CouponsAdmin(admin.ModelAdmin):
    def get_fieldsets(self, request, obj=None):
        fs = super(CouponsAdmin, self).get_fieldsets(request, obj)
        # fs now contains only [(None, {'fields': fields})] meaning, ungrouped fields
        filter_by_special_fields = (brands', 'categories', 'products')
        retval = [
            # Let every other field in the model at the root level
            (None, {'fields': [f for f in fs[0][1]['fields']
                               if f not in filter_by_special_fields]
                    }),
            # Now, let's create the "custom" grouping:
            ('Filter By', {
                'fields': ('brands', 'categories', 'products')
            })
        ]
        return retval

    form = CouponAdminForm

More information about fieldsets here

That did the trick:

Now, when an admin user created a new Coupon through this form (in other words: when a user clicked on the "Save" button on the page) I would get one queryset for extra field I had declared in my custom form (one queryset for brands, another one for categories and another one for products) but I actually needed to transform that information into a dictionary. I was able to achieve that by overwriting the save method of the Model's Form:

class CouponAdminForm(forms.ModelForm):
    brands = forms.ModelMultipleChoiceField(queryset=Brand.objects.all().order_by('name'),
                                            required=False,
                                            widget=FilteredSelectMultiple("Brands", is_stacked=False))
    categories = forms.ModelMultipleChoiceField(queryset=Category.objects.all().order_by('name'),
                                                required=False,
                                                widget=FilteredSelectMultiple("Categories", is_stacked=False))
    products = forms.ModelMultipleChoiceField(queryset=Product.objects.all().order_by('name'),
                                              required=False,
                                              widget=FilteredSelectMultiple("Products", is_stacked=False))

    def __init__(self, *args, **kwargs):
        # ... Yeah, yeah!! Not yet, not yet... 

    def save(self, commit=True):
        filter_by_qsets = {}
        for key in ['brands', 'categories', 'products']:
            val = self.cleaned_data.pop(key, None)  # The key is always gonna be in 'cleaned_data',
                                                    # even if as an empty query set, so providing a default is
                                                    # kind of... useless but meh... just in case
            if val:
                filter_by_qsets[key] = val  # This 'val' is still a queryset

        # Manually populate the coupon's instance filter_by dictionary here
        self.instance.filter_by = {key: list(val.values_list('id', flat=True).order_by('id'))
                                   for key, val in filter_by_qsets.items()}
        return super(CouponAdminForm, self).save(commit=commit)


    class Meta:
        model = Coupon
        exclude = ('filter_by',)

That correctly populated the Coupon's filter_by dictionary on "Save".

There was a little detail left (to make the admin form a little bit more user friendly): When editing an existing Coupon, I wanted the brands, categories and products fields of the form to be pre-populated with the values in the filter_by dictionary of the coupon.

Here's where modifying the __init__ method of the Form came in handy (keeping in mind that the instance that we are modifying is accessible in the self.instance attribute of the Form)

class CouponAdminForm(forms.ModelForm):
    brands = forms.ModelMultipleChoiceField(queryset=Brand.objects.all().order_by('name'),
                                            required=False,
                                            widget=FilteredSelectMultiple("Brands", is_stacked=False))
    categories = forms.ModelMultipleChoiceField(queryset=Category.objects.all().order_by('name'),
                                                required=False,
                                                widget=FilteredSelectMultiple("Categories", is_stacked=False))
    products = forms.ModelMultipleChoiceField(queryset=Product.objects.all().order_by('name'),
                                              required=False,
                                              widget=FilteredSelectMultiple("Products", is_stacked=False))

    def __init__(self, *args, **kwargs):
        # For some reason, using the `get_changeform_initial_data` method in the
        # CouponAdminForm(forms.ModelForm) didn't work, and we have to do it
        # like this instead? Maybe becase the fields `brands`, `categories`...
        # are not part of the Coupon model? Meh... whatever... It happened to me the
        # same it happened to this OP in stackoverflow: https://stackoverflow.com/q/26785509/289011
        super(CouponAdminForm, self).__init__(*args, **kwargs)
        self.fields["brands"].initial = self.instance.filter_by.get('brands')
        self.fields["categories"].initial = self.instance.filter_by.get('categories')
        self.fields["products"].initial = self.instance.filter_by.get('products')

    def save(self, commit=True):
        filter_by_qsets = {}
        for key in ['brands', 'categories', 'products']:
        # ... explained above ...

And that's it.

As of now (right now, March 19, 2017) this seems to be working nicely for what I needed.

As alfonso.kim points out in his answer, I can not dynamically filter the different fields unless I change the window's Javascrip (or maybe I use the ChainedForeignKey custom model? Don't know: didn't try that) What I mean is that with this approach I can not filter the select boxes on the admin web page removing products that only belong to the selected categories, for instance, I can not do things like "if a user selects a brand, filter categories and products so they only show elements that belong to that brand". This happens because there's no XHR (Ajax) request going between browser and server when the user selects a brand. Basically: the flow is you GET the form --> you fill up the form --> you POST the form, with no communication between browser <--> server while the user is clicking on "things" on the form. It would have been nice that if a user selects "Coca cola" in the brands select, the products select gets filtered, and removes plastic bags from the available products (for example) but well... This approach is "good enough" for my needs.

Please, be advised: The code in this answer could contain some redundant actions, or things that could have been written better, but so far, it seems to be working ok (who knows, maybe I'll have to edit my answer a few days from now saying "I was completely wrong!! Please don't do this!" but so far it seems ok) Needless to say: I will welcome any comment of suggestion that anyone has to say :-)

I hope this helps someone in the future.

Answer 2

You will need some javascript to fit the json dictionary into a nice HTML widget, and then process it in the Django handler.

If you want to use the "magic" of Django admin, you have to give it the input it needs to render that nice UI and create models for your discount system:

class Discount(models.Model):
  discount_type = models.TextField()
  discount_percentage = models.FloatField()

class DiscountElement(models.Model):
  discount = models.ForeignKey(Discount)
  manufacturer = models.ForeignKey(Manufacturer, null=True)
  category = models.ForeignKey(Category, null=True)