Project 0: The Django HTMX Stack

Project 0: The Django HTMX Stack

November 29 2021

Project 0. Fundamentals

Introduction

I built my first web application by following a Django crash course video that took a little over an hour. It was the best feeling in the world.

I didn't understand much of how Django worked or even the internet, but I was happy and went on to eventually learn these things on my own. This is a "by example" course, so the idea is to capture that same feeling.

We will build eight different web applications using HTMX and Django. All would be interactive, modern single-page applications without complicated Javascript frameworks.

Before we do though, it will help a lot if we went over how Django works. Why we needed single-page applications in the first place. As well as why HTMX is the perfect complement to use with the batteries included framework.

Prerequisites for the course:

  1. Python installed on your system with an understanding of basic syntax
  2. Comfortable with a modern IDE such as VS Code
  3. Familiarity with basic Git commands
  4. Familiarity with the command line. We would use Linux commands in this course. If you have Windows, I recommend downloading and configuring WSL from here.
  5. You have Docker installed on your system. You can install it from here. You don't need to understand how it works yet.
  6. You don't need an extensive experience with Django. If you have trouble following, you may need to do the official Django tutorial or something similar.

History

Like all good code, Django was developed to solve an urgent problem. Back in 2003-2004, Django creators were PHP developers who were maintaining large web applications and struggling. It was still early in the web and the life of PHP, so large codebases with complex database operations were difficult to manage.

The two creators were in love with Python and so decided to build a Python web framework. The aim of Django was to use it to manage a news organization web sites with it. Just like Rails was extracted out of BaseCamp, and React out of Facebook, Django was built to fit the functionality of a news organization.

It was designed to make common Web-development tasks fast and easy with an opinionated take on how to do things. As an overview, it has three main parts:

  1. An object-relational mapper. This is the part where we describe our database layout in Python code. This layout lives in a file called models.py.
  2. URLs module. URLs in Django drive what actions should be taken. At its heart, Django is about making your URLs clean and elegant. URLs function like a table of contents for your app. Each URL pattern maps to Python callback function. That would be your urls.py files.
  3. Views are the last step in a request in Django. They return what will the user see. Traditionally, an HTML template. They live in a views.py file.

Requests

The way it works is straightforward. A user makes a request to the server where our Django web application is. Depending on the URL of the request, Django routes this request to the right view. That view will have a function that return something back to the user to see, after possibly performing a database operation.

Back when Django first came out that something was always HTML. Let's say you make a request to a server where www.htmx-django.com lives. The server takes your request and then gives you back HTML . That HTML is what you see. We call this a "GET" request.

Sometimes, as in a form on a website. The user gives the server information. The server does a couple of things with that information and then responds back. We call this a POST request. For example, in a To Do application, the flow will go something like this.

Now - this pattern of request -> response -> request worked for a long time. There are big, successful web applications where it's just HTML going back and forth between user and server. There isn't anything complicated in between and every dynamic action reloads the page.

What Django does is manage this process for us. It connects each request from what URL it came from to a view. In the view, we process the information and save or retrieve data from our models.  The model is where the data is mapped to a database. It works very well for many use cases, but with one big limitation. Every dynamic action the user takes reloads the page.

Single-Page Applications

Let's say we are building a Twitter or a Facebook-like application. Every time a user likes a post, their whole feed will be reloaded. Even in a simple to-do application, any new actions like adding/deleting/updating items will reload the whole page. It works, but it is not the most user-friendly setup.

To fix this problem, web development has moved in recent years to single-page applications (SPA) using Javascript frameworks. The idea here is that instead of having each action associated with a request and a response, the server will just send a big chunk of Javascript that handles whatever the user decides to do and display the results to them without ever hitting the server.

Even if there is a database involved and you need to save or retrieve something from there. You don't send & receive HTML, rather you send and receive JSON, and then the Javascript handles that JSON data in the front end.

In that case, Django's (through a package called Django Rest Framework) role is to handle that JSON data on the backend. To send JSON when the front end asks for it, and to receive and process it when it gets from the front end.

There is no HTML traveling back and forth in that case. The something that travels between server and user here is JSON data, not HTML.

Now, SPAs fixed the problem of too many server requests and reloading the page for everything, but caused a few other problems. Here are a few:

  1. The Javascript code that gets sent on the initial request can be very heavy. Unlike a traditional web application, someone who requests the FAQ page is also downloading 15 other pages in the same request. For a traditional web application, only the FAQ page HTML will travel between server and user.
  2. SPAs are initially a blank page, which is then filled out by JavaScript. This is not ideal for SEO. Where search engine crawlers would just see the initial blank page.
  3. Data requests can get complicated very quickly. For example, the simple case of logging in a user requires a lot of overhead - just to make sure that passwords are not exposed in the request.

 

credit to htmx.org

There are fixes for these issues, but the fixes also have their drawbacks. In a perfect world, we would get to send and receive HTML with Django. Only updating the parts that got changed in the DOM, without reloading the whole page. That way we get all the benefits of SPAs without any of the drawbacks or complications.

I have good news for you. That perfect world is here. By using an HTMX/Django stack and the techniques in this course, we don't have to worry about APIs, complicated Javascript frameworks, or the limitations of reloading the page on each dynamic action. We get to have all the goodies of a SPA and Django without any of the limitations or overhead.

Exciting, right? So, let's get started!


So how does HTMX work?

In a traditional web application, there are two ways users can make a request to the server. They can make a "GET" request through <a> tag or a "POST" request when they submit a form. There are other actions than GET and POST, but let's keep it simple for now. After the request is processed, the server returns a new HTML template.

HTMX gives us two superpowers. The first superpower is the response of the server doesn't have to replace the whole page. It can replace just the pieces of the DOM you specify.

The second superpower is that a user can make a request to the server with anything in the DOM. Clicking a <div> tag? That works. Moving their mouse over something? Sure! Typing something in an input field? Yes. We can even make a request every few seconds, without the user doing anything. Like if we want to update stock prices for example. Practically anything you can imagine can trigger a POST or a GET request to the server via HTMX.

Let's explore a simple real use case. Let's say you have an email sign-up form, like the one I have at the footer.

Without HTMX, a user types their email and submits the form. This triggers a POST request to the server. A Django view then validates the data. If the data is good, the email is saved to our database and the view returns an HTML template with a success message. Otherwise, the view returns an HTML template with an error message if the user submitted nonsense or a duplicate email.

Either way, our page gets re-rendered, either with a success message or errors. Also, all the validations happen after the user clicks the submit button with no feedback before that. 

Wouldn't be great if we gave our user feedback as they typed before hitting the submit button? Even stopping them altogether from submitting nonsense? Also, we don't need to reload the whole page. We just need to update whatever HTML DOM that has an error message or a success message.

That's exactly what HTMX does. Let's explore the code for something like this where our user story is:

  1. Users should be able to put their email to receive updates 
  2. We should validate this email before submission to make sure that it is not a duplicate and is in the correct email format.
  3. We should give our user feedback before submitting.
  4. On successful submission or an error, we should let the user know without reloading the whole page

Here is the HTMX code,

<form hx-post="{% url 'email' %}" >
<div 
id="email_form" 
hx-target="#email_form" 
hx-swap="outerHTML">
    <h3>Email Form</h3>
    <label>Email Address</label>
    <input 
    name="email" 
    placeholder="Your email here..." 
    required = "required"
    >
   <button> Submit </button>
</div>
</form>

Here is what is happening in that form. The first line "hx-post" says that when the user submits the form. A POST request goes to the Django URL 'email'. This is what hx-post means. We can also use hx-get, hx-delete, hx-put for these request methods.

The fourth line, "hx-target" is saying that the response will only change the DOM element with an id of "email_form". The rest of the HTML on the page will not change. Since the id of the div and the hx-target are the same, we could have used hx-target="this". Python developers are explicit though, so we would always write down our targets.

The fifth line "hx-swap" is describing how will we change the DOM element (our target). outerHTML means we want to swab the whole element with the response.

To summarize, our HTMX code is saying that on form submission:

  1. Make a POST request to the URL named "email"
  2. Then swap the DOM element with the id "form-email" with whatever the server sends.

We won't go through the Django code in this chapter, but it is the standard pattern of Model/URL/View. The only difference is that in our view, we would send back the form and a success/failure message. Without HTMX we would redirect to another view or render a whole HTML template.

This takes care of user story #1 and #4. Where we used the HTMX superpower of only updating the HTML pieces that we want to be changed.

We still need to validate the email before submission & give our user immediate feedback. To do this, we need to add a few extra lines to our input field.

<form hx-post="{% url 'email' %}" >
<div
id= "email_form" 
hx-target="#email_form" 
hx-swap="outerHTML">
    <h3>Email Form</h3>
    <label>Email Address</label>
    <input 
    name="email" 
    placeholder="Your email here..." 
    required = "required"
    hx-post="{% url 'custom_email_validation' %}"
    hx-trigger= "changed" 
    hx-target = "#email_feedback"
    >
    <button> Submit </button>
</div>
</form>
<div id="email_feedback" > </div>

Here we are using the HTMX superpower of allowing anything on the DOM to trigger a request. We are putting a "hx-post" on an input field where when a "changed" event occurs, it triggers a POST request to a URL named "custom_email_validation". The response would replace the DOM element with the id "email_feedback".

We are explicitly saying what is the trigger to the POST request via the "hx-trigger" line even though it is the default behavior. We technically don't need that line. Also, in future chapters, we will do some magic with Django templates and forms, where our target stays the same.

For this to work in the backend, we would need to configure a new URL named "custom_email_validation" in our urls.py module. Also, we would need a view that returns the results of that validation. Again, we would go through all the Django logic and how to optimize it in a future chapter. This is just a quick overview of how HTMX works.

In our next chapter, we would build and deploy a "to do" web application. Where we would use the awesome power of Django forms with what we learned about HTMX to make a clean, elegant single-page application.

To learn more about the course or sign up for early access, please see here. I plan to update things on a weekly basis,

Further Readings:

  1. The History of Django
  2. Second-guessing SPA's
  3. What is JSON?
  4. What is the DOM?

Food for thought:

Go to the djangoproject.com, open up the development console. On Windows or Linux, open the console by pressing F12 or ctrl-shift-i simultaneously. On Mac, its option+cmd+i. Go to the network tab and refresh the page.

  1. How many network requests do you see?
  2. How long did the whole request take?
  3. What was the slowest request?

For reference, it took me 479 ms for me to load with 21 requests to the server. The slowest request was the font awesome CSS library.

Do the same thing with the reactjs.org

  1. How many network requests do you see?
  2. How long did the whole request take?
  3. What was the slowest request?

Now - repeat the process navigating different section of the site. Write down your observations.

Get notified about new HTMX/Django tutorials
You will only get an email whenever a post or a course is published!

Copyright © HTMX-Django 2021.

Built by Jonathan Adly. Contact for opportunities.