Aral Balkan

Mastodon icon RSS feed icon

Streaming HTML

Building the Streaming HTML counter example.

Estimated reading time: 25 minutes.

Kitten has a new experimental workflow for creating web apps called Streaming HTML that I want to introduce you to today.

Kitten, uniquely, enables you to build Small Web apps (peer-to-peer web apps). But it also aims to make creating any type of web app as easy as possible. The new Streaming HTML workflow is a big step in realising this goal.

So let’s jump in and see how Streaming HTML works by implementing the ubiquitous counter example.

O counter! My counter!

  1. Install Kitten (this should take mere seconds).

  2. Create a directory for the example and enter it:

    mkdir counter
cd counter

  3. Create a file called index.page.js and add the following content to it:

    // Initialise the database table if it doesn’t already exist.
if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

    // Default route that renders the page on GET requests.
export default () => kitten.html`
  <page css>
  <h1>Counter</h1>

  <${Count} />

  <button
    name='update' connect data='{value: -1}'
    aria-label='decrement'
  >-</button>

  <button
    name='update' connect data='{value: 1}'
    aria-label='increment'
  >+</button>
`

    // The Count fragment.
const Count = () => kitten.html`
  <div
    id='counter'
    aria-live='assertive'
    morph
    style='font-size: 3em; margin: 0.25em 0;'
  >
    ${kitten.db.counter.count}
  </div>
`

    // The connect event handler responds to events from the client.
export function onConnect ({ page }) {
  page.on('update', data => {
    kitten.db.counter.count += data.value
    page.send(kitten.html`<${Count} />`)
  })
}

  4. Run Kitten using the following syntax:

    kitten

Once Kitten is running, hit https://localhost, and you should see a counter at zero and two buttons.

Press the increment and decrement buttons and you should see the count update accordingly.

Press CtrlC in the terminal to stop the server and then run kitten again.

Refresh the page to see that the count has persisted.

What just happened?

In a few lines of very liberally-spaced code, you have built a very simple Streaming HTML web application in Kitten that:

In a nutshell, Kitten gives you a simple-to-use event-based HTML over WebSocket implementation called Streaming HTML (because you’re streaming HTML updates to the client) that you can use to build web apps.

HTML over WebSocket is not unique to Kitten – the approach is formalised with different implementations in a number of popular frameworks and application servers. And the general idea of hypermedia-based development actually predates the World Wide Web and HTML.

What is unique, however, is just how simple Kitten’s implementation is to understand, learn, and use.

That simplicity comes from the amount of control Kitten has over the whole experience. Kitten is not just a framework. Nor is it just a server. It’s both. This means we can simplify the authoring experience using file system-based routing combined with automatic WebSocket handling, a built-in in-process native JavaScript database, a simple high-level declarative API, and built-in support for libraries like htmx.

Kitten’s Streaming HTML flow – and Kitten’s development process in general – stays as close to pure HTML, CSS, and JavaScript as possible and progressively enhances these core Web technologies with features to make authoring web applications as easy as possible.

Let’s break it down

OK, so now we have a high-level understanding of what we built, let’s go through the example and dissect it to see exactly how everything works by peeling away the layers of magic one by one.

Let’s begin with what happens when you start the server.

During its boot process, Kitten recurses through your project’s folder and maps the file types it knows about to routes based on their location in the directory hierarchy.

In our simple example, there is only one file – a page file. Since it’s located in the root of our project folder and named index, the created route is /.

Pages, like other route types in Kitten, are identified by file extension (in this case, .page.js) and are expected to export a default function that renders the web page in response to a regular GET request.

Initial page render

export default () => kitten.html`
  <page css>
  <h1>Counter</h1>

  <${Count} />

  <button
    name='update' connect data='{value: -1}'
    aria-label='decrement'
  >-</button>

  <button
    name='update' connect data='{value: 1}'
    aria-label='increment'
  >+</button>
`

This renders a heading, the current count, and two buttons onto the page and sprinkles a bit of magic semantic CSS styling.

Notice a few things about this code:

Next, let’s take a look at the Kitten-specific aspects of this template, starting with the first tag.

Deconstructing the page

The first piece of magic on the page is the simplest, a <page> that has a css attribute specified:

<page css>

The page tag is transpiled by Kitten into HTML. In this case, since the css attribute is specified, it results in the following stylesheet reference in the head of the page:

<link rel="stylesheet" href="/🐱/library/water-2.css">

This, in turn, loads in the Water semantic CSS library that gives our page some basic styles based on the HTML elements we used.

🐱 Go ahead and delete the line with the <page> tag and see how it affects the display of the page, then undo the change. Kitten will automatically update your page in the browser whenever you save your file.

Kitten has first-class support for certain libraries, Water being one of them, that it serves from the reserved /🐱/library/ namespace. Instead of manually including these libraries, you can just use the <page> tag like we did here.

Most of the magic in this example, as we will see later, relies on a different library called htmx and its WebSocket and idiomorph extensions.

Components and fragments

Next in our page, we have a heading, followed by the Count fragment, included in the page using:

export default () => kitten.html`
  …
  <${Count} />
  …
`

This results in the Count function being called to render the fragment as HTML:

const Count = () => kitten.html`
  <div
    id='counter'
    aria-live='assertive'
    morph
    style='font-size: 3em; margin: 0.25em 0;'
  >
    ${kitten.db.counter.count}
  </div>
`

🐱 Kitten encourages you to split your code into components and fragments1. This becomes even more important in a streaming HTML workflow where you initially render the whole page and then send back bits of the page to be morphed into place to update the page. Breaking up your content into components and fragments enables you to remove redundancy in your code.

This fragment creates a div that displays the current count of the counter, which it gets from Kitten’s magic default database.

Kitten’s magic database

Kitten comes with a very simple, in-process JavaScript database called ­– drumroll – JavaScript Database (JSDB).

It even creates a default one for you to use at kitten.db.

🐱 You’re not limited to using the default database that Kitten makes available. You can create your own and even use multiple databases, etc., using database app modules. You can also implement type safety in your apps, including for your database structures.)

In JSDB you store and access data in JavaScript arrays and objects and you work with them exactly as you would with any other JavaScript array or object. The only difference is that the changes you make are automatically persisted to an append-only JavaScript transaction log in a format called JavaScript Data Format (JSDF).

It’s a common pattern in JSDB to check whether an array or object (the equivalent of a table in a traditional database) exists and, if not, to initialise it.

This is what we do at the very start of the file that contains the page route, creating the counter object with its count property set to zero if it doesn’t already exist:

if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

Once you are sure a value exists in your database, you can access it using regular JavaScript property look-up syntax (because it is just a regular JavaScript object).

This is what we do in the Count component:

const Count = () => kitten.html`
  …
  ${kitten.db.counter.count}
  …
`

So the first time the page renders, the count will display as zero.

After the Count fragment, we have the last two elements on the page: two buttons, one to decrement the count and the other to increment it.

But what is the magic that allows us to connect those buttons to the server, mutate the count, and persist the value?

Let’s look at that next.

A magic connection

At the heart of Kitten’s Streaming HTML workflow is a cross-tier eventing system that maps events on the client to handlers on the server.

Take a look at the two buttons declared in our page to see how it works:

<button
  name='update' connect data='{value: -1}'
  aria-label='decrement'
>-</button>

<button
  name='update' connect data='{value: 1}'
  aria-label='increment'
>+</button>

Both of the buttons have the same name, update. That name is the name of the event that will fire on the server when that button is pressed thanks to the magic connect attribute we’ve added to the buttons. Additionally, the contents of the magic data attribute will also be sent to the event handler.

The event handler in question is the only other bit of code in our pithy example:

export function onConnect ({ page }) {
  page.on('update', data => {
    kitten.db.counter.count += data.value
    page.send(kitten.html`<${Count} />`)
  })
}

In the onConnect() handler, we receive a parameter object with a reference to the page object.

Using the page reference, we set up an event handler for the update event that receives the data from the button that triggered the event and adds its value to the count property of our counter in our database.

Finally, we use a method provided for us called send() on the same page reference to stream a new Count component.

If you remember, the Count component had one last magic attribute on it called morph:

<div
  id='counter'
  aria-live='assertive'
  morph
  style='font-size: 3em; margin: 0.25em 0;'
>

This makes Kitten intelligently morph the streamed HTML into the DOM, replacing the element that matches the provided id.

Notice that unlike web apps that you may be familiar with, we are not sending data to the client, we are sending hypermedia in the form of HTML.

Streaming HTML is a modern event-based full-duplex approach to building hypermedia-based applications.

Its greatest advantage is its simplicity, which arises from keeping state on one tier (the server) instead of on two (the client and the server). In essence, it is the opposite of the Single-Page Application (SPA) model, embracing the architecture of the Web instead of attempting to turn it on its head. In fact, you can create whole Web apps without writing a single line of client-side JavaScript yourself.

And with that, we now know what Streaming HTML is and what each part of the code does.

Now, let’s go back to the start and review the process as we start to understand how things work at a deeper level.

High-level flow

Let’s go step-by-step, starting from when we launch Kitten to when the counter is updated:

  1. Kitten parses the page and sees that there is an onConnect() handler defined so it creates a default WebSocket route for the page and wires it up so that when the page loads, a WebSocket connection is made that results in the onConnect() handler being called.

  2. When a person hits the page, the onConnect() handler gets called. In the handler, we set up an event handler to handle the update event.

  3. When the person presses the increment button, it sends a message to the default WebSocket route. Since the button’s name is update, Kitten calls the update event handler, passing a copy of any data that was sent along.

  4. In this case, the data is {value: 1}. It is an object that has a value property set to 1. So we add the value to the count we are keeping in our database and send a new Count fragment back.

At this point, you might be wondering about several things:

The answer to both of those questions is ‘through the magic of htmx’.

So what is htmx?

Let’s find out!

Peeking behind the curtain: htmx

Earlier, I wrote that most of the magic in this example relies on a library called htmx and its WebSocket and idiomorph extensions. Let’s now dive a little deeper into the internals of Kitten and take a look at how Kitten transpiles your code to use this library and its extenions.

In our example, whenever either the increment or decrement button gets pressed on the client, the update event handler gets called on the server whereupon it updates the counter accordingly and sends a new Count fragment back to the client that gets morphed into place in the DOM.

There are three things working in tandem to make this happen, all of which sprinkle htmx code into your page behind the scenes.

First, whenever Kitten sees that one of your pages has exported an onConnect() event handler, it:

  1. Adds the htmx library, as well as its WebSocket and idiomorph extensions, to the page.

  2. It creates a special default WebSocket route for your page. In this case, since our page is the index page and is accessed from the / route, it creates a socket that is accessed from /default.socket. In that socket route, it adds an event listener for the message event and maps any HTMX-Trigger-Name headers it sees in the request to event handlers defined on the page reference it provides to the onConnect() handler when the WebSocket connects.

  3. It adds a send() method to the page reference passed to the onConnect() handler that can be used to stream responses back to the page. We haven’t used them in this example but it also adds everyone() and everyoneElse() methods also that can be used to stream responses back not just to the person on the current page but to every person that has the page open (or to every person but the current one).

Second, it goes through your code and, whenever it sees a form, it adds the necessary htmx WebSocket extension code so form submits will automatically trigger serialisation of form values. (We don’t make use of this in this example, preferring to forego a form altogether and directly connect the buttons instead.)

Finally, it applies some syntactic sugar to attribute names by replacing:

These little conveniences make authoring easier without you having to remember the more verbose htmx attributes. You can, of course, use the htmx attributes instead, as well as any other htmx attribute, because it is just htmx under the hood.

Progressive enhancement

Kitten’s design adheres to the philosophy of progressive enhancement.

At its very core, Kitten is a web server. It will happily serve any static HTML you throw at it from 1993.

However, if you want to, you can go beyond that. You can use dynamic pages, as we have done here, to server render responses, use a database, etc.

Similarly, Kitten has first-class support for the htmx library and some of its extensions, as well as other libraries like Alpine.js.

The idea is that you can build your web apps using plain old HTML, CSS, and JavaScript and then layer additional functionality on top using Kitten’s Streaming HTML features, htmx, Alpine.js, etc. You can even use its unique features to make peer-to-peer Small Web apps.

So Kitten’s implementation of Streaming HTML is based on core Web technologies and progressively enhanced using authoring improvements, htmx, and a sprinkling of syntactic sugar (collectively, what we refer to as ‘magic’).

All this to say, you can do everything we did in the original example by using htmx and creating your WebSocket manually.

Let’s see what that would look like next.

Goodbye, magic! (Part 1: Goodbye, syntactic sugar; hello, htmx)

Right, let’s peel away a layer of the magic and stop making use of Kitten’s automatic event mapping and syntactic sugar and use plain htmx instead, starting with the index page:

index.page.js

import Count from './Count.fragment.js'

export default function () {
  return kitten.html`
    <page htmx htmx-websocket htmx-idiomorph css>
    <main
      hx-ext='ws'
      ws-connect='wss://${kitten.domain}:${kitten.port}/count.socket'
    >
      <h1>Counter</h1>
      <${Count} />
      <button
        name='update' ws-send hx-vals='js:{value: -1}'
        aria-label='decrement'
      >-</button>
      <button
        name='update' ws-send hx-vals='js:{value: 1}'
        aria-label='increment'>
      +</button>
    </main>
  `
}

Notice, what’s different here from the previous version:

  1. The Count fragment now lives in its own file (with the extension .fragment.js) that we import into the page. This is because we now have to create the WebSocket route ourselves, in a separate file, and it will need to use the Count fragment too when sending back new versions of it to the page. Previously, our onConnect() handler was housed in the same file as our page so our fragment was too.

  2. We have to manually let Kitten know that we want the htmx library and its two extensions loaded in, just like we had to do with the Water CSS library (the css attribute is an alias for water; you can use either. Kitten tries to be as forgiving as possible during authoring).

  3. We wrap our counter in a main tag so we have some place to initialise the htmx ws (WebSocket) extension. We also have to write out the connection string to our socket route manually. As we’ll see later, our socket route is called count.socket. While writing the connection string, we make use of the Kitten globals kitten.domain and kitten.port to ensure that the connection string will work regardless of whether we are running the app locally in development or from its own domain in production.

  4. Instead of Kitten’s syntactic sugar, we now use the regular htmx attributes ws-send and hx-vals in our buttons.

Next, let’s take a look at the Count fragment.

Count.fragment.js

if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

export default function Count () {
  return kitten.html`
    <div
      id='counter'
      aria-live='assertive'
      hx-swap-oob='morph'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${kitten.db.counter.count}
    </div>
  `
}

Here, apart from being housed in its own file so it can be used from both the page and the socket routes, the only thing that’s different is that we’re using the htmx attribute hx-swap-oob (htmx swap out-of-band) instead of Kitten’s syntactic sugar morph attribute.

We also make sure the database is initialised before we access the counter in the component.

We’re carrying out the initialisation here and not in the socket (see below) because we know that the page needs to be rendered (and accessed) before the socket route is lazily loaded. While this is fine in a simple example like this one, it is brittle and requires knowledge of Kitten’s internals. In a larger application, a more solid and maintainable approach would be to use a database app module to initialise your database and add type safety to it while you’re at it.

🐱 A design goal of Kitten is to be easy to play with. Want to spin up a quick experiment or teach someone the basics of web development? Kitten should make that simple to do. Having magic globals like the kitten.html tagged template you saw earlier help with that.

However, for larger or longer-term projects where maintainability becomes an important consideration, you might want to make use of more advanced features like type checking.

The two goals are not at odds with each other.

Kitten exposes global objects and beautiful defaults that make it easy to get started and, at the same time, layers on top more advanced features that make it easy to build larger and longer-term projects.

Finally, having seen the page and the Count component, let’s now see what the WebSocket route – which was previously being created for us internally by Kitten – looks like.

count.socket.js

import Count from './Count.fragment.js'

export default function socket ({ socket }) {
  socket.addEventListener('message', event => {
    const data = JSON.parse(event.data)

    if (data.HEADERS === undefined) {
      console.warn('No headers found in htmx WebSocket data, cannot route call.', event.data)
      return
    }

    const eventName = data.HEADERS['HX-Trigger-Name']
    
    switch (eventName) {
      case 'update':
        kitten.db.counter.count += data.value
        socket.send(kitten.html`<${Count} />`)
      break

      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
}

Our manually-created socket route is functionally equivalent to our onConnect() handler in the original version. However, it is quite a bit more complicated because we have to manually do, at a slightly lower level, what Kitten previously did for us.

Socket routes in Kitten are passed a parameter object that includes a socket reference to the WebSocket instance. It can also include a reference to the request that originated the initial connection.2

The socket object is a ws WebSocket instance with a couple of additional methods – like all() and broadcast(), mixed in by Kitten.3

On this socket instance, we listen for the message event and, when we receive a message, we manually:

  1. Deserialise the event data.

  2. Check that htmx headers are present before continuing and bail with a warning otherwise.

  3. Look for the HX-Trigger-Name header and, if the trigger is an event we know how to handle (in this case, update), carry out the updating of the counter that we previously did in the on('update') handler.

For comparison, this was the onConnect() handler from the original version where Kitten essentially does the same things for us behind the scenes and routes the update event to our handler:

export function onConnect ({ page }) {
  page.on('update', data => {
    kitten.db.counter.count += data.value
    page.send(kitten.html`<${Count} />`)
  })
}

If you run our new – plain htmx – version of the app, you should see exactly the same counter, behaving exactly the same as before.

While the plain htmx version is more verbose, it is important to understand that in both instances we are using htmx. In the original version, Kitten is doing most of the work for us and in the latter we’re doing everything ourselves.

Kitten merely progressively enhances htmx just like htmx progressively enhances plain old HTML. You can always use any htmx functionality and, if you want, ignore Kitten’s magic features.

Goodbye, magic! (Part 2: goodbye, htmx; hello, plain old client-side JavaScript)

So we just stipped away the magic that Kitten layers on top of htmx to see how we would implement the Streaming HTML flow using plain htmx.

Now, it’s time to remove yet another layer of magic and strip away htmx also (because htmx is just a bit of clever client-side JavaScript that someone else has written for you).

We can do what htmx does manually by writing a bit of client-side JavaScript (and in the process see that while htmx is an excellent tool, it’s not magic either).

Let’s start with the index page, where we’ll strip out all htmx-specific attributes and instead render a bit of client-side JavaScript that we’ll write ourselves.

Our goal is not to reproduce htmx but to implement an equivalent version of the tiny subset of its features that we are using in this example. Specifically, we need to write a generic routine that expects a snippet of html encapsulated in a single root element that has an ID and replaces the element that’s currently on the page with that ID with the contents of the new one.

index.page.js

import Count from './Count.fragment.js'

export default function () {
  return kitten.html`
    <page css>
    <h1>Counter</h1>
    <${Count} />
    <button onclick='update(-1)' aria-label='decrement'>-</button>
    <button onclick='update(1)' aria-label='increment'>+</button>
    <script>
      ${[clientSideJS.render()]}
    </script>
  `
}

/**
  This is the client-side JavaScript we render into the page.
  It’s encapsulated in a function so we get syntax
  highlighting, etc. in our editor.
*/
function clientSideJS () {
  const socketUrl = `wss://${window.location.host}/count.socket`
  const ws = new WebSocket(socketUrl)
  ws.addEventListener('message', event => {
    const updatedElement = event.data

    // Get the ID of the new element.
    const template = document.createElement('template')
    template.innerHTML = updatedElement
    const idOfElementToUpdate = template.content.firstElementChild.id

    // Swap the element with the new version.
    const elementToUpdate = document.getElementById(idOfElementToUpdate)
    elementToUpdate.outerHTML = updatedElement
  })

  function update (value) {
    ws.send(JSON.stringify({event: 'update', value}))
  }
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

Here’s how the page differs from the htmx version:

  1. The htmx and htmx-websocket attributes are gone. Since htmx is no longer automatically creating our socket connection for us, we do it manually in our client-side JavaScript.

  2. The htmx-idiomorph extension is also gone. Since htmx is not automatically carrying out the DOM replacement of the updated HTML fragments we send it, we do that manually also in our client-side JavaScript.

    We do so by first creating a template element and populating its inner HTML with our HTML string. Then, we query the resulting document fragment for the id of its top-level element. Finally, we use the getElementById() DOM look-up method on the resulting document fragment to get the current version of the element and replace it by setting its outerHTML to the updated HTML fragment we received from the server.

  3. Finally, since we no longer have htmx to send the HX-Trigger-Name value so we can differentiate between event types, we add an event property to the object we send back to the server via the WebSocket.

Documenting the (overly) clever bits

There are two bits of the code where we’re doing things that might be confusing.

First, when we interpolate the result of the clientSideJS.render() call into our template, we surround it with square brackets, thereby submitting the value wrapped in an array:

export default function () {
  return kitten.html`
    …
    <script>
      ${[clientSideJS.render()]}
    </script>
  `
}

This is Kitten shorthand for circumventing Kitten’s built in string sanitisation. (We know that the string is safe because we created it.)

🐱 Needless to say, only use this trick with trusted content, never with content you receive from a third-party. By default, Kitten will sanitise any string you interpolate into a kitten.html string. So the default is secure. If you want to safely interpolate third-party HTML content into your pages, wrap the content in a call to kitten.safelyAddHtml() which will sanitise your html using the sanitize-html library.

The other bit that might look odd to you is how we’re adding the render() function to the clientSideJS() function:

function clientSideJS () {
  //…
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

You might be wondering why we wrote our client-side JavaScript code in a function on our server to begin with instead of just including it directly in the template.

We did so to make use of the language intelligence in our editor.

Given how little code there is in this example, we could have just popped it into the template string. But this provides a better authoring experience and is more maintainable.

Of course what we need is a string representation of this code – sans the function signature and the closing curly bracket – to embed in our template.

Again, we could have just added that logic straight into our template:

export default function () {
  return kitten.html`
    …
    <script>
      ${[clientSideJS.toString().split('\n').slice(1, -1).join('\n')]}
    </script>
  `
}

That does the same thing but it doesn’t really roll off the tongue.

I feel that templates should be as literate, readable, and close to natural language as possible and that any complex stuff we might have to do should be done elsewhere. And since in JavaScript nearly everything is an object, including functions, why not add the function to render the inner code of a function onto the function itself?4

OK, enough JavaScript geekery.

Next, let’s take a look at how the WebSocket route has changed.

count.socket.js

import Count from './Count.fragment.js'

export default function socket ({ socket }) {
  socket.addEventListener('message', event => {
    const data = JSON.parse(event.data)

    if (data.event === undefined) {
      console.warn('No event found in message, cannot route call.', event.data)
      return
    }

    switch (data.event) {
      case 'update':
        kitten.db.counter.count += data.value
        socket.send(kitten.html`<${Count} />`)
      break

      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
}

The general structure of our WebSocket route remains largely unchanged with the following exception: instead of using htmx’s HX-Trigger-Name header, we look for the event property we’re now sending back as part of the data and using that to determine which event to handle. (Again, in our simple example, there is only one event type but we’ve used a switch statement anyway so you can see how you could support other events in the future by adding additional case blocks to it.)

Finally, the Count fragment remains unchanged.

Here it is, again, for reference:

if (kitten.db.counter === undefined) kitten.db.counter = { count: 0 }

export default function Count () {
  return kitten.html`
    <div
      id='counter'
      aria-live='assertive'
      hx-swap-oob='morph'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${kitten.db.counter.count}
    </div>
  `
}

Goodbye, magic! (Part 3: goodbye, Kitten; hello plain old Node.js)

So we just saw that Kitten’s Streaming HTML workflow can be created by writing some plain old client-side JavaScript instead of using the htmx library (which, of course, is just plain old client-side JavaScript that someone else wrote for you).

But we are still using a lot of Kitten magic, including its file system-based routing with its convenient WebSocket routes, its first-class support for JavaScript Database (JSDB), etc.

What would the Streaming HTML counter example look like if we removed Kitten altogether and created it in plain Node.js?

🐱 Kitten itself uses Node.js as its runtime. It installs a specific version of Node.js – separate from any others you may have installed in your system – for its own use during the installation process.

Streaming HTML, plain Node.js version

To follow along with this final, plain Node.js version of the Streaming HTML example, make sure you have a recent version of Node.js installed. (Kitten is regularly updated to use the latest LTS version so that should suffice for you too.)

First off, since this is a Node.js project, let’s initialise our package file using npm so we can add three Node module dependencies that we previously made use of without knowing via Kitten.

  1. Create a new folder for the project and switch to it.

    mkdir count-node
cd count-node

  2. Initialise your package file and install the required dependencies – the ws WebSocket library as well as Small Technology Foundation’s https and JSDB libraries.

    Of the Small Technology Foundation modules, the former is an extension of the standard Node.js https library that manages TLS certificates for you automatically both locally during development and via Let’s Encrypt in production and the latter is our in-process JavaScript database.

    npm init --yes
npm i ws @small-tech/https @small-tech/jsdb

  3. Tell Node we will be using ES Modules (because, hello, it’s 2024) by adding "type": "module" to the package.json file (do you get the feeling I just love having to do this every time I start a new Node.js project?)

    Either do so manually or use the following one-line to make yourself feel like one of those hackers in the movies:5

    sed -i '0,/,/s/,/,\n  "type": "module",/' package.json

  4. Create the application.

    // Import dependencies.
import path from 'node:path'
import { parse } from 'node:url'
import { WebSocketServer } from 'ws'
import JSDB from '@small-tech/jsdb'
import https from '@small-tech/https'

    // Find the conventional place to put data on the file system.
// This is where we’ll store our database.
const dataHome = process.env.XDG_DATA_HOME || path.join(process.env.HOME, '.local', 'share')
const dataDirectory = path.join(dataHome, 'streaming-html-counter')
const databaseFilePath = path.join(dataDirectory, 'db')

    /** JavaScript database (JSDB). */
const db = JSDB.open(databaseFilePath)
   
// Initialise count.
if (db.counter === undefined) db.counter = { count: 0 }

    /**
  A WebSocket server without its own http server
  (we use our own https server).
*/
const webSocketServer = new WebSocketServer({ noServer: true })
webSocketServer.on('connection', ws => {
  ws.on('error', console.error)
   
  ws.on('message', message => {
    const data = JSON.parse(message.toString('utf-8'))
   
    if (data.event === undefined) {
      console.warn('No event found in message, cannot route call.', message)
      return
    }
   
    switch (data.event) {
      case 'update':
        db.counter.count += data.value
        ws.send(Count())
      break
   
      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
})

    /**
  An HTTPS server instance that automatically
  handles TLS certificates.
*/
const httpsServer = https.createServer((request, response) => {
  const urlPath = parse(request.url).pathname
   
  switch (urlPath) {
    case '/':
      response.end(renderIndexPage())
    break
   
    default:
      response.statusCode = 404
      response.end(`Page not found: ${urlPath}`)
    break
  }
})
   
// Handle WebSocket upgrade requests.
httpsServer.on('upgrade', (request, socket, head) => {
  const urlPath = parse(request.url).pathname
   
  switch (urlPath) {
    case '/count.socket':
      webSocketServer.handleUpgrade(request, socket, head, ws => {
        webSocketServer.emit('connection', ws, request)
      })
    break
   
    default:
      console.warn('No WebSocket route exists at', urlPath)
      socket.destroy()
  }
})
   
// Start the server.
httpsServer.listen(443, () => {
  console.info(' 🎉 Server running at https://localhost.')
})

    // TO get syntax highlighting in editors that support it.
const html = String.raw
const css = String.raw

    /**
  Renders the index page HTML.
*/
function renderIndexPage() {
  return html`
    <!doctype html>
    <html lang='en'>
      <head>
        <title>Counter</title>
        <style>
          ${styles}
        </style>
      </head>
      <body>
        <h1>Counter</h1>
        ${Count()}
        <button onclick='update(-1)' aria-label='decrement'>-</button>
        <button onclick='update(1)' aria-label='increment'>+</button>
        <script>
          ${clientSideJS.render()}
        </script>
      </body>
    </html>
  `
}

    /** The Count fragment. */
function Count () {
  return html`
    <div
      id='counter'
      aria-live='assertive'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${db.counter.count}
    </div>
  `
}

    /**
  This is the client-side JavaScript we render into the page.
  It’s encapsulated in a function so we get syntax highlighting,
  etc. in our editors.
*/
function clientSideJS () {
  const socketUrl = `wss://${window.location.host}/count.socket`
  const ws = new WebSocket(socketUrl)
  ws.addEventListener('message', event => {
    const updatedElement = event.data
   
    // Get the ID of the new element.
    const template = document.createElement('template')
    template.innerHTML = updatedElement
    const idOfElementToUpdate = template.content.firstElementChild.id
   
    // Swap the element with the new version.
    const elementToUpdate = document.getElementById(idOfElementToUpdate)
    elementToUpdate.outerHTML = updatedElement
  })
   
  function update (value) {
    ws.send(JSON.stringify({value}))
  }
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

    /**
  Subset of relevant styles pulled out from Water.css.
 (https://watercss.kognise.dev/)
*/
const styles = css`
  :root
  {
    --background-body: #fff;
    --selection: #9e9e9e;
    --text-main: #363636;
    --text-bright: #000;
    --text-muted: #70777f;
    --links: #0076d1;
    --focus: #0096bfab;
    --form-text: #1d1d1d;
    --button-base: #d0cfcf;
    --button-hover: #9b9b9b;
    --animation-duration: 0.1s;
  }
   
  @media (prefers-color-scheme: dark) {
    :root {
      --background-body: #202b38;
      --selection: #1c76c5;
      --text-main: #dbdbdb;
      --text-bright: #fff;
      --focus: #0096bfab;
      --form-text: #fff;
      --button-base: #0c151c;
      --button-hover: #040a0f;
    }
  }
   
  ::selection {
    background-color: #9e9e9e;
    background-color: var(--selection);
    color: #000;
    color: var(--text-bright);
  }
   
  body {
    font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif;
    line-height: 1.4;
    text-rendering: optimizeLegibility;
    color: var(--text-main);
    background: var(--background-body);
    margin: 20px auto;
    padding: 0 10px;
    max-width: 800px;
  }
   
  h1 {
    font-size: 2.2em;
    font-weight: 600;
    margin-bottom: 12px;
    margin-top: 24px;
  }
   
  button {
    font-size: inherit;
    font-family: inherit;
    color: var(--form-text);
     background-color: var(--button-base);
    padding: 10px;
     padding-right: 30px;
     padding-left: 30px;
    margin-right: 6px;
    border: none;
    border-radius: 5px;
    outline: none;
    cursor: pointer;
    -webkit-appearance: none;
   
     transition: background-color var(--animation-duration) linear, border-color var(--animation-duration) linear, color var(--animation-duration) linear, box-shadow var(--animation-duration) linear, transform var(--animation-duration) ease;
  }
   
  button:focus {
    box-shadow: 0 0 0 2px var(--focus);
  }
   
  button:hover {
    background: var(--button-hover);
  }
   
  button:active {
    transform: translateY(2px);
  }
`

    So that is considerably longer (although almost half of it is, of course, CSS). And while we haven’t recreated Kitten with a generic file system-based router, etc., we have still designed the routing so new routes can easily be added to the project. Similarly, while our client-side DOM manipulation is very basic compared to everything htmx can do, it is still generic enough to replace any element it gets based on its ID.

    I hope this gives you a solid idea of how the Streaming HTML flow works, how it is implemented in Kitten, how it can be implemented using htmx, and even in plain JavaScript.

    Maybe this will even inspire you to port it to other frameworks and languages and use it in your own web development workflow.

    At the very least, I hope that this has been an interesting read and maybe gotten you to consider how Web development could be made simpler, more fun, and more accessible.

    If you’re organising a web conference or similar event and you’d like me to present a keynote on the Small Web (peer-to-peer web apps), Streaming HTML, and Kitten, give me a shout and let’s chat.

    Like this? Fund us!

    Small Technology Foundation is a tiny, independent not-for-profit.

    We exist in part thanks to patronage by people like you. If you share our vision and want to support our work, please become a patron or donate to us today and help us continue to exist.

  1. A component or fragment in Kitten is just a function that returns an HTML element.

    In Kitten, your components and fragments can take properties (or ‘props’) and return HTML using the special kitten.html JavaScript tagged template string.

    In fact, the only difference between a component and a fragment is their indented usage. If an element is intended to be used in multiple places on a page, it is called a component and, for example, does not contain an id. If, on the other hand, an element is meant to be used only once on the page, it is called a fragment and can contain an id.

    You can, of course, pass an id attribute – or any other standard HTML attribute – to any component when instantiating it. When creating your components, you just have to make sure you pass these standard props to your component. ↩︎

  2. You would use the request reference, if, for example, you wanted to access session data which would be available at request.session if your request parameter was named request. In our example, since we’re not using the first argument, we prefix our parameter with an underscore to silence warnings about unused arguments in our editor. ↩︎

  3. For example, see the Kitten Chat sample application for use of the all() method. ↩︎

  4. In fact, if we wanted to get really fancy, we could have bound the render() function to the clientSideJS() function so we could have referred to the latter from the former using this:

    function clientSideJS () {
  //…
}
    
clientSideJS.render = (
  function () {
    return this
      .toString()
      .split('\n')
      .slice(1, -1)
      .join('\n')
  }
).bind(clientSideJS)

    Notice that we cannot use a closure – also known as an arrow function expression in JavaScript – because the this reference in closures is derived from the lexical scope and cannot be changed at run-time. Lexical scope is just fancy computer science speak for ‘where it appears in the source code’. In this case, it means that since we’d be defining the closure at the top-level of the script, it would not have a this reference. ↩︎

  5. It basically says “in the range of the start of the file to the first comma, replace any commas you find…” – yes, that’s the first comma, I know, but it’s sed, things were different back then – “…with a comma followed by the line we want to insert.” ↩︎