tl;dr — you can now find remote categories and see your tracked/watched categories in /world.

A new alpha version of NodeBB was tagged today: v4.3.0-alpha.3. The biggest change is to the /world route, which up until now showed a list of topics from outside of the local NodeBB instance.

New to this alpha release:

1. A quick search widget was added, allowing you to directly search for remote categories. There is no need to navigate to to the search page to discover new categories.
2. Your list of tracked and watched categories will show at the top of the page.
   • "Tracking" and "Watching" categories—both local and remote—is how content discovery happens in NodeBB. Tracked categories will have new content show up in the "unread" page, while watched categories take that a step further and notify you when new content is posted.
   • Tracking and watching a category will tell NodeBB to subscribe to that remote community for updates

At this time we're continuing to look for stability issues with the remote category integration. We'll be working on QoL fixes as we move into the beta phase this/next week.

Happy Tuesday!

Today we've updated the NodeBB community forum onto the remote-categories testing branch, which means that users on the open social web that identify themselves as "Groups" will be rendered in NodeBB as categories. Prior to this, they looked like users.

Here are some examples of remote categories:

• Comic Strips (on lemmy.ml)
• Star Trek Social Club (on startrek.website)
• Social Web Foundation (a WordPress blog)

ActivityPub "groups" and forum categories have quite a few things in common — they don't usually post topic themselves, they "contain" topics, and they are usually administered by a separate group of users (moderators!) In many ways, these groups lend themselves to categories much more easily than they do as users.

Notes:

• We will likely be releasing this as v4.3.0-alpha this Wednesday. Probably this means you don't want this on a live forum just yet.
• A lot of the backend logic is complete, but a lot of the frontend UX will be worked on.
• You can "search" for categories (via "in categories" in the search page), paste the full handle in order to instruct NodeBB to pull a new category in.
• You can now no longer mention a remote category. Instead, create your topic right in that category itself. As it should be .
• Remote content coming in that is slotted into a remote category will still show up in your "world" feed. That is still intended to be where discovery of content outside the local NodeBB instance will take place.
• Report any bugs or confusing behaviours (and there will be some) here.

Screenshots

For awhile now we've hosted https://try.nodebb.org as our demo instance for people to see a plain NodeBB instance.

I have added a federated category on that instance now. You can of course still make test posts to this instance, but you can also feel free to fill @federated@try.nodebb.org with garbage instead

Please note that the "try" instance of NodeBB is reset every 24 hours, which means all follow relationships, content, etc. will only last at most that long. If you follow the category or an account on that instance, those categories/users will "forget" you followed them.

I've just merged the activitypub branch of NodeBB into develop, meaning that v4 is on-track to be the next release.

It also means that v3.x has entered feature freeze, and that only bug fixes will be merged/committed at this time.

Exciting stuff!

ActivityPub introduced to NodeBB the concept of sharing/announcing/boosting, and we enthusiastically supported this as a method of content discovery.

While we were processing these activities (and publishing our own), we weren't actively tracking them internally. That has changed in the latest commits (soon to be RC5); shared content from both local and remote users are tracked internally and can be viewed in the user profile.

For example, here is c/comicstrips from lemmy.world. As it is a Lemmy community, it only shares content, and does not produce any of its own.

Here is a quick recap of how sharing is handled in NodeBB:

• When remote users boost/repost content, NodeBB interprets this as a share.
• When local NodeBB admins move topics out of "uncategorized" and into a category, it is interpreted as a share and this activity is federated out to the admin's followers.

At this time unprivileged users cannot share content.

I'm happy to announce RC1 (that's Release Candidate 1!!) of NodeBB v4. Since the last v4 pre-release (beta 4), we've made additional bug fixes and stability improvements.

Here is the list of new features and fixes since the first beta:

• Emoji!
• Ability to search for a user by their handle directly from the search bar
• When encountering links to websites that advertise an ActivityPub alternate, NodeBB will try to load that site in NodeBB first
• Categories now have their own avatars instead of serving the generic site brand icon
• NodeBB will attempt to backfill an entire topic when encountering a new object, reducing the need to visit the original site to get the entire conversation. (This is contingent on the other server supporting resolvable context collections)
• Chats are now federating out as limited-visibility posts on the fediverse
• Improvements to the title generator
• Custom Fields now two-way federating (custom fields are a feature new to the 3.x branch as well!)
• Better compatibility with nodebb-plugin-feed

Timeline-wise this brings us close to the end of our funding from @nlnet@social.nlnet.nl for the 2024 calendar year. We've exceeded our own expectations of what we could do this year, and we've already applied for funding from NLNet again to really let NodeBB push the boundaries of what the ActivityPub protocol can do!

The RC signifies that the software has reached a (self-proclaimed) stable state, and broader usage is encouraged. If you have any questions, concerns, or trouble upgrading your existing NodeBB to v4, please call that out here for support.

There is now a soft feature freeze in place while we sort out the last remaining (hopefully none ) bugs, and we'll aim to launch v4 after that.

Additional pages have been added to the NodeBB documentation regarding the following items:

1. Post visibility — how NodeBB handles incoming activities with public and non-public visibility, and how they are represented in NodeBB.
2. Conversational Contexts — how NodeBB builds out a conversational context when parsing incoming content.

tl;dr — you can now send me DMs. I blatently and unforgivingly abused the NodeBB chat system to make this work.

Ever since I started this project at the start of 2024, I knew that posts with limited visibility were going to be a sore spot. ActivityPub has the concept of "addressing", with the following valid entries:

• An actor (as:Person or similar) uri
• A followers collection (?!?!?!)
• The public collection

However, posts and topics in NodeBB have their privileges and access scoped to the category, which meant that while it is able to restrict visibility/posting/etc. to specific users, the system was not flexible enough to handle individual posts with different user visibilities.

Given that limitation, up until this week, if your object did not contain the public collection https://www.w3.org/ns/activitystreams#Public, it was automatically and unceremoniously dropped because there was no way NodeBB could display it to the targeted user without leaking it to other users.

I had gone through a couple iterations to try to figure out a way to make this work, but none stuck (see the follow-up technical post for more details), and I had just about relegated it to the "think on this for awhile" pile until a recent post by @scott@authorship.studio got me thinking outside of the box:

and your forums, direct messages, and inbox can have different feature sets.

Emphasis mine.

Unlike topics and posts, NodeBB's chat system is not constrained by the privilege system, and each chat room has its own collection of members, which in many ways made it a better fit!

I put together a proof-of-concept in a couple days, and we're test driving it now. So my apologies if in the past year you tried to DM me, and I didn't respond. I wasn't ignoring you, NodeBB just didn't know how to handle it, promise!

tl;dr — category actors in NodeBB now have better avatars/pictures! They're generated from the icon, color, and bgColor settings for each category.

Did you know that categories in NodeBB can also be followed on the social web/fediverse? Categories, as they are known here, are also actors on the fediverse, so you can find them just like any other account.

e.g. @general-discussion @support @nodebb-development @activitypub

@liaizon@social.wake.st opened an issue back in April (!!) pointing out that the avatar/picture served by NodeBB for category actors was not as good as it could be.

It's definitely possible to do something like this, although it is usually rather resource intensive. For example, when you paste a GitHub URL somewhere (e.g. Slack), it may try to unfurl the URL and display an image. GitHub blogged about how they did this using Puppeteer to generate the images.

I knew right away I didn't want to bundle what was basically an entire browser runtime into NodeBB's dependencies, so I needed a more lightweight solution.

I attempted to build my own SVG string and send that through as a data-uri, but not surprisingly, it was not handled by the implementations I tested.

Some more research this week led me to Vercel's OG Image Generation, which allows you to create images on the fly using Vercel Edge Functions. Since NodeBB is neither a Vercel app, nor do any people running NodeBB use Vercel, that ruled out this product.

However, more research into Vercel's implementation led me to Satori and resvg, a pair of programs that can generate an SVG from HTML, and a PNG from SVG.

From there, it was a fairly standard technical exercise to combine these with our font-awesome libraries in order to output our own implementation of this to generate colourful category avatars for its category actors.

For now, they are only generated on-demand when a category actor is requested, but if there are applications for the icons to be used elsewhere, then we could look into having them generated on direct file request.

Enjoy!

Last friday I quietly tagged a commit on the activitypub branch with v4.0.0-beta.1, which signals that the ActivityPub integration is now ready for beta testing.

For the most complete (yet readable) list of new functionality from the alpha, check out the "Road to Beta" project page.

tl;dr — some new features and a lot of fixes

• Editing a category now issues an Update(Actor)
• replies is now populated and responds with an OrderedCollection of direct replies (see the post about that)
• Proper sharedInbox support
• Better indicators for content that comes from non-local (aka "remote") users
• Conversational context synchronization mechanic (corresponding post for that)
• Moving a topic out of the "uncategorized" category will now federate out an Announce

By and large most incompatibilties have been resolved, although if you do find some issues, please do let me know in the corresponding bug report thread.

I'm looking to wrap up the year with some of the more difficult projects I've put off

• Object Integrity Proofs, which will also enable Inbox Forwarding
• Better handling of "Open in App" signals from third-party instances
• Post visibility support (or at least better handling so non-public messages aren'y unceremoniously dropped!)
• Ongoing integration with FEPs 400e and 7888

Now that the alpha version of NodeBB v4 has been released, it means that people are starting to look into using NodeBB for testing (or if they're really looking to have fun, production.)

The first step in enabling those intrepid individuals is providing documentation!

While a little bare-bones at the moment, I've written up a brief summary of what NodeBB brings to the ActivityPub landscape, as well as a run-down of various settings/toggles/switches, and how to switch over to the code branch that enables ActivityPub integration.

What I don't know is what's missing. There's a lot about NodeBB's integration that's very obvious to me, since I built it, but what don't you see that you'd like me to write about? Let me know.

Thanks!

This week I sorted out a feature that I've been putting off for far too long... content pruning!

ActivityPub had been enabled since March of this year, and in that time, the database had accumulated tens of thousands¹ of pieces of content from the fediverse. This was causing our database size to grow, but it was nothing entirely insurmountable, since it's mostly text.

However, the principle of the matter was that we were being pushed a mountain of content, most of which wasn't even consumed. For reference, when I ran the script against the data on this site:

2024-06-10T18:04:52.445Z [4567,4568/3631171] - info: [notes/prune] Found 32531 topics older than 30 days (since last activity).

We then take those topics and determine whether it received "engagement". Essentially whether a user reply to or liked a post within a topic.

Filtering for those, we get:

2024-06-10T18:07:31.860Z [4567,4568/3631171] - info: [notes/prune] 32252 topics eligible for pruning

... which essentially means ~99.14% of incoming content received zero engagement. That's not entirely surprising given that content streams in at all hours of the day, and I only reply or like a fraction of them.

In the backend, I'm allowing this value to be configurable, with the default being 30 days.

Next up, remote user pruning!

¹ Actually, the number ended up being closer to 116k pieces of content. In contrast, this forum has been running for about a decade and only recorded just under 100k posts!

Finally, you are now able to look up remote content and user profiles using the built-in NodeBB search tooling.

In the quick search bar and on the search page itself, you can paste in a URL to a post. If NodeBB can fetch it using the ActivityPub protocol, then it will be immediately parsed and returned as a search result:

If you change the search type to "In users", or use the search bar in the users page, then you can look up remote users using their URL or handle:

This change resolves the final hurdle stopping a brand new NodeBB from connecting to the fediverse. It wasn't possible to actually find anyone or anything in order to start those first follow relationships. Now it is possible.

Aside — I'm frankly surprised by how long it's taken for me to actually do this. It goes to show you just how much you'll put off doing something if it's not really critical.

@oplik0 and I are having a discussion regarding the conditions upon which origin checking should be done on S2S activity receipt.

The spec says this:

Servers should not trust client submitted content, and federated servers also should not trust content received from a server other than the content's origin without some form of verification.

Reasonable, if a little open-ended. I took this to mean that if an Activity(Object) is received, if Object is a copy of the resource itself, that origin checking is required. This is reflected currently in the NodeBB code where if typeof object !== 'string' then origin check is carried out.

Problem is, that's a code smell. It's not readily apparent why an origin check is only carried out under that condition. Compounding this is that not all activities need an origin check: Create, Update, and Delete typically don't (careful in my wording here, lest I say something way off-base).

@oplik0 advocates for a more explicit approach, where perhaps an object uri is dereferenced regardless, and an origin check is carried out if the specific Activity-Object type pair calls for it (and if the uri doesn't match).

P.S. Yes, @hrefna@hachyderm.io I know object can also be an Array... or undefined... or null... don't @ me!)

The latest change to the ActivityPub integration for NodeBB introduces the concept of "group actors" to categories.

This change means that categories can be configured to be visible to other applications connected to the social web (including Mastodon, Misskey, and also other NodeBBs!)

When followed, it means that any post made to the category will be automatically shared to anyone who follows that category.

NodeBB already supported this locally. You could set your watch state for a particular category to "tracking" or "watching", the latter allowing you to be notified of new posts.

Also of note is that remote replies to topics contained within a category will also be shared to the category's followers. This distinction means that you can effectively follow a specific category or sub-category in order to stay up-to-date on all discussions within that category.

As of today, users cannot create topics in a category from the social web (e.g. I cannot create a new topic in a NodeBB category), yet. That will come soon.

For the NodeBB community forum, the following categories are followable for now:

1. General Discussion
2. NodeBB Development
3. NodeBB Development > ActivityPub

Lastly, all of this follows the FEP-1b12 Group Federation enhancement proposal.

Building out NodeBB's fediverse implementation has been a huge eye-opener for me. Immersing myself in the dev community has only made me realize how much more there is to do and thoughtfully consider.

To that end, we've started consulting with @andypiper@macaw.social in order to ensure that our build-out follows fedi best practices and to ease any potential speed bumps that we might run into. Already we've shuffled our priorities somewhat to ensure that important topics like content moderation (and deletion!) and instance filtering are prioritised.

I also want to ensure that the onboarding experience is user-friendly, for admins and end users alike — a common refrain about ActivityPub software is the learning curve. Hopefully we can tackle this issue and share our learned lessons as we go along. What does that look like? Let us know!

Andy's DevRel talents and experience make him especially suited for this type of consulting. Sometimes you don't know what you don't know!

Spent most of the workday today adding in hostname checks to the NodeBB-ActivityPub integration in order to improve security. There's much to reflect on with the recent vulnerability disclosures, and many lessons to learn.

It absolutely sucks that Mastodon and Pixelfed contained these vulnerabilities, but their public disclosure allows the rest of the fediverse to learn from their mistakes and publish better software.

My #nodebbActivityPub implementation now handles incoming boosts, but I haven't gotten around to pruning stale content that wasn't interacted with... and I also followed @lisamelton, so I guess it's a race against time to either implement content pruning, or my hard drive fills up with her boosts.