Vlad's Roam Garden

Powered by 🌱Roam Garden

Your friendly neighborhood botnet

casual distributed computing

Lately, I've been contemplating how to make the distributed computing more accessible, more of a thing that a bit tech-savvy person can use of vs something that you only resort to as a larger company. Making it more "casual" in other words.

The back-story of my thinking veering in this direction is that I've noticed that many ideas I’ve been thinking about recently had one thing in common. They were relying on public compute infrastructure


When I was thinking about allowing people annotate URLs with structured metadata - my primary candidate for backend was Wikidata

In recent thinking on Bringing knowledge and conversation closer together - I've been relying on existing Matrix chat infrastructure

Why do I reach for public compute in my projects?

Some of my projects need backend! I try to go client-side/local-first to minimize complexity when possible, but it not always is!

My projects often have a quality of public good, where it’s less clear that one can easily fund the separate infrastructure. Relying on existing public infra can allow me to bootstrap the project and validate if it's viable/valuable

There are of-course trade-offs. In case of public infrastructure you have

No guarantees of reliability of commercial cloud

No, or minimal ownership/decision-making power

When people think about providing the infrastructure for public good projects - distributed computing often comes up as a potential answer. Most of the personal computing devices out there are idle vast majority of the time and there is a constant temptation to tap into that pool.

But creating a distributed computing system that can produce useful results and not expose you to undue risk is hard.

There is a variety of technical difficulties you face:

orchestration of execution

allocating work and handling failures (nodes will drop out, go offline temporarily mid execution, etc)

ensuring equitable access to computational resources - avoid one client hogging all the resources in the network


you need to trust the compute executing your tasks (or have systems in place to mitigate the risk). The compute nodes

can see your data/code

can return fabricated results

as a compute node you need to ensure that running arbitrary code from the network won't damage your system or steal your data (e.g. by sandboxing the execution)

Heterogeneity of the computing environments

and more

People have successfully solved these issues before, but solutions I've seen have taken trade-offs that leave me unhappy

BOINC solves the trust problems by

having only a small set of vetted organizations admitted to the project

employing techniques like replicating the same job multiple times and taking the majority as the result to minimize probability of invalid results

In case of BOINC people are also not really incentivized to cheat as contributions of compute are voluntary and the only benefit you get out of completing jobs is points on the leaderboard

There is a company that would pool idling GPU capacity from datacenters and allow you to interact with it as one compute cluster

Here only a few big compute providers who are vetted can particitipate

And of-course the companies build internal compute clusters all the time, as they can avoid trust problems by being the owner of all components in the chain.

Scaling the problem down

I was thinking about this recently (partially prompted by being interested in running OpenAI Whisper locally)

And one approach that occurred to me that seems promising is - "What if we scale the problem of distributed computing down?"

What if instead of thinking about how to organize all idle computers in the world into one big network - we tried to solve problem on the level of group of friends.

Your friends are happy to help you (especially if it's easy to do)

which side-steps the problem of "what incentivizes people to join the network"

You can trust your friends

to not steal your data 💬

or send you fake results

You can poke your friends

you can rely on communication and social norms to enforce resource allocation being fair and e.g. one person not hogging all resources

One potential variation on this is running things on a variety of personal compute devices you have

e.g. easily delegating things from a phone to a more powerful machine

Make it simple to start or join the network

Another constraint that seemed important for making the use of distributed computing "casual" is simplicity

There are various solutions that would allow you to set up a compute cluster across a set of machines that you control


But they usually require technical competency and take a while to set up.

If you were to ask your friends to spend an hour configuring their computer to join your custom distributed computing network, you won't get many takers.

Heterogeneity of compute nodes involved is often an issue as-well

So when thinking about how can things be better - making the setup and joining of the network be really simple - stood out as a core requirement.

With these considerations & constraints in mind - the following architecture came to mind

Web Worker execution & Matrix coordination

The experience I'm envisioning is:

You go to a web-page, log in

And your computer is ready to execute or submit tasks

You submit tasks by sending a message to a Matrix chat room

With UI/commands available for common/popular tasks

No setup or technical sophistication required.

Main components this relies on are

Web Worker

As a way to allow you to execute almost arbitrary code in a sandboxed manner with decent performance

Technologies like WASM and WebGPU, which you can call out to from the web workers further expand what kind of computation it's possible and practical to perform in the browser.

Matrix Chat

As a way to relay messages between different compute nodes in the network

Task execution flow

The entry-point web page is a Matrix Chat client.

After you log in - it starts listening for new Matrix events.

A user can submit a task as a Matrix event, that has a Web Worker URL and references to required inputs.

files can be transferred via Matrix's media storage or IPFS

When a client sees such a message - it will pick it up, send an ack message back to the channel and start executing the requested task

(to simplify deciding which node should take the task - one of the nodes in the network can randomly picked to be "coordinator" and would be responsible for allocation decisions)

When the result is ready - the node will post the result back into the channel, or will send it directly to the requester.

Why Matrix as a communication protocol?

The choice of matrix is not essential - any other messaging protocol would suffice, but Matrix is federated, reasonably easy to build on, and I have experience with it.

As with my other Matrix projects this is partially motivated by hope that common messaging layer as a way to solve interoperability

Example tasks

Though there are challenges with running this due to memory limits of WASM https://github.com/ggerganov/llama.cpp/issues/97

I'm thinking of building a prototype of this, initially focusing on transcribing voice messages via Whisper. Please reach out if you'd be interested in collaborating on this, have related ideas, suggestions or interesting applications.

Homomorphic encryption made a lot of progress in recent years, but still remains a lot slower then operations over unencrypted data


you still should be weary, as you can’t ensure the other people’s compute infrastructure is secure from intrusion. So you shouldn’t do anything super sensitive