Which of these appeals more: a new, free, Javascript framework for writing network-enabled 3D games, complete with integrated physics modelling and spatial audio… or, a complete, mature, dynamic programming platform that can implement the metaverse?
Why not both?
The new announcement from Croquet Corporation at the Game Developers Conference, GDC, in San Francisco is a very rare thing to come across the Reg FOSS desk's virtual – er – desk in multiple ways. Although it's far removed from our usual stuff about Linux distributions, virtualization, and so on, this is much more important and could impact many more people. We hope you'll bear with us for a few paragraphs while we try to explain why, because we think it's a genuinely big deal.
On one hand, this is trendy stuff – 3D gaming, immersive VR, the metaverse, and so on. It's a new Javascript framework for network-transparent multi-device 3D apps. It can now talk to Unity, which we've previously called "the most popular cross-platform game engine favored by video game developers". Although it's mainly in Javascript, it also supports Rust compiled to the cross-platform WASM runtime. It's FOSS, but the company has a revenue model. If it's not already apparent, Croquet's press release, and its website, are both highly buzzword-compliant.
Youtube Video
Croquet bills itself as "the OS for the Open Metaverse" and that is not an exaggeration: this could be one of most significant bits of software technology FOSS desk has seen so far this century. However, to get mainstream developers interested, the company is lightly disguising it, and marketing it as a tool for making it substantially easier to write cross-platform multiplayer games that work seamlessly over the internet, on desktop, laptop, tablet and smartphone. It is all that, but it's a lot more.
One thing Croquet isn't is a startup. Although this version of its eponymous product has been completely rewritten in vanilla Javascript, its first version was about 20 years ago, built on a transformative technology that was first released in 1985 and went open source in 1996. Despite the buzzwords, the tech is mature, with roots in designs that are older than Microsoft Windows. That also means that it's tiny, efficient, fast, lightweight, and in places, surprisingly simple. It's also got direct links to our story last week about the Xerox Alto and the three ways it influenced modern computers.
Croquet is a clever combination of some cutting-edge tech with some very powerful ideas that have been refined over decades. This is in part because Croquet was originally built by some exceptionally smart people, most of whom today act as its advisors. Among them are Alan Kay, who largely invented the graphical user interface; Dan Ingalls, who implemented the original Smalltalk and several important later derivatives, and on the side invented pop-up menus; David P Reed, who invented the UDP protocol; Mark P McCahill, who invented the URL and Gopher; and VR pioneers Julian Lombardi and Avi Bar-Zeev. We've written about Bar-Zeev before, but if you don't recognize the name, he built the original Google Earth (and much of Second Life), while Lombardi invented ViOS, probably the earliest attempt to commercialize the metaverse concept.
Croquet Corporation's founder and CTO is David A Smith, who wrote arguably the first ever first-person 3D shoot-'em-up The Colony, released in 1988 – four years before Wolfenstein 3D in 1992. He also did the 3D visualization tool for the movie The Abyss, which led to the first commercial 3D visualization app, Virtus Walkthrough. Alongside James Cameron, he's built projects with Tom Clancy, Michael Crichton, and worked for companies including Lockheed-Martin.
So it's really exceptionally safe to say that these folks really do know their stuff, because they invented it.
Croquet was originally implemented in Squeak. Derived from Apple Smalltalk-80 [PDF], released in 1985, Squeak is a Smalltalk whose VM is written in Smalltalk itself, by the same Dan Ingalls who implemented the Alto version at Xerox PARC.
There have been many implementations of Smalltalk, as well as later languages that it inspired. Notable among these is another Xerox PARC project, Self. Initially a dialect of Smalltalk, at Sun Microsystems this grew into its own thing: language rich, complete, and fast enough for a version of Smalltalk to be implemented in it. Self's just-in-time compiler turned into the Hotspot JIT in what is now Oracle's Java JVM. The most significant offspring of Self by far, though, is a version of it with curly-brackets syntax, whose creator described it as "a quickie love-child of C and Self. That is, of course, JavaScript.
We were surprised by the glowing terms in which Smith described it to The Reg:
JavaScript is an extraordinarily good language. I mean, it actually has three different languages it inherited its capabilities from. One was Self, which was itself built on top of Smalltalk. The second was obviously C, for the syntax, but the third was actually Lisp, for some of the things like closures and other capabilities. So, an extraordinarily powerful language. Most people don't understand that: they think it's a toy language, but in fact, it's probably one of the best languages out there. The other thing is, it's had billions of dollars of investment to make it very, very, very fast and very efficient, and it runs everywhere.
This is the point at which Croquet switching from Smalltalk to JavaScript suddenly made sense to us.
When this vulture first saw demos of Croquet in 2004–2005, it looked like a fun experiment in extending Smalltalk's two-dimensional, overlapping-window model into an 3D VR-style environment, using OpenGL to make it fast. This resulted in a translucent world, with windows that floated in mid-air. It looked to us a little like everything was underwater. It's intended for collaboration: each instance can talk to other instances over the network, synchronized via a lighweight protocol called TeaTime invented by the aformentioned Dr Reed. The Croquet environment could even embed other apps, such as a Mozilla-based web browser.
Right on the front page as it was in 2004, it gave the then relatively modest hardware requirements:
Our tests indicate that Croquet performs well even with Pentium II-450s and motherboards/components of that era as long as they have 256MB of RAM and a recent video card.
In the FAQ, they narrowed the latter down:
An nVidia GeForce 2 or better will work quite well and is reasonably inexpensive.
20 years ago, Croquet seemed quite large and complex. Now, specifications like these are trivial, and it runs on a far more heavily optimized, as well as ubiquitous, VM. Even the cheapest budget smartphone exceeds them by far, has hardware 3D rendering and fast, always-on communications.
Part of the problem, though, was persuading people to try it. Smalltalk is as much an operating system as it is a language, and most Smalltalk implementations run inside a self-contained VM, which is one of the things that has hindered its uptake. The main way to deploy a Smalltalk application is to ship an image of a VM with all of its components in place and ready to go. Smalltalk apps didn't look native on most OSes, and it's not trivial to keep Smalltalk code in a VCS such as Git, because Smalltalk apps are full of complex in-memory objects, not plain text.
But today, the most widespread programming language is a direct descendant of Smalltalk, which runs in a heavily optimized VM that's bundled with every browser on every computer and every fondleslab.
Smith told us:
So, it was pretty clear JavaScript would be a really great place to build. The browser would be the ideal place to build this next step, next-gen OS.
Of the current version, he said:
The system was greatly influenced by the earlier Smalltalk incarnation, but was a ground-up rewrite, based on many things we learned in earlier versions. Particularly important was the separation of the system into a model/view architecture.
The tooling is rich, mature, widespread and very well known. There are nearly 30 million developers in the world, and some 65 percent of them use JavaScript. It's a perfect fit. One of the company's demo videos shows VS Code editing a snippet of the code that controls the world the presenter is in, while the code is also visible in a live window in that world.
Youtube Video
This degree of integration was unimaginable in the old Smalltalk version. And now, this tool can talk to Unity, too.
All Croquet virtual environments are independent and can be protected, but they run in sync with all other instances of that world thanks to TeaTime messages sent to internet servers called "reflectors". Like the protocol, these are light enough to run on edge devices. Croquet Corp hosts its own, and charges for their use, so while the code is FOSS, there is a revenue model.
No matter how many devices they are running on, new instances can join at any point and sync within seconds. There is no separate client-side code talking to a complex server with state to be transmitted. Indeed, you don't need to write network-access code at all. As the documentation says (and the emphasis is in the original):
This synchronization is largely invisible to the developer. Creating a Croquet application does not require the developer to write any server-side or networking code.
For comparison, think of live migration of VMs from one host to another: a snapshot of its state is saved, and while the VM is loading into memory on the new host, the state is sent across from the old host, so that the VM can pick up where it left off within seconds. Source: The register