Introducing Synced Remote Images in Glamorous Toolkit

Until now, Glamorous Toolkit worked in a single image running on one processor. This was fine until we worked on figuring out what Moldable Development is, but now want to benefit from the multiple processors from our machines. That is why we are introducing support for parallel computations in Glamorous Toolkit.

This is achieved in remote images. Setting up a remote image is easy:

GtPharoLinkCodeSync default start

By default, a remote image is the same image as the current one only with a twist: it automatically synchronizes all code changes from the main image and it does not write anything on the disk. Of course, it's possible to work with other images that are not spawned from the current one, but this pattern is particularly interesting as we'll see in a minute.

In the meantime, take a look below. Starting the remote image shows up as a new process.

Use Cases

Ok, so what can we do with it? We think we can do all sorts of interesting things. Here are three.

Searching for code is one. For example, here we see a query that is run remotely:

Any query can be now run in the background simply by telling it to runRemotely and it will use the default remote image to do so.

#at:put: gtImplementors runRemotely

The result of the remote query is a promise, which is an object that knows the remote computation and that knows the resulting value. The inspector is smart enough to update itself when the value eventually arrives and shows an inspector inside the inspector. At least for now.

Another immediate use is that of running examples. For example, here we get an GtExampleGroup Object subclass: #GtExampleGroup instanceVariableNames: 'announcer ignoreAnnouncements' classVariableNames: '' package: 'GToolkit-Examples-Core' and ... guess what? We tell it to runRemotely and the result a promise that shows the results.

And the same can be done with tests.

About Syncing Changes

I mentioned an important aspect of the default remote image is that it syncs all code changes from the main image starting from the moment it was started. This allows us to keep the remote image around in an idle state and we can use it any time without paying the penalty of starting the image.

This was tricky to achieve. We wanted to not have to copy anything on disk to get a better performance. And because it's just not elegant to copy quaint files around. Starting the same image twice, is not a big deal. More challenging is to get it to not write to the disk.

Pharo is writing code changes into two places. On the one hand, we have the ancient mechanism of changes that are being written in a .changes file. On the other hand, we have the newer mechanism introduced by Epicea which can capture higher level changes, too.

Epicea was easy to adjust as it comes with the nice ability to writing to memory (using OmMemoryStore OmStore subclass: #OmMemoryStore instanceVariableNames: 'entries globalName' classVariableNames: '' package: 'Ombu-Stores' ), but for the other changes file we had to get sneaky. We settled on creating an in-memory file and getting Pharo write to it.

Initially we copied the changes file in memory to make the remote image be able to also read/write from/to it. That worked Ok with smaller changes files, but as soon as those changes files became larger, for example, when installing a larger code base on top of GT, the solution was slow. So, the final solution was to not copy the changes file at all, but to extend the stream to make it read from the changes file when it had to and to apply and read the new things from the memory file.

How do we serialize the changes? Those are just Epicea entities that the main image sends over the wire and replay in the remote image.

It works like a charm. Give it a try in the latest distribution of Glamorous Toolkit.