Glamorous Toolkit is now beta. This is the first usable incarnation of our vision of moldable development. We still have plenty of things we want to do, but now we can say that the large technical challenges are behind us.

Many things have changed since our previous release from a year ago. The GitHub repository says we addressed more than 900 issues spanning from deep technical issues to creating a whole new visible user interface. Essentially, much of the current system has been developed or redeveloped during last year.

Let's take a tour through the highlights.

Before we look at the cool technical details, let's pause for a second and consider our goal of making the inside of software systems explainable. To reach this goal, first and foremost, we focus on reimagining how developers read code. Glamorous Toolkit does bring innovations in how we write code, but it is the reading where we make the most difference. Developers alone spend 50+% of their time reading code. That is the single largest activity in software development and we reimagine how this effort is being spent from the ground up. The result is what we call moldable development and we are now confident in saying that it is a novel way of approaching programming.

To get a feeling of what it means, please scroll down and look briefly at the screenshots. Notice how they look rather distinct from one another. Software is highly contextual. We want the environment to mold to that context and depict it distinctly, too.

Ok, enough of that. Let's go on with the show.

Native windows

A good chunk of the effort went into making Glamorous Toolkit run in native windows reliably. We are now happy to announce that it now works for Mac, Windows and Linux.

This is an important step not only for the looks of it, but also internally. If until now we were bootstrapping inside the graphical support from Pharo, now we get to control the complete rendering. We are still able to spawn the old Morphic World interface on demand. In this way, people that are used to Pharo can enjoy the interface of the new environment while still being able to rely on old tools when needed.

Minimal frame

To be able a high degree of moldability, the environment's frame must impose little constraints.

In our case, the frame essentially knows a home tab that we can easily mold, a menu that we can make moldable, and a way to search that is already moldable. This is it. No elaborate global menus. No hardcoded views. We can do this because the actions are bundled with the context in which they apply.

Coder

Coder offers the means through which we manipulate static code. The work on Coder started from the need to mold the experience for the smallest unit of code organization: the method. We started from the editor for a single method and built the rest of the experience around it.

For example, below we see a search through the system revealing methods that can be individually viewed and modified in place. In this particular case, we expand one inner method only to look for references of an instance variable, and this, in turn, opens another list on the right. Using this simple mechanism we can transform any Playground into a powerful code query tool and a scoped code browser.

Coder can be molded in several ways. For example, below we see the baseline defining the dependencies of the Bloc project. Each dependency is defined through a string denoting the name of the dependency. Coder recognizes the API and allows us to expand the string right in place to browse the corresponding dependency.

The environment is classically seen as being specific to a language. We believe moldability must be much finer grained: all tools must be made adjustable within the context of code. In our case, we bring that ability to the individual instruction.

Playground and Inspector

Playground and Inspector are the core tools that enable live exploration. They were significantly consolidated. The core mechanism remained the same, but now they are complemented with a few abilities.

First, error handling. Inspector is extensible by third party code. At the same time, as it is the most basic tool we have in the environment, it is critical that these extensions cannot break the inspector. So, now we have a fine grained handlers that gracefully show errors without breaking the Inspector.

Another part is preserving the history in Playground. On each code execution we store the Playground's content on disk. This also works for code we are writing in the context of an object.

But, perhaps most important is that we brought the full coding experience into both Inspector and Playground. When inspecting an object, the Meta view offers a Coder on the class of the object. Particularly interesting is that the value of self is bound to the inspected object. This ability allows us to promote programming in Inspector in the presence of live objects. More recently, we went a step further and extended the Playground coding experience as well. This includes the ability to create classes and methods right in place.

A consequence is that we can now use the expansion mechanisms to peek into code wherever we are, including the Playground and Inspector. Through this, we do not go to where the code is. The code shows up where we are. That is the humane way.

Spotter

Spotter is the moldable search interface.

Just like how an object can tell the Inspector how it can be viewed, it can also tell Spotter how it can be searched. This simple ability can then be combined in many different ways to support various use cases from navigating through code or the file system, to pasting external urls to open them within the environment.

While Spotter is prominently available globally, we also integrated it with Inspector and Coder. For example, when editing a class, Coder offers a Spotter to search within the context of that class.

Debugger

Debugger saw significant improvements, too. The stack is made of the same snippets as found in Coder or in Playground. And it can be seamlessly integrated in the inspection flow.

Documenter

Documenter is the engine with which we assemble narratives. We use it for documenting classes, or we use it for describing business cases or architectural descriptions. For example, here we see a Coder with a document as a class comment. Clicking on a link opens up another document, essentially transforming the whole image into a wiki.

Documenter saw several developments, and it became faster and more usable. We achieved this by making the previews render lazily.

Pager flow

The different tools are wired together in a broader flow we call Pager that embeds tools as pages connected in a Miller columns interface. The novelty is that tools are now first class and can appear at arbitrary locations in a larger flow.

Presenter

Beside documents, another pervasive medium to express narratives are slideshows. Presenter offers exactly that, only in our case, the slides are live. Each slide is essentially a scoped environment and can be used for various purposes. For example, the Tour slideshow offers an overview of Glamorous Toolkit, each slide exemplifying a separate concept or tool.

Explainer

A narrative can take multiple forms. In the environment we already have documents or slideshows. And now we have a hybrid: A way to combine text with a target graphical scene and offer seamless links between the two.

For example, below we see an explanation text on the left. On the right, we see a piece of code denoting the definition of an inspector view and a preview of the rendering for a specific object. Hovering over the highlighted words visually link to the corresponding parts in the code and in the scene.

Visualizer

Visualizer makes visualization a first-class citizen in the environment. It contains a few programmable engines including Mondrian, for drawing graphs, an initial Plotter for drawing charts, or Connector, for identifying connections between various objects.

Transcript

Transcript is a basic console. Now, text is quaint, but we have to outgrow it. And, you should not use Transcript for debugging purposes either. But if you must, at least you can do it with style. For example, if you have an exception that you want to print, the whole exception stack is inspectable right from the Transcript. Why? Because we can, and because doing it any other way would be not glamorous enough.

GT4SmaCC: extensive parsing environment

The first non-Pharo language fully supported by Glamorous Toolkit is the one of SmaCC with which we define parsers for other langauges. For example, below we see the definition of the C# parser presented similarly to how we present Pharo code. Beside the editing abilities, the integration also comes with various debugging and inspection abilities, such as a simulator. Through these tools we make the construction and evolution of parsers innexpensive.

GT4PetitParser2: casual top-down parsers

PetitParser2 is another parsing framework. A top-down one. This is a different parsing technology than SmaCC and it is particularly interesting for more casual parsers for discovering structure in what appears less structured text (or data). It's like Regexp, but really cool. As we believe that developers should be able to tackle any data that comes their way, we built an integration for this engine, too.

GT4Famix3: multilanguage analysis platform

From the beginning, Glamorous Toolkit included parsing engines as seen above. Now, we also added support for FAMIX, the core meta-model from the Moose project for representing systems written in a variety of languages. Integrating this modeling ability in Glamorous Toolkit led to an extensive code analysis platform that can be used for steering agile architecture of projects written in various languages (or combinations of languages), such as JavaScript, Java, C#, C++, Swift or Delphi.

GT4Jenkins: integrating the continuous integration

We rely on Jenkins for our development. So, we integrated a client for it in the environment. This integration also serves a documentation purpose as it shows how we can create API browsers by customizing Inspector and Playground. Please take a look at the screenshot below. Now, please note how the tool does not require Pharo knowledge to be used.

Releaser: releasing deeply nested projects

Glamorous Toolkit is made out of many deeply nested subprojects. At the same time, we want that every commit anywhere in the dependency tree to lead to a reproducible release. That is the job of Releaser, our tool for dealing with the multiple facets of releasing deeply nested projects.

XDoc: executable documents

XDoc stands for eXecutable Document and it is a generic container for various types of documents. Essentially, it is content and a bit of metadata through which a player can decide how to present the content.

Currently, we support two types of contents:

1. The contents of a Playground are serialized.

2. Documenter documents like this one.

XDoc provides a simple bridge between the environment and the external world. A direct use case is that of sharing narratives created within the environment to the web. For example, this very document was created in Glamorous Toolkit and can be viewed on blog.feenk.com. Moreover, it can later be reloaded back into Glamorous Toolkit.

More about the graphical stack ... and other geeky details

As if the things above were not geeky enough, there are still a few things that are worth mentioning. We spend a ton of the effort on getting the low level plumbing into place. It was a rather large puzzle. The most important part is that the graphical stack is now standalone.

Pharo 8, headless VM and threaded FFI

While Glamorous Toolkit is implemented in Pharo, it comes with a complete and distinct graphical stack. To accommodate it, we needed to overhaul quite a bit of the default infrastructure. In particular, the challenge was to get native windows to work the way we wanted to. We moved to use the headless virtual machine (VM) for Pharo 8, and in the process, we also got ourselves a branded VM. The headless VM does not assume a specific rendering. Instead, it allows the image to choose it. This is a prerequisite for rendering to a native window and it works quite nicely.

In the classic Smalltalk setup, the VM runs in the UI thread of the operating system. In our setup, which is aligned with the preferred modern desktop app architecture, we let the operating system own the main thread and run the image in a callback. This also means that any other call from the image through the foreign-function interface (FFI) is nested. Code can be loaded reliably from Git repos through either Metacello scripts or through Iceberg. For this, Iceberg had to get compatible with threaded FFI. We worked with the Pharo team to make it happen, and we'd like to thank them for all the support.

Skia and Glutin

In the process of rearchitecting the infrastructure, we moved the graphical stack towards two new libraries: Skia as a backend for rendering, and Glutin for providing an OpenGL context and windowing support. We moved to these because they provide a strong cross-platform support. In particular, the move to Skia was particularly relevant because it is more nimble than Moz2D, the other library we used originally, and it is much simpler to deal with from a technical perspective.

Compositing rendering layers

On top of relying on Skia and Glutin, rendering is hardware accelerated based on compositing render layers.

In the picture below we see an example of how it works. On the left we see a test scene with multiple elements and effects, such as shadow. On the right, we see the scene decomposed into render layers, each of which can be rendered separately on different CPUs and GPUs.

UI Scripter

A graphical stack must also be testable. This is supported at several layers, but the most interesting is the end user interface layer. UI Scripter a small engine with which we can simulate and control a scene, including asynchronous interactions. The result of the scenario steps is depicted visually.

Monitor

And to maintain a better overview over the running system we consolidated multiple views in a tool we call Monitor. This can deal with processes and their stacks, higher level TaskIt services and their associated queues, low level events and a few other interesting things.

Also worth mentioning is the tool for profiling the performance of rendering. In particular, the profiler allows us to evaluate each individual frame.

All in all, it's a nice little utility that brings us joy, and it shows how a framework can ship with the tools necessary for helping one reason about its internal technical aspects.

No modal dialogs

As a matter of principle, we do not rely on modal dialogs. Not even one. Instead, we created a couple of interactions that allow us to embed interactions in context. For example, a pattern is to emply a contexutal dropdowns that can go away on a click.

Ok. That's a short summary. Now, please go ahead and download it. And let us know what you feel and think about it.

Have fun,

The feenk team