Glamorous Toolkit (GT) is built in Pharo, and it enables people to work with Pharo systems. Nevertheless, the goal of GT is distinct from that of Pharo and so is its realization.

Glamorous Toolkit is built by feenk. Our goal is to provide a complete and novel development experience to allow people to make the interior of systems explainable. All our work is free and open-source under the MIT license. We contribute two things towards our goal. First, we provide the actual environment. Second, and perhaps more importantly, through GT we enable moldable development, the approach we have authored.

Moldable development puts forward the hypothesis that the shape of software is essential and that it has to be customized to match the context of the developer. By now, we have a decade worth of experience and experiments in various scenarios. For example, we use it to assess and steer the architecture of legacy systems, or to explain the business logic to non-technical people.

Moldable development was embodied in the first generation of GT which was integrated into Pharo in 2014, and it showed how we can indeed provide an experience that is distinct from any other, including the classic Smalltalk one. We showed how by actively constructing tools while we program, we enable a new feedback loop: We can see and communicate the inside of software systems.

The new GT takes moldable development much further. Especially visible are new kinds of tools, like Documenter with its live notebooks, slide shows with interactive explanations, Playground with snippets, or Coder BaselineOf subclass: #BaselineOfGToolkitCoder instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkitCoder' with expandable editors.

Still, there are less visible parts. Underneath, we have a generic infrastructure for handling other languages. It comes with an integration of Famix BaselineOf subclass: #BaselineOfGToolkit4Famix3 instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkit4Famix3' , the meta-model through which we can represent various systems, and it also ships with an environment for SmaCC BaselineOf subclass: #BaselineOfGToolkit4SmaCC instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkit4SmaCC' , the parsing engine, that allows us to create parsers for new languages. For example, more recently we added parsers and importers for several languages including CPP SmaCCGLRParser subclass: #CPPParser instanceVariableNames: '' classVariableNames: '' package: 'SmaCC_CPP_Parser-C++ Parser' and JavaScript SmaCCGLRParser subclass: #JSParser instanceVariableNames: '' classVariableNames: '' package: 'SmaCC_Javascript_Parser' .

Then there are even more technical engines. Releaser BaselineOf subclass: #BaselineOfGToolkitReleaser instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkitReleaser' makes it possible to version deeply nested projects and repositories completely automatically. Visualizer BaselineOf subclass: #BaselineOfGToolkitVisualizer instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkitVisualizer' adds significant visualization abilities to the overall environment.

You can think of GT as a set of tools, but that would miss the point. GT is a whole environment that enables a new way of thinking about programming. We see the environment as a language, one that extends the base language with visual and interactive operators. Through this language the various components can be combined in many ways. For instance, examples BaselineOf subclass: #BaselineOfGToolkitExamples instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkitExamples' offer a different technical way to capture tests, but when combined with Documenter they can change how we document our systems and even how technical people communicate with non-technical people.

GT relies on the vm, language, and basic libraries of Pharo, but it comes with a completely separate graphical stack (Bloc) and engines for tools, such as syntax highlighting and completion. This stack is particularly relevant because it renders every scene through a single rendering tree Object subclass: #GtWhatMakesBlocSpecialExplanation instanceVariableNames: '' classVariableNames: '' package: 'GToolkit-Demo-WhatMakesBlocSpecial' . On the one hand, visualizations become first class citizens at a very low cost. On the other hand, this allows us to create new kind of interfaces.

A code size analysis

Currently, GT code is loaded on top of a full Pharo distribution. Pharo ships with 572131 lines of code. GT adds 827979. This is a significant amount of code. Let's take a closer look at it.

GT comes with a whole new graphical stack, and tools that completely replace the default tools of Pharo. Specifically, there are 32 projects from Pharo totalling 224602 lines of code that are not needed in GT. Still, we ship the complete Pharo code because we are in a transition period, and for the time being we want people to be able to easily access the default Pharo tools.

Now, if we take a closer look at the code from GT, there are a number of features that are not present in Pharo. For example, GT includes 23 parsers for other languages. These alone total 151613 lines of code. It includes 40795 lines of code to support code analysis of systems written in other languages. On top of that, we complement the already existing JSON support from Pharo with support for XML which adds another 69599 lines of code. We do that because we believe that a development environment should offer convenient support for various data formats.

We ship the extra libraries to support other languages and technologies out of the box. But, the size of the core of GT plus the size of the Pharo code we do rely on is about the same size as the size (even a little smaller) of the standard Pharo distribution. Let's explore a few key novel abilities made possible by GT.

New interactions

Consider this screenshot showing an integration for the Jenkins continuous integration server directly in the environment:

We have a Playground snippet on the left that is specific to Jenkins and that offers a form. The result of logging in is a set of objects that mirror the Jenkins API. The integration offers multiple inspector views, including the pipeline map.

While this tool is made out of a Playground and a few Inspectors, manipulating it requires no Pharo-specific knowledge. This shows how the environment can be molded not only for technical purposes while programming Pharo, but also to address new audiences.

Composable tools

We regard the environment as a language made of visual and interactive operators that can be combined in many ways. For example, in the picture below we see a Spotter BaselineOf subclass: #BaselineOfGToolkitSpotter instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkitSpotter' search opened in the context of an inspector that shows the preview as another inspector. The interface is shown in a dropdown. There are many such instances throughout the system.

Moldable coding experience

In general, the user interface follows an object-oriented decomposition that relies on localized, rather than global, actions. The coding experince follows the same pattern. The smallest piece in Coder is a snippet, either for a method or a script. The snippet is self contained and offers mechanisms to adjust the interface to the context of the code.

For example, below we see the baseline class defining the dependencies of GT. Each dependency is defined through a string denoting the name of the dependency. Coder recognizes the API and allows us to expand the string right in place to browse the corresponding dependency. This is achieved through a syntax highlighter that is only active in the baseline method and that adds the expansion adornments.

Searching for code patterns is also enhanced by snippets. For example, the picture below shows a Playground snippet holding a code query based on a fluent API that is part of GT as well. Executing the query offers the result browsable through method snippets. In our case, the expanded method is an example, annotated through <gtExample> and the snippet offers extra actions. Executing and inspecting the example shows the result on the right (a visualization in this case).

Programming in the inspector

Smalltalk is famous for the ability to program in the Debugger, in the presence of live objects. With GT, we are perfecting programming in the Inspector.

Consider the case below. In the first pane, we have a picture that detcted a few faces. We query the first detected face from the object Playground and get the result in the second pane. Here we switch to the Meta view and we get a full class Coder where self points to the inspected object. This then allows us to program right in place with full abilities. This is possible because we designed Coder to be flexible enough to fit in that pane.

Of course, code can still be modified from the debugger, but even in that situation, the inspector can augment the experience with detailed views.

Examples and live narratives

Technically, examples are like tests that return an object. This simple change enables us to compose examples out of other examples simply through message sends. That technical view is worthwhile, but the real power of examples reveals itself when we combine them with inspector views and live documents.

For example, here we see a document on the left that embeds a treemap visualization. On the right, we see another document showing an explanation (in this case, an explanation of the treemap algorithm). The pictures are drawn live and are embedded in the document through an annotation as seen the right. This annotation points to an example method and specifies which inspector view should be rendered in the document for the object returned by the example method.

These documents can play many roles. They can describe the business domain. They can explain architectural constraints. They can detail the inner workings of an algorithm. They can depict data. Or anything in between.

And they are highly maintainable. They include no dead code. Code and examples are always referred to. Examples act like tests, too, which means that they are ... well, tested. Furthermore, when examples are combined with views, there is little need for long text and this makes the cost of creating documents to be marginal.

But, perhaps even more important is that documents can be edited live during development. Documents can be exported through XDoc BaselineOf subclass: #BaselineOfXdoc instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfXdoc' to external mediums, but from the best exeperience is to consume documents right in the development environment, next to the code and other objects and documents they relate to.

This design addresses multiple problems at the same time. Documents evolve much better with the system. Creating and consuming documentation is integral part of the development process. And creating documentation is implies but a small extra effort of assembling existing examples and views.

Authoring and publishing

This article was authored in GT as a live document, it was published from GT and it is loadable in GT. The publishing and loading is achieved through XDoc BaselineOf subclass: #BaselineOfXdoc instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfXdoc' , the generic container for executable documents. To load the current article, one can simply paste the url in Spotter and work with the document locally.

XDoc is essentially an archive with content and a description of that data. The content can vary. For example, besides documents like this one, we also export the history of the Playground through XDocs as well.

Through this, the development environment is extended with publishing and content distribution abilities. With GT developers can author explanations about the interior of their systems in a live environment. And now, they can also distribute those narratives to other mediums.

One rendering tree

Many of the visible abilities of GT are made possible by the graphical stack behind it. Particularly powerful is the property of rendering everything in a scene through a single rendering tree. Why is this important?

Consider this scene:

We see multiple widgets. There is highlighted code. The larger panes are connected via thick lines and are arranged as a graph. There are lines that start from inside the text editor. Some panes hold example code, and others hold inspectors on the resulting objects. And they are all live. We can scroll, type, change views. This interface is only possible because all visual elements are part of one large scene that is internally organized as a tree. A single rendering tree.

The flexibility of this infrastructure has far reaching implications. We believe that the shape of our tools influences our ability to understand our systems. We also believe that the space of interfaces for software is highly underexplored. Until now tools were mainly focused on generic interfaces that barely changed over time or from system to system. Even in the Smalltalk world, the interactions have only marginally changed over the past 4 decades. Through moldable development, we advocate that the interface should change for each individual object. In the main distribution alone we have 2938 distinct inspector views each of which can benefit from a novel visualization. And these are only basic views. There are other countless unexplored ways to construct broader interactive interfaces through which to grasp the internals of systems.

The current visible part of GT is but an example of what is possile. GT is uniquely equipped to be a platform for exploring the space of novel system-specific development environments.

Native windows with accelerated graphics

One visible characteristic of the graphical stack is the support for native windows and the vector graphics support. This is achieved through the Skia library for rendering, and the Glutin library for native windows suport. Less visible is that the rendering is hardware accelerated based on compositing render layers. In the picture below we see an example of how it works. On the left we see a test scene with multiple elements and effects, such as shadow. On the right, we see the scene decomposed into render layers, each of which can be rendered separately on different CPUs and GPUs.

Scriptable, testable graphical stack

Building novel user interfaces require a strong graphical stack. Building complicated user interfaces requires a sound stack that can be scriptable and tested by design.

In the picture below we see an example of scripting, testing and visualizing the logic of dragging. The example simulates the drag event through low level mouse events, such as mouse down, mouse move and mouse up.

Modular, releasable system

While we bundle multiple things together, we still keep GT highly modular. It is currently built out of 159 distinct, yet combinable, components. The picture below provides an overview of these components and their interdependencies.

To manage this, we want to be able to release the entire GT in a traceable manner every time we commit code anywhere on this map. This is the role of Releaser. Releaser ... releases deeply nested projects. For example, here we see on the left a map of a release that was automatically created. On the right we see the code of one of the baselines from the map.

Through this mechanism we can release GT reliably and have it loadable cleanly in Pharo. In practice, this means people working on projects built on top of the Pharo runtime have the option of two distinct environments.

Handling of other languages

GT is a platform for creating system-specific environments created in various languages. The first language we support is Pharo to support the development of GT itself. The second language we support is the one from SmaCC BaselineOf subclass: #BaselineOfGToolkit4SmaCC instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkit4SmaCC' with which we define parsers for other languages.

Here is an example. On the left, we have the definition of the JavaScript parser. On the right, we see a simulator of one of the productions, and the live preview of the parser matches based on the example snippet.

Please note how the code expansion mechanism applies to SmaCC grammar productions rather than to Pharo message sends. This is possible because the whole coding experience is already based on SmaCC from the ground up. Together with SmaCC we already ship more than a dozen parsers.

Software analysis infrastructure

We believe that a complete environment must make it easy for developers to reason about any aspects of a system, including the non-functional aspects, such as architecture.

For this reason we integrate Famix BaselineOf subclass: #BaselineOfGToolkit4Famix3 instanceVariableNames: '' classVariableNames: '' package: 'BaselineOfGToolkit4Famix3' for its ability to model systems written in various languages. Additionally, together with the parsing facilities, we also extend the basic logic of Famix with complete importers and link the high level graph model with the abstract syntax tree and the source code.

Through this integration, GT becomes a highly flexible tool for reasoning about systems' structure. At the same time, the analysis workflow remains identical to any other activity we have in GT. This allows us to combine various sources and analyses and adapt to the needs that appear in practice when assessing software systems.

Distinct environment

If we define a Smalltalk-like system as being the language plus the environment, GT is a new such system. The tools and the experience are different, but more importantly, the workflows that it enables are distinct.

We see the environment as a language made of visual and interaction operators that can be combined in many ways to address various scenarios in a uniform manner and at low costs. In the process, we reimagined the coding experience with editors that can be augmented with sophsticated widgets live. We bridged the gap between development and documentation through live documents and examples. We created a whole new graphical stack that unifies the space of visualizations, widgets and editors, thereby laying the ground for new kinds of interfaces. We created an extensive and flexible analysis infrastructure to support software assessment for existing systems written in other languages. And we achieved all this with a relatively small code base.

The traditional Smalltalk environment was a revolution when it was created. It enabled humans to interact with the inside of software systems in novel ways that then led to a new way of perceiving programming. We see moldable development as yet another leap.