Extending the Pharo debugger with custom actions

One of the main goals of the current debugger from Pharo is to be easily extensible with new debugging actions and new user interfaces. Towards that goal it models all debugging actions as objects that can be dynamically added to various widgets from the user interface. A high-level view of the debugger is given below.

We can see that a debugger is split into three distinct components:

1. the session
2. the debugging actions
3. the user interface

The session is the base component of the debugger. Its role is to implement the low-level operations needed for working with processes and execution contexts. For simple actions one does not need to extend it. Each debugging action is encapsulated in a subclass of DebugAction. Debugging actions are then loaded by the user interface. Currently there are two different user interfaces for the debugger one written in Spec and the other in Glamour.

Next we will go through the process of creating, step by step, one debugging action. As a running example we will implement an action for reacting to doesNotUnderstand exceptions. This exception is raised when an object is asked to execute a method that is not implemented by its class or any superclass. Given that Pharo uses dynamic typing, this error is *caught* at run time and a debugger is opened. The debugger allows us to create the missing method. We can implement this feature using a debugging action.

First we need to create a subclass of DebugAction. We will name the class DoesNotUnderstandDebugAction and put it in the category ‘DebuggingActions’, as it should go into the main debugger:

DebugAction subclass: #DoesNotUnderstandDebugAction
	instanceVariableNames: ''
	classVariableNames: ''
	category: 'DebuggerActions'

To customize the action we need to override several methods. The first one is #id and it should return a symbol uniquely identifying the debugging action.

DoesNotUnderstandDebugAction>>#id
	^  #doesNotUnderstand

To change the label of the action we need to override #defaultLabel and to control the position of this action among the other debugging actions #defaultOrder.

DoesNotUnderstandDebugAction>>#defaultLabel
	^   'Create'

DoesNotUnderstandDebugAction>>#defaultOrder
	^   45

An important aspect of this action is that it should only be active when the debugger is opened as a result of a doesNotUnderstand exception. To achieve this we need to override #appliesToDebugger:. This method allows us to control when an action is active.

DoesNotUnderstandDebugAction>>#appliesToDebugger: aDebugger
	^ aDebugger session isInterruptedContextDoesNotUnderstand 

Last but not least we need to implement the actual logic for creating the missing method. This is achieved by overriding executeAction. In this case the logic is a bit complicated but you do not need to understand it in details (I also skipped the code for the helper methods).

DoesNotUnderstandDebugAction>>#executeAction	
	| msg msgCategory chosenClass |

	msg := self interruptedContext tempAt: 1.
	chosenClass := self 
		askForSuperclassOf: self interruptedContext receiver class
		toImplement: msg selector
		ifCancel: [^self].
	msgCategory := (self askForCategoryIn: chosenClass default: 'as yet unclassified').
	self  session
		implement: msg 
		classified: msgCategory 
		inClass: chosenClass 
		forContext: self interruptedContext.
	self debugger selectTopContext

Now that the action is ready we can load it in the debugger. Each widget of the user interface loads debugging actions that have on the class side a method with a particular annotation. For example, the SpecDebugger loads in the toolbar action annotated with debuggingAction; the GTDebugger uses gtDebuggingAction. In our case the following code is required:

DoesNotUnderstandDebugAction class>>actionType
	<debuggingAction>

That’s it. Now if we execute code that is calling a missing method (Morph new backgroundColor) we get the possibility to create that method.

More information about what makes this possible can be found at scg.unibe.ch/research/moldabledebugger.

Also the debugger is not the only extensible tool from Pharo. There are many more, as part of the Glamorous Toolkit.

Advertisement

Sharing code using Playground and Spotter

Playground offers a simple way to manage snippets of code locally on our machine. Nevertheless, from time to time we need to share these snippets of code with other people. We can do this using a pastebin like service or send the code through email/chat. This requires us to leave the image and do a lot of copy/paste. Copy/pasting code from the browser back into Pharo, for example, does not always work smoothly.

To ease sharing code between images Playground allows you to directly publish its current content to ws.stfx.eu, a pastebin for Smalltalk. With one click the code is published and a link to access that code is copied to the clipboard. You can send now only the link to the receiver.

Once the receiver has got the link she can open Spotter (shift+enter) and copy/paste the link. Spotter loads and displays the code. Pressing enter opens a Playground with that code.

This feature is already in Pharo 4. Below is a short screencast showing it in action:

Extending GTInspector with support for exploring HelpTopic objects

Help Browser is a tool from Pharo that provides an introductory help for the environment and its tools:

Internally, the content displayed by this tool is stored in HelpTopic objects. Each HelpTopic object has:

• a title;
• a content instance variable storing the help text;
• a list of subtopics (also HelpTopic objects);
• an owner topic.

Hence, the entire help is stored as a tree of HelpTopic objects. Help Browser provides a user interface for exploring the content of this tree.

When developing Help Browser one has to explore both the internal representation and the actual content of a HelpTopic object. An object inspector that focuses exclusively on object state only shows the internal representation of HelpTopic objects:

This, however, is not the best approach for exploring the content of a HelpTopic object, as ones has to permanently expand and collapse objects in the tree view. To provide a better mechanism for exploring the content of a HelpTopic object we can extend GTInspector with two custom views.

First, we add a view that shows the contents of a HelpTopic object by adding in the HelpTopic class a method that constructs the presentation and annotating that method with gtInspectorPresentationOrder:, so that the inspector can find it. The annotation expects an integer parameter used to order views. Creating this view requires just 6 lines of code:

HelpTopic>>#gtInspectorContentsIn: composite
        <gtInspectorPresentationOrder: 40>
        composite text
                title: 'Contents';
                display: [ self contents ];
                when: [ self contents notEmpty ]  

To create the view we first indicate that we need a text view and configure the view with a title. Then using the display: method we set the block used to get the actual object displayed by the text view; in this case the content of the topic object. Last by not least, we use the when: method to only display the view if the topic object has actual content.

Second, we add a view that displays the tree of subtopics from a HelpTopic object. We follow the same process as before. This time we need 8 lines of code:

HelpTopic>>#gtInspectorSubtopicsIn: composite
        <gtInspectorPresentationOrder: 50>
        composite tree
                title: 'Subtopics';
                display: [ self subtopics ];
                children: [ :each | each subtopics ];
                format: [:each | each title];
                when: [ self hasSubtopics ]

Here the display: method returns the root elements of the tree. children: is used to extract the children of a node. format: sets the block used to transform each node of the tree into a string.

Now, when inspecting a HelpTopic object we get directly in the inspector a simple tool for browsing the content of such objects. We can use this inspector to explore both the implementation and the content of HelpTopic objects.

The inspector includes, by default, both the Raw and the Meta views. While useful for a developer, showing these views to a user that is just interested in the content of a HelpTopic object introduces an overhead. To remove it we can use the filtering feature of the inspector and create an object inspector that only contains the two custom views previously created:

inspector := GTInspector new.
inspector presentationFilter: (GTInspectorMethodListFilter new
        addAndSelectSignatureFor: HelpTopic>>#gtInspectorContentsIn:;
        addAndSelectSignatureFor: HelpTopic>>#gtInspectorSubtopicsIn:).
inspector openOn: SystemHelp asHelpTopic. 

This transforms the inspector into a Help Browser:

HelpTopic is just an example of a domain object interesting from both an implementation and a domain-specific point of view. Many other domain objects have similar requirements. The GTInspector makes is possible to easily create custom views for objects so that they can be inspected from multiple points of view.

– Many thanks to Torsten Bergmann for starting this extension.

A bytecode debugger

While we interact with source code using its textual representation, to the Pharo virtual machine source code is just a list of bytecode instructions.

For most programming tasks we do not need to understand the execution of a program at the bytecode level. However, when building tools like compilers looking at the execution of a piece of code at the bytecode level is a must.

While the debugger allows us to control the execution of a program, the default debugger from Pharo shows the execution of a program at the level of its textual representation. This is of little use when what we care about are individual bytecode instructions.

To improve this we created using the Moldable Debugger framework a debugger that allows us to interact with the execution of a program at the bytecode level:

Instead of textual source code this debugger shows by default the bytecodes of the currently executing method. It further provides actions for stepping into bytecodes representing message sends, stepping over individual bytecodes and stepping to a selected bytecode.

The debugger further shows the stack of temporary variables for the selected stack frame (method activation). In the previous screenshot the stack contains six temporary variables: three method parameters, one local variable and two intermediary values needed by the following instructions.

Apart from temporary variables the debugger also show object attributes labeled with the index used to access them in the bytecodes and local parameters declared but not active in the current stack frame:

While it still can be improved this debugger took just 200 lines of code to create, on top of what’s already available in Pharo for inspecting bytecodes. With the right infrastructure in place creating custom debugger can actually become an affordable activity.

This debugger is available in the latest Moose image (from the stack toolbar menu of the default debugger select ‘Available Debuggers’->’Bytecode Debugger’). For more information about what makes this possible visit the Glamorous Toolkit page.

A first look at the Pharo debugger

The debugger is an essential tool in any programming environment as it allows developers to interact with a running system and explore its state. Given the seer size and complexity of today’s software systems it’s hard to imagine reasoning about the dynamic behaviour of an application without using one.

Considering that it is so useful we want the debugger to have lots and lots of features:

• manage breakpoints (conditional/data breakpoints)
• control execution (step through code, step over/into functions, etc.)
• field watches
• take snapshots
• inspect variables
• modify variables
• monitor the call-stack
• code edit/swap at runtime
• intuitive GUIs
• remote debugging
• support for working with threads
• reversible/back-in-time debugging
• …

By all means this is not an exhaustive list. What it shows is that we expect a lot from a good debugger. The best part is today’s debuggers do provide most of these features.

But what happens when we want something that’s not there, which is bound to happen sooner, rather than later? First, we might go in denial: ‘This features has to be somewhere. I just have to find it. I can’t be the first one to have this problem.’ So we start looking. We search online, ask colleagues, read documentation, etc. In the end, given the huge amount of information that’s out there, we find something that addresses our problem. It can be a workaround, a hack, a pice of code that we do not understand but works, some cleaver way of combining several tools, etc.  It can be a strange or ugly solution, but hey, if it solves the problem it is perfect.

I follow this reasoning a lot of times. I’m sure a lot of you do too. However, there is one small thing that few of us take into account: if there is a feature missing in the debugger why don’t I just implement it? I believe most answers boil down to ‘Because it’s hard’.

The debugger is just a software application. To extend it we have to understand it. And as it turns out this can be rather difficult. To see this let’s look at how the debugger is implemented in Pharo 2.0. As Pharo is a *Smalltalk inspired* environment, and Smalltalk prides itself on its live environment, one might think that extending the debugger is straightforward.

As it turns out the entire debugger is implemented in a single class, named Debugger. This contains both the low level logic of working with a process and the logic for creating and updating the user interface. Everything is nicely tight together and heavily optimised. The class has 1453 lines of code grouped in 158 methods that access 23 attributes (including inherited ones); it has 120 lines of documentation. To give you a feel of how the class is organised here is a Class Blueprint and a System Attraction view showing its internal structure:

We can see that methods are heavily interconnected. There are large methods both in the public and private interface. Attributes are accessed both directly and using accessor methods.

Does this mean this implementation of the debugger is wrong? After all it provides most of the features mentioned above,  the code is in one place and it has been like that for a while. Like many other things in software I believe the answer is change: as long as we don’t need to modify it or extended it, the implementation is just fine. However, as soon as we need to add new features, the current implementation only gets in the way.

Now the question becomes: ‘Why would one want to modify the debugger?’. Why not continue using the available one and when new problems appears hope to find somewhere something someone thought can be of help to you?

To answer this, consider for a moment the explosion of new frameworks and libraries that happened in the last years. With the right library or framework all our problems can be solved in one or two lines. We add layers over layers of abstraction to reduce the number of lines of code we need to write. Just then what happens when things do not work? Most of the times we are lost in a world of abstractions. The debugger does not serves its purpose anymore. The fixed semantics of what it does and show are so far away from the reality of our systems that it becomes less and less effective. And, given the highly particular situations we encounter, searching for solutions revels less and less useful insight.

“We become what we behold. We shape our tools and thereafter our tools shape us”.

Said almost fifty hears ago in the context of media it describes well how we currently work with debuggers. We create large monolithic debuggers with lots of features and little support for extension. Then, when we encounter new situations we haven’t considered before, we try to solve them using the available ones even if they are ill suited for that. We don’t attempt to adapt the debugger, as we created a debugger that does not allow us to do that.

What would happen if instead of creating large debuggers with lots of functionalities we would instead focus on moldable debuggers that have extensibility as their main feature?

Would we then, when building a large and complex framework, invest perhaps a fraction of our energy in creating a dedicated debugger for that framework? Now only could this help us to better understand the framework, but would surely make the life of our users much easier. And wouldn’t this then increase the value of our framework? Let’s say you have to choose between to parsers. They have more or less the same functionalities, but one also ships with a dedicated infrastructure for debugging; the other has none. Which one would you choose?

To make this vision possible we need to build moldable debuggers from the start. We saw in this post that the current implementation of the Pharo debugger is a rigid one. In the next post we will look at a new design of the debugger that encourages extensibility.

About this blog, about me and about my PhD

Hi random reader,

I am Andrei Chiș a PhD student at the University of Bern in the Software Composition Group. As part of my PhD I am working on reducing the effort needed to build domain-specific tools for development and assessment. The goal is to make tool creation a day to day activity that augments/completes the development process of software systems, much like testing. This is at core of Humane assessment, ‘a method for making software engineering decisions’.

There are many ways to tackle this. My current approach consists in designing and implementing moldable tools. By moldable tools I understand  tools created with one goal in mind: extensibility. Having this as the main feature makes it feasible to easily adapt/customise/mold tools to new situations and specific domains.

I am doing this using Pharo, an open-source Smalltalk-inspired environment. I chose Pharo as it is a dynamic environment where the user can change almost everything. This makes it an ideal platform to rapidly prototype and test new ideas.

The tool that I started with is the debugger. I believe it is a really important tool that most developers feel they will never extend or modify; they perceive it as very complex pice of software. This might be the case if one starts from a large and monolithically debugger. However, things can be very different if we have a flexible and extensible debugger. I am currently working on the “Moldable Debugger” Framework to show that this is indeed possible.

Using this blog I want to present the status of my work, discuss implementation/design issues and look at the possibilities that open as we move towards moldable tools. To begin with, I will focus on the debugger.

I hope you will join me in this journey,
Andrei