hpi-swa-lab/squeak-tracedebugger

A lightweight and interactive back-in-time debugger for Squeak to trace and retrace past method invocations and state changes. Powered by SimulationStudio.

squeak-tracedebugger's Issues

Stateful context search

Context search convenience could be improved with:

• an overview of all search results (list + counter)
• means to navigate between hits forward (✔️) and backward (not yet implemented)
• Also, the scope of the search should be made clearly unless the search is global.
• Open question: run and search? do we have the need to control how far code is evaluated (i.e. not too far) or is this just an optimization issue?

See also the notes from the 1st UX study.

Item to expand all tree items

This might be best implemented in the Trunk.

Current workaround: Hold and press the ▶ key for a long time while focusing on the tree. :-)

Performance measurements during tracing

Record the time consumed per trace. This might be a performance-critical change to the tracer. Note that we do not yet have any use cases for this.

Time slider?

Provide a simple horizontal slider in the debugger that can be used to select a time within the lifespan of the selected context.

See also PluggableSliderSpec from ToolBuilder-Kernel-ct.139/ToolBuilder-Morphic-ct.254 (inbox).

Efficient processing of queries for historic state changes

Implement a retracing simulator with vector semantics for simultaneous evaluation against all memory states of interest. Allow the client to specify a relevant time window. Provide a search mode with different strategies (forward, backward, binary) with streaming of results.

Text selection-independent representation of PC range

Use a not-yet-reserved text attribute for indicating the next PC range so that users do not need to press Where or Cmd + L all the time.

Discussion: How should this text attribute look like?

Missing simulation capabilities

TDBTracingSimulator

Single primitives:

• primitive 76 primitiveStoreStackp
• primitive 103 primitiveScanCharacters (optional)
• primitive 105 primitiveStringReplace (optional)
• primitive 145 primitiveConstantFill (optional)
• tbc
• when an unsupported primitive is hit, write a note to the Transcript

Implementing an optional primitive means to accelerate its execution time.

TDBRetracingSimulator

We have a general issue here with primitives that directly access the state of any object which was changed at a later point in time. A solution to this problem would be temporal rematerialization (#23) of the state in question, maybe in a copy of the original object.

Known affected primitives:

• BitBltPlugin/primitiveCopyBits - which variables next to sourceForm are affected?
• tbc

Both simulators

Groups of primitives:

• changeClass, flushCache

  Here is an example that will currently crash your image with a VM failure:

  Simulator cleanUpContextClasses.
  [Simulator evaluate: []] traceAndDebug.

  This snippet will trigger the construction of a new (simulator) class during the simulation.
  Possibly related: [squeak-dev] primitiveClass infected after primitiveIdentityHash on wrong receiver

• forwardIdentity

• immutability

• control primitives (: What would we do with new processes? See also: [squeak-dev] [squeak-dev] Debugging through #fork)

• FFI

• See also the overrides of testGroups and expectedFailureGroups in TDBMemorySimulatorKernelTest and subclasses.

• Handle failing primitives gently (don't crash the simulator, for instance, see #testAtPutOutOfBounds)

Optimize navigation

Current bottlenecks:

• scrolling/tree display: TDBCursor>>#traceFor: (via TDBCursor>>#childContextsFor:, 68%) - forego generators in TDBTrace>>#withAllDescendants, use binary depth-first search
• stepping: TDBTrace>>#traceAtTime: (13%) - inline binary search? don't use #associations.

Further optimization potential:

• We are still rebuilding the tree after every step (#wrapRoots), which helps us fulfill all invariants, but is terribly slow. Updating the trace iteratively would be hard with respect to the current limitations of the PluggableTreeMorph API, automatic expansion of relevant nodes, and coroutines (#14).- [ ] use non-linear search algorithms for stepping operations (i.e., binary search with unknown size)
• All stepping operations could be optimized by rolling out an exponential search mechanism. For this, we would need to define an absolute criterion for "whether a timeIndex belongs to the right step".
• optimize inspector updates (could be possible with the help of #21)
• not yet optimized: stepBack
• Reduce truncationLimit in snapshot inspectors (requires upstream contribution)

Epic: Trace Manipulation

Contributes to #12 [#10].

Steps:

• #24
• Manipulate historic state to reevaluate (#25)
• Programming in the debugger to reevaluate

Implement process buttons

Make the Proceed/Abandon buttons in the debugger work. Contributes to #36/#37.

Further stepping buttons

Stepping operations that are not yet implemented might include:

• back through
• run method to end
• step until criterion is fulfilled

To not overload the UI, provide these options via a menu (either a menu on a normal stepping button or as a menu button via https://github.com/hpi-swa-lab/squeak/issues/58)

Track object lifecycle inside tracer?

I.e., record when an object was created and, maybe, when it becomes unreferenced.

We don't have any use cases for this yet, but it might be an interesting feature.

Trace Morphic world cycles

Implementation:

• BlockClosure >> #traceAndDebug
• a small "trace recorder" UI

Support coroutines

Currently, coroutines cannot be traced and break tracing because the architecture relies on a constant sender per context. Optimization is also an issue here. We could implement the missing look-ups in a second, slower run. See #coroutines.

Further notes:

• correct context update when stepping to new sender - it needs to be registered as trace

Also discuss how we could still display parts of the trace that are currently not in the active stack.

`TDBProxy` limitation: Retracing of side effects will not continue after return

Example:

array := Array streamContents: [:s|
	aTdbProxyForAnArray do: [:ea | s nextPut: ea]].
array first someState.

One might expect that the element arrays would be further TDBProxys after executing this snippet, or that the entire handling of the array would be continued in the retracing simulator. However, none of this is currently the case and the final expression accesses the current version of the item's state instead of its historic state.
Hypothetically related if we wanted to go the second route: LinqLover/SimulationStudio#51

At the moment, a workaround is to wrap the entire expression with a message send to the proxy, e.g.:

aTdbProxyForAnArray in: [:theTdbProxyForAnArray |
	array := Array streamContents: [:s|
		theTdbProxyForAnArray do: [:ea | s nextPut: ea]].
	array first someState].

Maybe we should at least document this pattern somewhere.

See TDBObjectExplorerWrapper>>#contents for a real motivating example.

Fork memory

Create a copy of the memory at a certain time index. The copy can be continued independently of the original memory.

Context search: Only display reachable signatures/classes in the dialog

Currently, search is scoped to the tree below the selected context (and I think this is the way that makes most sense). However, search suggestions should only be collected for all actually reachable hits.

Also pre-select the current value when bringing up the dialog.

Enhance trace with stub methods

Display methods that were not actually activated (primitive methods/quick push primitives, FFI methods) in the trace, too.

Implementation:

• How to fake contexts?

Scenario: Explore Morphic construction/rendering

• Roles: System learner (understand the entire process), system expert (find a specific bug)
• Goal: Explore the construction/toolbuilding process or the rendering/drawing process of a complex morph
• Examples:
  • ToolBuilder build: (Inspector on: #(1 2 3))
  • Project current world imageForm
  • Localize the text run that causes an invalidation in a NewParagraph during the setting inside a TextComposer (see hpi-swa-lab/babylonian-programming-smalltalk#55).
• Problems:
  1. Only linear stepping possible, no back-in-time display of prior construction/rendering states, no binary search to trace back certain changes to certain stack frames.
  2. Irrelevant stack frames distract understanding of the relevant process (e.g., Canvas clipping, Morph iteration).
  3. Intermediate result is hardly accessible: Need to reevaluate aMorph imageForm for every (relevant) step or to navigate to aCanvas form > "screenshot" and update the inspector manually (bug in Form >> #=).
• Possible solutions:
  1. :
     • display the entire trace at once
     • step into details later in the context of the original event state
  2. filter trace by method category/class/package
  3. similar to printbugger:
     • overview of single drawing steps of #imageForm (maybe sandboxed) from which the user can jump to the responsible stack frame
     • slider for efficient time-traveling (maybe fish-eye slider)

Epic: Spawn trace

Drag a context out of a debugger to watch it in a separate debugger.

Steps:

• #27
• Discuss: What to do with still-active context? Copy them?
• #28
• Implement UI for drag-and-drop

Contributes to #9.

Scenario: Debug Morphic Layouting

• Roles: System outsider (nearly zero prior knowledge about the system), system expert (already knows precisely where to search)
• Goal: Understand the reason for an AssertionFailure signaled from #doLayoutAgain
• Example: Layouting of an etoys script morph (see EToys-ct.403 and "Etoys car example.1.cs")
• Problems: Narrowing down when the property #doLayoutAgain is added to the morph requires stepping into all details or performing a binary search by starting a new debugger whenever it was stepped too far.
• Possible solutions:
  • step into details later in the context of the original morph state
  • find all instructions in the trace that add the property #doLayoutAgain

Improve support for proxies

In theory, it should be even possible to debug ProtoObject new yourself without that the UI crashes. This, however, is currently neither possible in the normal debugger nor in the TraceDebugger because there are several places that inexorably send messages to the receiver of the selected context. This issue is for making sure that no debugger pops up while stepping through ProtoObject new yourself etc. Also, print-its in the inspector should work.

Epic: Organisational trace filtering

Filter units (disallow-list or allow-list, maybe combinable):

• packages
• class categories
• classes
• message categories
• selectors

Requires:

• First prototype: tree-wide type-to-filter
• Filter UI (could be a simple UIManager dialog)
• trace filtering
• cursor filtering

Contributes to #9 #11 #12.

Reevaluate trace

Build an API to reevaluate a trace from an earlier point in time in a forked memory (#27).

In the context of the UI, this would be relevant for the following operations:

• accept contents to inspector field (currently changes the present state, unclear semantics)
• debug it from code pane/inspector field (could spawn another trace debugger)
• return value from context (not yet implemented)
• skip context (not yet implemented)

Technically, TDBRetracingSimulator could detect state changes and offer the client to reevaluate the trace automatically. See "retrace side effects" in the design process document.

Could either be solved via rematerialization (#23) into a copy or by melting a tracing simulator and a retracing simulator properly.

Epic: State queries

Contributes to #10 #11.

Steps:

• #20
• Inspector integration: Jump to previous/next change (#104)
• History explorer

At some point, optimization will be needed in order to only trigger changes to the UI when there was actually a change in a historic value.

Rematerialization

Build an interface on TDBMemory to rematerialize objects from an earlier state.

We are still lacking the requiring scenarios for this, though.

Epic: Inspect historic state

• Display historic states in debugger
• Inspect historic states in inspectors/explorers
  • Make inspect it/explore it usable (do not simulate tool construction) (solved with snapshot inspectors/explorers via #42)
  • Provide TDBProxyForCursor with a time index following the cursor position

"Step until" button

Add a simple "step until" button to the debugger that steps until a given expression by the user evaluates at the current time to true. Would benefit from #20.

Also implement the "run to here" item in the code pane menu.

Additional ideas:

• User can enter a non-boolean expression that is checked for changes
• Discuss dealing with side-effects
• What about dedicated "step over until", "step through until", and "step back until" buttons?

Mechanism to hide bottom contexts

Only display a subset of a trace in a debugger.

The hidden contexts could be displayed again via something like a "Full Stack" button.

Scenario: Explore Morphic event handling

• Roles: System learner (understand the entire process), system expert (already knows most static artifacts, wants to fix a specific bug)
• Goal: Explore the relevant event handlers for a MorphicEvent that is propagated through the world (including composed handlers, submorphs, event filters, rejecting handlers, etc.).
• Examples:
  • Debug menu invocation (e.g., world menu)
  • Find all activations of capture event filter in PluggableMenuButtonMorphTest >> handleMouseRedUp (https://github.com/hpi-swa-lab/squeak/issues/65)
• Problems:
  1. Currently, this requires stepping into every possibly relevant detail and either viewing all methods of interest in temporal order or noting them down manually.
  2. Finding an entry point for debugging event handling is hard.
     Traditional workarounds for this problem include:
     • provide a scripted entry point manually:

       self currentHand handleEvent: (MouseButtonEvent new setType: #mouseDown position: 50 @ 50 buttons: MouseEvent redButton hand: self currentHand)

     • set a breakpoint somewhere in the middle of the process, restart debugging from the bottom, and reset the event manually
• Possible solutions:
  1. :
     • display the entire trace at once
     • bookmark relevant stack frames
     • filter trace by method names/class names
     • step into details later in the context of the original event state
  2. record trace before debugging it: analogously to the profiler tool, enable system-wide tracing while performing a relevant option

Epic: Debug Morphic world cycles

Contributes to #9.

Requires:

• #18
• #16
• #17

Mechanism to exclude methods from tracing

Some expressions/processes have a large computational foot stamp because they execute expensive methods such as ClassDescription>>#packageInfo that, at the same time, do not include any relevant side effects to the debugged system. As a "last resort" optimization performed by the user of the tool, it might be helpful to declare an exclusion list of methods that should be ignored during tracing. The tracer would be responsible for running all invocations of these methods outside of the simulator. See LinqLover/SimulationStudio#54.

Discuss a proper UI:

• analogously to filters, select a context to exclude it by example
• would benefit from #64
• Challenging: Exclude method but not block called by function

Scenario: Explore regex matching

• Roles: System learner, system expert
• Goals:
  • Retrace the matching process of a string against a regular expression
  • Understand the reason for backtracking in the matching process
  • Program in the debugger to fix a bug/add a new feature and explore the updated behavior
• Examples:
  • Survey all sends to/invocations of #matchAgainst: in the matcher
  • Browse invocations of RxMatcher >> #matchAgainstMarkerAt:nextLink:
  • In RxMatcher >> #matchAgainstLookbehind:positive:nextLink:, remove a particular #position: send
• Problems:
  1. Once you missed an important detail, going back is impossible without restoring the prior state of the matcher's source stream manually.
  2. At one time, you can only see the current route of links, excluding all past/future routes before or after backtracking or checking lookaround links.
  3. After editing a method that has already been executed, the debugging session needs to be restarted and manually navigated to the previous position.
• Possible solutions:
  1. step into details later in the context of the original event state
  2. :
     • display the entire trace at once
     • filter stack frames by selector (e.g., #matchAgainst:)
     • filter stack frames by return value (e.g., false)
  3. different accept options in the debugger:
     • reevaluate trace and select the equivalent stack frame
     • reevaluate from the current stack frame
     • open updated trace in a new window

Epic: Behavioral trace search/filtering

Contributes to #12.

Search for:

• #44 (sends to selector)
• accesses to instance
• accesses to a variable/binding
• returns with specific value

Steps:

• create UI
• design & implement query language

Possible follow-up for #19.

Epic: Integration into the base system

• fix Object >> #tdbHalt
• trace button in normal debugger to open a trace debugger for the next send

Robust user interrupts during stepping

When the user presses Cmd + . while a step is being executed, interrupt the tracer gently. At the moment, one has to complete the current step message inside the cursor manually from the interrupt debugger before abandoning it. For an elegant solution, this would require an UserInterruptException provided by the Trunk (see https://github.com/hpi-swa-lab/squeak/issues/131).

Inspector issues

• support "add field"/"edit field getter" items for custom fields

  For the required inspector hooks (dispatch compilation cue construction to inspector), see inspector-compilercue.cs, which only needs to be contributed upstream.
  For the remaining todos on the TDB side, see the comment in TDBInspector>>#fieldListMenu:shifted:.

• fix interactive print-it (currently triggering simulator)

  InteractivePrintIt & Co. need to dispatch to the requesting model (upstream).
  Alternatively, this feature could be turned off by changing the preference temporarily or overriding printIt:result: in the models.

Improve convenience of debugger explorers

Referring to the experimental preference #shouldReplaceInspectorsWithExplorers.

• Explorers are wider than inspectors. Maybe rearrange the inspector pane so that we only have one evaluation pane in the middle between both explorers?
• Contextual evaluation currently does not work in the explorer panes.

Epic: Replace the default debugger

Possible follow-up/alternative for #36.

• Relativized tracing (opt-in toggle, definition of skip-lists not to trace, ...)
• AppRegistry/ToolSet that always uses TraceDebugger
• Subclass DebuggerTests
• Implement missing legacy features on trace debugger - pre-debugger window, #38, and all other not-yet-implemented buttons, menus, and shortcuts

Add support for existing `SimulationContext`s

At the moment, the TraceDebugger will eliminate any other custom context customizations in the code to be traced. For instance, the following does not run the code sandboxed when you trace it and step over the assignment:

| x |
x := 0.
Sandbox2 debug: [x := 1].

The cause for this behavior lies in TDBTrace>>#enableSimulatorDuring: where the customizations of all contexts are via simulator customize: context. We should preserve other custom context simulators and combine all simulators instead via the COR mechanism.

Epic: Self-supporting tool

The TraceDebugger can be used to debug itself.

Requires #14 or giving up generators in the codebase (which, on the other hand, might also improve performance).

The Equivalent of Breakpoints

How could a tracing equivalent of breakpoints look like? Brainstorm useful mappings to trace queries #19/#22.

Revise window titles

Currently, custom title strings for exceptions/the debugger invocation are not displayed in the title. On the other hand, we do not want to leave the timeIndex display. Think about a proper solution. This problem might also relate to the snapshot inspectors.

Optimize tracing

Current bottlenecks during tracing are:

• TDBTrace>>#maxTimeIndex (42%) - cache?
• TDBTrace>>#traceAtTime:ifAbsent: (39%) - inline binary search? don't use #associations.

Remaining potential for optimization:

• SimulationContext>>#customize: (1.7%) - hard-code relevant subset of copyFrom:
• TDBMemory>>#object:priorTo:atSlot:put: (19.4%) - avoid thisContext, anything else?
• TDBTrace>>#updateMaxTimeIndex:from: (7.5%)
• TDBTrace>>#traceAtTime:ifAbsent: (18.4%) - single index for all traces?
• TDBTrace>>#enableSimulatorDuring:: Currently, we are customizing all contexts there, which makes every forward-step in a large tree an expensive operation. Would it be fine only to customize the top context instead? Or could we at least exclude dead contexts (this would still be an optimization from O(n) to O(√n))?

Ideas for reducing space consumption:

• Use dense table memory layout for certain objects/slots (i.e., Context/pc)?
• Reduce default size of TDBTrace.children? Currently, over 90% of all trace instances do have 2 children or less:

  sizes := self cursor rootTrace withAllDescendants contents collect: [:t | t children size] as: Bag.
  sizes openAsMorph.
  (sizes count: [:size | size <= 2] ) / sizes size asFloat.

  

Brainstorming: Enhance tree view

• highlight next context that is about to be enabled?
• highlight similar contexts to the current one (same receiver, selector, method)?
• display deprecations strucked-out?
• experimental: upside-down display (to be supported by Morphic)
• Morphic: indentation-aware scroll-into-view (do not scroll back horizontally when selecting a deeply indented context)

All-exclusive filter causes image freeze, but mitigable through interrupt + proceed

Steps to reproduce:

• trace it: 1 yourself
• add filter for package Kernel
• freeze! but Cmd + . & Proceed fixes it.

Additional testing opportunities

TDBCursor

• analogously to #testTraceSimple, add acceptance test for recorded states somewhere?
• test #isContextAlive:
• test stepToHome: with foreign contexts (see examples in stepToHome:)
• test stepThrough: from sender

Kernel tests

• These tests currently do not include the building of a trace (only side-effects are traced).

  We should discuss whether this reveals a bad design of the tracing simulator (the tracing simulator is not responsible for building the trace; the cursor/trace does that) or whether we should talk to the cursor directly in some of the kernel tests.

• Smart smoke assumptions for kernel tests: For instance, reevaluate each test twice and compare the second execution to the trace of the first one. Reevaluation #24 would give us further possibilities here.

• Compare trace with normal execution? This would not work for details of SimulationContext.

UI

• test spawn & process termination

General concerns

• Improve primitive tests in the simulated code (such as WriteBarrierTest)
• Run all tests without useProxiesAlways (CI config?)

Inspector field error texts are not displayed properly

A virgin fraction (Fraction basicNew) looks like this in a regular debugger/inspector:

In the TraceDebugger's inspector, it looks like this instead:

---

An error from a custom inspector field looks like this in a regular debugger/inspector:

In the TraceDebugger's inspector, it looks like this instead:

Find context

Part of #22.

Use cases:

• debug '[a-z]' asRegex, jump to RxCharSetParser (contributes to #12)
• debug Morphic rendering, jump to Canvas>>#drawString:* (contributes to #11)
• debug Morphic construction, jump to PluggableCheckBoxSpec (contributes to #11)
• debug Morphic event handling, jump to #doubleClick: or Editor (contributes to #9)

Detect state changes during retracing

Detect state changes inside the retracing simulator and design a RetraceSideEffect exception that can be used to cancel or permit the state change or to apply it in a forked memory (#27).