sliminality/ply

CSS inspection aided by visual regression pruning

Ply

Ply is a lightweight DOM and CSS inspector. It introduces novel CSS analysis techniques to help less-experienced developers inspect and learn from production websites of interest, when existing inspector interfaces are too overwhelming. Ply is research software, initially under development through Design, Technology, and Research at Northwestern University.

Several of Ply's core features have been released as Inactive CSS in Mozilla Firefox 70. Give it a try! You can also read our award-winning UIST 2018 paper, Ply: A Visual Web Inspector for Learning from Professional Webpage, or watch the talk.

Novel features

In order to help less-experienced developers navigate the complex DOM and CSS structures found on production webpages, Ply implements the following novel features:

• Visual relevance pruning (CHI 2017 SRC paper, talk): While inspecting the styles for an element, users can click a button to dynamically rule out styles with no visual impact on the page. Useful for reverse-engineering how certain visual effects are achieved, without wasting time on irrelevant properties.
• Implicit dependencies (UIST 2018 paper, talk, slides): Many CSS properties depend upon each other in unintuitive ways (e.g. z-index does not apply if position is static). Ply uses visual relevance pruning to test for implicit dependencies, and provides explanatory tooltips to reinforce the relationships between properties.
• DOM component isolation: Rather than viewing the entire DOM at all times, users can select a node on the target webpage, and inspect only its subtree. Useful for inspecting a component of interest.

Additional features under development include:

• Multiple element inspection: Click on multiple elements in the DOM viewer to inspect their styles side-by-side. Useful for exploring parent-child relationships, positioning contexts, or many other interactions between CSS properties on distinct elements.
• Style organization cues: Using visual regression pruning, Ply can heuristically identify when a CSS rule serves as a "base style" for multiple elements on the page, and when another rule serves as a "refinement" style on top of shared base styles. Useful for understanding how professionals structure their CSS to minimize repetition and capture visual similarities and differences between elements.

Design rationale

In our needfinding studies, less-experienced developers were frequently overwhelmed by the complexity of the Chrome Developer Tools Element interface.

Accordingly, Ply's user interface has been carefully designed to minimize visual clutter and extraneous cognitive load:

• Truncated content: Long text nodes are truncated by default in the DOM view, with the option to expand their contents.
• No closing tags: DOM nodes are displayed Python-style, with no closing tags. Indentation dictates the node hierarchy.
  • By not rendering closing tags, we cut vertical screen space by up to 50% for block-level elements.
• Nodes are collapsed by default: When a user selects a node for inspection, very rarely do they want to see the entire subtree at once. Ply collapses DOM nodes by default, allowing the user to explore the DOM structure step-by-step, at their own pace.
  • Auto-expansion: Container elements, which wrap a single child for styling purposes, frequently confused users in testing. Namely, users would neglect to expand container nodes, missing large portions of the DOM as a result. When a user expands a DOM node with a single child element, Ply will auto-expand its child recursively, until reaching a leaf node or an element with multiple children.

Structure

This repository only includes the Ply frontend and proxy server. The system communicates with the target webpage via the Chrome Remote Debugging Protocol; all browser-facing functionality is located in the chrome-remote-css repository.

The prototype implementation of the visual regression pruning algorithm can be found in chrome-remote-css as well.

Getting started

The following instructions assume familiarity with Git and Node.js tooling. If you're not even sure how to download this project, check out this guide to get up to speed.

Install the chrome-remote-css extension. Ply depends on this extension to debug pages remotely.

Once you've cloned this repository:

yarn install     # install dependencies
yarn run server  # start the proxy server
yarn start       # start the inspector

Then navigate to http://localhost:3000 in Chrome, and you should have a blank inspector.

To start inspecting a webpage, open a new window or tab, and navigate to the site you want to inspect.

Click the browser extension icon to activate chrome-remote-css, and select an element on the page using the cursor. The element and its subtree should appear in Ply.

Note that you'll need to keep the inspection target tab open, so that Ply can update and request CSS styles as you inspect.

Please file an issue if you encounter any difficulties running the project.

ply's Issues

Implement pruning options

Related issue: sliminality/chrome-remote-css#15

• Reset to pruned: add a new action, APPLY_MASK
• Re-prune: extend the PRUNE_NODE action with a flag to overwrite existing mask
• Reset all: add a field to APPLY_MASK called reset

Shadowed CSS properties should reference the effective settings

When properties are shadowed in a BASE_STYLE, users should be able to see arrows (or some kind of highlight) on the effective settings of those properties, higher up in the cascade.

Node v10, v12 not worked by deps issue. v8 is okay

Please update readme prequirements or bugfix package.json lib dependency
Bug details:
Module:
node-sass
Path:
ply/node_modules/node-sass/build
CLI:
g++ '-DNODE_GYP_MODULE_NAME=binding' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DBUILDING_NODE_EXTENSION' -I/home/user/.node-gyp/12.16.2/include/node -I/home/user/.node-gyp/12.16.2/src -I/home/user/.node-gyp/12.16.2/deps/uv/include -I/home/user/.node-gyp/12.16.2/deps/v8/include -I../../nan -I../src/libsass/include -fPIC -pthread -Wall -Wextra -Wno-unused-parameter -m64 -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++1y -std=c++0x -MMD -MF ./Release/.deps/Release/obj.target/binding/src/binding.o.d.raw -c -o Release/obj.target/binding/src/binding.o ../src/binding.cpp
stderr:
../../nan/nan_object_wrap.h: In static member function ‘static void Nan::ObjectWrap::WeakCallback(const v8::WeakCallbackInfoNan::ObjectWrap&)’:
../../nan/nan_object_wrap.h:124:26: error: ‘class Nan::Persistentv8::Object’ has no member named ‘IsNearDeath’
124 | assert(wrap->handle_.IsNearDeath());
Solution:
$ nvm i v8.17.0

run_times make recursion error when program uses threads

[Tested on a ESP32]

When using decorator run_times and the program also run threads, simple_cron may fail with 'recursion error':

This test program fails most of the times. If you delete the run_time decorator (or comment out the counter thread), it runs OK:

import _thread
import time
from scron.week import simple_cron
from scron.decorators import run_times

def nowplus(seconds):
    '''Compute now plus seconds
    '''
    future = time.time() + seconds
    year, month, mday, hour, minute, second, weekday, yearday = time.localtime(future)
    return {
        'weekdays': weekday,
        'hours': hour,
        'minutes': minute,
        'seconds': second
        }


def counter():
    '''A simple thread'''
    counter = 0
    while True:
        print(counter)
        counter += 1
        time.sleep(15)
        
                    
print("Starting ...")
_thread.start_new_thread(counter, ())
simple_cron.add("null task", lambda *a, **k: None, seconds=0, minutes=range(0,59,5), removable=False)
simple_cron.add("Cosa 1", run_times(1)(lambda a, b, c, d: print("Hello, World")), **nowplus(10))
simple_cron.run()

print("Waiting for 'Hello, World'")
time.sleep(30)
print("End")

`

Handle UA styles

Currently, we don't have any way to visually distinguish UA default styles from normal styles. (We should maybe not even display these.)

Add tooltip component for rule annotations

Users should be able to hover a ? hint and see a description of the annotation.

Implement this for the BASE_STYLE annotation.

Make status message more informative

Currently, the DOM view always says Connecting... when there is no active node.

We should update this to an enum:

• Connecting... before socket connection
• Connected to server, waiting for node. after successful socket connection
• Could not connect to server. after failed connection, with a button to reconnect
• Connection lost. after closing, with a button to reconnect