Giter Site home page Giter Site logo

karta_proto's Introduction

Karta

image

Karta, most fundamentally, is intended to be an augmented file browser and allow for visual and conceptual connections to be made between files in the file system. This makes it potentially a great tool for file and project management, as a sort of in-between layer between other, more specialised software. It would allow compositing and packaging up of some files and data to sequence them into new files. For example, exporting a sequence of image nodes into pdf's or videos. This would provide a great base set of functionality from which Karta can then be specialised into different use cases depending on what there is demand for.

This repository contains the prototype or proof-of-concept for Karta. It was made in the Godot 4 game engine with its internal scripting language GDScript. Therefore a copy of the newest stable version (Godot 4.0.3 as of writing this) is required to edit and run the project. This version is considered to have reached its goal of illustrating the concepts behind Karta and is no longer in active development. If and when the idea is pursued further, the application will be rewritten from scratch (likely not in Godot) to provide a more stable and testable foundation to develop on.

karta_proto_demo.mp4

Core Features

This prototype implements the a somewhat scattered set of features. They are unpolished and often buggy. Their purpose is to map out the design space for a future iteration of the app, to give a clearer picture of what the desired architecture should be.

Keyboard shortcuts

F - Focal tool G - Move tool T - Transition tool E - Edges tool D - Draw tool

TAB - New node menu

P - Toggle performance mode

X - delete node (warning: buggy)

F5 - Toggle node debug info F6 - Toggle display of node names

Core tools

The current active tool determines what clicking with the mouse does.

  • Focal - clicking on a node sets it as the focal node and moves into its context or local graph. Nodes are despawned, spawned and moved accordingly.
  • Move - clicking and dragging on a node moves it. This updates its relative position in its connection to the Focal. If the Focal is moved, all connected node positions are updated relative to it.
  • Edges - clicking and dragging from one node to another creates a new edge between them. The type and group of the edge is defined in the pop-up menu under the tools panel.
  • Transition - moves into a Focal context, but only if there is an edge of type "transition" connecting them. More on this feature below.
  • Draw - draw rectangles into the scene layer. More on this below.

Focal Nodes

The basic structure of Karta is that it is a view into a directed graph, made of nodes and edges. The network is always viewed from the context or point-of-view of a single node, displaying that node and its connections with one layer of depth. Like a local graph.

Nodes don't have an absolute position: instead, their position in the graph is always defined as a relative position to the current Focal node. These relative positions are stored in the edges between nodes. This means that the Focal node, the focal point of whichever context you are in, always sets the positions of its connections.

The aim of this feature is flexibility and reusability. Depending on what you're working on, you might want to organise existing nodes in different arrangements, yet it's preferable to not have to duplicate nodes to do this. Relative positions allow for the reuse of nodes in vastly different layouts, and for nodes to always know the contexts in which they are used and relevant.

Basic Node Types

The most basic node is a Base Node. It contains no data except the data common to all nodes, such as the node name.

The other two most basic nodes are the Text and Image nodes. Both of these are resizable. The Text node is just a text box for writing (non-rich) text. The Image node specifies a path in the file system to an image and then loads and displays that image. These two nodes illustrate how Karta could also be used as a digital whiteboard or canvas, for laying out visuals and text in a dynamic way.

Edges, the connections between Nodes

Edges between nodes can be one of three different types, which change how they are treated by the app.

  • Base - generic, two-way connections
  • Parent - establish a hierarchical connection between two nodes. Only implemented for Scenes and Rectangles.
  • Transition - mark an edge as a transition. More on this in the section about Transitions.

Edges can also have a group, which can be any text. These currently do nothing, but the intended use is for filtering. In a future version it might be better to just allow arbitrary attributes on all edges.

Performance mode

When Performance mode is on, the current active Focal node will have its data displayed in a separate view. In this case, the graph background. The ability to display this in a separate window was planned for this prototype, but is apparently not something supported by the game engine. You will have to imagine it.

Currently this is implemented for Image and Scene nodes. If an Image node is set as the Focal, that image will be displayed in a different view. If a Scene node is set as the Focal, it will display all of its child ObjectRectangles.

Scenes, Rectangles and Transitions

This is an experimental feature developed during a course on Interactive and Immersive Art. Turning Karta into a state machine.

Scenes display their children rectangles in the performance view. Each Scene is therefore a state, an arrangement in which a set of objects exist. If a transition edge is defined between two Scenes, moving between them using the Transition tool will smoothly animate the rectangles between two states. If there is an uneven amount of rectangles, the excess will be faded out or in.

The Draw tool can be used to automatically create these rectangle nodes by drawing them in the graph background. Click and drag to create a rectangle, and the corresponding node will be added to the graph automatically.

There is also a Properties node, which displays the data of whatever other node is currently selected. When selecting a rectangle, the Properties node displays the position and size of it and allows manual editing. The update isn't instantaneous. The rectangle will update when another rectangle is created of when the user reenters the current context.

Other notes

  • Nodes and edges are stored in the default appdata folder in Godot's resource ( .tres) format.
  • There exists a bug where sometimes a node's edge index gets erased, and its connections are no longer spawned when they are supposed to. This has proven to be very hard to debug and is not worth fixing at this point, because this prototype is not meant to be used for serious work anyway. Saving nodes is therefore disabled.

Future work

Written on 25.5.2023.

As mentioned before, this iteration of Karta is no longer in active development. The foundations are shaky and hacked together and have proven, during the last couple feature implementations, to not be scalable. And as much as I love Godot, I've come to believe that it might not be the best tool to make Karta with.

However, I am quite excited about the ideas present here. The local graph based interface and way of working, I believe, lends itself very elegantly to many use-cases. It is universal enough to be extendable to many domains, as I hope the state machine feature shows. It feels like a more intuitive way to work with data, in a way that more closely models how ideas flow in the mind and how patterns are constructed. At least for me.

karta_proto's People

Contributors

teodosin avatar isokissa avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

karta_proto's Issues

Node duplication

Add the ability to duplicate a node.

Its type and properties will be copied, but it will be given a new id and be its own node. One of Karta's principles is that it should discourage duplicates, so perhaps a temporary wire could be connected between the nodes that isn't removed unless one of them is edited.

Add types and labels to WireBase

Types would be, for example

  • Connection (two-way, regular connection. This is the default currently)
  • Parent (establishes parent-child relationships, could allow children to be collapsed intuitively into the parent)
  • Stream (one-way, used for transmitting data between operator nodes)
  • Sequence (perhaps could be merged with stream? Or should they be different?)

Labels/groups serve to group nodes and their connecting wires together. This is what adds ordering in node clusters, which currently have no way to be arranged in a fixed order.

This enhancement is crucial to the foundation of this software.

Refactor actions to implement the Command pattern

Critical enhancement that will allow for an undo/redo stack.

Requires a thorough mapping of what user actions should and should not be refactored into Commands, and how the Commands are grouped.

Try porting the node graph to graphEdit

graphEdit offers additional features that may or may not be useful. Would likely require modification on the engine level to make conform to this program, so at that point we may just as well implement the wanted features directly.

Implement Command history console

Dependent on #28 and #36

A console view in the ui layer where executed commands can print messages. Serves as a visualisation for the undo/redo stack. Labels in the console can be clicked to revert to a particular point in the list of actions, perform the operation again or some other action.

Could perhaps be also implemented as a node which is by default pinned to the ui layer.

Keep a set amount of images in memory

Currently every image is fetched from the file system every time it's node is spawned. This slows down the changing of focal nodes when multiple images need to be loaded. Images aren't currently stored in memory at any resolution once their container node is deleted, so some sort of cache is needed here.

This could be implemented simply by adding a reference to each loaded image to an array of references. Because Godot uses reference counting, this should keep all the images in the array loaded. Have a maximum amount after which the oldest reference is popped from the array.

References should be added to the array upon despawning the host node.

Implement right-click menu on nodes

Implement a right-click menu for nodes.

Display each option alongside their keyboard shortcut, if it exists.

Delete / X
Duplicate / D

(for example)

This issue requires a different method for connecting wires between nodes. Right-click dragging won't do.

Finish implementation of pinning

The basics of pinning are already there, just need to be polished and finished.

Pinned nodes do not disappear.
Pinned nodes maintain standard zoom level.

There are quite a few preliminary UX decisions that need to be made. So finishing this feature may not be as simple as it seems.

Implement first operator node - Composite

Implement a node that composites two or more images on top of each other.

  • Offset for each layer
  • Stretch mode for each layer
  • Base resolution of resulting image taken from bottom layer

Add ability to search nodes

A keyboard shortcut opens the search bar and typing in search terms brings up nodes with matching texts and/or names. Brought up nodes can then be freely connected to nodes on the graph or among each other.

Refactor dataAccessInMemory to save and load resources instead of json.

  • Function to save nodes using resources
  • Function to load nodes using resources
  • Save and load a settings resource
  • Delete resource file and all connected edges when node is deleted
  • Save and load edges using resources
  • Save and load typeData as subresources within nodes
  • More frequent autosave

Implement Node: Properties

Implement a properties node which exposes properties and statistics of the currently selected node, excluding itself. When this node gets selected, it still displays the data of the node that was selected previous to it.

If multiple nodes are selected, display the data of the first one. This can be rethought in the future to, for example, display the properties that are shared between all selected nodes.

Has an input to which a node can be connected to always display the properties of that node. Also a button that "pins" the data, meaning it automatically creates the edge between the socket and the node. This feature requires the implementation of Sockets #38.

Implement Sockets

Implement Sockets, which would provide an alternate target for edges. Important for all nodes which can take in inputs and operate on data. Sockets would limit the amount of edges that can connect to them. They must have a parent node which connects them to some of their properties.

Consideration is needed for whether a Socket can be the source of an edge. Can Sockets also serve as outputs for specific data, or would that functionality be better suited as an optional filter inside an edge?

Open image in Krita

Implement an option to open an image from a node into Krita for editing. Saving the image automatically updates the node.

Implement stable diffusion node

Implement a node that uses stable diffusion (locally preferably, API also available) to generate an image based on a prompt. The prompt will be a text node that is fed in to the generator node.

The FocalPanel color isn't updating properly

Two bugs:

1: If there is no focal node and a newly created node is set to be the focal node, the color on its FocalPanel doesn't light up properly.
2: When setting another node to be the focalNode, the previous focalNodes FocalPanel color doesn't update.

Visual enhancements for wires

  • Add bezier curvature to wires (if feasible)
  • Control points to the ends of wires, next to connecting nodes. This is where you would connect from, connect to, and disconnect from.
  • Indicate direction of source to target

Separate undo stack for navigation

Since navigation between focal nodes is so important in Karta, there should be a way to quickly return to the previous focal node (and maybe camera position).

Navigating the node graph would use a separate action stack than when modifying it. And these two can be reversed and forwarded separately and at will.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.