Programmable Ink Lab Notes
title
Crosscut Extension Design Jam • Marcel's Notes
dated
Feb 2023
author
Marcel Goethals

Part of the Crosscut Extension Design Jam

Floatyness

One of the things that is perhaps hard to articulate but that I think feels good about crosscut is it’s ability to maintain constraints without “floatyness”. If I construct a midpoint line in crosscut, it’s behaviour is much more straightforward and predictable compared to the same construction in a relaxation based solver like Relax-overveld.

Relaxation always solves constraints globally, and so has a tendency to move points that (from the users perspective) it didn’t need to. Crosscuts ability (and limit) to solve-constraints locally, makes the behaviour much more legible. I think there is often a sort of mental “down-streamness” to certain constraints: the midpoint feels downstream to the two endpoints.

Algebraic Expressions

Crosscut meta-ink visualises dataflow, but it mostly feels quite tedious to construct graphs which are much more elegantly and legebly written as basic math expressions. I made some attempts at mapping algebraic expressions to crosscut style dataflows. There are I think two different approaches possible:

Marking values Constant

In the midpoint example, we could mark B as constant, or not changeable by the computer. In the sketch below B is marked with a yellow dot. This expression would map onto a well formed Crosscut dataflow. If we do not mark any value as Constant, we could fallback on using Split operators (see the section below).

Which side of the = sign a value is on would be semantically meaningful though. These two constructions would result in different data flow graphs

Moving values to the other side of an expression would rewire the data flow.

Moving values to the other side of an expression would rewire the data flow.

Expressions with (implicit dataflow markup)

An alternative approach is to mark expressions up with data flow. That is, visualise how change in one variable would impact the others. There are probably sensible defaults in most cases. But we could expose affordances in the case that the user would want to change that behaviour.

Interestingly, we could probably visualise that flow both on the expressions and as an overlay on the drawing itself. So this approach would play well with Monotonic downwards flow by default (see below). Note that this would still use crosscut style backwards flow, no need for global relaxation!

Split Operators

One of the limits of the crosscut model is that change can only ever be propagated into one output. Nodes have two inputs and one output (in both directions). But from a users perspective it might sometimes be useful to be able to “push change” into two directions. Split operators could be a way of doing that inside the crosscut model.

Split would need to use History to sensibly compute change.

If 7 changes to -> 6 We can compute the Δ to be 1.

Split the difference: 0.5 and push that change into A & B.

Running 3+4 = 7 Backwards using split [-]

Running 3+4 = 7 Backwards using split [-]

Introducing splits would allow for a version of a distance calculation that would push change into both points!

For addition/subtraction the implementation of split is trivial. But for other operators splitting change in a sensible way becomes increasingly complex. Though there is a naive way of doing this which involves always pushing every value equally.

For addition/subtraction the change is always a diagonal vector.

For addition/subtraction the change is always a diagonal vector.

For multiplication/division pushing that change is not so obvious.

For multiplication/division pushing that change is not so obvious.

We could implement a generic split wrapper around any function using an interative method.

Non-local information and Circularity

Another limit of the crosscut model is that, because of the way dataflow is explicitly encoded in the graph, it isn’t possible to solve constraints using non-local information. This significantly limits the types of things you can do. For example, it isn’t possible to solve two equations with two unknowns because this would introduce circularity.

There are a number of potential usecases that would require this capability, one example is a voltage divider. Which could be constructed in a similar way to thinglab.

I came up with two possible extensions to solving this problem within the framing of the crosscut evaluation model.

Propagating Guesses through the graph

For example, if we want to constrain a point to a circle, by constraining the length of a line:

We could have spreads propagate a guess through the graph. Once a guess hits a hard constraint that guess could backpropagate through the graph. This would essentially be a limited form of relaxation, that would maintain locality much more!

This approach is based on propagator networks.

Constructing Bridges

Alternatively, the fact that we need non-local information to solve some equations, means that we could pipe-in that information from across the graph using direct links. Effectively bridging a cycle. This is analogous to rewriting the equations in different terms, or constructing a new point of view.

In the example below, these equations work fine when calculating forwards, but because B is constrained by two inputs, we cannot push change backwards into it in the crosscutian model. Instead we can bridge the gap by constructing a path that takes into account that non local information. This idea is based on the Constraints paper by Steele & Sussman.

It’s worth noting though, that this wouldn’t solve the the point on a circle example above, because that requires some kind of minimisation behaviour.

Monotonic downwards flow by default

These ideas could point to an interesting direction which is that the whole model is represented by a spreadsheet style graph, that flows monotonically downwards by default. The graph grows in order of construction by the user. New facts are derived Monotonically from old facts. The user draws two points connected dy a line. The two lines intersect, so we can derive an intersection point. The intersection point is derived from the “Assumptions” given by the user.

Sometimes though, we might want to change our “reasoning”, because the position that the intersection ended up is different that we want it to be. So we bridge backwards and change our “Assumptions” based on where we want the conclusions to end up. That is, we temporarily turn a Result into a new Assumption, and an Assumption into a Result.

These “bridges” give a more detailled control than global relaxation, because we specify very clearly which values are allowed to be changed from the point of view of the source change.

This kind of flow requires the user input to always be completely “free variables” though. The case below would require some kind of additional system, like minimization on the user input that would keep the point on the line. @James Lindenbaum suggested that we could think of user inputs as just pushing around deltas but I’m not sure I understand how that would work in this case.

One way of doing spatial queries (like the example that @Alessandro Warth showed) in this model would be to introduce “stratification” in the flow. Queries only query results that appear downstream from that point, but ignore everything that appears upstream. (An element is always placed below its lowest dependency).

If we allow collections of values to pass through the wires, This model would also be a way of distinguishing what should be generated as multiples vs what should stay single. Something that we couldn’t figure out in Crosscut.

Draw lines between a single point and all the intersections

Draw lines between a single point and all the intersections

Draw lines originating at each intersection to a point with a certain offset.

Draw lines originating at each intersection to a point with a certain offset.

This approach has very similar semantics to the visual rule demo that I did a few weeks back!

Algebraic Geometry

I did some experiments on using Implicit Functions to render geometry. A big advantage of this would be that we would get a bunch of functionality almost “for free”:

There seem to be different rendering approaches possible:

  1. Rendering using Quadtrees & Marching Squares
  2. Dual Contouring:

Dual Contouring Tutorial

  1. Rendering on the GPU using a fragment shader:

Inigo Quilez

Meta-ink, Pointing to things

Having yellow lines feels very direct, in that it allows the user to “point” to the thing being referenced, it quickly turns into a hairy ball of mud.

A realistic system would probably use a number of different approaches in combination.

Most drawing apps highlight the relation between two elements on the screen highlighting both.

Most drawing apps highlight the relation between two elements on the screen highlighting both.

Tolerance to contradiction

James ponted out that one of the things that makes relaxation work well is that you can write constraints that are contradictory and that the system will just “deal with it’ rather than complaining. While dealing with contradiction wouldn’t flow naturally out of a crosscut like system, there are ways we could be tolerant to contradiction, by relaxing constraints.