title

Crosscut Extension Design Jam

dated

Feb 2023

author

Proposed by Alex Warth

Prompt & Questions

I think that it would be interesting for us to do a design jam on Crosscut-related ideas. Here are a few questions that we could use as prompts:

• What if Crosscut’s operators were multi-directional instead of bidirectional? E.g., the “add” operator only has forward and backward computation — the operand at the top is treated as a constant. What if this weren’t the case? How might we back-propagate a change that is coming in from the right? Should the user be able to specify which of the operands (top or left) would change? Or that they should both change? If so, how would that change the model of computation? Are there things that are impossible to do with the original Crosscut that this sort of thing would make possible? Does the added complexity pay for itself in terms of gained expressiveness?

• One way to get the multi-directional operators that I mentioned above might be to use relaxation for all computation. (Suspension of disbelief, please! Let’s assume that it’s possible to make this perform well enough to be interesting.) Some of the experiments that we did in my research group at HARC suggested that relaxation doesn’t work well when complex constraints are expressed as a combination of lower-level, primitive constraints. And as @Marcel Goethals pointed out recently, composing large expressions using Crosscut’s operators doesn’t feel great. But what if we could express complex constraints using a more natural language? (Like mathematical expressions/formulas?)

• In the original Crosscut, a user can create geometric constructions that have some interesting dynamic behavior, and then use multiple instances of these constructions in a bigger model. But what about the operators? Shouldn’t the user be able to create new ones? A “level-2” way to do this may be to use existing operators in the “implementation” of the new operator. Is this already possible in the original design? (All of the examples of reuse that I’ve seen include concrete lines, whereas what I’m talking about here is abstracting a network of “meta” components.)

  • (A “level 2 1/2” way to do this might be to write JS (or …) formulas for the forwards and backwards computations for whatever bricks you create. I like this idea, because it would make a lot of the primitive bricks not so primitive — “add”, “sub”, “mul”, “div”, etc. could all be written in a very user-friendly way.)

• I spent a few minutes thinking about how to build a Crosscut model to solve @James Lindenbaum’s “countersunk screws” problem. It didn’t seem like a trivial task, but I imagine that it might be, if only I had a better set of operators. So:

  • What are some new operators that would help me solve this problem?
  • Is it possible for me to build them in a level-2 way? If not, what are some changes/extensions to the language that would enable us to build them?
  • It may be necessary to add operators with more than 2 operands — e.g., one that computes the angle formed by 3 points A-B-C. Does it make sense to define such operators with only one “backwards” computation? (@Peter van Hardenberg and I had a chat about this and we think so, but it would be useful to make something that we can play with so we can see how it feels.)

• An operator that would be useful for the “countersunk screws” problem is which computes the distance between (left) and (top). But for this operator, computing in the backwards direction is highly ambiguous, because the new (left) could be any point along a circle. @Marcel Goethals told me that at some point you guys had an implementation in which the backwards computation would pick a solution whose x (or y) was equal to that of the (top) input. Another alternative would have been to choose a solution that’s along the line between the original (left) and (top). But any single solution is not likely to be what the user wants. So the question here is how can we make this work? Are there some modifications that we could make to Crosscut’s model of computation that would make it possible for us to provide a “dist” operator that is both useful and has predictable behavior?

• Suppose we add support for time traveling in Crosscut — i.e., the user can rewind to any point in time, and see their dynamic model as it existed then. Are there interesting opportunities that we could explore, e.g., synergies between this feature and Crosscut’s abstraction mechanism? (I rewind to a previous version, drag a part of my model into its own project/sketch, then go back to the “present” and add multiple instances of that guy into my model.)

James short list

• directionality of operators like “add”
• conveyance of multiple values/properties through the lines
• conveyance of values through concrete ink
• labels? other things besides points/lines?
• hand-drawn ink vs vectors
  • is this a small easy change or huge one?
  • seems like affordances for moving strokes around (some intent-preserving splining, etc) would be necessary

Marcel Goethals short list

Some open questions:

• How to deal with non-bijective operators (Like sine wave)
• How to deal with non-commutative operators: In Crosscut, “division” only works backwards with a constant denominator. — You’d either need a second division operator, or the ability to pick which input is constant.
• How to deal with the problem of cycles.
• How to do multiple values & ranges of values? — It’s easy to pass collections through the graph, but it’s not obvious how to get multiple values in and out.
  • We tried collections, spreads and repeaters, but didn’t find a satisfying answer.
• Should meta-ink be auto-layout?

Proposed studies:

• I’d like to try to do a serious prototype of Symbolic Math as a way to solve some of the issues described above.

  • See this example: Which works in Crosscut if 3 is constant, but breaks if you make it a variable because you introduce a cycle.
  • My intuition is that we can use Equality Saturation to rewrite cyclical graphs into valid Crosscut style graphs. In the example, the cycle is mathematically equivalent to basic multiplication, so we can trivially compute the answer without falling back on relaxation.

• What if instead of using 2d points and lines, we allow the user to pass functions into concrete ink? This could be a way of doing collections and ranges.

  • Let’s say you wire up a sine node to some concrete ink.
  • Then if the input is a single value, the concrete ink is a 2d point (like in Crosscut)
  • But if we introduce the notion of a free variable or a range, the concrete ink could “stretch” to plot the whole function, rather than only one position on the screen.
  • That might also be a way of implementing all geometric primitives like lines, curves and circles in “level 2”

• count is a kind of spatial query. What other spatial queries should we have? How should they work?

• images? labels? as objects (multiple properties that can be written to / read)

• ways to work with sets, map/reduce, iterators, etc.

  • hopefully w/o losing the ability to visualize the entire computation
  • converting string to numeric value may be a good example to think about

Post-Design Jam Notes

• Alex’s notes
• James’s Notes
• Marcel’s Notes