title

ink track update / thoughts on research direction

dated

March 29, 2023

We want a digital notebook that combines the best of hand-drawn sketching and note-taking with the power of the dynamic computing medium.

With Doodles, the Bullet Journal Studies, and more recently Atelier, we have worked on inking aesthetics, performance strategies, and intent-preserving deformation.

With Inkbase we gained conviction that such a tool would be great to have and to use. Properly designing the system was out of scope, so it had a poor inking experience, a poor programming experience, and no basic app features (erase, undo, notebooks, navigation, etc.).

Programming Model Explorations

Next we set out to find a good programming model. Having already explored constraint solvers with Rectoverse and preceding studies (unification & linear solving, relaxation-based, delta shipping, other unconstrained error minimization methods), we set out to look for programming models particularly suited to programming with a stylus on a tablet.

With Crosscut we found that constructing programs via drawing “circuits” on the canvas with meta-ink is a delightful experience. Encapsulation of a drawing where the internal logic is hidden so it can be composed into a larger drawing/program without “laminating” the contents feels super promising.

However:

• ceiling on what you could do was quite low
• operators have a weird handedness/directionality that makes behavior unintuitive and allow the user to get themselves into trouble
• encapsulation needs to be expanded to allow for cleaner explicit public/private component design
• need to be able to work with more/arbitrary properties

With Untangle we found that gestalt-style/example-based spatial queries are super powerful, and SAT/SMT solver assistance is super helpful for certain cases.

With all these different computational models in-hand, it was clear there is no one-size-fits-all answer; instead we need an architecture that allows multiple models to be composed together.

Recent Work

Habitat was originally conceived as an attempt to:

1. figure out how to compose different programming models/mechanics together
2. work through a cohesive design for the system
3. create a dogfoodable app

That turned out to be too much scope. Habitat proved that different computational models could be composed together, but there wasn’t time to be thoughtful about the architecture or design, or build any real app features.

Next Steps

Ultimately, we still want a digital notebook as described above, however:

• we are certain we aren’t yet close enough to make one in a single project — too many major questions and way too much scope
• we have done multiple projects where we punted on big questions and built an app anyway—we don’t feel can learn much from yet another project in that vein (e.g., inkbase again but with different programming model)
• we have done multiple projects where we focussed narrowly on a computational model but not the overall app design—we don’t feel we can learn more right now from yet another full-project computation model experiment (e.g. crosscut or untangle again but with yet another new programming model)

What we need to do now is a series of tightly scoped studies—short quick projects to answer open questions and start to converge toward a cohesive overall architecture and design.

If we can answer these key questions and figure out a cohesive design—both for the app in general, but especially for the inking and programming affordances and gesture space, then we could be ready to implement a usable app as a larger project.