ai ui ei ei o

collectively we’ve probably spent many lifetimes of people’s time trying to figure out a good user interface for software. who would win: the apple HIG or one streaming chat boi? the latter is making a compelling case right now.

I’m generically concerned by this because of two primary reasons:

1. taste appears to be a very difficult property for language models to develop. there’s certainly some evidence that you can make something that has a strong visual identity (eg nano banana), but i still find models sense of taste to be lacking.

2. as model capability increases, I feel like they’ll drift further towards organization that doesn’t make a ton of sense to humans

the latter seems a bit fanciful, but I think it’s pretty likely. models can inspect their internal state to some extent, but trying to have it translate that into a visualization seems unlikely to be very fruitful. a big part of user interface design is about guiding people towards optimal outcomes in most cases, while not completely obscuring out-of-distribution use cases (within reason, based on underlying capability)

a particular example — public API design has a ton of inputs, but safety is a big one. not just safety for the end-user; you don’t want to make it easy for people to do the wrong thing, or break their shit, or whatever. you also need to encode safety for yourself. external callers don’t have observability of your system state, and your service’s SLA is now a bound on theirs. you need to prevent noisy neighbors, you need to ensure one large API consumer doesn’t interfere with everyone else.

there’s a lot of intentionality to both visual interface and programmatic interface is what im getting at

AI’s understanding of this is mixed, to be sure. it is foundationally unwise, even as it is incredibly intelligent. i fear this gap is going to get larger rather than smaller as model capability improves.

bonus

one thing that I haven’t seen a lot of people talk about is UI design for text models. i’m still pretty convinced that screenshots are a bad way for a model to interpret stuff at scale, and text representations are superior. i think it’d be really interesting to see some sort of standard data tag that contained a text-based summary of the data encoded in a UI component, primarily intended for LLM usage.