The demand for software engineers keeps growing. Over the next couple decades, the actual number of people who will need to learn programming, in various languages, will equal the entire sum of all previous programmers. That means: a huge number of people will be learning shell, c, python, r, assembly, javascript…

While the ‘first wave’ of programmers were those who found keyboard-shortcuts a time-saver, this ‘second wave’ will need to draw from the pool of visual thinkers, as well. A truly visual IDE would provide numerous advantages for that group, and it would aid comprehension and engagement with non-programmers.


The rest of your team trickles onto the work-chat-channel, and each of you reviews the docket. A few of them begin asking each other questions, their QA appearing in branches of the chat-history. You watch the right side of your screen, where whiteboard-thumbnails display some others’ wiring-drawings for different parts of the project.

You click on one programmer’s whiteboard-thumbnail, to zoom-in and watch them code live for a moment. They have been drawing boxy circles, with labels, and different kinds of arrows, also labeled, between them. The main functional blocks and flow-control are taking shape. You settle your cursor on one of the arrows, to see a pop-up box with your coworker’s comments about that arrow’s meaning and purpose. You right-click, and go to — the video-scroll at the bottom of your screen leaps to the point where your coworker wrote the comment. You can start filling-in the details, there.

You zoom into the arrow and the two boxy circles it joins. You say “yes” to the prompt, before it finishes asking you if you are starting a new branch. You scribble your branch name atop your whiteboard-view, while explaining what you are doing: “These are the internal operations that pass outputs, their outputs’ types, conditions for their use, and the functions receiving that data.” Inside the outputting circle, you lay smaller circles, naming them out loud as you go along. You label the data types, and draw conditionals’ arrows among them. You finish wiring outputs, writing comments, linking mentions of functions in those comments to the functions themselves, for clarity, and responding to the prompts that ask you about garbage collection and tests. You click done, to push the branch.

You move to another coworker’s whiteboard. They are taking a list-display of input files, and operating on them manually, creating action-samples to construct a filter. After manually filtering a few files, they highlight the action-sample snippet of one of those operations on their video-scroll, and click generalize. You hear their translator-bot explain the generalization they are performing, as they label and generalize each file’s qualities. “Like this file, but generalize to any capitalized name preceding the dot.” “Like this file, and only within directories named in this other file…”

You scroll back in their video, to their manual operations, click new branch, and leave a flag. You name the flag “an example that may break your generalization, here.” You link ‘here’ to the point in their video where they mentioned ‘filtering within directories’, and write a comment on it. Skipping back to the list-display of inputs, you write an example filename, and draw its parent directories. You explain out loud, while drawing a box with the source filename, “If your directory was added to the source file’s list, but its parent directory was also added…” Your coworker must have seen the flag, and watched as you were doing all this — their message appears in the corner of your screen: “Ah, thanks — generalizing that now!” You relax back, and click ghost branch, with a link to their message as your final comment.

Why Bother?

Emacs saves you precious seconds, according to its tutorial, by providing keyboard shortcuts for everything. This, apparently, keeps your hands away from the mouse, which would waste time. Yet, shell manual pages are organized alphabetically, rather than by operation, so you must scroll through and read each one, to find the option you seek. (If I know the meaning I want, and I need to find the options that have that meaning, I should look in a thesaurus, not a dictionary.) There is a disconnect, here.

Computers should do what they are good at, so that humans can do what we are good at. A computer could automatically generate the fields of syntax for a command, so that you don’t have to spend a few minutes reviewing the man page. Computers are good at that kind of thing. As soon as you type ‘grep’, your IDE could generate ghostly fields to the right of your cursor, each labeled according to grep’s syntax: ‘options’ ‘pattern’ ‘patterns from file’ ‘applied to files…’. A right-click on the ‘options’ field would display all the options available to grep, arranged under headers declaring their function, like in a thesaurus. That would save a lot more time than keyboard shortcuts.

Additionally, a visual IDE could rely on a palette of commands, so that functions are laid within code using a drag-and-drop. In shell, you could highlight a snippet of code, and drag it to an empty palette box, to create an alias. Drag a copy of it from the palette, into another command’s syntax, to say ‘the output of this aliased command is fed as input to this syntax field’. People who are familiar with mouse+keyboard operations, from games, photo and video editors, would have an environment adapted to their style. That’s why Mac OS is supposed to be superior, right? [:

Most People are Visual Thinkers

Most of the people needed for our ‘second wave’ of programmers will be visual thinkers. And, with a visual IDE, code would benefit from the strengths of visual thinkers. If a visual IDE lets a user drag outputs to their inputs, typo-bugs disappear. (You could script “If this, then that, else this-other,” and leave the inputs vague for a while. Later, you drag the appropriate section’s output into the appropriate field, telling your IDE what this, that, and this-other are… If a function is re-named, its drag-and-drop wiring is preserved!) Scripting layout and comments would be unified at the IDE-level, and then matched to the respective language by the IDE itself. (Commenting with /* or ## becomes irrelevant — and should be irrelevant! That’s something the computer ought to fit to your work, while you declare comments in a comment-box, like a sticky note…) Bugs related to flow, dependencies, garbage collection, and namespaces would be easy to catch.

Chris Granger’s light table, and Bret Victor’s talk “the Future of Programming”, are early steps toward this visual IDE. Festo, the german robot manufacturer known for its biology-inspired designs, relies upon the same principle. (at 3:00min onward, in that video.) There are games for learning to code, and interfaces that let you write simple instructions with a drag-and-drop palette, but none of these is sufficient for visual thinkers to write production-quality software.

Whoever develops a visual IDE, opens programming to a swath of new designers. And, with a visual interface, non-programmers would be more able to see and understand what code is doing. In stead of discovering too late that their client wants something else to happen, programmers could avoid costly and time-consuming misunderstandings, by displaying the code as whiteboarded wirings, and generating visual demonstrations of that code in operation. Time saved with keyboard shortcuts pales in comparison to time lost reading dysfunctional manual pages, and correcting simple bugs. If you are interested, I’d be glad to work with you!

Easily distracted mathematician