Tuesday, June 15, 2010

Squeak Etoys on iPad

In preparation of making Etoys work on the recently announced OLPC tablet, I ported it to the iPad. Here is a video—read on below for some details:

This might look exciting, and it certainly is, but it feels actually a lot more clunky than it looks. You may have noticed the little target circles I added whenever a finger touches the screen. That's to know where exactly the finger hit. It's really hard to hit anything, took me a while of training to hit the colorful halo handle buttons on first try. We really need to redesign the user interface before giving this into the hands of kids ...

But for now, some technical details: John McIntosh ported the Squeak Virtual Machine to Apple's touch-based OS last year (source available at isqueak.org). I modified it slightly to enable multi-touch and keyboard input. Also, I rewrote his code to deal with touch events in Squeak, and added multi-touch handling to Morphic. Fortunately, Morphic was designed to handle multiple "hands" (pointing devices) from the beginning, so adding this was much easier than in a system that assumes there is only one mouse. That's why moving multiple objects at the same time, and painting with more than one finger, just works once the events are in the system.

So far this is just an early test. We should work on improving the Etoys touch UI in next year's release. The Sugar menu bar works fine, but everything else is way too small. At least we have the luxury of being able to test Etoys already—getting the rest of Sugar running on a touch device might take a while. Hopefully OLPC will have developer machines soonish. If this test has shown one thing, then that there is lots of work to do (and it may even be necessary to start over).


sebastianconcept said...

Isn't great to see how much adaptable squeak as technology is?

It can run in almost whatever that has bits in it :)

MrStevesScience said...

What about a Hand Magnifier to show where your finger is pointing? I've used this when playing with scripts. Even with a mouse on a large screen those little arrows are hard to hit.

Vanessa said...

Steve: doesn't work, because we only know where the finger is once you touch. Having no mouse-over events is one of the reasons we need to redesign the UI for touch.

Anonymous said...

What ar the steps to install Squeak into the iPad, is thre like an installable package? Is like any other MacOS package?

Vanessa said...

Yes, an iPad application is similar to a Mac OS one. However, an unmodifed iPad only runs signed apps (modifying it to run anything is called "jail breaking"). Applications are signed by Apple once they are accepted into the AppStore. Then anyone can download and run them. For testing, an app can also be signed by a registered developer. This will then only run on the devices of beta testers. That is how I ran it.

C. Scott Ananian said...

I've been thinking some about what a "Next Generation" Sugar (on a multitouch machine) would look like. Since you're the only person who seems to be thinking about multitouch, I wonder -- what if "Sugar II" was built in Squeak? Assuming y'all come up with a killer design for the object browser in a multitouch world, why not use that for everything?

What's missing? Well, the packaging people would likely prefer to have some form of "save as source" instead of writing out binary system images, but I think that's already been worked on to some degree. Perhaps it would be best if squeak grew some gobject-introspection support so that arbitrary GNOME libraries could be accessed easily. What else?


GerryJ said...

Re Steve's comment, how about presenting a zoomed box of the nearby screen rectangle so you can do a better finger point next time, with option to try again at bottom or top.

Unknown said...

Nice Work!

Two questions... well three:

1) Do you think that the iPad could be a viable OLPC/OTPC solution? Why, or why not?

2) Is the source for your port available or at least an ad hoc distribution.

3) I am retired, with 3 grandchildren (10, 11, 14), a background in computers and an iOS developer... can I help?

Vanessa said...

Dick: I think having a touch UI is essential for a good kids computer. A mouse or trackpad is no good substitute. Having a keyboard and stylus would be nice but I guess they are not essential. The tablet form factor has the largest chance of driving the price down quickly, since there are so few parts, and every machine can be the same (no custom keyboard). So OLPC is right in going in that direction. A sunlight-readable low-power display would be nice. Not sure if a sealed device like the iPad or an open, easily maintainable one is better. OLPC has focused on the latter.

The source is available, but not in readily buildable state. My VM changes have been merged into the official iOS port. My adhoc ipa is in the experimental Etoys downloads. You might also try the Etoys-Touch-Emu app there which should work on Mac, Windows, and Linux.

Help would be very welcome. I have had no time to work on this since May. A first step would be to get acquainted with Etoys and Squeak. At least as Objective-C developer you know the Smalltalk syntax already :) You should subscribe to the Etoys developers list and we can take it from there.