Dr. Bert Freudenberg

Squeak Etoys on ARM-based OLPC XO-1.75

2011-08-29T17:25:00.004+02:00

First post this year, yikes! The last one was about ESUG 2010 in Barcelona, now I just returned from ESUG 2011 in Edinburgh. While I was there, a package with the shiny new XO-1.75 prototype arrived.

Incredibly, the pre-installed Etoys simply worked! Never mind the change in processor architecture, the Fedora folks have done a great job compiling the Squeak VM for ARM and so Etoys just works. Of course that's just as it should be, but it's still awesome. And e.g. Squeakland's own Etoys-To-Go would not have worked, as it only includes binaries for Intel-compatible processors.

Another great addition is a 3-axis accelerometer. The Linux kernel's driver exposes it as a file at /sys/devices/platform/lis3lv02d/position. Gotta love the unix design of exposing devices as files. All it took to make this usable from an Etoys project was just an object with ax, ay, and az variables that get set with one simple textual script:

Another simple script can use this to control a ball (the "rebound" script just keeps it on-screen):

Fun all around—it's a bit a hard to see the yellow ball in the Video, but Jakob enjoys it anyway:

Also, uploading from Etoys directly to Squeakland using Wifi just worked. Yay!

Update: If you want to try my uploaded project on your XO-1.75, you need to save it once from Etoys, quit Etoys, and run it again. Otherwise it won't work - it was signed by my key so the Etoys security sandbox prevents it from opening the accelerometer device. The saved copy will be signed using your key so no sandboxing happens.

ESUG 2010 in Barcelona

2010-09-20T19:21:00.002+02:00

This year's conference logo was designed by my good friend Patty Gadegast.

I just returned from the European Smalltalk User Group conference in Barcelona, Spain. It was a really nice experience. There was too much going on to report everything here, so I will just pick some favorites.

Photo by Bert Freudenberg

The event was hosted by citilab Cornellà. It started off with a Camp Smalltalk over the weekend. I already met quite a few people there. I couldn't mingle as much as I hoped to because I had to get the first Etoys 4.1 release candidate out of the door:

Photo by Adriaan van Os

Close by was "Yokohama Wok", a Japanese/Spanish restaurant with the best all-you-can-eat buffet imaginable. You could have everything from freshly cut ham to sushi, grilled steak or seafood, bread, pasta, rice, fruits, cake, desserts.

Photo by Bert Freudenberg

I talked to Stef (president of ESUG) and gave him a Squeak Etoys button, which he ended up wearing the whole week:

Photo by Bert Freudenberg

We also played together in a 2-on-2 Magic game (which we promptly lost ...):

Photo by Bert Freudenberg

On Monday I gave my Squeak Community Update talk, outlining what has happened in the Squeak and Etoys communities lately. Got some nice comments afterwards, including the request to give an Etoys demo the next time. I of course used Etoys to give the presentation, but did not really include an Etoys introduction for people who had not seen it before. But I got a slot in the "show us your projects" session on Tuesday where I made up for that with a 10 minute demo.

Photo by Adriaan van Os

Gonzalo Zabala and his students from Argentina presented Physical Etoys:

Photo by Adriaan van Os

I also liked the Xtreams presentation by Martin Kobetic:

Photo by Adriaan van Os

I was session chair on Wednesday morning, so I could see Travis' update on Pango text rendering from the first row. Would love to have that in Squeak, but it only builds easily on Linux:

Photo by Adriaan van Os

But the most exciting thing on Wednesday was of course that Physical Etoys won the ESUG Innovation Technology Award:

Photo by Adriaan van Os

On Thursday, I participated in a panel discussion about open-source licenses, organized by Julian Fitzell and Jason Ayers of Cincom.

Photo by Adriaan van Os

In the projects session, Ricardo demoed some of his Etoys work done during Google Summer of Code:

Photo by Adriaan van Os

Besides showing his graphing tools, the comic-like bubbles were a favorite with the audience:

Photo by Adriaan van Os

Dale showed the beginnings of Bibliocello, a repository for Monticello packages that can actually analyze them. You get to search implementors and senders across all packages, take statistics etc.

Photo by Adriaan van Os

And at the end of the day, an exciting demo was given by HwaJong Oh, a Smalltalker and iPhone developer from Korea. He demonstrated Drag-and-Drop for Squeak Tools, e.g. dragging the object held in an instance variable directly to another inspector.

Photo by Adriaan van Os

He also used cool animated mind-maps for his introduction:

Photo by Adriaan van Os

The highlight on Friday was Lukas' Helvetia presentation. I particularly liked the integration of PetitParser with the Smalltalk tools.

Photo by Adriaan van Os

All in all it was a rather refreshing conference at a great location with interesting people. Looking forward to next year's ESUG :)

Squeak Etoys on iPad

2010-06-15T17:12:00.011+02:00

In preparation of making Etoys work on the recently announced OLPC tablet, I ported it to the iPad. Here is a video—read on below for some details:

This might look exciting, and it certainly is, but it feels actually a lot more clunky than it looks. You may have noticed the little target circles I added whenever a finger touches the screen. That's to know where exactly the finger hit. It's really hard to hit anything, took me a while of training to hit the colorful halo handle buttons on first try. We really need to redesign the user interface before giving this into the hands of kids ...

But for now, some technical details: John McIntosh ported the Squeak Virtual Machine to Apple's touch-based OS last year (source available at isqueak.org). I modified it slightly to enable multi-touch and keyboard input. Also, I rewrote his code to deal with touch events in Squeak, and added multi-touch handling to Morphic. Fortunately, Morphic was designed to handle multiple "hands" (pointing devices) from the beginning, so adding this was much easier than in a system that assumes there is only one mouse. That's why moving multiple objects at the same time, and painting with more than one finger, just works once the events are in the system.

So far this is just an early test. We should work on improving the Etoys touch UI in next year's release. The Sugar menu bar works fine, but everything else is way too small. At least we have the luxury of being able to test Etoys already—getting the rest of Sugar running on a touch device might take a while. Hopefully OLPC will have developer machines soonish. If this test has shown one thing, then that there is lots of work to do (and it may even be necessary to start over).

How to write a Sugar activity in Squeak/Etoys

2010-01-04T13:27:00.002+01:00

Milan just started a blog describing how to write activities in Squeak, see his first post. There are already a few of these, most notably the games by Robert Hirschfeld's students at the University of Potsdam, Germany. But so far the process is only documented in mailing list posts. Way to go, Milan!

Peaceful Holidays ...

2008-12-22T15:43:00.012+01:00

... and a Happy 2009 to everyone from me and my family.

Presented by my XO-hugging USB men (who seem to have more memory capacity than me at times).

Featuring a seriously tree-infested Etoys project, with snow (for those living south who cannot enjoy the white fluff this time of the year).

And see you at 25C3 I hope!

Emulating the latest stable OLPC XO software

2008-12-11T00:04:00.014+01:00

Even with XO laptops readily available now there are quite a lot of reasons why one would want to emulate it on another machine. One being to hook up a projector. Unfortunately there are quite a number of hoops (*) one has to jump through to make it work.

Anyway, I made a virtual machine that allows me to emulate the XO in VMWare on my Mac, running Sugar in the XO's native 1200x900 resolution, scaled down to a nice physical size in a window on my regular screen (fullscreen works, too). Sound works (even Tam Tam), Browse works (so networking is good), and after setting a working Jabber server I do see other XOs in the neighborhood view (Chat worked fine). Camera and mic are half working (Measure crashes, Record shows blank picture, but reportedly does record video), and a "Sugar restart" does not actually restart Sugar, but apart from that it seems fully functional, and much nicer than the emulations I had used to date.

Click to see actual screenshots (calibrated to match the XO's physical extent using the Ruler activity on my MBP's 110 ppi screen):

And here you can get that virtual machine (665 MB, 2 GB unzipped): VMWare-8.2-767-bf.zip

I made this using VMWare Fusion, which I found to be much better at running Linux clients than Parallels Desktop (I had been using that for 2 years). Give it a try, it's free as in beer for 30 days. No, I don't get paid if you buy it.

Update: Reportedly it does work in VMWare Player on Windows and Linux, too (see comments). And maybe someone can make an appliance for even easier use?

(*) Now to the hoops:

I started with the 767/ext3 image from http://download.laptop.org/xo-1/os/official/
extended to 2 GB by appending /dev/zero
(jffs2 compression gives roughly 2 GB too)
enlarged the partition to full 2 GB
(using fdisk and ext2resize)
mounted that in a Fedora 10 virtual machine
copied over the F10 kernel, initrd, and modules
(olpc kernel wanted AMD instructions)
edited grub.conf to use that kernel
and appended a root=/dev/sda1 kernel arg
(the fedora kernel wants to use LVM otherwise)
unmounted
created new virtual machine
(that disk, 1 CPU, 256 MB RAM, NAT networking)
booted into that new system
installed Perl
(for vmware tools installer)
installed vmware tools
(to get the X driver)
(but none of the kernel modules, would need make/gcc/etc.)
deleted Perl
(to restore the default sw environment)
copied the existing xorg-vmware.conf to xorg.conf
(to get 1200x900 resolution w/ 200 dpi)
booted into Sugar
(looks really nice so scaled down)
installed activities
(took a long time, maybe it's my DSL)
tested a bit
rm -r ~olpc/.sugar
(to remove my personal data)
should have deleted sshd host keys, too, but didn't
shut down
zip
upload
...
...
...
still no profit? ;)

Enjoy.

European grassroots meeting

2008-12-05T22:42:00.005+01:00

I went to Brussels last weekend, meeting with folks from OLPC Europe, OLPC Boston, and some other European grassroots (me representing OLPC Germany e.V.). We mostly discussed how to implement the new Give-1-Get-1 program as well as Give-many here in Europe. Some notes are available.

And, while sitting together at a nice Cafe on Sunday I made this little Etoys project to celebrate OLPC Europe. Just click here if you are on an XO or have the Squeakland plugin installed (as everyone should of course). For those who were there and took the huge file home on their USB drives - this is a fixed version that does not keep growing indefinitely while animating.

For the Etoyers out there - if you put a copy of a player in one of its own variables repeatedly, this creates an evergrowing "linked list" of player copies. Bad idea. Reset the variable first (by assigning dot for example) then do the copy: see my example.

OLPC review now online, English too

2007-04-20T15:30:00.000+02:00

The extensive OLPC article by c't magazine is now available online in both original German and an English translation. It's a thorough review (they had a B1 machine) with some interesting photographs in it, like the Squeak Etoys screen in reflective and backlight modes with microscopic images revealing the working of the XO's incredible LCD.

Croquet SDK 1.0 released

2007-03-27T16:58:00.000+02:00

Get it while it's hot from the equally new Croquet Consortium web site.

In-depth review of XO in German c't magazine

2007-03-20T11:36:00.000+01:00

There's a glowing review of the OLPC project and its XO machine in the current issue 07/2007 of c't magazine. The in-depth article by Dr. Jürgen Rink describes the project's history and educational ambitions as well as its current prototype hardware and software. One very interesting detail is a comparison of the XO's novel dual-mode display in low light and bright sun light, at normal size and magnified:
On the left, under indoor lighting, the colored backlight shines through holes in the reflective layer. On the right, when brightly lit outdoors, the reflection is so strong that the backlight is not even visible anymore, thus creating a gray-scale image. The photographs show one of the example Etoys projects.

The magazine is available now at kiosks until next week, or via mail order. In a few weeks the article should be available online via click&buy.

Croquet for Business: Qwaq Forums

2007-03-14T11:03:00.000+01:00

So Qwaq came out of "stealth-mode" and reveiled what they have been working on for a while now, Qwaq Forums:

Qwaq Forums, the company's first product, is a secure virtual workspace application that significantly increases the productivity of distributed teams by bringing critical resources together in virtual places, as if they were in an actual physical location. A highly interactive and persistent environment, Qwaq Forums enables users to work, collaborate with others, and identify and solve problems.

And I'm proud to say I contributed a little, which most probably will find its way into the next Croquet release.

Update: Here's a few nice stories of fellow bloggers who have seen Forums already.

From Steve Borsch's Connecting The Dots:

Qwaq will get traction only because they completely understand that giving someone a semi-trailer truck (i.e., an engine like There or Second Life) doesn't do much good if the person has a small garage and needs a vehicle to go get groceries and tool around (90% of collaborators). Qwaq Forums is a powerful, easy to use and navigate, co-creation space that the rest of us can use.

Glyn Moody writes on Open dot dot dot:

One of the benefits of using Croquet as the basis of its products is that the protocols are open, and this allows Croquet-compatible products to interoperate with Qwaq's. This means that the dynamics of the Croquet ecosystem are similar to that of the Web, which is never a bad thing.

And in The Culture of Collaboration, Evan Rosen writes:

Unlike most traditional web conferencing which works only while a session is underway, Qwaq Forums is persistent. This means authorized users can access the virtual space any time. Team members in another time zone may wake up to find the results of real-time collaboration that occurred while they were sleeping.

Interactive OLPC XO Display Simulation

2007-03-04T20:14:00.001+01:00

Many people still have not seen the innovative display of the OLPC project's "XO" laptop. It has twice the resolution of a regular LCD (200 dpi), and works in bright daylight in gray-scale reflective mode. It's impossible for me to increase your screen's resolution by software, and I cannot make your display reflective, but here is an interactive simulation of the backlight mode with its interesting color pattern. This pattern is the source of a lot of confusion about the "color resolution" of the display. The LCD has 1200x900 square pixels, but the backlight puts a full color through each pixel. It is not made of red, green, and blue sub-pixels like a regular LCD, but the first pixel is full red, the second green, the third blue, and so on. The DCON chip (Display CONtroller) selects the color components from the full-color frame buffer.

My simulation of the DCON achieves the same effect by selecting either the red, green, or blue color component in each pixel. Just move the mouse pointer around to see how different colors are reproduced. You'll notice strong diagonal patterns, but remember, on the actual display the pixels are only half as large. Note that the actual DCON optionally applies a bit of anti-aliasing in hardware which is not simulated here. It helps reproducing fine structures and depicts colors more accurately. Additionally, the simulation shows a magnified image to better illustrate the principle, but it is not accurate because the reflective area of each pixel is not depicted. Maybe I can add this in a later version.

I made the simulation using Squeak / Etoys, which is one of the programming environments on the OLPC machine, but also works on Windows, Mac OS X, Linux, and many more systems. If you run the simulation on the actual laptop (download the project, place it in /home/olpc/.sugar/default/etoys/MyEtoys, run Etoys, choose Load Project), then you should close the small simulated screen and just leave the magnified view open.

For the interactive simulation, download Squeak (this version installs both, a regular application and a browser plugin), then click here to run the simulation in your browser, or download the project file, launch Squeak, and drop the project into it.

Intel-Mac users beware, the plugin is not supported directly yet. To see the project in Safari, you have to quit Safari, set it to open in Rosetta (select Safari in the finder, press Cmd-i), and reopen. Or, use the download method, Squeak itself is running fine on Intel Mac, it's just the browser plugin that's making problems.

OpenGL in a Workspace

2007-02-16T10:51:00.000+01:00

On some modern Linux systems, Croquet does not work anymore because OpenGL failes to initialize. Now, I originally wrote that code, and it worked fine for years. So it can't possibly be buggy, right? Jens Lincke of impara tracked it down to the "Composite" extension that is enabled by default nowadays. With Composite disabled, it works, enable it, and it does not.

So I turned to NVIDIA for help, thinking their driver might be buggy. Had to give them an easy way to reproduce the problem, this is the snippet I came up with:

| ogl green |
ogl := OpenGL newIn: (0@0 extent: 100@100).
green := 1.
[[
        ogl glClearColor(0, green, 0, 1).
        ogl glClear(16r4000).
        ogl swapBuffers.
        Sensor waitClickButton.
        green := 1 - green.
] repeat] ensure: [ogl destroy]

Beauty, eh? ;-) I guess nobody has done this in a workspace for a long time. Stop it with Alt-.

Anyway, NVIDIA could reproduce the problem, and found our bug:

[...] the app is trying to create a depth 24 child window of a depth 32 parent and the app specifies neither a border pixel nor a border pixmap.

Doh! I forgot to specify the border! We were just lucky that this did not happen before. Jens and y.t. made a patch, should be in the next VM. And big thanks to NVIDIA developer support!

Stripped XO

2007-01-30T23:13:00.000+01:00

Early in January, my laptop was the star of a photo shooting for the German issue of MIT's Technology Review. We took off the plastic enclosure of the "brick", it was pretty interesting, for example to see how the whole display in its metallic housing is carefully held by rubber mounts.

This photograph was published in the magazin's current issue (02/07), along with a shot of the main board (which sits behind the display). The accompanying text not only provided a description of the parts, but also highlighted some design decisions that makes it unique hardware-wise. OLPC's educational goals were already reported on in the previous issue.

Image courtesy of http://www.heise.de/tr/magazin/

The colors are off for some reason after uploading to blogger - they were fine on my disk. Sorry.

Etoys kid-tested on XO

2007-01-21T23:22:00.001+01:00

I brought my green machine home this weekend, and my twins had fun with it. Enormous fun in fact for the two 7-year olds, pounding on TamTam furiously. I couldn't bear it anymore after half an hour or so.

Instead, I showed Jakob how to make a little figure bounce around on the screen in Etoys, while his sister went to practice her cello. He painted a simple head, and then we used the "forward by" and "bounce" tiles in a tiny two-line script making it move around. I made the mistake of pointing out that the "bounce" tile can produce some noise when bouncing. Endless fun trying the different noises ensued. Oh well.

Disturbed in her practice by these noises, Sophie came over and wanted to paint, too. So we saved Jakob's project and started a new one for her. I sat back to work on my email and let her brother teach. She spend like half an hour just painting the figure. The paint tool showed that it is not tuned to the XO's display resolution yet, it's far too small. But not giving up that easily, Sophie was erasing and repainting it over and over until she was satisfied with her "cow girl". Then Jakob proudly told her how to let it move and bounce, he had rembered almost everything needed. Together they quickly made it work, and just started exploring the noise-making possibilities again when we were saved by the call to dinner ...

OLPC talk at design school

2007-01-12T15:26:00.001+01:00

I gave a talk about the $100-laptop at the Magdeburg school of Industrial Design. We did some very inspiring projects using Squeak, Etoys, and Croquet together before. The designers always come up with interesting ideas, even though not everything is directly implementable by us developers.

Carola Zwick, dean of the school, wrote a book Designing for Small Screens that certainly gives valuable insight for OLPC developers, and she provided (though indirectly) some very important infrastructure for the OLPC office: her group designed the chairs they are sitting on. I got the actual invitation by Christine Strothotte, who got her PhD doing computer graphics in Smalltalk just a few years before I got mine from the same school. She's teaching interaction design nowadays. I'm looking forward to doing an OLPC-related project with these great folks.

A student took some photographs during the talk. Also, from his blog post it seems I convinced him of the merits of the OLPC project (it was a lively discussion). Thanks for posting, Cheng!

Sophie, Tweak on the OLPC laptop

2007-01-11T15:22:00.001+01:00

I just installed Sophie on my green machine. Sophie is a project of the Institute for the Future of the Book, is implemented in Squeak (just like my Etoys activity on the laptop) using Tweak as its UI framework (which is the original topic of my blog). Tweak is also the base for the next-gen Etoys.

Installation went pretty smooth. I downloaded the cross-platform zip file using the Web activity from Sugar
and unpacked it using the command line. The first start of Sophie failed, but after replacing the failing plugin with one from the pre-installed Squeak it started and worked. Yay!

This is an excellent example why it's a good idea to have a regular X11 installation on the kid's laptop: a lot of software will just work, even if it is not correctly integrated into the Sugar UI.

Michael Rüger of impara (a Squeak shop leading Sophie development here in Magdeburg, Germany) came over and made a little book, downloading two logos directly from the web (Sophie can do that!), adding a bit of text and color ... Tweak performance is not exactly blazing on the XO machine, I think we made the right decision to not use the Tweak-based Etoys but stick to the proven Morphic-based one. Of course one could optimize it a lot, but who has time for that? Anyway, it was useable - click the image to get a larger view:

OLPC, graphics, and more

2006-12-19T13:41:00.000+01:00

I just returned from a nice productive visit to California, where I brought back my very own $100-laptop. Got the camera working in Etoys (see picture on the right, showing Yoshiki and me) with just a few patches to Diego's code.

We also went to visit Keith Packard and Carl Worth of Cairo fame in Portland. Had a nice chat about Twin and then hacked away on a Pepsi version of image compositing operators. A very interesting meeting it was, with the old folks (Alan Kay, Jim Gettys) and the younger ones all sharing their thoughts.

And back to OLPC, read this nice essay titled Free Laptops: Creating, Producing and Sharing a Revolution.

OLPC video from NECC

2006-10-22T18:05:00.000+02:00

Just found a nice video on youtube about the $100 laptop at NECC 06, including a cameo of our very own Kim Rose (at 3:21) while the speaker is mentioning "creativity" - very apt, I'd say.

Performance Profiling

2006-09-29T10:49:00.000+02:00

Andreas Raab wrote a nice primer on profiling in Squeak. MessageTally might be the single most undervalued Squeak performance tool - a mandatory read for every serious developer.

Squeak for every child

2006-09-11T23:44:00.001+02:00

Lately I work on Squeak integration in the One Laptop Per Child (OLPC) project, perhaps better known as the "$100 laptop". The whole etoys group came over to OLPC's office in Cambridge. Squeak looks surprisingly well on the display prototype, and also etoys are reasonably fast. Ian Piumarta took some nice pictures, which might very well be the first photos of the actual display in the wild.

Croquet SDK 1.0 Beta out

2006-04-20T21:37:00.000+02:00

Get it at opencroquet.org!

Be a magician

2006-04-07T22:36:00.000+02:00

A very nice post about the Magic of Smalltalk.

Smalltalk Party Brussels 11 March 2006

2006-03-05T12:35:00.000+01:00

Only six days left to the Smalltalk Party in Brussels! I'll be there, and more interesting people, too. You should come, too!

Sophie movies

2006-03-01T20:57:00.000+01:00

Steve is posting screen clips of progress in Sophie (a.k.a. the Future of the Book), which is developed in Squeak with a Tweak UI. It's looking nicer and nicer every week.

Squeak Elections

2006-02-12T01:04:00.000+01:00

The first-ever elections are held in the Squeak community. Candidates have to be nominated by February 17, same deadline for getting certified at SqP if you want to vote.

Croquet & Tweak at NASA

2006-01-25T12:23:00.000+01:00

Ted Shab posted a job offer to the ECOOP list. Part of it was this:

We are currently building a [...] testbed for user-experience exploration in Squeak (Smalltalk) (really). The resulting framework will be used by the various NASA centers [...] to build distributed, multi-mission systems for planning and executing a variety of NASA missions, including robotic (e.g. Mars rovers & deep- space probes) and manned (e.g. the new Moon/Mars exploration effort, including the Crew Exploration Vehicle currently being designed).

Andreas Raab commented:

FWIW, this project uses Croquet and Tweak. If you ever wanted to work with a bunch of really sharp guys this is definitely a place to consider.

Sounds cool :)

Simplified TeaTime

2005-12-12T15:05:00.000+01:00

The next Croquet release, code-named "Hedgehog", will be centered around a real replicated object model, rather than the ad-hoc meta sends in Jasmine. See Andreas' and David's OOPSLA presentation (PDF) for an overview.

Tweak Interview

2005-10-19T19:03:00.000+02:00

Andreas gave an interesting interview about Tweak.

Easy as Pie

2005-08-15T14:21:00.000+02:00

Children "discover" Croquet's collaboration model called TeaTime.

Connecting fields

2005-05-12T15:13:00.000+02:00

Quick Recipe

To connect field a in obj1 to field b in obj2, use this:

obj2 startScript: #b: when: {obj1. #aChanged}

Now for the whole story ...

Problem

A colleague of mine wanted to make a drop-down list, where the options are not just set once, but provided and updated by the application. So, of course, when the options in the application changes, the items of the list widget have to be set to this new value. Nothing easier than that, just write a handler:

onOptionsChanged
<on: optionsChanged in: app>
listWidget items: app options

HOWEVER, he wanted to build this programmatically, not using a separate method. So, he easily came up with the following:

listWidget startScript : #items: withArguments: {app options} when: {app. #optionsChanged}

HOWEVER, this does not work as intended because the arguments to the script are evaluated only once, rather than every time the script is triggered. Well, this is what blocks are for, right? So this indeed works as intended:

listWidget startScript: [listWidget items: app options] when: {app. #optionsChanged}

HOWEVER, using blocks as long-lived scripts is discouraged. They're hard to identify in inspectors, hard to debug, etc. Alas, there seems to be no easy way around them. Or is there?

Solution

I've seen this problem a few times now, and the solution is so simple that I keep forgetting about it (which is why I spell it out here). This is how to wire the two fields:

listWidget startScript: #items: when: {app. #optionsChanged}

Doh! Where are the arguments? Well, the current value of a field is actually a parameter of the field change event (the previous value is the second one). Most of the time we just ignore it, since it's easy to get at the current value, but nevertheless, it's there. So, when #items: is triggered by the change event, its argument is the current value of the changed field, options. (This, btw, is a difference between #startScript: and #perform:, script arguments are optional, whereas method arguments are mandatory).

Of course, you can use the same technique in a regular method:

onOptionsChanged: newOptions
<on: optionsChanged in: app>
listWidget items: newOptions

But the earlier version at the top seems a bit more readable to me.

Scripts in Croquet

2005-04-21T00:46:00.000+02:00

Tao asked for a "Tweak & Croquet" tutorial. I don't have time right now to actually write one, but here's some sample code anyway.

Using scripts is easy and useful, even without the Tweak GUI. Just use #startScript: to run some method as a script. Inside a script, you can use loops and anything you like, just throw in a wait to account for time. Like, to animate the color of a frame, you could use this method (just add it to your TeapotMorph):

animateColorFor: aFrame
    [
        0 to: 360 do: [:hue |
            aFrame material color: (Color h: hue s: 1.0 v: 1.0).
            self wait: 0.01]
    ] repeat

This changes the color every 10 ms, and you can start it from the initializeDefaultSpace method:

    self startScript: #animateColorFor: withArguments: {someFrame}.

Here is something that does not loop forever, but finishes after one cycle:

jump: aFrame
    | v g |
    v := 0@1@0.
    g := 0@-0.05@0.
    [aFrame translation y >= 0] whileTrue: [
        aFrame translation: aFrame translation + v.
        v := v + g.
        self waitTick.
    ].
    aFrame translation: 0@0@0

Here we wait for the end of the frame - waiting is essential, because we do want to change the position only once in a frame. This can be run in response to a pointer click like this:

    self startScript: #jump: withArguments: {someFrame} when: {someFrame. #pointerDown}.

Or, if you wish, you could make a Tweak button for it. Here's the whole initializeDefaultSpace method doing it all:

initializeDefaultSpace
    | space cube |
    space := TSpace new.
    space addChild:  TLight new.
    self makeFloor: space fileName:'floor.BMP'.

    cube := TCube new.
    cube material: TMaterial new.
    space addChild: cube.

    self startScript: #animateColorFor: withArguments: {cube}.

    self startScript: #jump: withArguments: {cube} when: {cube. #pointerDown}.

    self initializeTweakWorld: [
        | button |
        button := CButton new.
        button label: 'Jump'.
        button openAt: 100@100.
        self startScript: #jump: withArguments: {cube} when: {button. #fire}.
    ].

    ^space

Hope that helps ...

Tweak Tutorial

2005-03-13T19:06:00.000+01:00

Andreas posted a BankAccount and ATM tutorial. This nicely demonstrates some of the basic Tweak concepts such as fields, events, triggers, and handlers, as well as introducing UI aspects like players, costumes, updating etc.

Print-Quality Screenshots

2005-03-07T23:19:00.001+01:00

For high-quality prints you need high-quality screenshots. This means very high resolution, and nice anti-aliasing. Just grabbing the screen produces rather unpleasant results (screenshot, 80 KB, 800x600 pixels). With normal OpenGL rendering you get rarely more than screen resolution, and anti-aliasing quality very much depends on your graphics board. So what to do?

Tiled Rendering comes to the rescue. Instead of rendering the whole image at once, we render smaller portions of the scene, and then arrange the tiles into a large picture. However, just pointing the camera at each tile will not work as intended, the perspective would change from tile to tile. What is needed instead is to construct partial viewing frustums that together exactly recreate the whole frustum. This sounds like a lot of math, but actually it is quite simple:

gluPerspective: fov aspect: aspect zNear: near zFar: far tile: rect
| cotangent radians w h |
radians := (fov/2.0) degreesToRadians.
cotangent := radians cos / radians sin.
h := near / cotangent.
w := h * aspect.
self glFrustum(rect left*w, rect right*w, rect bottom*h, rect top*h, near, far).

So I just extended the existing gluPerspective:aspect:zNear:zFar: method with a tile argument that gives the sub-rectangle in the viewing plane, where the whole picture ranges from -1 to 1. The only problem is to hand down that extra argument into the method. I ended up copying all the methods in the call chain, adding a tile: parameter. With the methods in place and a utility method to construct the right sub-rectangles, I could render the image from the first screenshot again:

| m c f |
m := CroquetGlobals theTeapotMorph.
c := m activeCamera.
f := c root snapShot: m ogl camera: c tiled: 12.
f := f magnify: f boundingBox by: 0.5 smoothing: 4.
JPEGReadWriter2 putForm: f quality: 90 progressiveJPEG: false onFileNamed: 'croquet-bf-4800.jpg'.

Instead of 800x600, this renders 12x12 tiles, creating a 9600x7200 pixels image. The result is scaled down with smoothing to yield an anti-aliased 4800x3600 image, finally compressing as JPEG gives a 1.5 MB hires screenshot.

Beauty, eh? Never noticed that guy on the bridge before ;-)

Change events are special

2005-02-17T16:15:00.000+01:00

I just spent a few hours chasing a very mysterious bug, where an event handler was properly called the first time, but never again. This was a rather unspecial on:in: handler:

onFooInBar
<on: foo in: bar>
...

After initialization it was properly registered in the event map, but at some point the event map entry just vanished.

There also was another on: handler which did some stuff whenever bar changed. To "call" that stuff, we just signaled a barChanged event somewhere else:

self signal: #barChanged.

It turned out this innocuously looking line was responsible for the trashed on:in: handler! What happened?

Well, an on:in: handler must always be registered to the object that currently is occupying the bar field. But what if bar changes? Then we must unregister the foo handler in the old bar, and re-register it with the new bar. That's why the system installs a barChanged handler behind the scenes, which normally receives the new and old values of bar as arguments.

Now, can you see what happened? If we just signal #barChanged, then the arguments are nil! The current handler will be unregistered, but no new handler gets registered instead!

Lesson learned:

Do not signal a change event by hand!

Of course, we could have constructed a proper change signal like this:

self signalChanged: #barChanged from: bar to: bar

but why bother? It's much cleaner to not misuse the system-defined change event. I ended up defining another event, #barModified with a proper handler, which is signaled from both the #barChanged handler and other places.

About Scripts

2005-02-09T15:26:00.000+01:00

We were looking into another Croquet performance problem the other day so we fired up a message tally (world menu - debug - start MessageTally). Curiously enough, 70 percent was taken by ScriptScheduler>>runActiveScripts! Unfortunately, the tally did not further differentiate this item.

But what are scripts, anyway? Everyone knows that Smalltalk is all about objects and messages, so what the heck are scripts?

Well, Croquet and Tweak are not just using Smalltalk as you know it (and the underlying Squeak still is pretty much vanilla Smalltalk-80), but instead improve on it by implementing a new enriched object system. There still are objects (the entities of the system) and messages (their means of communication). But where in Smalltalk methods are invoked synchronously by a message send, we now have asynchronous method invocations as well, which are called "scripts".

Synchronous in this context means that the sender sends a message, which invokes a method in the receiver that is immediately processed. Only after finishing the method the control returns to the sender. In contrast, an asynchronous send only schedules a method invocation for later processing, control is immediately returned to the sender. Here is an example (#perform: is synchronous, #startScript: asynchronous):

Transcript perform: #show: withArguments: #('1').
Transcript startScript: #show: withArguments: #('2').
Transcript perform: #show: withArguments: #('3').

If you execute this snippet from inside Tweak, it prints "132". Outside of it just "13" is printed because the second invocation is only started but never executed. There is no ScriptScheduler running to manage the execution of scripts. Inside Tweak there is one, and Croquet does it in a similar fashion.

Historically the term script in Squeak comes from the Etoys environment, where kids make objects and specify their behavior using scripts. One can have multiple objects and multiple scripts for each. All these scripts are running in parallel, at least from the user's point of view. Surprisingly enough kids don't have any problem with that, whereas concurrency normally is a hard problem even for seasoned programmers.

One reason why the parallel execution of interacting scripts is no problem in practice is the underlying scheduling policy: Scripts are never interrupted by other scripts. Other scripts are only executed when your script finished, or when it gives up control explicitly by waiting for an event. You can write your script almost as if it was the only process on the machine. Here's an example script:

| p |
p := CRectanglePlayer open.
1 to: 500 do: [:i |
p x: i.
self waitTick].

Looks just like how we did animations back when, right? Move a bit, wait for vsync, repeat. But the best part is this: While the loop is executing, you can perfectly well use anything else in the world! It's running "in parallel" to everything else on screen.

Another advantage is that you can easily have thread-local storage that way. Say, on mouse click you want to change an object's color to red, and on mouse up set it back to what the color was before. In most GUI frameworks you would have to implement both a mouse down and mouse up handler, and use an instance variable to store the previous color on mouse down. Not so in Tweak. The mouse down handler would look like this:

| oldColor |
oldColor := self color.
self color: Color red.
self waitUntil: #mouseUp.
self color: oldColor.

Much cleaner, in my book. Anyway, to get back to the original problem: So scripts are run as separate processes, but MessageTally's spyOn: method normally only takes samples in the process it was run in, which is the main UI process. Fortunately, there is a new method spyOnScript: which does the Right Thing. When starting the message tally from the Tweak project builder's debug menu, this new method is used, and indeed, we got a much more meaningful tally, and could spot the performance hog immediately.

Lend me a Hand

2005-02-08T11:35:00.001+01:00

The last Tweak updates broke our Tweak-based Croquet application, objects did not respond to clicks anymore. I first suspected changes in Croquet, but the latest updates there didn't look suspicious. But neither did the last couple of Tweak updates. Investigating further and sprinkling debug output here and there I discovered that pointer events were offset. Moving the Croquet window to the upper left corner confirmed this, all of a sudden everything worked normally. What was happening?

Well, living in a shared world isn't easy. Morphic, Croquet, and Tweak interact very closely in the TeapotMorph, which bridges between those three worlds. Without Tweak (that is, without calling initializeTweakWorld in your initializeDefaultSpace method), the TeapotMorph uses the regular Morphic event dispatching mechanism which leads to calling the event handling methods keyDown:, mouseDown: etc. which in turn dispatch the events to the activeCamera, that is, into the Croquet world.

When using the Tweak overlay, this does not happen, but all events are translated into Tweak events and dispatched into the Tweak world. The initializeTweakWorld method not only sets up the Tweak world, but also registers event handlers (onKeyDown, onMouseDown etc.) that are called when no Tweak object handles the event itself, like when clicking outside a Tweak window. These event handlers translate the Tweak events back into a Morphic event and call the regular non-Tweak TeapotMorph event handling methods, from where processing continues as before.

Now translating the events back and forth also involves offseting them, because Tweak uses relative coordinates (every Tweak object defines its own coordinate system) whereas Morphs use global coordinates. Translating into Tweak was still working fine, the Tweak objects were acting just as expected. However, the back translation was wrong. In fact, the event we got back already was in Morphic coordinates, so when we applied the offset again it was way off. The problem is that events were not explicitely passed around, but rather we used the active hand's last event.

The Hand in Squeak is the object that injects user interface events into the world. Its visual representation is the mouse pointer, but keyboard events come from the hand, too. Because there can be multiple hands in a world, there's a global variable ActiveHand which is set to each hand while processing events initiated by that hand. Multiple hands, you ask? Yes, there was collaboration even before Croquet in Squeak (so each user in a shared Morphic world had a separate hand), and also multiple mice can be used independently, if that is supported by the VM (I once wrote a plugin for that using the XInput extension under Linux).

Now until last week, Tweak also did set ActiveHand to its current hand while processing an event. This had some odd side effects, for example, when some code expected ActiveHand to be a HandMorph (as is the case in Morphic) or a CHandPlayer (as in Tweak). In particular, when running a Tweak overlay in Croquet, which still largely assumes a Morphic environment, this led to confusion. But we pretty much ironed out all these wrinkles.

Except for the back-translation of events coming from the Tweak world, which happens in a Tweak process, so we fully expected the active hand to be a Tweak hand in this handler. But Andreas removed all references to ActiveHand from Tweak in an otherwise innocuously looking update (comment: "Various preps for upgrading to 3.8") so we did, in fact, by referencing ActiveHand get the original Morphic event instead of the expected Tweak event.

Just removing the offset would not work because you might have or might have not loaded that update. So we need to access the active Tweak hand, but without accessing the ActiveHand global. This is easiest if you are programming in Tweak, because every CPlayer has a field aptly named hand that refers to the active hand. Outside of Tweak you must do it the hard way, which is getting the hand from the active process. I'll talk about Tweak processes in another post, this one is already getting too long.

So, to make a long story short, if you fetch Croquet updates now, everything should be working fine again. There are two new methods in TeapotMorph, morphicHand and tweakHand, that return the right sort of hand (of course, the latter returns nil if Tweak is not running). They are used in the two methods I found where it actually matters, namely bringing up the halo which needs a morphic-hand, and back-translating events, which uses the tweak-hand.

That's all for today. Happy Tweakin'!

Tweak Roadmap

2005-02-04T16:08:00.000+01:00

Andreas talks about the next steps in Tweak.

Why Smalltalk?

2005-02-04T02:19:00.000+01:00

Because it's powerful.

Dynamic Textures

2005-02-02T14:51:00.000+01:00

We've been wondering for a while, why screen updates are more expensive than expected in the Tweak overlay. Now I debugged into this and it turns out we're uploading the whole texture even if only a small part was changed. The relevant code is in OGLTextureManager>>uploadTexture:dirtyRect:. A partial upload using glTexSubImage2D() is only performed if the texture in use is not static.

Having found out what's going on it's easy to fix, a one-line change in TeapotMorph>>glRenderTweakCostume:on: does the trick:

texCache isStatic: false.

As simple as that - I just posted the update.

Profiling + Vector Math = Performance

2005-01-28T19:00:00.000+01:00

So there was a student who implemented a flocking simulation for a virtual fishtank using Croquet. Worked fine with 20 fishes. With 50 fishes it became rather sluggish. Putting in more than 100 fishes the framerate could be measured in seconds per frame. So the rendering in Croquet is too slow, right?

Not quite. There are two things you have to keep in mind when it comes to performance:

When in doubt, measure.
You are in doubt.

So we fired up a message tally (world menu - debug - start MessageTally). Turns out only 12 percent of the time were spent in rendering. So it's not the rendering at all. A whopping 80 percent was taken by the flock's step methods. The leaves of the message tally showed that 15 percent of the time went into each of the methods #x, #y, and #z!

So we looked at the code. Every fish was told to swim in the main direction of its neighbors, that is the fishes within a certain radius. Processing continued only for those fishes in range.

Sounds quite reasonable, doesn't it? The only problem was that to find the neighbors for a fish, the distance to each other fish was compared to the radius. Not only does this result in a quadratic complexity (every fish was tested against every other) but the distance check was this:

(((thisFish position x - otherFish position x) *
(thisFish position x - otherFish position x))+
((thisFish position y - otherFish position y) *
(thisFish position y - otherFish position y))+
((thisFish position z - otherFish position z) *
(thisFish position z - otherFish position z))) sqrt

This guy knows how to take the distance of two points in 3D, and he knows that the square is the product of a number with itself. Very well, but very slow, too.

Vectors in Croquet are represented as FloatArrays, and for a good reason. The reason is that you can compute sums, products, etc. of a zillion numbers very rapidly, just by invoking a single method. This is way faster than doing a component-wise computation on your own. There's a drawback too, which is that because the internal representation of a float in a FloatArray and the one in a normal Float object differs, it is rather expensive to read or write individual elements from such an array.

So to make your math go fast, use vectors instead of home-brew arithmetic. The whole mess above can be replaced by this:

(thisFish position - otherFish position) length

which is way faster.

Of course, one can further improve this by omitting the square root inside #length, by using #squaredLength and comparing to the squared radius, but that would be nit-picking. And of course one should reduce the overall algorithmic complexity from O(n²) to O(n log n), but that's just common sense.

The point I want to drive home is this: Use the class library wisely. If something is too slow, chances are you're doing it the wrong way. Look for examples in the image, there almost always is one. For a nice example of vector math look at the particle system (TParticle).