[Cuis] Time to break Morphic?

[Juan Vuletich]

Hi Phil,

(abridged and interleaved)

Phil (list) wrote:
> The basic idea is pretty simple: right now the Squeak display is just one large dumb frame buffer.  Makes perfect sense for the systems that Smalltalk was originally designed to run on... not so much for most systems in service today.  My thought was that a first step to modernizing it would be to break it up into multiple overlapping dumb frame buffers based on the host system's native view hierarchy.  This requires a very minor change to the VM, a plugin, and changes to the Smalltalk drawing logic.  For CogDroid the VM change is two lines of code in the interface logic (i.e. no need to touch the VM itself.)  The work to be done is in the plugin (relatively minor) and Smalltalk (the bulk of the work.)  The performance win from this is that you take a lot of the display management out of the hands of Smalltalk and the VM: overlapping, reordering, and moving views with alpha blending, damage detection and redrawing all becomes free (even better: on some systems it gets a multithreaded boost) as far as the Smalltalk system is concerned (though obviously it still needs to happen in the image for any Morphic structures that would not be implemented as a native view that are contained within one... the good news is that the scope of redraw is significantly reduced.)  If any of this sounds familiar, it is conceptually somewhat similar to how Croquet deals with OpenGL re: native OpenGL views (only dumber... Croquet uses the GPU for drawing, my initial approach would not change the 2D drawing logic within Smalltalk) and how pretty much every desktop GUI today works re: nested, overlapping views.
>
> So a basic example of how this would work can be illustrated by a Morph sitting on top of a Transcript window.  Currently, any change to either results in a redraw of portions of both in Smalltalk.  Simply moving either one causes a redraw of both (and more.)  The approach I'm proposing would make the Morph and the Transcript independent hardware-backed views that would be unaware of each other, from a pixel point of view, at the Smalltalk level.  The only downside I can see is that a view not implemented as a native view (i.e. one that exists in the traditional root view aka the display) cannot be positioned above another native view.  But as long as major UI elements like windows, menus, and stand alone morphs get implemented as native views I don't see why that would be a problem.  Taken to the logical extreme, if every UI widget were implemented as a native view (i.e. close, minimize, scrollbars, etc.) there would be practically no overhead to damage detection and redraw left in the image: the host view would determine when it needs to redraw and the image would do so on demand.  Most of the redraws should be the result of the view actually changing on the Smalltalk side (i.e. something changed within the UI) rather than redrawing as the result of damage from another unrelated view changing.
>
> Where local coordinates come into play is that native views are based on local coordinates (at least all the GUIs I can recall developing for recently.)  If Morphic used local coordinates, I'm pretty confident that once the infrastructure was in place, just about any Morph could be turned into a native view by changing one line of code (i.e. subclass from a hardware-backed parent rather than a software-based one) without knowing or caring that behind the scenes its pixels were being dumped into a different frame buffer.  Since we haven't changed the framebuffer-based drawing model and Smalltalk is still in control of every pixel drawn, behavior would still remain bit identical across platforms with and without native views.  I'm thinking that to keep things as simple as possible, all keyboard/mouse input handling would still be handled by the root view for now.
>
> Of course once implemented as native views, nothing says they have to all necessarily remain dumb frame buffers as it would be a fairly straightforward matter to start adding a higher-level drawing interface to further improve performance (as well as the look of the Smalltalk system) by taking advantage of the drawing capabilities that modern GUI's provide.  Though one downside to this would likely be that you'd lose the bit-identical results as you changed platforms.  It's also possible that frame buffer performance could be improved by being smarter about how it interacts with the host environment via the plugin.  But those are projects for another day... I think the basic approach outlined above will keep me busy for a while.
>   

Wow! All this sounds great!

> ...
>> Right now I have a set of about 20 change sets that add 'position' and 'extent' instance variables to Morph, and reduce a lot the use of the now obsolete 'bounds' instance variable. This is not finished yet, but it could be better for you to start playing with this than to wait until Morphic 3 is ready for prime time...
>>     
>
> Since the plan is still to implement Morphic 3 as the drawing toolkit for Cuis at some point in the future, I'd very much like to use this as a starting point rather than doing a lot of throw away work on the existing Morphic system and having to redo it down the road.  As long as it sort of mostly works (i.e. I don't need feature complete, just usable) I can live in an alpha/beta state until it's fully baked and possibly help out a bit (at the very least, I can provide feedback/bug reports).  Your approach sounds like exactly what I'd want for native views as that matches how things work in the host GUI.
>   

Well, right now the state is not good enough, because the bounds 
instance variable is there, and many methods use it. I mean, it is not 
local coordinates yet. I'll publish the changes in a few days anyway, so 
at least you can take a look.

> Thanks,
> Phil
>
>   

Cheers,
Juan Vuletich