8: Mouse Handling
This entry is part 8 of a 12-part series on WPF
3D and the Mouse
Many 3D applications want to offer interactive capabilities
using the mouse:
- Click on an object in the 3D scene to "select" so that
further actions can be applied that specific item.
- Click and drag to rotate, zoom or pan.
- Show coordinates or other information when hovering over
The mouse lives in a 2D world. The objects in your scene
are in a 3D world. You need a few tricks to bridge the gap.
The Need for an Overlay
WPF makes it fairly simply to get mouse notifications for
any Visual. It is therefore tempting to just add mouse event handlers to the
Viewport3D object, somewhat like this:
The problem with this approach is that the Viewport3D will
only receive mouse events when the mouse is actually hovering over one of the
rendered triangles in the scene. When the mouse is over the background, no
events are sent.
For some situations (such as picking a 3D object, or a 3D
scene which fills the entire screen), this is fine. For others (such as
interactive rotation of a model centered on the screen), this may not be not so
The usual fix for this problem is to overlay another element
directly on top of the Viewport3D. The overlay must be transparent to allow
the Viewport3D to be completely visible. The mouse event handlers should be
placed on the overlay instead, as the Viewport3D will receive no mouse events
at all. Because the overlay and the Viewport3D have the same 2D coordinate
system, all the math works out just fine. In XAML, this approach might look
something like this:
<Viewport3D Grid.Row="0" Grid.Column="0" >
(model stuff goes here)
<Canvas Grid.Row="0" Grid.Column="0"
From 2D to 3D
Inside the mouse handlers, you want to take the 2D
coordinates of the mouse click and find the 3D object where that click
occurred. WPF 3D makes part of this work simple with
VisualTreeHelper.HitTest(). After that, the handling will depend greatly on
For example, in Sawdust, every 3D scene is generated on the
fly, constructed from a data structure which was built from the solid modeling
code. When the user clicks on the 3D model, a specific piece of wood is
selected so that further operations can be applied directly to it. My code for
OnViewportMouseDown() looks something like this:
On mouse click, select the specific board
/// where the click
if (vstuff.models == null)
// extending the selection.
// don't unselect all first.
if (rayMeshResult != null)
PartialModel found = null;
pm in vstuff.models)
if (pm.mesh ==
found = pm;
if (found != null)
I included all the code, even though most of this routine is
specific to my application. In particular, vstuff, PartialModel, IsSelect(), Select(),
Unselect(), and UnselectAll() are all elements of Sawdust. Your app will have
different things, depending on the manner and data structures used to generate
your 3D scenes.
The key is the RayMeshGeometry3DHitTestresult returned by VisualTreeHelper.HitTest().
The members of this object contain plenty of helpful information. In my
particular case, I use the MeshHit member to search my data structures so I can
find the original information used to construct that piece of the 3D model. My
life would be a bit easier if MeshGeometry3D had something like a Tag member,
so I could just tuck a reference to my PartialModel object right there for easy