Composition Topics: What Is Composition?
Articles Blog

Composition Topics: What Is Composition?

October 23, 2019

Hello Windows Insiders
I’m Paul Gildea I’m a program manager in the Windows UI team Today we’re going to be talking about Windows
Composition and how to use it in a XAML application Today we’ll be talking about visuals and brushes
as the basic building blocks of our APIs We’ll talk a little bit about animations and
effects To do that, we’ll do a high level 101 basics
coding to give you a breadth of the API capabilities and a simple demo First, let me show you the desired outcome:
We’ll be building a basic sample here where we’ll create an image, spin it, and add some
effects (which is really cool). The first thing we need to do, we need to
be able to create Composition Objects. The way we do that is we use a shared instance
to a compositor object that XAML creates for us in the framework. The reason that we want to get an instance
to that specific compositor is because we want to be able to interop with the same visuals
that XAML provides. Having that shared instance allows us to create
our own visuals but then we can interop them with XAML’s visuals as well. So I’m going to write a line of code here
and we’ll try to unpack this to understand what’s going on here. The first concept is whenever you create a
visual, if you call .compositor on that visual, you’ll get a back pointer to the compositor
that created it. One of the things we need to do is we need
to get access to a visual that XAML creates for us so we can get that shared instance. One of the ways we do that is we use this
class in the Windows.UI.XAML.Hosting namespace called ElementCompositionPreview So there’s some methods in this class to allow
us to get some visuals that XAML has created and set our own visuals within visuals that
XAML creates for us. One of the ways we do that is we call GetElementVisual
on this. This is a UI element. Essentially every UI element has some form
of a handoff visual associated with it that XAML creates for us. So by calling GetElementVisual on this, which
is the main page (this is just basically our page – also a UI element), we can get access
to that visual – that handoff visual and then get access to the compositor that is the shared
instance that XAML creates for us. So it’s a big line of code, but basically
it’s how we get access to the shared compositor that XAML creates for us. Once we have our compositor, we want to build
out some of our composition object primitives. One of those primitives is a visual. Visual is a base class, so we have some derivations
of that. We have SpriteVisual which are boxes that
holds content. We have ContainerVisual that allow you to
group visuals together into trees. We’ll talk about that, but we’ll get into
more details later in another video. The next thing that we want to do here is
– I want to create a SpriteVisual. I’m gonna’ give that visual a Size, Offset,
and CenterPoint to basically move that CenterPoint from the top left corner to the middle of
that visual. You’ll see why when we do more examples – when we start manipulating the
visual later on in this demo. So I’ve created my visual. The next thing
I need to do is to parent that guy underneath that handout visual that XAML creates for
us so I can get that visual into the scene of our application. The way we do that is we use that ElementCompositionPreview
class as well and we call SetElementChildVisual – we pass in the UI element which is just
the main page here and we pass in the visual that we’ve created on the app side to say
“Hey, can you put that on our handoff visual?” What I’m gonna’ do here – I’m gonna’ run this
application now so we’ve created that visual, we’ve given it a Size, an Offset, and then
we’ve added it to the scene, but we don’t see any content. That’s because by default, our visuals don’t
have content inside of them. So one of the ways we can get content inside
a visual is we use brushes. So let’s type in this next line of code here
and let’s create a thing called a ColorBrush. We have other types of brushes, too, so you
can look through our API – you can see the types of brushes, but color is the most basic
one so I’m just gonna’ create a ColorBrush here with the color red and I’m gonna’ take
that brush and I’m going to apply it to our visual using the Brush property. So now when I F5 this application, we should
see a red visual in our application kind of in the middle there, offset from the left
hand corner – so there you go. I’ve created a visual and now we’ve put it
inside our XAML application. So… That’s well and good and one of the things
we really wanna’ talk about now is how do we manipulate visuals in a more interesting
way? One of the ways we can do that is we can apply
animations. Our animation system is interesting in that
we have two types of animations: We have your basic keyframe animation – if
you’re familiar with XAML storyboards, you’ll be very familiar with keyframe animations
where you define a starting point and an ending point, duration, possibly interpolation function
to go across those values – but you’re basically sending keyframes and values. We also have expression animation which we’ll
get into a little bit later, but let’s move forward with the basic keyframe animation. The next step here is I’m going to create
a scalar keyframe animation – we have different types of keyframe animations – for the types
of properties that we have that we can manipulate. In this instance we’re just creating a scalar
keyframe animation and we’re inserting an ending keyframe with a value. So that value is 360. So the thing that we want to do is we want
to take that red visual that we have on the screen and we just want to rotate it 360 degrees
so we’re gonna’ be targeting a scalar property of a visual that’s going to allow us to rotate
the visual. We give that guy a duration – so we’re going
to do it for 3-ish seconds and we’re going to give it an IterationBehavior so that this
thing will just kinda’ keep kicking off so it doesn’t only do it once. The next thing we need to do is we need to
take that animation – because this is just a description – just a description of an animation
– it doesn’t actually do anything so we need to do – we need to apply it to a property
and then I’ll call StartAnimation so that it will actually kick off and execute. So the next line of code here is we get our
visual and we say StartAnimation and the property that we want to target is the RotationAngleInDegrees
property. This type is a scalar type so that’s why we created our scalar keyframe animation. So we hand in that animation so now with the
execution of this function here, we should start to see our KeyframeAnimation here going
from the current value that the visual’s at to 360 degrees within 3 seconds and it’s going
to keep iterating indefinitely (essentially). So now you have seen how we can add animation
to our application. Cool. So the next thing we’re gonna’ talk about
is how do we get more interesting content into our visual. As I’ve said before, brushes are the way that
we get content into visuals, so one of the things we need to do is we need to figure
out how do I get, say, pixel data, into a brush, so I can brush it onto a visual. One of the core underlying components that
we have – we have this CompositionDrawingSurface – this is a surface that allows us to draw
pixel data on it. One of the things we need to do is say “hey
how do I load – say, an image from a URI, get that pixel data, and draw it onto a surface
and then how do I get that surface content into a brush?” We have a helper library that gets all those
steps for you – and we’ll go into it in another video in more detail. But if you understand that process – how do
I load image data, take that pixel data, draw it onto a surface – this little helper library
takes care of that for you. Once we have our CompositionDrawingSurface,
we need a brush – and luckily, our compositor, we have a CompositionSurfaceBrush. The next line of code here is – we’re gonna’
create that SurfaceBrush and we’re gonna’ pass in that SurfaceBrush – which has our
loaded pixel data that we talked about – going through that process of going from a file
on disk all the way to drawing the pixel data onto a surface, and then we apply that surface
brush to our visual. So now when we hit F5, we should see our visual
having more dynamic content – or in this case, the picture of a nice little flower. Awesome. And it’s still animating, and it looks pretty
good. So this kind of leads us up to the next set
of API areas that are very interesting in the composition world: We’re gonna’ start
talking about effects. And oddly enough, effects needs some kind
of input, pixel data, that they can run over the top of to produce some output – which
is typically pixel data, but remember we want to actually start and create brushes. So let’s talk a little bit about that process
– and with some framing of code. So our next line here, we’re gonna’ say we
have to be able to create an effect description. So the next thing we need to do is we need
to create an effect description. The effect description that we leverage in
composition are the same exact effect description supplied by Win2D so if you’re familiar with
that library, you’ll notice that in this application here, we’re referencing the Win2D library. The way you do that is that if you right click
here and look at NuGet packages, search for Win2D and then you can install that into your
project. We’ve already done that and we’ve referenced
the correct namespaces so here we can start defining effect descriptions. And if you look at their documentation you’ll
be able to tell which effect descriptions are supported with Windows Composition and
which ones are supported by default in Win2D. Right here, the description that we have is
the SaturationEffect description and what this does is it is essentially a shader that
lets us apply saturation to some image data. Right here, we create it and give it a name
– what we want to be able to do is we want to be able to refer to this effect because
we are essentially building a little effect graph. It’s a simple one but we’re building one up
and the concept is the same when we start to chain these things together. So we give it a name so we can refer to it
later. We set the Saturation property to zero – because what we want to do is we want to
be able to take away all the saturation in that image that we have and then we pass in
this CompositionEffectSourceParameter – essentially what this does is this gives us a bind-able
spot in our effect graph that we can set our input data into and allow the effect to run
over that data. That allows us to set it dynamically at runtime and you’ll see it later when we
set the effect. So all this does is this just creates – this
is a description of the type of effect that we want to have. The next thing we need to do, we need to actually
kind of compile that – to do that, we need a factory that compiles that and generates
our effect brushes. So the next line is we create our CompositionEffectFactory.
The way we do that is we take that description and we offer our compositor – you see – CreateEffectFactory
and then we’re also doing something interesting here – we’re specifying a property that we
want it to be animatable. So what’s going to happen here is when we
initialize the effect factory, it’s gonna’ say “Ah. The saturation property of this effect
description up here – composition will be able to set that property.”
So we’ll be able to animate it with different types of animation, which we’ll go into a
little bit later.” So once we can create our factory, now we
need to be able to take that factory and then start creating effect brushes from it – so
the next line of code here – you’ll see we’re taking our effects factory, we’re creating
a brush, and then we’re setting the source parameter of that brush and remember when
we said before, our composition effects source parameter – this is kind of that bind-able
spot, and we’re taking that surface brush data, which is our image data, which we talked
about before, and we’re running it through that saturation effect – so the output of
that effect should be the output of our brush – the content that is actually in the brush
and so the nice thing with visuals, is that brush is the object that we use to paint on
content, so we just pass in that saturation effect brush which has taken our image data,
applied an effect on top of it, and the output of that brush, we’re gonna’ brush that onto
the visual. So now when we hit F5, we should see our rotating
picture but the content of that picture should look at little different – that content is
actually desaturated. Here’s a neat example of how we can apply
effects to our visuals using the same components that we have in our toolbox in the Windows
Composition namespace. Awesome. So we added an effect. Now the next thing we want to do, we wanna’
do something really cool with effects. Remember before we added in that animatable
saturation property, so let’s go animate the saturation from 0 to 1. or from 1 to 0. We could do that with keyframe animation,
we could set that for a duration to match the rotation of our visual, but let’s do something
a little bit more interesting. The next thing I’m gonna’ do is I’m gonna’
create an expression animation. An expression animation is a way for you to
create an animation that uses an expression to basically generate values. Those values can be mapped to different properties
and the values can reference other Composition objects that have other values and properties. So it’s just a big generic way of saying “Hey,
I can map a property from a composition object, use that as input for my expression, and use
that to generate an output that isn’t bound by time.” Because animations we typically think of it’s
0 to some value over some duration of time. But you’ll notice here that when I create
this expression animation, I’m not specifying any duration for how long this animation should
run. What I am setting is I’m setting this thing
called a reference parameter and that reference parameter is the visual, so it’s actually
that composition object – that visual that we have and I’m just giving it a name here
so we can refer to it in our expression. So what I’m doing here is something very simple
expression – but essentially what I’m doing is I’m taking that rotation, angle, and degrees
property, that’s the property that we’re subsequently animating with our keyframe animation – those
values are going to change over time, but what I’m doing here is I’m saying “Hey, let’s
take those values, divide it by 360, and subtract it from 1.” So what’s gonna’ happen is – as that visual
rotates, the saturation of the effect is gonna’ change at the same time – which is kinda’
pretty cool (if you think about it). So this last bit of code here to wire it up
– and so one of the interesting things here is that you’ll see I’m targeting the saturationEffectBrush,
not the visual, that’s because I want to animate and manipulate the effect, but brushes are
composition objects, so that means we can animate them with our animation system. So
not only can we animate things like visuals, we can animate effects, and so basically anything
that derives from Composition objects – we can animate that guy. So let me hit F5, so hopefully we should see
on the screen a rotating image, and the saturation change as the image rotates – and so there
you go. Awesome. So I’m gonna’ close this guy here. So today we talked a little bit about how
to do Windows Composition in a XAML application., We talked about visuals. We talked about brushes. We talked about animations – and effects. Just kind of a high level tutorial, a smattering
of features that we have. We’ll go into more details in each of these
areas, so I highly recommend that you follow us on Twitter to get announcements and news
and also check out this code example on our GitHub repository and there you’ll find a
whole bunch of samples that our team is creating to demonstrate how to use Windows UI, use
Composition, and use XAML. Thanks very much and we’ll see you next time.

Only registered users can comment.

  1. Really good video. Thanks for demonstrating keyframe vs expression animations. You saved me a bunch of msdn reading time. Subbed ! Cloning the git repo now…

Leave a Reply

Your email address will not be published. Required fields are marked *