or how I wrote an editor for a single SpriteKit API
tl;dr: I built an editor to warp an image using native SpriteKit
A couple of weeks ago I was asked to simulate an effect shown to me in Adobe After Effects in SpriteKit. The effect modified the shape of an image over time, a warp effect, whilst it looked simple, it probably wasn’t.
Update with Video Warping: https://medium.com/@nthState/warptool-with-video-4471704857f8
So rather than write anything straight away, I looked at my options:
1. Spine — I downloaded it, couldn’t figure out how to use it quickly enough, That may be my fault, but it wasn’t obvious, so I moved on.
2. After Effects Export — I also did think about just exporting frames from After Effects, but then you’re stuck with frames at fixed sizes.
you can’t easily change, and your project will rapidly increase in disk size.
3. Unity — I then thought about using Unity, but again, it’s a lot of setup for something which seems like SpriteKit can/should be able to handle.
4. SKWarpGeometry — I stumbled upon this API by a quick google of “spritekit warp” and found a new(ish) API in SpriteKit.
The image below shows what the API can do, and the animation section piqued my interest.
Press enter or click to view image in full size
It looked like what I needed, but the setup required manual entry of an array of float2’s — and I’m far too lazy to manually figure out all inputs.
What I needed was a little tool to just tweak the inputs, view the results and export. Sounded simple enough…
Requirements
I want to be able to load in an image, setup columns and rows, pull around the control points and export the results as Swift code.
I opened up Xcode and created a new macOS project using the Game project, with SpriteKit & Swift. I spent a few hours putting together a skeleton of what I needed, used an NSSplitView and container views to separate out my view controllers, the first version looked like this:
Press enter or click to view image in full size
At this point, I could load an image of disk, show a simple inspector, and show an item in the timeline….nothing worked, as in, it was just faked, you couldn’t select the yellow handles on the image, changing the items in the inspector did nothing.
But I knew I had to prove that moving the control points warped the sprite.
Press enter or click to view image in full size
I was working on this in the evenings and at the weekend so after some short time, I could drag the control points and warp the image…it slowly started to come together.
I knew I wanted to be able to click on a control handle (yellow square) and move it around, and see the sprite change accordingly. With this in mind, I knew I had to have an Inspector…just like in Xcode where whatever you select has a context sensitive list of attributes that you can tweek.
I also knew that from reading the Apple document, I could have any number of warps applied in sequence, so I decided to have a timeline/layer system. Again, this should be context sensitive.
Out of that I also knew that the entire scene could have any combination of columns * rows, so there needed to be an overarching model to store those, so I filled out a data model like this:
DocumentModel
— WarpModel
— — ControlPointModel
The DocumentModel would store:
* The image to use (Actually, just the URL to the image)
* Number of columns & rows of control points.
* The layers of warps
The WarpModel would store
* A list of ControlPoints
The ControlPointModel would store
* Position, simple X,Y co-ordinates
I wanted my tool to be able to save the structure of any image I decided to warp, so I needed to implement Swift 4’s new Codable interface. It’s really simple to use and can encode and decode your data into JSON easily.
“It’s easy to see how a simple idea can turn into a lot of features”
I then (there’s a lot of then’s isn’t there) needed to prove that I could animate from one geometry into the next geometry.
That meant having to have multiple layers. Each layer has it’s own set of control points…but I want to animate between them. The layers should have different start/end points. If they started at the same time, you would get blended results.
Press enter or click to view image in full size
Exporting the code is the most important feature of the app, I didn’t want to just be able to see the results of the warp, I needed to export it and use it in another project.
Everytime I make a change to the data, The app re-generates the Swift code, which you can copy/paste as needed.
Press enter or click to view image in full size
Features
Menus
Again, I found myself adding features, adding layers at certain positions, duplicating layers if you want to move control points by a little over time.
Shifting control points up and down, and rotating control points.
Press enter or click to view image in full size
Press enter or click to view image in full size
Press enter or click to view image in full size
Undo/Redo
Another feature I didn’t intended on building was Undo/Redo, but it really makes you structure your code into re-usable components. I’m happy I did it.
Grid toggle
When developing the app I found that I wanted to toggle the grid on and off as in playback mode, or when scrubbing the timeline I wanted to see the result, not the grid obscuring the sprite. I also wanted to see by how far the warp has deviated from the original source image, so I added an onion-skin type feature where I can see the original, but turn it off if needed
Container Views
Container views allow you to embed a view controller reference into another view, this allows you to seperate out your views.
Press enter or click to view image in full size
Timeline
The timeline is made up of a custom drawn NSTableColumn, I draw increment dashes at 100px intervals, as I decided to lock 1 second to 100 pixels. I’m not going to go into scaling the timeline, it’s just overkill for now.
Inspector
I’ve written inspectors before, in my apps: Flux and Phonique, each time I write one, I start from scratch, and they’re getting better and quicker to implement, the previous two were written in Objective-C, this time I wrote in Swift and it felt like it almost wrote itself, in fact, most of this app felt like that.
Marching Ants
A cool trick I found was that I could use a SKShapeNode and a shader to draw the marching ants.
Physics
A button I haven’t talked about: Physics, this button exists purely for fun. I realised that the control points are just nodes, and can have physics forces applied to them. In turn, their position affects the warp. It can make your image turn into a nice gloopy image :-) The nodes are wired up behind the scenes with Spring constraints.
Saving/Loading, File Wrappers
macOS Apps allow you to save/load files, but I didn’t want just to save out a simple JSON file.
File Wrappers allow you to create a directory-type structure and store multiple files. So, for the WarpTool I store a JSON file of the data, and also a QuickLook Thumbnail. This means that when the file is saved, I take a screenshot of the design area. Save that as the QuickLook thumbnail, and macOS is then smart enough to show that image as the file’s icon.
Presentation/Model Separation
I think this comes with practice rather than anything else, separation of concerns is a really good idea, for example:
The main design area of the app is a SpriteKit View, as such I deal with SKNodes/SKSpriteNodes etc.
I have a DocumentModel, it’s job is to keep track of the Image, WarpLayers, Number of rows and columns etc.
I can make it so that when I update the image, I also update the SKSpriteNode for the original image,…..but eventually you get to a point where you have something which is purely data entangled with how it looks like on the screen.
So I split my models into Model/Presentation. I have the DocumentModel, but also a DocumentNode. The DocumentNode manages all of the drawing for the DocumentModel. This also means that if I decided to use SceneKit instead, I could just add a presentation for SceneKit and swap them in/out as needed without affecting the model.
KVC/Observing/NS
Key-Value-Coding is the glue that makes the App possible, by using NS classes…NSNumber you can observe changes on a property. I have tried observing primitive types like Bool/Float wit no joy. If you haven’t played with KVC, do it! It’s great!
Camera
I’m using an SKCameraNode to adjust the design view over the selected image with some padding. There’s so much you can do if you spend time on it, like zooming, panning, but I just wanted something that would just work.
When the image loads, I get its size, calculate the max of the height/width and then set the scale of the camera.
let minimum = min(imageFrame.size.height, imageFrame.size.width) + paddinglet scale = min(physicalFrame.size.width / minimum , physicalFrame.size.height / minimum)
cameraNode.setScale(1/scale)
Modes
The app has four modes, `Design`, `Animating`, `Scrubbing`, `Physics`. The app can only ever be in one state at a time.
* Design mode is where you can edit the handles
* Animating mode runs the WarpLayers through SpriteKit’s SKAction.warp method.
* Scrubbing mode is a mix of animating and design. Given a point on the timeline, I’m able to calculate the WarpModels to use and interpolate between to WarpGrids.
func lerp(_ x: Float, _ a: Float, _ b: Float) -> Float { return a * (1.0 - x) + b * x
}- Physics mode, implemented just for fun set the SpriteKit worlds phyiscs speed from 0 to 1. Simple.
UI Design
I made the design as simple as I could, as little design as possible. It should be a quick tool to load up, do what you need to do, grab the output and close it.
Press enter or click to view image in full size
Live streaming
Just as a footnote more than anything, I read Noopkat lessons on Live Streaming here I thought I’d give it ago.
At first I was just streaming the screen, no video and no microphone. Then I turned on my microphone and it felt like a different world. All of a sudden I had to start talking about the problem at hand, thinking about the ways in which I could solve it. It really made me think about what I was doing more, I’d recommend it.
However, I did end up deleting my videos as I messed up my configuration, the videos were illegible. If anyone would like to see me try again, let me know!
Bugs
Yes, I’m sure I’ve got bugs, The app was made just to prove a point, 10 points if you find one!
Prediction
I don’t think SpriteKit gets enough love, I love it, of course there are bugs , but with SpriteKit you can get something up and running so quickly! I think Apple will give the SpriteKit editor a revamp soon, maybe including a feature like this, and maybe even more powerful to include bones and automatic geometry generation.
Conclusion
It’s easy to get carried away. All I wanted to do was prove if SpriteKit could do what I wanted it to do, and it definitely can.
You can follow me/contact me on twitter at: https://twitter.com/nthState or check out what I make on my website: www.chrisdavis.com
Thanks for reading!