After completing the Metal pipeline, I’ve been working on other things the last week.

I’ve been doing a lot of UI updates, while removing my old theme code, I had a think about my colour app scheme in general. While Dark/Light mode is now handled by the system, you still define the colours in the App assets. And I worked to make sure it was more unified throughout the app, so all common things will have the same colour schemes, eg all views have the same background colours, table cells have the same colour, everything uses the same tint etc. It’s looking much nicer and more coherent.

I’ve also revamped all the icons, I was previously using Google’s material design icons, but since the app was released, Apple has vastly extended icon support via SFSymbols . These have many advantages, they’re built in, scale nicely and there’s some possibility to animate them (I’ve not attempted that yet). I have replaced all my icons with them, and it looks much, much better. I even found some replacements for icons I designed myself I couldn’t really find suitable ones from Material.

I’ve also been going through the code, improving things where I can, applying things I’ve learnt since, and sometimes I came across code I no longer understood, and took the opportunity to go through some of those and refactor to make things more legible. I was already stricter with typing than a lot of obj-c code I’ve seen, and this pays off with refactors, as type errors are caught by the compiler and you can back out of things if it looks like you’re refactoring too much.

I’m working on the screen connection/disconnection code, it was always a bit flaky and a source of frequent crashes, especially when there were conflicts of resolution between the apps current configuration and the connected display.

In short, I’m going to do a week or so’s testing, and I hope to have the new version released soon. No doubt Apple will flag some issues on submission but let’s see.

After a final sprint. everything renders from beginning to end in Metal. There’s still some tweaking and fixing to be done, some things don’t quite look the same as before, but now the whole pipeline is done, it’s much easier.

The video player is acting strange, with some strange flicker, and the colours aren’t quite right. It’ll probably turn out to be something simple, I hope.

While the OpenGL rendering code is still there, it’s mostly to cross reference against the Metal now to double check my implementations.

As I had some frustrating times with Metal at points, I would spend some days updating the UI to take my mind of the Metal work, so a good deal of the UI is now updated as well.

The system Light/Dark theme is now supported and works better than my implementation.

I’ve replaced most of the icons from Google’s material design with equivalents from SF Symbols, which are much more flexible in their usage.

I replaced my own colour picker with Apple’s own, it’s much better than the last time I looked at it, and now much better than my own. I will also look to remove my Custom photo browser, but I’m not sure it’s so feasible.

And I put dropbox support back in, and google drive, well anything iOS supports importing from using their Document Picker, which automatically adds support for any apps on your device.

Just had a frustrating 2 days dealing with a simple scaling matrix.

I’m removing all openGL and GLKit code, so have to migrate all the GLKit code to simd utilities that Metal uses.

I’d grabbed a matrix_scale function in the simd code to replace a AAPLMathUtilities, but it simply did not work how I expected. It took adding debug code to see the matrix values to pick up on the problem.

Many of Apple’s demo projects include AAPLMathUtilities, this has an proper equivalent to AAPLMathUtilities, matrix4x4_scale, which finally solved my issue.

It’s incredibly frustrating to debug, while the Metal debugging stuff is cool in XCode, it just had a blind spot here.

I hate matrix math.

Still, solving this, removed a huge hurdle, I’m not so far away from completing the transition to Metal.

Very pleased with my progress. Mostly due to the way the app is architectured, it took a while for me to remember how it was all pieced together, and to adapt it for metal. But now it’s coming back together in my head and with a few key things done, progress is now very rapid.

I used a lot of inheritance, with a key super class doing a lot of heavy lifting, once I’d adapted it for Metal, there’s really only tweaking on the child class to adapt to their particular quirks.

I’m still on the source units, rendering images, video and text, but due to the way they all work, I’ve had to implement most of the pipelines. So it generates texture directly, say using the MTKTextureLoader, to generating core text, which used an FBO in openGL, that’s now a custom render pass in Metal, I just needed to wrap them the same way, such that subclasses don’t care if they’re in an FBO or not.

I used vertex Array objects in OpenGL, but now that’s much better handled in Metal, I define the structs for vertices, with position, texture coordinates and maybe colours, and define a MTLVertexDescriptor and it’s much more elegant.

Overall, I’m hoping to get an update out much sooner than I expected, I’ll disable the very new features I was working on, but they should come rapidly after.

Progress on migrating to Metal is progressing, with a bit of a slow down over Christmas, I’m getting back into it.

I made a few false steps, and perhaps got a bit too into investigating some of the new features in Metal 4 that I got distracted from just writing the simplest pipeline to start with.

I was also hopeful I could have done some nice mixing and matching of rendering techniques using protocols, but that also while possible, is probably too complex. I’ve ended up making 2 parallel rendering pipelines, one for Metal and one for OpenGL, just copying the classes and removing all the openGL code for the metal pipeline so it compiles, then figuring out how to replace everything with Metal.

There’s some small differences to the APIs, probably the biggest one is switching from using openGL contexts, to using Metal’s Command queues, buffers and encoders. With the context, you can just call opengl code without being aware of the context, with the Metal approach, we have to manage the buffers and encoders and pass them around to the code that’s using them. I do see some potential benefits to this in the future though.

It still broadly fits with my rendering model, so the adaption isn’t too painful.

I’ve now just managed to make a simple full render pass, from generating a texture to getting to a drawable, or rather I’ve done it in theory and the code compiles, now I’m learning about Metal debugging!

Having done all the admin and project reorganization without getting too far into it, I’ve started working on migrating to Metal. This took a bit of brushing up, I haven’t really worked with it much apart from a few projects a year or two back, so I had to remind myself of a few things.

I’ve just converted the shaders, or, rather, converted the code from OpenGL to Metal, I get to the tricky part next, which is doing the rendering pipelines, which are very different to OpenGL. There’s also been some updates to best practises in Metal 3, so I’m going to have to learn those as well.

I think I have much in place for the types to share between CPU and GPU, but I’m sure I’ve missed some things, or it’s not quite all there with the new features in Metal, but once I figure out how to do one renderer, the rest should be much simpler, the first one is the hardest.

What you see above is my kanban, it looks a bit tatty as it’s gone through a move and been stored away for many years, but it still had all my taks from when I stopped the project, so I just put it back into use, after creating new tasks for the upgrade of course. Once done, I can ressurect some of the old features I wanted to implement.

With 3 test failures out of 235, things are going well, 2 of those are openGL, so they may no longer be relevant when replaced by Metal.

I spent a day just tidying up the project. This was started before XCode had folders, and it used groups extensively to keep things organized. In Xcode it looked fine, but if you look in the file system, everything was flat. I reorganized the file system to reflect the project structure and converted XCode to use folders (In most cases, I still have a few groups in there).

I can’t really think of anything else to do that I can put off the conversion to Metal, so it goes up on the board, and that’s what I’m going to do next.

Most of the Unit tests still run, there seems to be an issue importing files for testing, and there’s some issues with openGL which is going to be replaced.

There’s an issues with opening the core data store, but I think I know what that is, Apple enforced some things around to do with secure coding I remember, from when I abandoned the project I think I might have the fix for that somewhere.

The UI Tests all fail, I wasn’t too surprised, although, I am actually surprised to see anything at all is rendering, dialogs do appear to be triggered, but there are many issues. I want t o revamp the whole UI anyway, it had proto light/dark themes before Apple implemented their own support with what I think is a much better solution with the asset catalogue. And I was doing some other stuff it looks like UIKit doesn’t treat the same as before.

I’m debating what to with the repo, I want to start backing up to github (of course I keep local backups!), but there’s so much history and some of the old pods are causing issues as they have some binaries too large for github. I think I might just create an new orphan branch, and figure out what to do with the rest later.

The Reality Augmenter builds again! It’s nowhere near ready for a new release, but I was kind of dreading this step.

I had to strip out old dependencies, update the project settings etc. Some of the dependencies didn’t exist anymore, or no longer support what I used from them.

The google cast SDK no longer does direct video streaming so that had to go. The Dropbox SDK I used is long gone, I think there’s better facilities but for now I had to remove it. I had found Alamofire still worked, but I discovered I only ever used it for dropbox as it had an OAuth thing. I had a dependency for User Guiding, MPCoachMarks, that doesn’t look it’s going to work without effort.

I was lucky I suppose, the app was a learning project for me and I wanted to learn *everything*, so I usually avoided dependencies, and not to toot my own horn, but the project is well organized and commented, and the code that relied on dependencies was suitably isolated from the rest, that’s good usage of protocols for you. There were a few bits of glue here and there, but I was fearing I might have to do some major rewrites, nope.