Just registered a table at the Zug Expat Expo on April 3rd. Anyone in Switzerland interested in seeing and talking to me about the reality augmenter or what projection mapping could do for you or your business, come and talk to me!

New version has been submitted to the app store for review, this version will add a new feature to add slide shows with a variety of transition styles. An incredibly useful feature, make a sequence of images representing your brand, products or whatever and the Reality Augmenter will now just cycle through them. For best results, crop or manipulate images to be all the correct aspect ratio for the surface you wish to display on.

Slideshow creation apps also exist for iOS, and you can use the resultant videos, but most existing video and slideshow solutions use fixed output aspect ratios and resolutions, with my built in slideshow, the aspect ratio is fully customisable.

I’ve still got many features left to come, I want to make the Reality Augmenter the most versatile and user friendly projection mapping app on iOS (and in the future, other mobile platforms). I feel I’ve already included simple features not included in other apps that make it a truly standalone projection mapping mobile app. I’m ultimately looking to reduce the complexity and cost of simple projection mapped installations.

As I’m going through testing and putting the finishing touches to my next release, I thought I’d let you all have a peek into adding a seemingly simple feature. I’m about to release a new version with a slideshow feature added, so you can just assign an album as a source, and my app will just go through the photos. I’m just doing some testing manually before submitting to the app store.

It turned out to be an interesting problem. The code to do it really isn’t all that complex, load the photo as a texture, and do matrix manipulations to achieve the transitions, or use a shader. I ran into some troubles using the GLKTextureLoader, but they turned out to be only related to the simulator. I started using the GLK tools more as it provided an easy way to load texture asynchronously, and while I already had openGL code to load images to texture, I didn’t fancy the extra work of writing all the stuff to make it load textures in a work thread. Ultimately, if I ever want to convert this to a multi platform app, then the less I rely on Apple specific frameworks, the better, however, pre written frameworks are a massive time saver.

The biggest problem was on how to present a slideshow maker to the user in my context. Having a predetermined video format of 16:9 or 4:3 would make it a lot easier, but with my app, i want my users to have total freedom, but I don’t want to present a complex interface. The app has to cope with formats that apply to any presentation. Beginners need to have something to walk them through a simple setup, but more advanced users need to be able to tweak the settings. A modern slideshow consists of portraits and landscapes, mostly in 4:3, but it’s so easy to crop a picture now, it’s not a reliable standard, nor is it one I want to encourage. With projection mapping, any surface can be a presentation surface, and not likely to adhere to a 4:3 or 16:9 aspect ratio. Eventually I decided to adopt a broad approach, anticipating my users to supply images in a singular aspect ratio, my app will broadly guess, landscape, portrait or square, or the majority of anything in between.

And then, once you accept that while there are ways to work around aspect ratios, which my app offers, letterboxing is another feature you have to write in. I though I might be able to do it with some extra drawing of black boxes, that concept does not work with something like a fade transition. Your textures must go through an addtional step to letterbox them before going through to a shader, easily achieved through a few lines of code, but a processor and memory intensive operation. here’s the code:

- (UIImage *)addLetterboxToImage:(UIImage *) inputImage
{
    CGFloat inputWidth = inputImage.size.width;
    CGFloat inputHeight = inputImage.size.height;
    CGFloat fitWidth = self.textureBounds.size.width;
    CGFloat fitHeight = self.textureBounds.size.height;
    CGFloat inputAspect = inputWidth / inputHeight;
    CGFloat fitAspect = fitWidth / fitHeight;
    UIImage *letterboxedImage;
    if (inputAspect == fitAspect) {
        // No need to do anything
        letterboxedImage = inputImage;
    } else {
        CGFloat scale = inputAspect >= fitAspect ? fitWidth / inputWidth : fitHeight / inputHeight;
        UIGraphicsBeginImageContext(self.textureBounds.size);
        CGFloat newWidth = inputWidth * scale;
        CGFloat newHeight = inputHeight * scale;
        CGFloat newOriginx = (fitWidth / 2.0) - (newWidth / 2.0);
        CGFloat newOriginy = (fitHeight / 2.0) - (newHeight / 2.0);
        [inputImage drawInRect:CGRectMake(newOriginx, newOriginy, newWidth, newHeight)];
        letterboxedImage = UIGraphicsGetImageFromCurrentImageContext();
        UIGraphicsEndImageContext();
    }
    return letterboxedImage;
}

In short, most of my work for this feature went into UI work, the actual code to make it was relatively simple.  Here are the logs:

40 minutes ago: Sliders now change dimensions.  I think that's about it, few more cosmetic changes but all the functionality is there.
63 minutes ago: Change maximum size for the slider to that of the largest dimension of the config, same as we use to calculate.
64 minutes ago: Code clean
80 minutes ago: Basing dimension size, using the largest dimension of the config.
2 hours ago: Save slideshow on return to editor fields.
3 hours ago: Slide show fields editor now updates after slider views.  Removed log messages.  Still a problem with the largest and smallest dimensions selector.  If a large one is scaled down, it could end up being smaller than the smallest image.
4 hours ago: Code tidy
4 hours ago: stop loading texture twice.
4 hours ago: Don't forget to delete the new textures.
4 hours ago: Removed the now unused ivars.  Current solution to just recreate the image with correct dimensions means we won't run into any problems with fades.  The generalised methods for VAOs, coords and colours are still useful, so we'll keep them in.
5 hours ago: Letterboxing done asynchronously so the whole next texture loading cylce is off the main thread.
5 hours ago: Added method to do the letterboxing, it's being called but I've not got the dimensions right yet and it's making them too large.
17 hours ago: Added ivars for vert coords of current and next texture.
17 hours ago: Added new ivars for the stuff we're going to need to do letterboxing.
18 hours ago: Use more generalised method to create the VAO for the slideshow.
19 hours ago: Don't forget to pass in the colourBuffer! colour buffer generator now works.
20 hours ago: Move properties to ivars for things we never access outside of class
20 hours ago: Added facility to add second set of texCoords to the VAO.
20 hours ago: Ah ha! you need to keep track of the allocated size, and use that for glBufferData when creating array sizes dynamically.  Otherwise sizeof() gives the size of the pointer, apparently..
24 hours ago: Trying to add a count to the colours is confusing.  Didn't know you cant create arrays with dynamic number of elements, malloc is used instead, but I'm getting some very strange results man....
25 hours ago: Added a count so we can use the general method for larger cases.
25 hours ago: Added more general methods to create vaos, tex coords and colour buffers.
26 hours ago: Added a colourOnly bool to the basic shader, so we can make coloured quads without switching shaders in the slideshow.  This way we can add black bars for when we have to do an aspect fit.  We should be able to do aspect fitting through matrices or seperate VAOs for each texture.
26 hours ago: Fixed right left transitions.
29 hours ago: Remove debug comments.
2 days ago: Can update dimensions from the pre configured options.
2 days ago: Compare all dimensions by area
2 days ago: Added a convenience property to get the current definition from the app delegate
2 days ago: Can select rows in the dimensions editor
2 days ago: Remove unnecessary observers, analyse dimensions on previous edit storyboard and pass on to the dimensions editing.  It's not possible to change the source in this view, just being over zealous.
2 days ago: Dimensions editing locks out of all images in a slideshow are the same format and size.
2 days ago: Removed comment
2 days ago: Set new path, name and dimensions when slideshow is changed.
3 days ago: Creating a new slideshow uses new analyse methods.
3 days ago: Removed keys for recommended format and just store the key with the relevant values, then it provides an easy way to access the correct array of size values.
3 days ago: Improved size analysis, so a preferred format is found, and the sizes are sorted. Also used it to replace information coming from the albums collection view.  Cleaned up code.
3 days ago: More annotations added
3 days ago: Added some annotations to properties we can be sure of the type.
3 days ago: Shouldn't loop in set, just use containsObject
3 days ago: Code cleaning and removal of comments.
3 days ago: Observers added and removed as appropriate, stops error on going back from dimensions editor.
3 days ago: Disabling fields now works.  Cell index paths couldn't be worked out when it was called, so they've been hard coded. Analysing is better too, but not sure it should be on the source.
4 days ago: Fleshed out the analyse class methods, starting to implement editor fields updates.
7 days ago: Going to move analyse dimesnions method to a class method on the SlideShow, it's too useful and should be used during creation and updating.  Provide constant dictionary keys.
8 days ago: More improvement to the dimensions, started to put code in to work out the dimensions.
9 days ago: Added table view for picking dimensions, but not sure if I like it at all.
9 days ago: Avoid division by zero
12 days ago: Added multiplier and unit to speed values in slide show editor.
12 days ago: Support for new multiplier for the speed.
12 days ago: New group for general purpose utility views like the enum and slider views.
12 days ago: MAde slider view more customisable, can add a multiplier and a unit for the label.
12 days ago: Can edit crop style, speed and transition speed. Fixed localized language switch.
2 weeks ago: All current transitions now correct
2 weeks ago: Fader works!  Typo in the shader means only the first texture was referenced.
2 weeks ago: Trying to get fader to work, doesn't seem to be picking up second texture, but it's def receiving the mix value
2 weeks ago: Can change transition styles, all slide overs work correctly, but the fade isn't, and the slides with both textures  have the underlying textures moving in the wrong direction.
2 weeks ago: Added fade transition
2 weeks ago: Cleaned unused code, added some commenting.
2 weeks ago: Rendered objects could get out of sync with each the external display and editor views.  So unified timing to be app wide.
2 weeks ago: Added seperate methods for rendering the transition for clarity.  Unified transition styles to switch cases for all up/down/left/right cases.  Removed debugs
2 weeks ago: Transition correct up down.
2 weeks ago: Enable fbo, transition now works, just need right transitions.
2 weeks ago: Fix order of operations
2 weeks ago: Slide show advances through the album, but the animations aren't showing yet, probably a mistake in the translation matrix.
2 weeks ago: Moved code to check if transition finished to start of draw method, so texture can be deleted at right time, the next texture swapped in to the current texture, and call the next texture load so it loads asynchronously ASAP.
2 weeks ago: Method renaming
2 weeks ago: Removed ugly cut paste code with nice little loop.
2 weeks ago: Swapped out individual const assignments to a much more sensible enum.
3 weeks ago: Playing with fbo
3 weeks ago: Some commenting and tidying.
3 weeks ago: Set delegate for the glkview, now it's rendering, but the texture has a blue tint, or is only blue.
3 weeks ago: Fix some clear colors.
3 weeks ago: Not unbinding texture and vao was caisomg exc_bad_Access, but still not drawing.
3 weeks ago: Added code to load first texture in the slideshow synchronously to save complex threading we don't need to deal with yet.
3 weeks ago: Pass round the cgimageref rather than the UIImage for more compatibility between iOS and OSX, we need to retain and release the cgimageref, this doesn't seem quite right, I would have thought the property would have retained it, but maybe with thread interoperability it's not quite so simple.
3 weeks ago: Texture loader now loads the texture, we need to keep a handle on the UIImage until texture load is complete.  Keeping the cgimagref is not enough, or maybe required a retain, either way it's working now.
3 weeks ago: Stopped crash by keeping image in a property while loading, looks like the image was getting released.  It's still a bit messy, but now there's another error reported by the textureloader.
3 weeks ago: Added a glktextureloader and set it up so we can load textures asynchronously, but it's causing a crash for some reason
3 weeks ago: Base draw code put in and mechanism to fetch photos in place, view controller draws result, now we just need to work out how to do a slideshow.
4 weeks ago: Renderer is created and photo album retrieved.  There's going to be some fundamental differences on how this works in iOS and in OSX. iOS will work exclusively through the Photos framework, and OSX will be a bit more fast and loose, probably ignoring photos altogether.  We'll cross that bridge later, let's just write iOS.
4 weeks ago: Was only storing the localizedName, we should have been storing the localIdentifier as well, easy fix, path stores the localIdentifier, and name provides the human readable localizedName
4 weeks ago: Fixed storyboard segue, slideshow editor now updates all detail text labels with appropriate information.  Gonna move to rendering for a bit of change of pace.
4 weeks ago: Added localizable strings for transition and crop styles.
4 weeks ago: Added slideshow editor table view class and bound cells.  Configured embedded segue to forward the slideshow
4 weeks ago: Fixed constraints so we get a view split in two with glview and editor fields getting equal space
4 weeks ago: Thumbnail saved before database save, so cell picks it up.  If database fails to save, thumbnail is deleted.
4 weeks ago: Slideshows save with thumbnails, but don't seem to retrieve them all the time, and they don't seem to delete the thumbnail.
4 weeks ago: Now we get all the sizes of the assets in the slideshow, but what shall we do with them? calculate an average size? they should probably be hung onto, but collections aren't static, so it wouldn't be a good idea to store them in the database.
4 weeks ago: Can get result of create slideshow action
4 weeks ago: Can activate the album picker for slideshows from the source view
4 weeks ago: Slideshow album picker now only shows non empty collections with at least one image and get a preview too.
4 weeks ago: Most of the view controllers in for the slide show, not sure if I should be making spaghetti with the view controller to pick albums or use multiple copies of it in the story board.  I'm probably going to use it for a new custom image picker too.
4 weeks ago: Added photos framework and started on album or group picker.
4 weeks ago: Basic slideshow editor put in, another glkview and container view with embedded table view.
4 weeks ago: Fleshing out even more, but going to switch to updating storyboard so can have data to work with, we'll implement this feature in OSX second as iOS seems to be doing much better. OSX will require a slightly different approach too.
4 weeks ago: Starting to flesh out the renderer
5 weeks ago: Added enums for slideshow styles, and a new attribute for the transition speed, so we can say how long images stay up, and how long they take to transition.
5 weeks ago: Make UniqueID optional, but keep uniqueIDString mandatory, the uniqueID is set on insert, and reset on fetch, so it's always gonna be there
5 weeks ago: Move code to render images to a texture to the top level texture unit class
5 weeks ago: Added all relevant shared source files for the slide show.
5 weeks ago: Keep screensaver on secondary display disconnect too.
5 weeks ago: Moved the screensaver to the app delegate, so we can screensave and connect to an external display automatically.

Time to have a look at my GIT statistics again a couple of years in. This project started out as a learning exercise, but eventually developed into a fully fledged application in iOS and OSX versions, so there’s a lot of change in the code.

The Reality Augmenter project currently has 1812 commits. With GitStatX, I can look at how those commits break down by hour, days, months and years, giving a peek into my natural working cycle. Looking first at commits by week:

I still stick to a working week, but out of that, Friday is my least productive day, followed by Monday, Tuesday seems to be when I’m busiest. I do occasionally put in work on the weekend, but not regularly. Next let’s look at time of day:

As we can see, I’m not much of a morning person, I don’t really get going till midday, then my commits go up till a peak of around five-six o’clock, probably because I’m thinking about dinner and committing before preparing and eating. I continue working late into the early hours of the morning. Usually after I’ve bashed out some larger tasks during the day, I’ll spend the evening tidying things up, tweaking things and adding comments.

Next we’ll look at the files and content in the project, I’ve got 2 different counts for lines of code, GitStatX lists it as 177859 lines, but Project Statistics says 39124, I’m more inclined the latter is a more realistic evaluation that ignores XCode specific stuff. Here’s the graph from GitStatX:

It keeps getting bigger, but a few things I remember about it. The small hump at the start is where I was learning a lot, but I was very badly organised with some terrible code in there, a total rework followed with los of reorganisation and consolidation. The project grows steadily until April 2014, that’s when I released the first beta of the OSX version, at which point it was all about bug fixing. Several more beta releases follow until my 1.0 release in November, a few more point releases follow in November, but the big jump is when I started thinking about iOS support and started setting up the project to support it’s future development. Since then the project grows steadily, with small jumps for new features.

I’m currently finishing a new slideshow feature for iOS that I should done and submitted for review in the next couple of days. I’ve still got a few more features I want to do, and to rework some aspects of the UI, but I’m pretty close to what I want it to be. I think it’s a great little tool that should appeal to a lot of people, but I need to spend more time selling it rather than coding it.

Graphs in this post were generated by GitStatX, and some details from “Project Statistics for XCode“

It’s good to remember the iOS Simulator isn’t always perfect. I wanted to use the GLKit framework to simplify texture loading in my app using GLKTextureLoader, it seems simple to use, and once I worked around a few errors making sure the context was in the right state I got the textures loading, but they came out a shade of blue.

No errors were reported, no obvious reason, it doesn’t look like channels have been swapped, it’s just a shade of blue. Checked all the documentation, everything says it should be fine, my images are in the correct format, no one else reports the same problem, I’m pulling my hair out.

I didn’t think to try it out on an actual device, where lo and behold, the texture is actually displaying just fine! The problem lies only in the simulator!

I know the simulator has some restrictions, and opengles performance cannot be compared to an actual device. But my existing texture loading code was working just fine, it was fairly standard code you could find in many examples. The GLKTextureLoader must be doing something a little different, some little wizadry behind the scenes, or has it not implemented something properly that’s only revealed by a small difference in how opengles is implemented on a device and the simulator? I don’t have a full range of devices available, could this problem on an actual device? I doubt it though, so I filed an apple bug for the simulator, and I’m going to move on.

Spam bots are really demoralising, you see your pages getting regular but suspiciously stable page hits day by day, do some digging, and it’s all coming from some site in Russia. It’s not too hard to remove them from your Google Analytics, this page gives a good breakdown, but if you’re aren’t getting to many page views yet, you get the real picture…

Been making lots of changes to the website today.
I’ve now enabled comments through the Disqus plugin, so you can comment on postst, but pages will stay locked out.
I’ve added share buttons for posts and pages through the Simple Share Buttons Adder plugin, I’m not sure I’m happy with the style of them, but I might change that in the future.
As noted in my previous post, I joined the Amazon Associate program, although I since learned that they have different programs for different regions. I’m now signed up for most of them and have the geniuslink plugin installed so any Amazon links should take you to the closest regional store. So far I’ve only added links for the correct apple AV adapters for iOS on the Reality Augmenter iOS page, and indicated them as Amazon links. I will endeavour to keep any associate links relevant to their position and clearly marked.
These changes should make things for the better, opening up my website to more users and start to get some better visibility for my apps, and now it’s easier to let me know your thoughts!

I just joined the Amazon Associates program, I’ll clearly mark any links through it. I want to provide links to things like adapters and projectors or other useful products that may be relevant to users of my software and seeing as Amazon offer a small kickback for doing so, I’m going to take advantage. I want this website to stay focused on my customers, so it’s not about to have adverts plastered everywhere, but if I can make it convenient to purchase a hdmi adapter, I think that’s a good thing.

While my app goes out for review, going to take a break from it and learn Swift and Metal for a possible new app.

Well had a late night going through my openGL code line by line, reworking a few things from the top down, simplfying code, reconsidering a few things and generally tidying things up and it’s fixed my problems. Just submitted the app for review now, hopefully it should go through in a week or so (According to http://appreviewtimes.com/).