This of course set my wheels spinning and made me wonder why apps like AirVideo couldn’t do the same. Being friends with several well respected iOS devs, I asked them what they thought. One (who will remain nameless) emphatically stated that AirVideo didn’t work because they “were doing it wrong” in his own typical cocky style response. I asked how he would suggest doing it, to which I got a rundown of different video frameworks on the platform with no specific details (again, typical). Being suspicious it couldn’t be that easy, I set off to ask a few more developers. Everyone I spoke to seemed to think it would be possible somehow. Sadly, they were all mistaken.

My first course of action was to try the MediaPlayer framework, as it’s the older API and thus had better sample code. After going through the motions to get an MPMoviePlayerController instance, I found that on my iPad the same thing happened as does in AirVideo, complete blackness. Wondering if it might be an iOS version thing, I then tested on three other devices ranging from an iPhone 4 down to a 3G still on 3.0 and still no dice.

My next thought was to use AVFoundation. Unfortunately this is limited to iOS 4, so it ruled out playback on my iPad which is where I wanted it most. Despite that, I set out to learn how to use it. Freaking A did I not expect how hard that would be! Not only are the docs extremely lacking on how to use AVPlayer, there isn’t any sample code available either from Apple. Searching the web, apparently nobody had ever thought to write a blog post on it either. Apple’s developer forums were a bit of help, but just barely. After a bit of trial and error getting a CALayer to render properly in my rudimentary test app, I was up and running with a basic video clip. Then upon switching to an iTunes DRM video, I was back at a black screen again. All that effort, still no dice!

Feeling mighty dejected at this point, I decided to see if it working was a browser only thing somehow. My first test was to fire up iCabMobile (a custom Safari implementation). Not to my surprise at this point, it shot up an error immediately about unsupported content type. On a long shot, I decided to test out a UIWebView. Again, still no luck as it errored out too.

At this point, it seems the only way to play DRM’d iTunes content is either via syncing it in iTunes or via MobileSafari over HTTP. With the later being supported, that’s still a big win, but I’m very disappointed there doesn’t seem to be anyway to do it in a custom app. I’m going to keep digging to see if there is a way to do it in a jail broke app, but I doubt it.

If anyone has any tips here, I’m definitely all ears!

-Jeremy

P.S. – after I get a little sleep, I’ll write up a blog post on using AVPlayer & AVPlayerLayer so others don’t endure the same pain I did.

One of the leading theories people had was that the iPad was using SkyHook to do a lookup based on cell tower triangulation. I knew this was an immediately flawed concept based on prior knowledge of how it works. What Skyhook and similar services do is interface with the gsm modem to get the towers in the area. It’s not even really triangulation, basically they get the cell tower ID from the modem and then lookup that in their database. This is why you see a relatively big circle on 1st gen iPhones, as it’s not a very precise lookup since towers cover a several square mile area each. Not only was my location being shown with pinpoint accuracy, since my iPad was talking to my iPhone via wifi there is no possible way for it to have gathered the gsm data directly.

Another theory was that the iPad was doing a lookup on the IP address. While this method is a last ditch fallback employed by SkyHook, this is also flawed because one can drive a long way and still retain the same IP. As previously indicated, the location pinpointed me following a curve in the road, something even triangulation couldn’t do let alone IP lookup.

Not having an explanation which seemed plausible, I decided to do some experimentation this evening with the help of a friend. Over dinner, we found the location was perfectly spot on (see second photo below). Firing up an app which showed the location data directly, I observed a couple curious things. The first thing was that altitude data wasn’t being reported. Based on this, our assumption was that it wasn’t getting actual gps data from the phone via some “magical” method. The second thing was that the location accuracy was <300ft (see the first photo below). This further confirmed the hypothesis that tower data was not being used.

Still stumped, we decided to drive around and see if any patterns emerged. Driving around, the accuracy wasn't as accurate as before and would jump every few seconds to where we were. When it did track the road, updating in virtual realtime, we were in densely residential areas. The farther we were from any homes or buildings, the less accurate it was, leading up to a total loss of location when we were on the 280 heading north.

That's when it dawned on me how it worked and I was embarrassed to have not immediately thought of it before. Just because I was associated to the wifi access point created by MyWi running on my phone, doesn't mean the iPad can't see the numerous other access points in the area. One of Skyhook's differentiating features is that they have one of the largest databases in the world mapping wifi access points to their geo location. Using technology they call WPS (short for "wifi position system"), it can then triangulate my location with much greater accuracy (since wifi aps have a very small coverage area compared to a cell tower) than any other non-gps based solution and hence what I was observing.

All in all nothing extraordinary or new, but still a nice unexpected surprise none the less!

-Jeremy

P.S. This blog post 100% penned via the Wordpress app running on my iPad using the onscreen keyboard. Surprisingly easy to do even without a physical keyboard.

Today I spent most of the day catching up on my blog reading and I stumbled across an article about a guy using an iPhone app named Brushes to design this month’s cover of The New Yorker. Highly impressed by the output and the video of it being created, I decided to give it a shot.

Realizing of course that I am definitely not an artist and that using just a finger on the iPhone screen isn’t the most precise way to dry, I don’t think my first work is too bad considering. I’ve done a lot of sketches in the past with various pen tablets and while the experience wasn’t anywhere near as easy as with them, it was pretty fun and rewarding that such a feat could be accomplished on my iPhone. A company named Ten One Design seems to make a stylus for the iPhone for a very reasonable price ($14.95) and I think I might pick one up not just for this purpose.

I think what impressed me the most is that all this was possible using a $199 “smartphone”. Times are changing and when we step back and think about what was possible with a cell phone not even 10 years ago, it simply is amazing, IMHO.

The above image was modified after creation in Brushes to add in the frame and thought bubble. The additional elements were added using Photogene and then the image was uploaded to Flickr via Mobile Foto (both also iPhone apps). See below for a time-lapse video of the creation of the coke zero image created using the Brushes Viewer desktop companion app.

1. I want my email everywhere at every time, with or without network access, on a range of devices. Sitting at my desk with my main desktop? Sitting in a meeting with my laptop? Sipping a coffee with my iPhone in my hand? On a plane flight to some far away place? All of those should be no challenge to doing something as simple as reading an email, right?
2. I like to organize my email, especially mailing lists which I subscribe to hundreds of at last count. Some lists are important, others less so and should be organized accordingly. As having things “in view” can be distracting when I don’t need to see them, I need the ability to have a nesting system or some other way to have multiple views into my mail. These “views” should be synchronized across all devices and possibly have the ability to be device specific.
3. When someone sends me an email, it should show up instantly in every client, not five minutes later, not an hour later, certainly not forcing me to restart the app just to see the email. The system should also support some type of push system as constantly “polling” puts excessive load on the server and sucks up precious power reserves in the case of mobile devices.
4. I need the ability to find old stuff quickly and painlessly, again on any device. We can have Yahoo! search teh net in under 100ms for anything I could dream up for a search term but even my quad core Xeon can’t accomplish a similar search across an infinitely smaller dataset in under five minutes? Huh? You’re kidding right? Guess I won’t even try it on my iPhone then as I know my battery will then run dead before the search finishes in that case.

Seems like fairly modest needs right? Should be doable? Well, I am amazed to say it seems to be about as simple as sending a man in a balloon to the moon.

Currently I am running my own Courier IMAP server, using Procmail and all it’s powerful mojo to sort my mail, and using Mail.app as the actual MUA. The problems I run into are as follows, in no special order and certainly not all inclusive:

1. My mail server is constantly thrashing the disk. Despite being on a dual G5 w/ plenty of memory and decently fast I/O, it seems the combination of commands Mail.app sends to Courier seems to give it fits. Have two clients hitting it at once and the machine drops to a snail crawl. Heaven forbid you add that third client and the machine starts sparking and flames shoot out the back. Of course running it on another OS would probably help as OSX doesn’t have the best kernel support for things like this, but that’s not the point. I’ve also thought about moving over to another IMAP server implementation, but as yet I haven’t found one which I feel confident will be better performing without enough negative aspects to scare me away.
2. Mail.app seems to not like to pull subfolders with consistency. It’s extremely common for me to restart the app and suddenly have hundreds of unread emails suddenly appear across folders. This definitely is a problem no matter how you look at it. Of course there are multiple possible causes to blame, but that’s irrelevant. I don’t care if the IMAP spec is brain dead in how it specifies IDLE support, that’s not my problem. Having a mail client which shows new mail without fail should be one of it’s developer’s top priorities.
3. Procmail while very powerful is extremely error prone. I’ve had several occasions where things backfired. My dream would be to someday have something which said “hey, you moved this email into this place, would you like me to move emails like it in the future for you?”, but then I snap back to reality and realize that’s a geek’s wet dream and not anything which will really work as you would expect any time soon.
4. Keeping multiple devices and offline copies of emails in sync seems to be error prone at best. Isn’t this exactly the problem IMAP is supposed to solve? Tell me again why there is not a single IMAP client which handles this painlessly? Oh, right, that’s because the IMAP spec is crap. Please fix, ok? KTHXBAI!

So my question is, is there something I have missed? If you have a solution I am more than ready to apply my hand to my forehead and yell “Do’t” with enthusiasm. Sound off in the comments with what works for you.