Yesterday, Google organized an invite-only summit for mobile web and tools developers. As an engineer working on Firefox Developer Tools, I was invited to be on a panel about mobile testing and tooling. This is my attempt to write down my answers in hope they’ll be useful for someone out there. I suggest reading them as separate, individual mini essays.
And, of course, ramblings below are my own and not my employer’s.
Testing
There is this accepted notion among mobile developers that you have to run your tests on physical devices. For example, PhoneGap team has a wall with different phones attached to it and every-time a commit lands in the repo a MacMini connects to each and every device and runs tests on it. And, while I agree that there’s no other way to go about this today, I feel sad that everyone just accepted this state of affairs as necessary evil.
Physical walls full of phones don’t scale and, for the most part, we shouldn’t need physical devices for all our tests. Simulators should be smarter and more powerful. For example, it’d be great if we could simulate hardware limitations allowing you to run tests in an environment when the phone is almost out of memory or when the network is shit, etc. All mobile-specific events can be simulated as well (we’re getting there with FirefoxOS Simulator). This way you can run 90% of your tests in a simulator and reserve the actual hardware for very specific corner cases.
Interoperability
One question people often ask me (and I assume other devtools people) is why don’t we just use the same debugging protocol so that you could have a truly cross-browser Developer Tools experience and shouldn’t we standardize the protocol just like we do with other web technologies.
Remote debugging protocols will probably be standardized at some point in future but it won’t happen anytime soon. And it’s not because Mozilla or Google want to stifle innovation and lock developers in but because we all want to iterate as fast as we can. When Firefox team started working on our debugging protocol Chrome already had their own but it didn’t have everything we wanted. So we designed our own and started shipping features.
Standardization is good but it also slows everyone down and web developer tools are not at the point where anyone can afford that. Browser vendors have different priorities. We want our protocol to handle files and large JSON blobs so we could send FirefoxOS apps over it. Chrome team, obviously, has different priorities. So do Opera and IE. And this is how it should be. The world where everyone is doing the same thing would be extremely boring.
In addition to this, some parts of the protocol are tied to specific engine implementation details so, if you’re a third-party tools developer, you kind of have to write code specific to each platform. And you can always shim parts of the protocol.
Other issues
Someone mentioned yesterday the fact that stack traces are so different across different browsers and environments. They’re not standardized and, as far as I know, TC39 (or whoever is responsible for that part of the platform) doesn’t consider them to be a high priority. But, I think, they should be.
On desktop, reproducing a reported issue is relatively easy. You have three major operating systems, five major browsers and hardware that’s mostly irrelevant. On mobile everything is very sad. You get a bug report and either you hunt down the exact device with the exact version of Android (iOS, etc.) with the exact version of your browser or you’re kind of screwed. One solution would be to automatically send exceptions information from your web application to your servers but, when I was helping Sentry folks with it a year ago, I realized how much of a pain in the ass this was.
Issues like this are the source of great embarrassment for the web platform. We can show shiny new debuggers all we want but then you deploy your app and—boom—welcome back to the medieval ages. So if you’re a web developer please find your local browser/VM representative and make sure they understand that the issue is important. Contrary to popular belief they do listen to your feedback so be loud (but constructive, of course).
Developer Tools in a year
What is the one issue I would like to see resolved in a year? I would like to lower the barrier of entry for mobile web developers who actually want to use our tools. Today, in order to debug a goddamn web application in Firefox for Android you need to install Android SDK and adb and then run a few commands to do some magic connection bullshit and then maybe pray a little.
This is insanity. What we need is a Firefox add-on (or Chrome extension or whatever IE/Opera/Safari have) that adds a button to the toolbar that connects to your phone. That’s it. Nothing more complicated.
Fortunately, this is not just a wishful thinking. Both Mozilla and Google are working on this and Safari already has a relatively easy and streamlined process. So we’re getting there.
In conclusion
When I was a web developer I could complain about web developer tools and feel good about myself. Then I went to Mozilla to hack on Firefox Developers Tools and things changed. I still rant about the sad state of web and mobile tooling but now I have nobody to complain to. All I can do is get back to work.
So back to work.