Two weeks ago I blogged about how someone had taken an add-on of mine that was in production and uploaded a derived version to https://addons.mozilla.org (AMO) without changing its ID. This means that because the AMO web site identifies add-ons by their ID, it would not allow me to upload my add-on. I tried to solve this problem by engineering a solution to change my add-on’s ID.

I thought this would work and even provided some sample code for my ideas in my previous blog entry; however, there were just too many problems to overcome.

The sample code might work if Firefox’s UI gave users a pleasant experience when an add-on’s automatic update facility installs an add-on with a different ID. (I’m not blaming Firefox here; it’s probably not a supported scenario.)

Instead this is what the user sees:

Compare this to what they should see:

Notice anything missing? 5 points if you said the "Restart Firefox" button. It’s pretty important.

"OK," I say, full of resolve, "let’s just create an update with the same ID that downloads the new version for them." Essentially, we use an update with the same ID to get around the problem laid out above, but all that update does is to download the newest version of the add-on automatically. Finally, it uninstalls itself and then restarts Firefox. Here’s what I envisioned for the user experience:

1. Joe Bloggs sees there’s an update for one of his extensions. Firefox guides him through the update process, at which point he sees the picture above to restart Firefox to complete his changes.
2. Joe clicks "Restart," and as soon as this intermediate version of the extension springs into action and downloads the newest version of the extension. Firefox doesn’t have time to show its windows again before the download completes, and Firefox is forced to restart by the intermediate extension.
3. Joe sees his windows and tabs being restored, and he has the newest version of your extension.

Unfortunately, it never quite worked out that well. This is closer to the end user experience:

1. Joe sees there’s an update for one of his extensions. Firefox guides him through the update process, at which point he sees the picture above to restart Firefox to complete his changes.
2. Joe clicks "Restart," and as soon as this intermediate version of the extension springs into action and downloads the newest version of the extension. Meanwhile, Joe sees his windows and tabs being restored. All of a sudden, all of his windows disappear.
3. Joe at this point could try and relaunch Firefox, but that wouldn’t work since Firefox is technically already restarted. If he waits long enough he’ll see Firefox return, but all of his windows and tabs have gone.

Whether these were technical limitations of Firefox itself or problems in my code I never confirmed. What I knew was that the investigation had cost a lot of time, and I was very nervous about deploying something that could potentially frustrate a large percentage of my user base.

Turns out I made the right call. Nick Nguyen, who was my liaison at AMO, allowed me to take over the extension that used my add-on’s ID. No more problems, and definitely no more hitting my head against a brick wall.

So if you have deployed an add-on publically, and you’re thinking of changing its ID, don’t do it. There’s a better way to achieve what you want.

The Situation

In early 2008 I wanted to upload NAMFox onto AMO for the reasons mentioned above, and because the server it was already hosted on wasn’t very reliable. I set up a new account and went through the process to upload my add-on, when I encountered an error saying that I didn’t have permission to upload the add-on.* At the time I thought it was because the ID of my extension (superfastoz@neoseeker.com) didn’t match the email I registered with. (This was not the case; read further….) I gave up on trying to host it at AMO and continued using the same server the product had always used.

…Until a year later, when after some significant code rewrites I decided to try my luck again with AMO. I got the same error message and decided to investigate further. This time I discovered that it was actually because an add-on with the same ID already existed on AMO. I certainly didn’t remember uploading it before, and the previous project owner professed that he didn’t upload it on AMO either. I began writing a piece of software to download all the add-ons on AMO and check their IDs for conflicts, but I figured it would be a much more efficient use of my time to enlist some at Mozilla for help.

Enter Nick Nguyen, who helped me figure out that another user had uploaded an add-on with the same ID as mine. I was a bit disappointed; then I figured out that someone had taken NAMFox back in 2007, amended it to work with GameFAQs instead of Neoseeker, and then uploaded it to AMO with the same ID. I was flabbergasted. I had wasted so many hours trying to get NAMFox onto AMO, only to find this. Unfortunately all Nick and I could do was to email the original developer of that account to see if he would be willing to relinquish ownership of the project. Unfortunately I can’t just take it over, even though there is currently no download on the page, nor have there ever been (0 total downloads since 2007). If we didn’t hear back from the developer in a couple of months, I could perhaps take over the project then.

Well…

1. We haven’t heard back from the developer, and it’s been about two weeks now.
2. A couple of months is such a long period to go without a release or an update, especially since the add-on enhances a web site which is prone to change (and break NAMFox) without notice.

There was another alternative—I should change the ID of my add-on. Then I could easily upload it to AMO for people to use. There are a couple of problems with this:

1. Firefox treats two different add-on files as the same add-on if they have the same ID and as two separate add-ons if they have different IDs. This means that if anyone who had the old NAMFox installed downloaded the new version with a different ID, their Firefox would have two instances of the add-on running, the old one and the new one.
2. The same problem occurs with automatic updates. I can tell Firefox to download this updated NAMFox with a new ID, but now the customers that download the updates will have two instances of the add-on running.

In theory there is a way around this limitation, but I haven’t tried it…

The (Tentative) Solution

DISCLAIMER: I have not used the following techniques/code when releasing my add-on into the wild, so you should be careful if you’re going to adopt it.

UPDATE: Don’t try this! My Findings

The great thing about Firefox is that you can control just about every part of the browser, including the add-ons that the users have installed. If you’re like me, you’re probably thinking you can use this to get around the two limitations I listed above. In fact, I believe we can.

We’ve already identified the two scenarios that users can get your add-on with a new ID: through a direct download and through automatic updates.

If a user doesn’t have your add-on installed already, then we can agree that there is no problem.

If a user does have your old add-on installed already, then the new add-on must recognize this and uninstall it. This action is required regardless of whether the user used automatic updates to install the new add-on.

Turns out this is actually quite simple and only requires a few lines of code:

This code exists in the new add-on; for me it runs in an event listener for the window’s load event. It tries to find the old add-on with a certain ID. If it exists, it’s uninstalled and Firefox is restarted to commit the change. When Firefox restarts, only the new add-on remains.

I’m planning to try this soon. I’ll update this post with the good, the bad, and the ugly after I do.

*The AMO team did a great job of cleaning this error message up. You’ll now see this message, which is much clearer.

The first step is obviously to download the extension, available at http://fireunit.org. I put a "Hello World"-type test together just to see the results. It should be no surprise that I saw the results in Firebug’s new Test panel when I opened the HTML page in Firefox.

So far so good. I noticed that FireUnit supported running tests from multiple HTML files, so I tried that based on what I found on the FireUnit examples wiki page.

I was a bit disappointed to see the following chrome error in Firebug when I ran this page:

What I should’ve done immediately was to download the source, but I was already sucked into hacking my local version to get it to work. I eventually did download the FireUnit source, but I still received errors like "nsHttpServer is not defined." (I tried to discover the cause of this problem, but after a couple hours of tweaking the httpd.js file, I decided to cut my losses and try something else.)

Not to be deterred (mainly because I really like the idea of FireUnit), I set out on writing an alternative to running test suites. What I created wasn’t pretty, but it was functional. I overwrote the fireunit.runTests function to take each argument and make a synchronous XMLHttpRequest for that specific page. Once I received the page, I would import all of the scripts on that page (i.e. importing the scripts in all of the script elements that had a src attribute). After all the scripts were imported, I would eval() the remaining inline scripts on the page which contained the tests themselves. Finally, I would repeat the process for the next test file, until all tests in all test files had run. It took a while to get this just right, but now I’m happy building tests instead of test infrastructure.

Another way I’m being unorthodox with FireUnit is that my tests are actually HTML files embedded in the add-on itself. This offers a few advantages over using a local HTML page but also comes with its own set of disadvantages. The main advantage is that I don’t have to deal with enabling the UniversalXPConnect privilege in every test function or (more importantly) the code I’m testing. The main disadvantage is that Firebug will not show the source for script files that exist in chrome (probably with good reason); perhaps this is just a Firebug preference that I’m missing.

Thoughts on FireUnit? I like what the extension aims to do and what it allows me to do already, albeit with some heavy tweaking. I know it’s in its early stages, but I can’t imagine that everyone would be willing to spend as much time as I did to set up the environment. On the other hand, it’s possible that not everyone will run into the same issues I did, especially with the nsHttpServer. I can’t think of much to improve besides making it much easier to run tests from multiple HTML files. About the only thing I would enjoy it would be great to see a QUnit-like HTML report for tests, with each module being a separate HTML file.

Now back to testing…

If you’re not familiar with the architecture of Firefox add-ons, here’s an extremely abridged top-down view:

• Most add-ons are implemented with a combination of XUL (XML User Interface Language) for the UI and JavaScript to support it.
• Instead of running the context of a web page, the JavaScript runs in the context of a window.
• In order to share state across multiple windows, you must use and/or implement XPCOM components.

XPCOM components provide services or objects that JavaScript running in a Firefox web page normally does not have privileges for. For example, there are services for file I/O, preference reading and writing, and writing to Firefox’s JavaScript console. You can also build your own XPCOM components (in JavaScript, C++, among other languages) that enable you to share data across the entire Firefox session or place logic that doesn’t fit well within the context of a single window.

My use of IDEs for add-ons shifted from Notepad++ to Visual Studio in early 2007 because I needed a better way to organize and search my files, but now I am glad I made the switch for other reasons. The hard about using XPCOM is that unless you have memorized your own interfaces as well as the interfaces of all the components you use, you often find yourself referring back to your IDL files or Mozilla’s documentation for the exact name of the attributes and functions you need. This arguably slows your momentum when working with XPCOM. But with the new improvements to Visual Studio’s JavaScript IntelliSense, this problem (almost) fades away completely.

Consider the following XPCOM interface:

Creating an XPCOM component requires an interface specified in XPIDL (shown above) as well as an implementation (not important for this discussion). The interface above should be straightforward to interpret even if you are not familiar with XPIDL. The example shows a simple tracing interface which abstracts away the API pain points of logging data to Firefox’s JavaScript console. It inherits from nsISupports (the XPCOM equivalent of IUnknown), and its methods take very simple parameters. The scriptable attribute specifies that this interface is callable from JavaScript and hence usable in our add-on. Here’s some sample code to show from JavaScript how we can invoke this interface.

The indexer on Components.classes returns the object that follows the given contract ID. The contract ID is defined in the implementation (and we hope the documentation); from there we get the service that corresponds to the ITrace interface (this is like a QueryInterface call) and call the error function with the exception that we caught. Of course, there is zero IntelliSense help because Visual Studio doesn’t know what Components.classes and Components.interfaces are, nor would it have any idea how to find the XPIDL file for a given service to find what functions ITrace actually exposes. We can do much, much better.

Let’s expose a simple function called "error" that we can include from any other JavaScript file that allows us to do the same thing:

Great! Now we have a utility function, we can replace our ugly code before:

Still, though, no love from IntelliSense:

Here we can use the the reference tag in our JS file to enable Intellisense, like this:

Very nice! If we flesh out this xpcom-interop file more, our development time can really improve.

Something you need to be aware of is that sometimes the XPCOM interop file can become too complicated for Visual Studio to parse correctly because it references objects that it might not know exist (e.g. Components.classes, Components.interfaces, Components.results). If this is the case, add an xxx.debug.js file in the same directory as the xxx.js file and write all of your documentation there. For example, I could keep xpcom-interop.js like this:

(Note that because we reference Components.classes outside of the logError function we see this when updating IntelliSense (Ctrl+Shift+J).)

And I could also create a new xpcom-interop.debug.js file for documentation purposes:

Notice that I don’t include any code for the logError function this time. This file is intended for documentation purposes only; it will never be included with the final Firefox add-on because it doesn’t do anything. If you go back to the default.js file and refresh IntelliSense (Ctrl+Shift+J), IntelliSense will display the same help for the logError function in addition to displaying the prefService variable. When you place the reference tag at the top of a file, Visual Studio will look for filename.debug.js before checking the actual file that the tag references, which is very useful in these situations.

Ctrl+Shift+J Is Your Friend!

Fiddling with JS IntelliSense can be frustrating if you’re not aware of how to get it to refresh itself. Ctrl+Shift+J should work. If not, you can find it in the Edit menu under the IntelliSense menu item. Also, you can bind a shortcut to the command "Edit.UpdateJScriptIntelliSense".

Resources

1. Format for JavaScript Doc Comments
2. Hotfix to Enable -vsdoc JS files

So in my Firefox plugin development phase, I’ve been working with a lot of AJAX to enable new “Web 2.0″ functionality in some sites that I go to. Recently one of my testers encountered a bug with this AJAX that caused non-ASCII characters (e.g. ») returned by the responseText property of the XMLHttpRequest object to turn into replacement characters.

This has to do with the MIME encoding used by the server to return the response you’re looking for. This is encapsulated in the Content-Type response header. If that header doesn’t contain the correct information to actually parse the response, then the XMLHttpRequest object will return these “replacement characters” (uFFFD) to substitute the characters it doesn’t recognize in its responseText.

One way to override this Content-Type so that the responseText will have the “expected” text is to use the overrideMimeType method on the object, which, as its name suggests, “overrides the mime type returned by the server (if any). This may be used, for example, to force a stream to be treated and parsed as text/xml, even if the server does not report it as such. This must be done before the send method is invoked.” (XULPlanet – http://www.xulplanet.com/references/objref/XMLHttpRequest.html#method_overrideMimeType)

For example:

// getXmlHttp() = browser-friendly way to create XMLHttpRequest

var xmlHttpRequest = getXmlHttp();
xmlHttpRequest.overrideMimeType(”application/xml; charset=ISO-8859-1″);

// Do other work normally

The syntax for the String input for the overrideMimeType method is the same as the Content-Type response header. Setting the content-type as application/xml will also allow you to use the responseXML property of the XMLHttpRequest even though the server might not report the page as XML. Hope this helps anyone that’s been experiencing these sorts of problems.