Herdict and its Tasty, Anonymized, Aggregated Data

Nothing sucks on the web like not being able to go to the site you want. Page not found and 404 errors are an inconvenience that entirely halt your workflow. What’s worse than not being able to access a site is not being given relevant information to fix the problem. When users are presented with an error message, they tend to do whatever will make the error go away to get back to their task. Page not found errors can’t be dismissed, because they’re shown instead of the content wanted.

What creates an added level of frustration is not being given information on what the problem is. When users get a Page not found error, they likely have two questions in mind:

  1. Is this problem on my end, or not?
  2. If the problem is on my end, how can I fix it?

These are questions that have been hard for browsers to answer. Currently, Firefox’s network error pages aren’t incredibly useful. They’re certainly not as useful as Chrome’s, which use Google Link Doctor to find possible matches both for subdirectories and domains. That won’t necessarily tell the user if the problem is on their end or not, but it will help if the problem is a typo.

So how could a browser tell users if the problem is on their end or not, without infringing on their privacy? One project that currently takes a stab at this is Herdict, which Johnathan Zittrain’s been working on at Harvard University’s Berkman Center for Internet and Society. What Herdict does is let computer users tell the “herd” – via a Firefox extension – what sites are accessible. The aggregated data can tell if a site is down (because no one can access it), or blocked by a firewall (because only some people can access it), or likely on the user’s end (because everyone else can access it). Not only does that answer the question of “is this problem on my end,” but it may start to answer questions like “is this problem only experienced by my country, network provider, or device?”

Useful stuff! Does it have a place in the browser, and specifically in Firefox? I think that getting and submitting anonymized data should have an increased role in the browser, and especially where it promotes transparency and information to the user. Mitchell Baker has been writing about data, and how Mozilla could be treating aggregated, anonymized data as a public asset that should be freely available. Especially in situations where sites are being blocked and censored, giving users knowledge of the situation seems to align with Mozilla’s goals of transparency and viewing the web as global public resource that must remain open and accessible.

One way something like Herdict could be incorporated is through those Page not found errors. If there were an option on these to submit anonymized data, we could build a pretty accurate view of accessibility information for a website and share it. Allowing users to submit data when there’s a problem is something many programs do already – especially for crashes. This is good design; it makes users feel better by registering the annoyance they feel as a useful data point to developers. Here’s some sketches of what it could look like to incorporate Herdict’s aggregated accessibility data with these error messages:

1. No available information on a site:

2. Site is blocked due to local firewall:

3. Site is down for a country:

4. Site is down for everyone:

No Comments

Chime in Leave a Comment

  1. Curtis Bartley says:

    I think it makes sense to say what service is being used, either when the user is submitting a website-is-inaccessible report or when Firefox is reporting that a page my be inaccessible based on location, e.g. “this page has been reported inaccessible in China”. If Mozilla is running the service, it wouldn’t be necessary, but if it’s a third party service, e.g. Herdict or Google LinkDoctor or whatever, I think the users should be informed. This could be done on the “more info” pages so it’s available to the curious but doesn’t clutter the primary screens. But it ought to be pretty easy to find out what’s going on.

  2. Curtis Bartley says:

    It would be nice to be able to diagnose a network problem as being local (due to a firewall, for example), but in my experience there’s often no way to tell. All Firefox knows is that it tried to make a connection and couldn’t.

  3. Rob R. says:

    Superb ideas. Then follow on to add options to automatically report when sites are down. (with a sub-option to stop in private browsing mode).

    (Similarly, include a box on the browser 404 page to “always report sites as down”.)

    This may lead to some false positives if you temporarily have local configuration problems (especially for people’s home page), but that will disappear in the aggregate.

  4. Ian says:

    Also suggest adding links to Google (/Yahoo!/Bing/etc) cached copy, and also Archive.org.

  5. Meph says:

    There is just one problem with the first mock up. The question mark icon looks as though it does the same thing as the blue link.

  6. Atul says:

    Awesome mockups, yo. I’ve added a link to this blog post on https://wiki.mozilla.org/Firefox/Projects/Herdict.

  7. fowl says:

    Another thing that could be included on error pages is the complete url, as typed, in a large font. That would make spotting typos easier.

    It has always bugged me that firefox makes very little (visual at least) distinction between “dns server timed out” “dns server nxdomain” “tcp time out” and “http 404”.

  8. John says:

    Anywebsite I type into browser comes back as
    “proxyserver is not connecting”Sometimes the wording varies abit and starts with Firefos is configured to a proxy server tha is refusing connection.

    There is one site i get and that is kitco.com

    any ideas?

Trackbacks for this post

  1. More on SSL errors | Owl's Portfolio

Comments are now closed for this article.