The US government's web traffic
analytics.usa.govIf anyone would like to see, use or contribute to the code we used to build the Dashboard, we're on GitHub: https://github.com/GSA/analytics.usa.gov
And also open-sourced the adapter we use to get the data out of Google Analytics and into a nice clean JSON API for our client-side static app to consume:
There was a bit of a kerfuffle around Healthcare.gov sending data to analytics services (despite that being industry standard practice), so I'm curious if there's any push to ditch Google Analytics, etc., for .gov, and move analytics services in-house.
Can you be more specific about what you mean by "kerfuffle"? This is the same healthcare.gov that was/is sending PHI data to 3rd parties analytics and ad services (doubleclick, etc)?
Man I love the fact that you guys continuously contribute to this community. I hope whatever you are doing spread to other parts of the government. I really enjoy seeing the government engage the community in this way.
clap clap clap. Keep up the great work.
JFYI, under operating systems, "Macintosh" should be "OS X". Macs are the hardware, OS X is the operating system that runs on the hardware. :)
Also, Apple hasn't called their product line "Macintosh" for over fifteen years. It's an anachronism as blatant as replacing Windows with "IBM PC".
Very nice, open-source FTW.
Everything we do at 18F is open source, usually as public domain. Interested in working on this stuff? Come work with us: https://18f.gsa.gov/2015/02/25/We-Are-Hiring/
what's the difference between 18F and USDS? i applied to USDS and never got a reply.
Here's an article that describes the difference:
http://www.washingtonpost.com/blogs/the-switch/wp/2014/08/11...
So did I, months ago. No reply.
404 on the link to the 1,350 domains: https://catalog.data.gov/dataset/gov-domains-api
Should be fixed momentarily, if it's not already. Thanks for pointing that out!
28.3% on IE, 41% on Windows 7. Wow.
It's odd, being in the tech bubble, you forget how many people there are that really don't think about their technology (or work somewhere for whom the most important things are cost and stability).
It would be interesting to take a poll of what people think the numbers would be like before they saw them. I would have guessed a lot more Chrome/FF.
What's wrong with Windows 7? The majority of the world runs Windows and 7 is a fine choice. It's rock solid and there's really no reason to upgrade to 8.
Maybe they're worried that Windows 7 is only 41%?
I think W7 is the de facto for businesses now.
Nothing is "wrong" with it, I'm just surrounded by people who obsess about tech all day and go after whatever is newest and shiniest - if you're not running chromium on your 15" retina macbook pro then you're obviously incompetent.
Turns out for most people stability and cost are much more important than having something incrementally better.
Not passing judgment on one or the other, just thought it an interesting realization.
For what it's worth, I run Windows on a virtual machine to do some stuff and I use 7.
Turns out for most people stability and cost are much more important than having something incrementally better
Or, rather, people have different definitions of better. For many people, cost and stability are huge components of their personal calculation. I think it would be more appropriate for you to say:
having something incrementally NEWER
I work in tech and take the latest shiniest things very seriously. But when it comes to PCs I don't put Windows 8 in that category, for me it's an egregious regression in UX quality and pleasure of use than Win7. Hope springs eternal for windows 10, but for me Win8 is so packed full of annoyances that I would never consider it to be an upgrade from 7.
Have to agree here... my company even develops primarily for Win8, so its not as if I'm not used to using it, but I still vastly prefer Windows 7.
I'd love to see the stats for HN for browser use.
On February 20, I had a story hit the front page of HN on a brand-new blog that didn't receive traffic from anywhere else.
Here are some screen shots from my analytics.
OS: http://i.imgur.com/oZNOxCK.png
Browser: http://i.imgur.com/84DGWHz.png
For the amount of *nix people I see showing out on here, I'm surprised that it's only 10% - although for the Web in general, that's an absurdly high number, still!
Some percentage of Windows visits could be browsing at work, maybe?
Hmm...
HN could use the user's ISP to guess whether they were home or at work, then correlate that with OS data.
That's actually an interesting idea!
>stability and cost
If this were truly the case, they wouldn't be using Windows. It's mostly inertia in large organizations and for the general public.
I'm not a sys admin by any means, so I may be out of the loop here - but is there a viable alternative for management of tens of thousands of systems and users, outside of Active Directory?
As good as *nix/Apple stuff is, is it really feasible to run it at scale across an org, without building custom deployment tools each time?
There were giant UNIX networks before there were giant Windows networks, using things with names like NIS/YP and NFS. IMO building custom deployment tools is kind of the point of using not-Windows, but I'd wager Canonical, RedHat, and/or Oracle have something for central management and deployment of Linux systems.
I'm actually surprised at how low the IE number is - for government websites. Obviously you can't compare it to the traffic on HN or Techcrunch, where like 70 percent must be on Chrome and 5 percent on IE, but seeing how some analystics sites still put IE at 50 percent market share, I think that's quite impressive.
Or NetApps' market share numbers are crap, and StatCounter's (and others') numbers are a lot closer to reality - which is that Chrome dominates the web browser market.
I slapped this together recently using Chartbeat's realtime data (so the sources are major sites – nytimes.com, foxnews.com, espn, etc): http://percentoftheinternet.com
http://percentoftheinternet.com/windows http://percentoftheinternet.com/chrome http://percentoftheinternet.com/ie
...etc...
Raw data is here: https://api.chartbeat.com/cbtotal/?v=2
NetApps numbers appear to be based on a small number of corporate sites, which are skewed towards US media companies. They weight the numbers by the CIA World Factbook data for internet usage in a country but they've never given a reason to believe this approach doesn't exaggerate the distortions present in their sample of data.
If you want reliable numbers, nothing beats direct measurement for a comparable site. I've never worked on a site where traffic was anywhere near NetApps’ but I've found Akamai’s numbers to be reasonably close for sites which target a general audience:
IE8 is just 3%, great news. On par with what we see.
Most people whinge about IE8 support, yet their mobile experience is awful. 25% of these hits are mobile, we see close to 40-50% for us, with much higher for hospitality clients (bars, restaurants), though they usually get <3k sessions a month.
Similar IE8 and mobile numbers here. IE8 support used to be a main focus because 3% of a really big number is still a pretty big number. Then we started paying attention to mobile stats and realized how much we were missing in the mobile experience.
What would someone who really thought about their technology run? Why would the most important things for an operating system not be cost and stability?
e: Realise this comes off as confrontational. Genuine questions!
Stability and features tend to fight each other.
It is interesting to see the IE numbers, I suspect it is because many government computers do not have other options. When I was in the military it was all we had. CAC cards also only work properly in IE, on government sites at least, so that will likely also skew the numbers. We of course were running extremely outdated versions too, so that may be part of the higher 8/9 numbers then you might expect.
As someone that has had a CAC, it does work on non-IE browsers (including on non-Windows OSes) for most government sites.
It does but half the crap they have us use runs on flash and only works in ie8...
I would have guessed much higher for IE (closer to 50%). I'm glad that it is ~30%, as that means we have a relatively competitive market with a diverse array of quality products to choose from (and yes, that includes recent versions of IE).
If you'd like to build things like this Dashboard, and help fix the services Americans depend on, we are hiring. We have an amazing core already, and are growing fast. Read more about the U.S. Digital Service and apply at: https://www.whitehouse.gov/usds
Our pitch is pretty simple: Fix Stuff That Matters.
While you might be hiring, it was remarkably hard to find any details on job listings. (edit: I don't think I actually found any.) The closest I could find was by following many links and eventually getting a web form where I could give my contact info for joining the Digital Service.
Is there a chance you could put a more convenient link directly to your listing of jobs (or a friendly page that links directly to USAJOBS, or explains about that painful tar baby) at the bottom of the main 18F page?
This 18F blog post talks about the different types of roles people play on our team: https://18f.gsa.gov/2015/02/25/We-Are-Hiring/ . The application form for the Presidential Innovation Fellowship program also calls out specific skills and domain expertise they're looking for: https://pif.gsa.gov/ . Cheers!
From our Website:
There are several ways you can plug in to the big problems we’re solving. You'll drive vision for a better government, working alongside policymakers, agency leaders, and program teams to build world-class digital services.
Product Managers -- We are looking for product managers who can balance user needs with business requirements and apply technical knowledge to bring world-class digital services to the American people. You'll need to manage complex projects to hit tight deadlines without sacrificing quality. If you feel comfortable reviewing engineering design documents, project plans, and contracts, and you can communicate technical concepts to various audiences up and down the command chain, we're looking for you.
Engineers -- Our engineering leads will drive technical teams to build large-scale, innovative products that serve the American people. You need to be an expert strategizer, fixer, and builder with the engineering chops to roll up your sleeves and push your own code. You'll provide leadership on major technology projects, manage project teams with focus and vision, contribute to product strategy, and support strong teams of engineers.
Designers -- We are looking for designers from an array of disciplines — experts in research, UX, service design, product strategy, visual/UI design, and content strategy. Whatever your flavor, you embrace complexity and revel in drawing simplicity and clarity out of thorny challenges. You are a relentless advocate for the needs of users. You will help lead teams and prototype, test, and iterate on services and products. You will help build the Federal Government’s capacity to deliver services to the American people.
Here it is as a clickable link: https://www.whitehouse.gov/usds
I'm curious: where is that work, geographically?
Many positions are DC-based (at least some part of every month), but several of our digital service teams (including the largest and most mature Digital Service team at GSA called 18F https://18f.gsa.gov) have remote offices and allow remote work. You can indicate your geographic limitations on the application.
Are there ever opportunities for freelance/contract work?
Gov.UK has something similar.
The detail: https://www.gov.uk/performance/site-activity
By department: https://www.gov.uk/performance/web-traffic
All Gov.UK and digital services performance data: https://www.gov.uk/performance
Really interesting to see a spike in people switching from desktop to mobile and tablet at Christmas - and then switching back afterwards!
Or prison visits for that matter:
I love this.
Top Pages right now:
Where's My Refund? - It's Quick, Easy and Secure. 4,600
"You take ~40% of my money every year. And you took too much this year. Give me a little back, please?"
National Weather Service 1,858
"What's the weather gonna be?"
National Weather Service - Forecasts by Region 1,846
"Will it rain today?"
Internal Revenue Service 1,672
"Let me spend a few hundred hours figuring out how to pay you for the weather."
NOAA - National Oceanic and Atmospheric Administration 1,155
"What is the weather today, though, seriously?"
myUSCIS - Case Status 1,053
"Please let me in so that I can work in your private sector."
The United States Social Security Administration 921
"Where's my check this month?"
What on earth are you doing that you pay a 40% effective tax rate and spend a few hundred hours doing your tax return?
Medium volume day trading (hundreds or thousands of transactions per day), where you make $500k per year (which will get you to the 40% marginal tax rate, 31% effective). Have to account for the basis of every trade, all short-term gains/losses, and let's assume they're doing that accounting work themselves (the hundreds of hours is a stretch though), and that they're not plugging a csv file into a program to take care of it for them so it's all manual gain/loss matching.
That's about as close as I could come without getting silly. Even oil companies like Exxon don't end up paying 40% (35% effective I think in their last fiscal year).
If you make $5+ million per year, and have nothing to offset any of that, you can end up paying over 40%. Pretty rare group of people likely to fall into that.
Add the ~8% spent by the employer for Medicare and Social Security, and the 31% effective rate becomes ~39%, doesn't it? For income above $200K, there's also an "Additional Medicare Tax" of 0.9%.
I always hear this comical assumption that the employee would automatically receive the matching Social Security and Medicare tax amounts if they were repealed but do you honestly believe that to be true and it wouldn't just go towards padding margins or buying more office supplies?
Employers by definition want to pay employees as little as possible, only enough so they stay.
I think you may have failed to think of freelancers, who are often employed by their own company.
Secondly, the specific effects of repealing a tax doesn't make it any less real.
I was casually lumping in state income tax. That, the Obamacare 3.8% investment surtax, &c., and the situation is reasonably painful for a whole lot of people.
Taxes are around the lowest they've ever been.
What's the point of this comment, all you're doing is trivializing how people use government websites/programs. Also there is absolutely no way your effective tax rate is near 40% unless you're in the 99th+ percentile of earners, a little melodramatic, no?
> "You take ~40% of my money every year. And you took too much this year. Give me a little back, please?"
In my experience, most people are trying to get the biggest refund possible, rather than minimizing the amount they under/overpay.
Hmm, I was expecting to see what sites government employees were visiting the most. Probably not for a few years at least :)
There's a similar service in place for Gov.UK: https://www.gov.uk/performance/site-activity
Nice to see IE gets updated a lot quicker in the US than it does in the UK (though there aren't browser usage statistics available for Gov.UK sites I don't think...)!
Browser stats for gov.uk sites: http://data.gov.uk/data/site-usage#totals
That is one very sexy government website.
Wow beautiful site, nice work to all involved! I found there's a good blog post about how it all works too: https://18f.gsa.gov/2015/03/19/how-we-built-analytics-usa-go... It's all static pages with JSON data pulled in from reporting services that use Node. Very cool!
This data is really valuable to gauge OS & browser usage. Like anything, I would expect it to have some sampling bias in it, but I can't think of better sources for this data. Google, perhaps. (does Google release this type of data?)
I would love to see some history to understand what's growing, what's in decline, and how rapidly.
What scares me is Safari's Usage is higher than the Mac usage. Does that mean people are using the old Safari on Windows?
Safari (20.4%) < Mac (9.1%) + iOS (16.7%)
Probably not an issue of Safari for Windows, but not accounting for the mobile users. That sums to 25.8% so around 5.4% of Mac + iOS users are using something other than Safari (the vast majority of that is likely Mac users).
The non-Safari visits from Mac + iOS are actually 5.4% of all visits, or about about 21% of Mac + iOS visits.
(5.4% / 25.8% = 20.9%)
Yes, thanks. That's what I meant to write.
Are DHHS and Ed not on there yet? Or do they just not rank very highly?
They're on there - they just aren't popular enough to rank. Some DHS services are there (like USCIS).
The petitions are ranking pretty high...
Really surprised Safari accounts for 20% of the visitors on Gov sites but MacOS only 9.1%. Means 10.9% of those are Windows Safari users.
Mobile safari?
http://i.imgur.com/H3E8NMh.png
They can't get a number sorting algorithm right? :/
Look slightly to the right and you'll see why.
It is obviously sorted by % use.