D3, Conceptually - Lesson 2: Charts

We start this lesson with a simple (and completely contrived) problem to solve. Word has it the world might be ending; some aliens have contacted the leader of the world (pretend, will you) and demanded to be shown a bar chart of the even numbers up to 10. Some sort of test, we gather. If we can't deliver, they promise to destroy the world. A far-fetched request, but it sure beats a whale. Really, who wants a whale?

So, how to begin? As everything, with our data: var evenNumbers = [0, 2, 4, 6, 8, 10]; Good, good. Now, the aliens also demanded the graph have a width of 600 pixels and height of 100—those numbers please them (don't ask how they share our definition of pixels). Putting those in variables won't hurt: var width = 600; var height = 100; And why not arbitrarily decide the bars will be stacked vertically, each extending to the right in proportion to the represented number. And we'll use the full 600x100 pixels. If we have six bars, that must make each of them height / 6 pixels tall. Hardcoding 6 doesn't really strike me as a good idea, considering the example we set above. Plus, the aliens might want more numbers in the future. Insteand, we make sure however many bars we have will fit: var barHeight = height / evenNumbers.length; What about the width of each bar? Well, that depends on the number it represents. We have 600 pixels to work with; giving 10, the largest datum, a width of 600 pixels will work fine (and zero, obviously, gets zero). A function for that would be: var barWidth = function(number) { return number * (width / 10); }; "Aha!", you might say, "shouldn't that also vary based on the data, and not hard-coded? Aren't you setting a bad example for the kids?". And I would reply with a proper function (and, possibly, a not-so-proper gesture): var maxDataValue = d3.max(evenNumbers); var barWidth = function(number) { return number * (width / maxDataValue); }; A rect element in SVG is defined by its width, height, x, and y attributes. We have two of them (width and height), so lets get the others.

Remember, in SVG the top-left corner is (0, 0). x-coordinates increase to the right and y-coordinates increase down the page.

x is quite easy, actually. All bars should be aligned along the left axis, so they all have an x of 0. y takes a bit more thought, but not much (save those brain cells for something else). To place the bar for 0 at the top of the chart, and that for 10 at the bottom, the i-th even number should be offset by i * barHeight. Or, in a function: var barY = function(index) { return index * barHeight; }; Now, we know from the last lesson that barY is probably going to be provided as an argument to attr, and when a function is provided to attr it gets two arguments: first the datum, then the index. So we change the function to: var barY = function(datum, index) { return index * barHeight; }; Putting all of this together, and choosing arbitrary colors, we get:

<div id='chart'></div> var width = 600; var height = 100; var root = d3.select('#chart').append('svg') .attr({ 'width': width, 'height': height, }) .style('border', '1px solid black'); var evenNumbers = [0, 2, 4, 6, 8, 10]; var maxDataValue = d3.max(evenNumbers); var barHeight = height / evenNumbers.length; var barWidth = function(datum) { return datum * (width / maxDataValue); }; var barX = 0; var barY = function(datum, index) { return index * barHeight; }; root.selectAll('rect.number') .data(evenNumbers).enter() .append('rect') .attr({ 'class': 'number', 'x': barX, 'y': barY, 'width': barWidth, 'height': barHeight, 'fill': '#dff', 'stroke': '#444', });

New syntax! You might have noticed the attr function here was passed an object literal. Bet you haven't seen that before. Join the club. D3 is under constant development and new features get added with some frequency. Your author tries to keep up, but can't always be bothered. Hopefully you pick better role models. Passing a map to the attr function (among others) is shorthand for calling attr for each (key, value) pair in the map.

Contrary to the example we've set, SVG provides more primitives than the lowly rect. And you know what that means: more contrived examples. Let's consider a new graph - a self-descriptive one. A plot of circles, where the x-axis is the radius of the circle and the y-axis is the number of pixels in that circle (its area).

The circle element is just what we need. Like the rect before it, and all the future SVG elements we render, it supports a set of presentation attributes.

Huh? What? Afraid you fell asleep in class and missed the definition of the word? Nope, the teacher is just making you sweat. The presentation attributes are the attributes which determine how a shape is rendered. We've already seen fill and stroke. There are more to come, but we won't be exhaustive. The SVG standard is, while not exactly readable, easy to browse and full of examples. Consider browsing it over coffee. Or not, actually. Don't do that.

While the presentation of a circle is the same as a rect, the positioning isn't. We need three attributes: The circle is centered at (cx, cy) and has a radius of r. So, building it much like the last one with a few changes: var circleX = function(radius) { return radius; }; var circleY = function(radius) { return Math.PI * radius * radius; }; Let's try radius values of [1, 2, 3, 5] as a starting point for the graph.

<div id='chart'></div> var width = 600; var height = 100; var root = d3.select('#chart').append('svg') .attr({ 'width': width, 'height': height, }) .style('background', 'white'); var circleRs = [1, 2, 3, 5]; var circleX = function(radius) { return radius; }; var circleY = function(radius) { return Math.PI * radius * radius; }; var circleR = function(radius) { return radius; }; root.selectAll('circle') .data(circleRs).enter() .append('circle') .attr({ 'cx': circleX, 'cy': circleY, 'r': circleR, 'fill': '#ffcdcd', 'stroke': '#666', });

Hrm, not the best graph. We have a few obvious issues: first, 100 pixels of height is not going to cut it at this scale if we want to have larger circles. Second, our elements are scrunched along the left. Thirdly, our y-axis is reversed from a normal graph. The third is easy to solve - instead of mapping the radius to Math.PI * radius * radius, we subtract that value from the height of the graph: height - Math.PI * radius * radius. Now top is bottom and bottom is top. The first two issues are both classes of the same problem - a scale where 1 unit along the axis is 1 pixel is tiny. Miniscule. Laughable. Infinitesimal. No, wait, that isn't right. We can definitely both see and measure the circles, so they must not be infinitesimal. Got carried away with my adjectives.

When drawing a graph on paper, your scale is more likely to be closer to 10mm to 1 unit (we don't use imperial units here; we are snobby intellectuals). This is fixed by scaling our computed cx and cy values by an arbitrary number - 4 sounds good. Never done me wrong, trusty 4. We'll also need to change our bounds; if the largest circle is has a radius of 5, we would like height - 4 * Math.PI * 5 * 5 to not be negative; 350 is big enough for that. Now, our new (and maybe improved) chart:

<div id='chart'></div> var width = 600; var height = 350; var root = d3.select('#chart').append('svg') .attr({ 'width': width, 'height': height, }) .style('background', 'white'); var circleRs = [1, 2, 3, 5]; var circleX = function(radius) { return 4 * radius; }; var circleY = function(radius) { return height - 4 * Math.PI * radius * radius; }; var circleR = function(radius) { return radius; }; root.selectAll('circle') .data(circleRs).enter() .append('circle') .attr({ 'cx': circleX, 'cy': circleY, 'r': circleR, 'fill': '#ffcdcd', 'stroke': '#666', });

In the tradition of all my favorite textbooks, I leave deciding if that is an improvement as an exercise for the reader. It really isn't much of an improvement. Not much could save the chart, really. Terribly dreadful, that.

While the previous example wasn't bad code, it isn't exactly idiomatic D3. It works, but D3 provides helpers to make our code easier to read and our intent clear. For example, d3.scale. Note, in the last example, we inserted some magic values in our attribute functions (like 4 and height). But these were separate from the data we actually cared about - our cx is just the radius on some arbitrary scale, and the cy is likewise a number on some scale. In case the name, and this hammering of a point, did not clue you in, d3.scale and its many members are designed to assist us.

Before we introduce our first scale, we'll define what a scale is. A scale is a function that maps input values to output values. Being mathematically-minded, the proper name for input is domain, and that of output is range. Neither the domain nor range need be numbers, but starting there will be easiest. The simplest scale, then, is the identity function. We used that, implicitly, in the first circle example (the "bad" one). We mapped our data (radius) values to a value along some axis, then scaled that axis using the identity function.

In the second, questionably improved, example, we scaled that axis by multiplying by 4. Our scale function was then function(x) { return 4 * x; };. This is an example of a linear scale (as, an astute reader will note, is the identity scale). D3 provides d3.scale.linear for just this purpose.

The documentation for it is useful, but you can save the for later. The short of it is thus: d3.scale.linear() returns a new object that is also a function. The two methods of current interest on the object are domain and range. Both take an array of values that specify, respectively, the domain and range of the scale function. An example would prove instructive:

<div id='root'></div> var root = d3.select('#root'); var scale = d3.scale.linear() .domain([0, 10]) .range([0, 100]); root.selectAll('div') .data([0, 10, 5, -5, 100]).enter() .append('div') .text(function(d) { return 'The scaled value of ' + d + ' is ' + scale(d); });

The first two lines are, hopefully, exactly as expected - an input of 0 (the first element of the domain) maps to an output of 0 (the first element of the range), just as 10 maps to 100. The third value, 5 should also make sense - our scale is linear, so an input value halfway between the domain values of 0 and 10 maps to an output value halfway between the respective range values—0 and 100.

The other two may be surprising, but follow from above. If an input value falls outside the domain, the scale pretends the domain and range are extended to include the value. If you don't like this behavior, you can call .clamp(true) on the scale to force all values to fall within the domain.

Linear scales actually accept an array length at least 2 for the domain and range. Specfying more than 2 values is a more advanced feature which may prove useful later, but now only serves to confuse. Read the docs or attempt to guess the semantics if you want, or just carry on with your life. All are valid paths.

We can use our scalar powers (sweet! only a Levenshtein distance of 4 from super!) to improve the second circle example. Now, we have a lot of options available to us - anything that results in 1 mapping to 4 will work (as our scale just multiplied by 4). So what values to use for the domain and range? Why not the simplest - a domain of [0, 1] and a range of [0, 4].

<div id='chart'></div> var width = 600; var height = 350; var root = d3.select('#chart').append('svg') .attr({ 'width': width, 'height': height, }) .style('background', 'white'); var circleRs = [1, 2, 3, 5]; var xScale = d3.scale.linear() .domain([0, 1]) .range([0, 4]); var yScale = xScale; // Our x and y use the same scale. var circleX = function(radius) { return xScale(radius); }; var circleY = function(radius) { return height - yScale(Math.PI * radius * radius); }; var circleR = function(radius) { return radius; }; root.selectAll('circle') .data(circleRs).enter() .append('circle') .attr({ 'cx': circleX, 'cy': circleY, 'r': circleR, 'fill': '#ffcdcd', 'stroke': '#666', });

And I'm not going to let it rest until that code is good (even if the source of the chart is contrived). The yScale used in the example is, frankly, an embarrassment. Look at our circleY function again: var circleY = function(radius) { return height - yScale(Math.PI * radius * radius); }; We still have parts of our conversion from axis to pixels (our original motivation for scales) outside of the scale definition. I'm speaking, of course, of that pesky height - that begins our function. So, to fix it, we need to put on our thinking caps.

We need two input values and two output values for a scale. We have a trivially easy one - the input value of 0 maps to the output value of height. That was fun! Plug and play sure is easier than Windows 95 led me to believe... and we can just put in another input value to get another output value.

But what input value? We can choose one arbitrarily and it will just work. Why not 1, which maps to height - yScale(Math.PI * 1 * 1) or simply 337.4336blahblahblah. Or actually simpler in the form of height - 4 * Math.PI. Just put that in our code, right? Problem solved? var scaleY = d3.scale.linear() .domain([0, 1]) .range([height, height - 4 * Math.PI]); Not so fast. The only thing we've accomplished is moving a magic number from one place to another. The 4 introduced into this function comes from our xScale, and the Math.PI comes from our mapping of data values to y values. We have conflated two things: the mapping from data to y, and the mapping from y to pixels. We can make this more clear by introducing functions: var dataToY = function(d) { return Math.PI * d * d; }; var yScale = d3.scale.linear() .domain([0, 1]) .range([height - 4 * dataToY(0), height - 4 * dataToY(1)]); var circleY = function(d) { return yScale(dataToY(d)); }; Now our path from the data bound to each circle to the final resting y is clear.

Another option would have been a different set of data values. Instead of an array of radius numbers, it could have been an array of objects, each with a radius and area property. This would have removed the need for the dataToY function at the cost of more verbose data. There are reasons for keeping the bound data simple and reasons for binding very complex objects, and no easy rule to follow.

Before moving on to more advanced data bindings, we'll feature one last example that introduces some additional elements of SVG used in most visualizations. We'll start with a description of the problem, the complete solution, and follow it with a explanation of some of the new bits. So, let's graph!

But what? Histograms. Simple, and easy. And practical, too. Pretend (or not, as necessary) that you play a tabletop roleplaying game as some character that hits things with other things. You just got a sweet new things to hit other things with - an eldritchian quadruple axe (it's pretty cool, trust me). When you hit things with this thing, you roll 2d4 + 2d6 for damage, assuming the enemy fails their sanity check, and half rounding down otherwise. Now, you received DJs boombox of "Call Me Maybe" in your last quest (for those not familiar with the game, that is a cursed amulet that makes everyone within 50m fail their sanity checks) so the math here is going to be easy. Which is good, because some members of your party just stare at you blindly when you say probability mass function and you just want to demonstrate how awesome this new axe is. You need a pretty picture to illustrate.

A picture of a histogram, of course.

We'll simulate 500 attacks with your axes and graph the resulting histogram. And we'll pick, arbitrarily, 640x480 pixels as the size. With a title and nicely-labeled axes for our axes.

<style type='text/css'> svg { border: 1px solid black; background: white; } .axis .domain, .axis .tick { stroke: #000; fill: none; } .title { fill: #666; font-family: Helvetica, sans-serif; /* Helvetica is cool, right? */ text-anchor: middle; font-size: 24px; } .bar { fill: #fcc; stroke: #444; } </style> <div id='chart'></div> var width = 640; var height = 480; var root = d3.select('#chart').append('svg') .attr({ 'width': width, 'height': height, }); // Render the title. var titleHeight = 50; root.append('text') .attr({ 'class': 'title', 'x': width / 2, 'y': titleHeight / 2, }) .text('Skull-splitting power!'); // Simulate 500 rolls of the axe. var rollDie = function(numSides) { return 1 + Math.floor(Math.random() * numSides); }; var MAX_ROLL = 4 + 4 + 6 + 6; var rollHisto = d3.range(MAX_ROLL + 1).map(function() { return 0; }); for (var i = 0; i < 500; i++) { var rolled = rollDie(4) + rollDie(4) + rollDie(6) + rollDie(6); rollHisto[rolled]++; } // Render our axis. var yAxisWidth = 50; var xAxisHeight = 50; var xScale = d3.scale.linear() .domain([0, rollHisto.length]) .range([yAxisWidth, width]); var yScale = d3.scale.linear() .domain([0, d3.max(rollHisto) * 1.2]) .range([height - xAxisHeight, titleHeight]); var xAxis = d3.svg.axis().scale(xScale); root.append('g') .attr({ 'class': 'x axis', 'transform': 'translate(0,' + (height - xAxisHeight) + ')', }) .call(xAxis); var yAxis = d3.svg.axis().scale(yScale).orient('left'); root.append('g') .attr({ 'class': 'y axis', 'transform': 'translate(' + yAxisWidth + ',0)', }) .call(yAxis); // Render the dice bars. root.selectAll('rect.bar') .data(rollHisto).enter() .append('rect') .attr({ 'class': 'bar', 'x': function(d, i) { return xScale(i - 0.5); }, 'width': xScale(1) - xScale(0), 'y': yScale, 'height': function(d) { return yScale(0) - yScale(d); }, });

Phew! That was quite a bit of code, and much of it new. First order of business: this example used CSS. Yes, CSS. You see, I know I mentioned this in the last lesson as an advanced topic. Congratulations, your SVG is now advanced! Gold star! Styling your charts the same way you would style your documents is done for the same reason - keeping presentation separate from content is good. It is a little more complicated with CSS as some things you think of as presentation in CSS are, conceptually, part of the content in SVG. Namely, the position and dimension of an element.

Some readers may have spotted another difference: some of those CSS properties aren't valid. True, text-anchor isn't a style property for HTML elements, but it works just fine for SVG. The complete list of all valid properties to put in CSS is available in the spec.

In addition to text-anchor, there is another commonly used attribute: dy. The dy of a text element is the amount, in any units, the text should be offset from its defined y position. How is this useful? Vertically aligning text. There are some "magic" values one can use with dy to get a variety of effects. A dy of 0.3em vertically centers the text along the defined y position. A dy of 0.7em aligns the top of the text at the defined y position. And, obviously, a dy of 1em moves the text down a line from the defined y attribute. These numbers aren't actually magic, but follow from the definition of the em unit.

The next new bit is the text element used for the title. Just to confuse you, text in SVG behaves nothing like in HTML, nor like rects in SVG. Ignoring any centering or offsets, the x and y attributes determine the left and baseline, not top or mid, positions of the initial character. So far, so good. The big annoyance: it does not wrap. It will overflow as necessary and you can't easily prevent this. It sucks. For serious text layout in SVG, I suggest two good options. If you know your data, lay your text out by hand, assuming everything will fit (which it will, because you can modify as needed). If you don't know your data, I suggest using HTML - either with absolute positioned nodes or with a foreignObject (to embed HTML within SVG). Neither of the HTML options are great, mind you, but they do work.

Another option for laying out unknown text is to render, offscreen, the text you wish to layout and use some of the DOM interface methods to query the size of the text, such as getComputedTextLength. Alternatively, you can use the getBBox method (on the SVGLocatable DOM interface) to get the height as well.

Lets talk about the presentation a bit: we have a title that takes up the top of the chart, and x-axis that takes up the bottom, and a y-axis on the left. Each bar is centered on the dice roll it represents, and our data on how many times the dice summed to n is in rollHisto[n]. The area for drawing bars in then spans from the end of the y-axis to the edge of the chart.

You may have noticed a few variables scattered about the code - in addition to the width and height of previous examples, we additionally have xAxisHeight, yAxisWidth, and titleHeight. Knowing this, our x scale should be obvious; the value 0 maps to the edge of the y-axis, and the value rollHisto.length - 1 (the highest data value to graph) should be close to, but not at, the edge of the chart. Hence: var xScale = d3.scale.linear() .domain([0, rollHisto.length]) .range([yAxisWidth, width]); Which, I might add, is a lot more obvious than the constant-less version with a range of [480, 50]. The y scale is similar, with the added complexity of it being inverted; a value of 0 maps to the highest y-value, namely height - xAxisHeight.

Going back to our "not-so-rubbish" circle example, we could improve it yet again. Instead of defining the xScale and yScale as having domains of [0, 1], we can define them as having a more natural domain. Like [0, d3.max(circleRs)] for the xScale and [0, dataToY(d3.max(circleRs))] for the yScale. The ranges, then, would be [0, width] and [0, height]. Our scale is no longer perfect, but we could change the definition of width and height to get the original definition. This isn't as simple as I've led on, hence ignoring it. But, generally, domains and ranges should be natural from the domain of data you wish to render to the range of pixels you have to render in.

The scales we have also let us do some nice tricks. The center of each bar is at xScale(i), as our scale is defined. This means the left edge should be halfway between i - 1 and i - which we can put directly into the definition as xScale(i - 0.5). The width of each bar, then, should be the distance between i - 0.5 and i + 0.5. As our scale is linear, each bar has the same width and we simplify this to xScale(1) - xScale(0). As before, so again, the y-scale is defined in a similar fashion, using the yScale to determine the y and height.

The next wrinkle is our axes - the most confusing, and new, piece of code. The easiest part to explain is the wonderful g element. We use this element mostly for two purposes: it can have children elements, and it allows one to apply a transform to those children. This element forms the bread-and-butter of many interesting graphs, paralleling the uses of a div element in HTML. It allows elements to be grouped together, both in the structure of the document and in the presentation.

We commonly use the transform attribute just mentioned for its translate form - to offset a set of elements from their normal position by a set amount in the x and y directions. The other forms, which allow a full range of affine transformations, won't be covered here.

So how does that help us? Well, the d3.svg.axis helper isn't the smartest, or most configurable, kid in the shed. This is by design (I think). d3.svg.axis is a function that takes a selection and appends elements to it that resemble a labeled axis. By default, drawing a line from the start to the end of some scale along the x-axis, positioned with a y of 0, with ticks and labels below. The simplistic interface doesn't allow us to specify where this is positioned. If you browse the documentation you won't find any such method. So, g to the rescue! Translating all the elements rendered for the axis down by height - xAxisHeight gets the desired position.

You'll also notice the docs don't mention anything about the presentation of the axis elements, so I'll clue you in. There is a path element that runs parallel to the axis with a class of domain and many line elements forming the perpendicular tick marks with, conveniently, a class of tick. The text labels have no class. Given all of this information, we can easily use CSS to change the display, even hiding bits if we don't like them with a stroke and fill of none.

To determine which way the axis runs, and which side the labels get placed on, you use the orient method. And, to define the scale used, the scale method is used.

The rendered axis goes from the start of the scale to the end, so if you had used a not-so-obvious scale definition, it would not work out so nicely. Like in our previous example of a y-scale with a domain of [0, 1].

And that is about it. Except that pesky call method we called with the d3.svg.axis object. This is a helper method; selection.call(fn) is equivalent to fn(selection); return selection; - enabling method chaining.

A bit cleaner, if not shorter. In the next lesson, we'll dive back into data bindings. As you may have learned to expect, I've hidden a bit of complexity in previous descriptions. But, trust me, it was for the best. So, next time, "you got data in my data!" and other adventures in binding land.