D3.js Intro

Code-District Meetup

Dan Isla
Data Scientist at JPL
July 9, 2015
https://github.com/danisla/d3-talk

About Me

Showcase some of our visualizations

Visit us at the IT Data Science web site:

http://itds.jpl.nasa.gov/

What is D3.js

Creating document structure

Getting D3

Get from CDN:

https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js

Get via Bower:

bower install d3

Get via npm

npm install d3

Github wiki: https://github.com/mbostock/d3/wiki/API-Reference

Skeleton DOM for Visualization

<html>
    <head>
      <link rel="stylesheet" type="text/css" href="style.css">
      <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js"></script>
      <script src="script.js"></script>
    </head>
    <body>
        <h1>D3</h1>
        <div id="chart"></div>
    </body>
</html>

Solarized stylesheet compiled from .scss

Source: http://ethanschoonover.com/solarized

/* light is default mode, so pair with general html definition */
html, .light {
  background-color: #fdf6e3;
  color: #657b83; }
  html *, .light * {
    color: #657b83; }
  html h1, html h2, html h3, html h4, html h5, html h6, .light h1, .light h2, .light h3, .light h4, .light h5, .light h6 {
    color: #586e75;
    border-color: #657b83; }
  html a, html a:active, html a:visited, .light a, .light a:active, .light a:visited {
    color: #586e75; }

.dark {
  background-color: #002b36;
  color: #839496; }
  .dark * {
    color: #839496; }
  .dark h1, .dark h2, .dark h3, .dark h4, .dark h5, .dark h6 {
    color: #93a1a1;
    border-color: #839496; }
  .dark a, .dark a:active, .dark a:visited {
    color: #93a1a1; }

html * {
  color-profile: sRGB;
  rendering-intent: auto; }

/*# sourceMappingURL=style.css.map */

Experiment using codepen.io

http://codepen.io/

  • Great for simple editor and instant feedback of changes.

Controlling HTML DOM Elements

Change style of elements on the page

  • Chained method calls
d3.select("h1")
    .style("font-size","48px")
    .style("color","#2aa198")

Change text of elements

d3.select("h1")
    .text("Hello D3")

Add elements to the page

d3.select("body")
    .append("img")
    .attr("width","400px")
    .attr("src", "http://apod.nasa.gov/apod/image/1507/2015Jul8_plutoNH_nasajhuaplswri950.jpg")

Binding data to the DOM

Bind from Javascript Data Structures

  • Use the .data method to bind data to a selection.
  • Use the .enter method to select "placeholder" elements that will be added in the future.
d3.selectAll(".item")
    .data(["Earth","Mars","Saturn","Jupiter"])
    .enter()
    .append("div").classed("item",true)
    .text(function(d,i) { 
      return d;
    })

More Complex Data Structures

  • Use the function alternatives for method arguments to transform data.
d3.select("h1").text("Relative Planet Sizes in our Solar System ... and Pluto")

var data = [
      {name: "Mercury", color: "#839496", radius: 1516},
      {name: "Venus", color: "#b58900", radius: 3760},
      {name: "Earth", color: "#268bd2", radius: 3963},
      {name: "Mars", color: "#dc322f", radius: 2110},
      {name: "Jupiter", color: "#d33682", radius: 44423},
      {name: "Saturn", color: "#b58900", radius: 37449},
      {name: "Uranus", color: "#586e75 ", radius: 15882},
      {name: "Neptune", color: "#268bd2 ", radius: 15882},
      {name: "Pluto", color: "#b58900", radius: 715},
    ]

Add the items and their text to the DOM

d3.selectAll(".item")
    .data(data)
    .enter()
    .append("div").classed("item", true)
    .style("font-size", "24px")
    .text(function(d) { 
      return d.name;
    })

Use CSS styling to visualize the data

.style("color", "white")
.style("background-color": function(d) { 
    return d.color 
})
.style("width": function(d) { 
    return (d.radius/10) + "px" 
})

Drawing SVG graphics with D3

What is SVG?

  • Scalable Vector Graphics - SVG
  • html and css-like markup of graphics elements.
  • Programatic definitions via JS and event binding for interactions and transitions.
  • Portable and consistent rendering across devices.

SVG Basics

  • rect: Rectangle with attributes x, y, width, and height
  • circle: Circle with attributes: cx, cy, and r
  • Coordinate system starts from the top left

SVG Hello World

<svg width=400 height=300>
  <circle cx="20" cy="20" r="10"/>
</svg>

Add a border to the SVG box:

svg {
  border-width: 1px;
  border-style: solid;
}

SVG Hello World with D3

d3.select("h1").text("SVG Hello World")

d3.select("#chart")
    .append("svg")
      .attr("width",600)
      .attr("height", 400)

Add the first shape, a rectangle.

.append("rect")
  .attr("x", 0)
  .attr("y", 0)
  .attr("width", 200)
  .attr("height", 200)
  .attr("fill","#002b36")

Add a circle inside the rectangle.

d3.select("svg")
  .append("circle")
  .attr("cx", 100)
  .attr("cy", 100)
  .attr("r", 100)
  .attr("fill", "#268bd2")

Planet Sizes with SVG

d3.select("h1").text("Relative Planet Sizes in our Solar System ... and Pluto")

var data = [
  {name: "Mercury", color: "#839496", radius: 1516},
  {name: "Venus", color: "#b58900", radius: 3760},
  {name: "Earth", color: "#268bd2", radius: 3963},
  {name: "Mars", color: "#dc322f", radius: 2110},
  {name: "Jupiter", color: "#d33682", radius: 44423},
  {name: "Saturn", color: "#b58900", radius: 37449},
  {name: "Uranus", color: "#586e75 ", radius: 15882},
  {name: "Neptune", color: "#268bd2 ", radius: 15882},
  {name: "Pluto", color: "#b58900", radius: 715},
];

Add SVG and dark background

d3.select("#chart")
    .append("svg")
      .attr({width: 640, height: 500})
    .append("rect")
      .attr({
        width: "100%", 
        height: "100%",
        fill: "#002b36"
      })

Bind the data and create overlapping circles.

d3.select("svg").selectAll("circle")
.data(data)
.enter()
.append("circle")
  .attr({
    cx: 10,
    cy: 10,
    r: function(d) { 
        return d.radius/100 
    },
    fill: function (d) { 
        return d.color 
    }
  })

Using Scales

  • Scales are the D3 way to let the library calculate mappings between different domains and ranges.
  • Helps you dynamically layout elements in your SVG graphics.
  • Lets your visualization scale automatically to your dataset.

API Reference: https://github.com/mbostock/d3/wiki/Scales

Types of Scales

  • Quantitative Scales - for continuous input domains, such as numbers.
    • Good for gradients and interpolation between discrete points.
  • Ordinal Scales - for discrete input domains, such as names or categories.
    • Good for dynamic positioning of elements.
  • Time Scales - for time domains.
    • Good for time-series data, x-y plots.

API Reference: https://github.com/mbostock/d3/wiki/Scales

Ordinal Scales

  • Think of as a mapping from one set of values to another.

For example, planet names to radii:

var scale = d3.scale.ordinal()
    .domain(["Earth","Mars","Jupiter"])
    .range([3963, 2110, 44423])

console.log(scale("Earth"));
// Output: 3963

Dynamic Positioning with Ordinal Scales

  • Use the .rangeBands scale to create evenly-spaced discrete intervals

https://github.com/mbostock/d3/wiki/Ordinal-Scales#ordinal_rangeBands

Planet sizes with Ordinal scale positioning

d3.select("h1").text("Planets of our Solar System ...and Pluto")

var data = [
  {name: "Mercury", color: "#839496", radius: 1516},
  {name: "Venus", color: "#b58900", radius: 3760},
  {name: "Earth", color: "#268bd2", radius: 3963},
  {name: "Mars", color: "#dc322f", radius: 2110},
  {name: "Jupiter", color: "#d33682", radius: 44423},
  {name: "Saturn", color: "#b58900", radius: 37449},
  {name: "Uranus", color: "#586e75 ", radius: 15882},
  {name: "Neptune", color: "#268bd2 ", radius: 15882},
  {name: "Pluto", color: "#b58900", radius: 715},
]

var w = 800,
    h = 400;

Create the ordinal scale

// Scale mapping position in data array to width of chart.
var xScale = d3.scale.ordinal()
    .domain(d3.range(data.length))
    .rangeRoundBands([0, w], 0.3);

SVG chart with dark background

d3.select("#chart")
    .append("svg")
      .attr({width: w, height: h})
    .append("rect")
      .attr({width: "100%", height: "100%", fill: "#002b36"})

Create circles using ordinal scale for positioning

d3.select("svg").selectAll(".planet")
    .data(data)
    .enter().append("g")
    .classed("planet", true)
    .append("circle")
      .attr("cx", function(d,i) { 
          return xScale(i);
      })
      .attr("cy", (h/2))
      .attr("r", function(d) { 
          return d.radius/1000;
      })
      .attr("fill", function (d) { 
          return d.color;
      })

Add some text labels

d3.selectAll(".planet")
  .append("text")
  .text(function(d) { return d.name})
  .attr("fill","white")
  .attr("font-family", "sans-serif")
  .attr("font-size", "18px")
  .attr("x", function(d,i) { return xScale(i) - 20})
  .attr("y", (h/2 + 80))

Events and Transitions

  • Use events to add interactivity to visualizations.
  • D3 has methods to add events to keep everything in sync.

Add mouseover event

.on("mouseover", function(d) {
    d3.select(this)
      .style("opacity", 0.5);
})
.on("mouseout", function(d) {
    d3.select(this)
      .style("opacity", 1.0);
})
  • Note that the data d is passed to the event handler, different compared to standard Javascript events.

Add transitions

Use a transition to animate the size of the circles.

.transition()
  .attr("r", function(d) {
    return d.radius/1000;
  })
  .duration(1400)
  .ease("elastic")

Another transition to animate the text.

.transition()
  .attr("font-size", function(d) {
    return 18;
  })
  .duration(400)
  • When using transitions, make sure to set the initial state when you create the element. In this case, r and font-size to 0

Working with Data

Reading Remote Data

D3 library handles AJAX calls to download and parse remote data from sources like CSV and JSON.

Example datasets:

Near-Earth Comet Example

  • D3 converts CSV to JSON objects keyed by the row header.
  • Dump the data from the callback to see its structure
// Near-Earth Comets - Orbital Elements: https://data.nasa.gov/Space-Science/Near-Earth-Comets-Orbital-Elements/b67r-rgxc

// Terminology: 
// MOID: Minimum Orbit Intersection Distance
// Node: Oribt angle of intersection

d3.csv("https://data.nasa.gov/api/views/b67r-rgxc/rows.csv", function(data) {
  d3.select("body").append("pre").text(JSON.stringify(data, null, 2));
})

Data fetch and SVG setup.

d3.select("h1").text("Near-Earth Comets")
d3.csv("https://data.nasa.gov/api/views/b67r-rgxc/rows.csv", function(data) {
  var w = 600,
      h = 400;

  d3.select("#chart").append("svg")
    .attr({width: w, height: h})
  .append("rect")
      .attr({width: "100%", height: "100%", fill: "#002b36"})
  d3.select("svg")
  .append("circle")
    .attr({cx: w/2, cy: h/2, r: 50, fill: "#268bd2"})
});

Create the scales

// Copy MOIDs to array and convert to float.
neo_moid = data.map(function(i) { return parseFloat(i["MOID (AU)"]) })

// Copy node intersection angles to array convert to float.
neo_node = data.map(function(i) { return parseFloat(i["Node (deg)"])})

// Make scale to map moid distances to center circle.
xScale = d3.scale.linear()
.domain([d3.min(neo_moid), d3.max(neo_moid)])
.rangeRound([0,150])

// Make scale to map moid distances to color gradiant.
colorScale = d3.scale.linear()
.domain([0, d3.max(neo_moid)*.33, d3.max(neo_moid)*.66, d3.max(neo_moid)])
.range(['#B58929','#C61C6F', '#268BD2', '#85992C'])

Add circles for the Near-Earth objects

d3.select("svg").selectAll(".neo")
    .data(data)
    .enter()
    .append("circle")
      .attr("cx", function(d) {
        var mag = xScale(parseFloat(d["MOID (AU)"])),
            angle = parseFloat(d["Node (deg)"]);
        return ( (mag+55) * Math.cos(angle)) + w/2;
      })
      .attr("cy", function(d) { 
        var mag = xScale(parseFloat(d["MOID (AU)"])),
            angle = parseFloat(d["Node (deg)"]);
        return ( (mag+55) * Math.sin(angle)) + h/2;
      })
      .attr("r", 5)
      .attr("fill",function(d) { 
          return colorScale(parseFloat(d["MOID (AU)"])) 
      })

Add event to display lables.

.on("mouseover", function(d) {
    d3.select("svg")
      .append("text")
      .classed("neo-name", true)
      .text(function() { 
        var label = d["Object"];
        label += ": moid=";
        label += parseFloat(d["MOID (AU)"]).toFixed(3) + "(AU)";
        label += ", node=" + parseFloat(d["Node (deg)"]).toFixed(3) + " (deg)";
        return label;
      })
      .attr("fill","white")
      .attr("x", 5)
      .attr("y", (h-5))
})
.on("mouseout", function(d) { 
  d3.selectAll("svg .neo-name").remove();
})

Completed Example

// Near-Earth Comets - Orbital Elements: https://data.nasa.gov/Space-Science/Near-Earth-Comets-Orbital-Elements/b67r-rgxc

// Terminology: 
// MOID: Minimum Orbit Intersection Distance
// Node: Oribt angle of intersection

d3.select("h1").text("Near-Earth Comets")
d3.csv("https://data.nasa.gov/api/views/b67r-rgxc/rows.csv", function(data) {
  var w = 600,
      h = 400;

  // Copy MOIDs to array and convert to float.
  neo_moid = data.map(function(i) { return parseFloat(i["MOID (AU)"]) })

  // Copy node intersection angles to array convert to float.
  neo_node = data.map(function(i) { return parseFloat(i["Node (deg)"])})

  // Make scale to map moid distances to center circle.
  xScale = d3.scale.linear()
    .domain([d3.min(neo_moid), d3.max(neo_moid)])
    .rangeRound([0,150])

  // Make scale to map moid distances to color gradiant.
  colorScale = d3.scale.linear()
    .domain([0, d3.max(neo_moid)*.33, d3.max(neo_moid)*.66, d3.max(neo_moid)])
    .range(['#B58929','#C61C6F', '#268BD2', '#85992C'])

  d3.select("#chart").append("svg")
    .attr({width: w, height: h})
  .append("rect")
      .attr({width: "100%", height: "100%", fill: "#002b36"})
  d3.select("svg")
  .append("circle")
    .attr({cx: w/2, cy: h/2, r: 50, fill: "#268bd2"})

  d3.select("svg").selectAll(".neo")
    .data(data)
    .enter()
    .append("circle")
      .attr("cx", function(d) {
        var mag = xScale(parseFloat(d["MOID (AU)"])),
            angle = parseFloat(d["Node (deg)"]);
        return ( (mag+55) * Math.cos(angle)) + w/2;
      })
      .attr("cy", function(d) { 
        var mag = xScale(parseFloat(d["MOID (AU)"])),
            angle = parseFloat(d["Node (deg)"]);
        return ( (mag+55) * Math.sin(angle)) + h/2;
      })
      .attr("r", 5)
      .attr("fill",function(d) { 
          return colorScale(parseFloat(d["MOID (AU)"])) 
      })

      .on("mouseover", function(d) {
          d3.select("svg")
          .append("text")
          .classed("neo-name", true)
          .text(function() { 
            var label = d["Object"];
            label += ": moid=";
            label += parseFloat(d["MOID (AU)"]).toFixed(3) + "(AU)";
            label += ", node=" + parseFloat(d["Node (deg)"]).toFixed(3) + " (deg)";
            return label;
          })
          .attr("fill","white")
          .attr("x", 5)
          .attr("y", (h-5))
       })
     .on("mouseout", function(d) { 
          d3.selectAll("svg .neo-name").remove();
     })
});

Mars Images Example

// Mars Image Database - https://msl-raws.s3.amazonaws.com/images/image_manifest.json
d3.csv("https://msl-raws.s3.amazonaws.com/images/image_manifest.json", function(data) {
  d3.select("body").append("pre").text(JSON.stringify(data, null, 2));
})

Using Layouts

D3 Layouts

  • Programatically layout elements using provided library functions.
  • Higher-level abstraction of common visualization components.
  • Pie charts, force charts, histograms etc.

https://github.com/mbostock/d3/wiki/Layouts

Force Chart Example

var force = d3.layout.force()
    .nodes(nodes)
    .links(links)
    .size([w, h])
    .linkStrength(0.1)
    .friction(0.9)
    .linkDistance(20)
    .charge(-30)
    .gravity(0.1)
    .theta(0.8)
    .alpha(0.1)
    .start();
  • Layout is aware of the nodes and links and uses the given parameters to update it on every tick event.
  • You still add your elements and style them as usual, layout will perform the translations and positioning for you.
  • Overview of various force layout capabilities: http://mbostock.github.io/d3/talk/20110921/#26

Complete Example

var   w = 400,
      h = 400;

var circleWidth = 5;

var palette = {
      "lightgray": "#819090",
      "gray": "#708284",
      "mediumgray": "#536870",
      "darkgray": "#475B62",

      "darkblue": "#0A2933",
      "darkerblue": "#042029",

      "paleryellow": "#FCF4DC",
      "paleyellow": "#EAE3CB",
      "yellow": "#A57706",
      "orange": "#BD3613",
      "red": "#D11C24",
      "pink": "#C61C6F",
      "purple": "#595AB7",
      "blue": "#2176C7",
      "green": "#259286",
      "yellowgreen": "#738A05"
  }

var nodes = [
      { name: "Parent"},
      { name: "child1"},
      { name: "child2", target: [0]},
      { name: "child3", target: [0]},
      { name: "child4", target: [1]},
      { name: "child5", target: [0, 1, 2, 3]}
];

var links = [];

for (var i = 0; i< nodes.length; i++) {
      if (nodes[i].target !== undefined) {
            for (var x = 0; x< nodes[i].target.length; x++ ) {
                  links.push({
                        source: nodes[i],
                        target: nodes[nodes[i].target[x]]
                  })
            }
      }
}

var myChart = d3.select('#chart')
        .append('svg')
        .attr('width', w)
        .attr('height', h)

var force = d3.layout.force()
    .nodes(nodes)
    .links([])
    .gravity(0.3)
    .charge(-1000)
    .size([w, h])

var link = myChart.selectAll('line')
    .data(links).enter().append('line')
    .attr('stroke', palette.gray)

var node = myChart.selectAll('circle')
    .data(nodes).enter()
    .append('g')
    .call(force.drag);

node.append('circle')
    .attr('cx', function(d) { return d.x; })
    .attr('cy', function(d) { return d.y; })
    .attr('r', circleWidth )
    .attr('fill', function(d, i) {
        if (i>0) { return palette.pink }
        else { return palette.blue }
    })

node.append('text')
    .text(function(d) { return d.name})
    .attr('font-family', 'Roboto Slab')
    .attr('fill', function(d, i) {
        if (i>0) { return palette.mediumgray}
        else { return palette.yellowgreen}
    })
    .attr('x', function(d, i) {
        if (i>0) { return circleWidth + 4 }
        else { return circleWidth -15 }
    })
    .attr('y', function(d, i) {
        if (i>0) { return circleWidth }
        else { return 8 }
    })
    .attr('text-anchor', function(d, i) {
        if (i>0) { return 'beginning' }
        else { return 'end'}
    })
    .attr('font-size',  function(d, i) {
        if (i>0) { return '1em' }
        else { return '1.8em'}
    })

force.on('tick', function(e) {
    node.attr('transform', function(d, i) {
        return 'translate('+ d.x +', '+ d.y +')';
    })

    link
        .attr('x1', function(d) { return d.source.x })
        .attr('y1', function(d) { return d.source.y })
        .attr('x2', function(d) { return d.target.x })
        .attr('y2', function(d) { return d.target.y })
})


force.start();

Overview of capabilities: http://mbostock.github.io/d3/talk/20110921/#26

Additional Resources

Gallery: https://github.com/mbostock/d3/wiki/Gallery

Another space vis: http://joshworth.com/dev/pixelspace/pixelspace_solarsystem.html