6 Practical Research Design for Real People
Week 6:
TL;DR: , do your homework
readings (links) & lectures ～ assignments due ～ live session agenda
TL;DR: , do your homework
readings (links) & lectures ～ assignments due ～ live session agenda
Figure 18: Week 6 “reading” time estimated at 250 words per minute
Readings
Lectures
- ↓ 6.1 Introduction to Questioning
- ↓ 6.2 Weekly Video
- ↓ 6.4 Importance of Good Research Design
- ↓ 6.4.1 Induction vs. Deduction
- ↓ 6.4.2 Linear Model of Research Design
- ↓ 6.4.3 How Do We Sell a Slide Projector?
- ↓ 6.4.4 Alvarez Story of the Meteor
- ↓ 6.4.5 Iterative Research
- ↓ 6.5 Qualitative Data
- ↓ 6.6 Design Thinking in Practice
- ↓ 6.6.1 IDEO Shopping Cart
- ↓ 6.6.2 A Consultant’s View
- ↓ 6.7 Design Thinking Is Asking the Right Questions
- ↓ 6.7.1 Ask Them!
- ↓ 6.7.2 Asking Better Questions
- ↓ 6.7.3.1 Common Questions Aren’t the Best
- ↓ 6.8 Expert Mind vs. Beginner Mind
- ↓ 6.9 Accumulated Wisdom
- ↓ 6.10 Elegant Questions Are an Art Form
- ↓ 6.10.1 Beautiful Questions
- ↓ 6.11 Week 6 Wrap-Up
6.1 Introduction to Questioning ↑
| Players | Slug | Excerpt |
|---|---|---|
| φ λ | research design | It’s sort of the prep before the painting. And as everyone knows who’s ever painted a wall, it’s the prep that really counts in how that wall’s going to look after you’ve painted it. And so is true of research design. |
6.2 Weekly Video ↑
| Players | Slug | Excerpt |
|---|---|---|
| η χ | eternal peril | what if you get a question wrong then you are cast into the gorge of eternal peril |
| ι θ | African or European | Stop! What is your name? It is Arthur king of the Britons. What is your quest? To seek the Holy Grail. What is the airspeed velocity of an unladen swallow? What do you mean, an African or European swallow? Huh? |
6.4 Importance of Good Research Design ↑
| Players | Slug | Excerpt |
|---|---|---|
| κ σ | dumb up front | it is amazing how often really smart, really capable people just screw this up and don’t pay enough attention to the research design at the front of the project. |
| ω ψ | Bratislava | It’s not necessarily the best place to spend February and March. And she came home with boxes and boxes of data. And then, guess what she had? Boxes and boxes of data. A couple months later, she called me and said, oh my god, what am I going to do with all this data? Well, actually, in a really significant way, it wasn’t data. She didn’t know what her research design was. She didn’t know why she was collecting that data. And you know what? She had to go back to Bratislava for another year. |
| μ ε | distracting data | even when data becomes cheaper and easier to collect, it’s still a distraction, unless it’s organized for a specific purpose. |
| μ χ | urban resilience | we want that competition to generate the best ideas that could possibly out there about urban resilience. And so in order to help us kind of figure out how to design that, could you go out there and look at five or six recent competitions that got a lot of press attention and tell me what are the key design criteria for us to use? |
| υ π | implied RD | I tried to explain to them the problem in their implied research design. By the way, they didn’t like hearing this, as any client wouldn’t. |
| ι ψ | small sample | here’s one problem. Your sample size is too small. Five or six recent competitions, that’s not enough to really know what works. So could we go for a bigger sample size? |
| υ φ | nasty selection | we got a nasty kind of selection bias going on here. So you’ve asked me to look at the competitions that got a lot of press attention, but maybe those aren’t the ones we should be looking at. |
| δ ι | conditioning on a collider | here’s where we just ran into a wall– with the core research design issue. Look, I tried to explain to them, you cannot possibly determine the causes of success by looking only at successful cases. |
| ι ρ | high blood pressure | they just couldn’t grasp that you couldn’t determine the causes of success by looking only at successful cases. And as you can imagine, my blood pressure was going up, my heart rate was going up, and I was sweating. But at the end of the day, it was a core research design issue. And there was absolutely nothing I could do about it. So here’s the thing. We know proper inference, proper inference from data is not always intuitive. |
| ω θ | silly mistakes | even great statisticians sometimes make the same mistakes when they’re out in regular life. Even very smart people sometimes make very silly mistakes. |
| ν μ | safe bet | In early 2013, the Ravens defeated the 49ers by 34 to 31. That predicts a 65% fall in the S&P 500. There’s the data. You tell me if people aren’t going to bet on that. You know they will. |
6.4.1 Induction vs. Deduction ↑
| Players | Slug | Excerpt |
|---|---|---|
| β λ | induc | Do we want to start with a theory or start with data? |
6.4.2 Linear Model of Research Design ↑
| Players | Slug | Excerpt |
|---|---|---|
| φ σ | NA | NA |
6.4.3 How Do We Sell a Slide Projector? ↑
| Players | Slug | Excerpt |
|---|---|---|
| π μ | NA | NA |
6.4.4 Alvarez Story of the Meteor ↑
| Players | Slug | Excerpt |
|---|---|---|
| α ξ | NA | NA |
6.4.5 Iterative Research ↑
| Players | Slug | Excerpt |
|---|---|---|
| α ψ | NA | NA |
6.5 Qualitative Data ↑
| Players | Slug | Excerpt |
|---|---|---|
| ν ε | NA | NA |
6.6 Design Thinking in Practice ↑
| Players | Slug | Excerpt |
|---|---|---|
| χ α | NA | NA |
6.6.1 IDEO Shopping Cart ↑
| Players | Slug | Excerpt |
|---|---|---|
| η ι | NA | NA |
6.6.2 A Consultant’s View ↑
| Players | Slug | Excerpt |
|---|---|---|
| σ α | NA | NA |
6.7 Design Thinking Is Asking the Right Questions ↑
| Players | Slug | Excerpt |
|---|---|---|
| α ξ | NA | NA |
6.7.1 Ask Them! ↑
| Players | Slug | Excerpt |
|---|---|---|
| γ ψ | NA | NA |
6.7.2 Asking Better Questions ↑
| Players | Slug | Excerpt |
|---|---|---|
| γ ξ | NA | NA |
6.7.3.1 Common Questions Aren’t the Best ↑
| Players | Slug | Excerpt |
|---|---|---|
| ο μ | NA | NA |
6.7.3.2 ↑
| Players | Slug | Excerpt |
|---|---|---|
| ζ ι | NA | NA |
6.7.3.3 ↑
| Players | Slug | Excerpt |
|---|---|---|
| ρ σ | NA | NA |
6.7.3.4 ↑
| Players | Slug | Excerpt |
|---|---|---|
| ι τ | NA | NA |
6.7.3.5 ↑
| Players | Slug | Excerpt |
|---|---|---|
| γ ν | NA | NA |
6.8 Expert Mind vs. Beginner Mind ↑
| Players | Slug | Excerpt |
|---|---|---|
| ξ δ | NA | NA |
6.9 Accumulated Wisdom ↑
| Players | Slug | Excerpt |
|---|---|---|
| ι ψ | NA | NA |
6.10 Elegant Questions Are an Art Form ↑
| Players | Slug | Excerpt |
|---|---|---|
| ξ υ | NA | NA |
6.10.1 Beautiful Questions ↑
| Players | Slug | Excerpt |
|---|---|---|
| ξ θ | NA | NA |
Bibliography
Creswell, John W. 2009a. “Chapter 1: The Selection of a Research Design.” In Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, Calif. : Sage Publications, c2009. http://www.study.net//r_mat.asp?crs_id=30124014&mat_id=50272415.
Creswell, John W. 2009b. “Chapter 3: The Use of Theory.” In Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, Calif. : Sage Publications, c2009. http://www.study.net//r_mat.asp?crs_id=30124014&mat_id=50272418.
Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. “Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks.” Proceedings of the National Academy of Sciences 111 (24): 8788–90. doi:10.1073/pnas.1320040111.
Panger, Galen. 2014. “Why the Facebook Experiment Is Lousy Social Science.” Medium. https://medium.com/@gpanger/why-the-facebook-experiment-is-lousy-social-science-8083cbef3aee.