Assignments Due

Week 6 checkpoints should be submitted by Monday, 02/12/2018 at 11:59 PM PST.

Perform ～ Compose ～ Review

Perform

Group 2: Facebook Critique (50%)

Is the “Facebook Experiment” lousy science? Galen Panger thinks so, and he had the gall to make some bad publicity for Facebook. Now their credibility is on the line with other principles and stakeholders who each have different views, which may be sympathetic, indifferent, or hostile to Panger’s argument. How do the interests at stake for each actor affect what they feel they know about the Facebook Experiment? How does the science relate to what people say about the science?

Before meeting with your group to discuss this project for the first time, read the little (2 page) study by Kramer, Guillory, and Hancock (2014). Jot down your general impression, how you would describe it in a nutshell, and whether you thought it was interesting, important, and credible. Save those notes to discuss with your team.

After reflecting on Kramer, read Galen Panger’s (2014) comments about the same study. The order matters here, so be sure to read Kramer et al. first to avoid being swayed one way or the other by Panger’s conclusions.

In this assignment you will deconstruct the scenario Panger describes using the decision making approach we’ve been studying. Panger is merely one of the perspectives on the events in question, and he takes the role of the social science critic.

Your team will take the perspective of one of four types of actors:

  • Facebook executives
  • Facebook data scientists
  • Facebook users
  • Stakeholders – choose one of:
    • Advertisers
    • Mental health professionals/advocates
    • Competitors

Your task will be to first read Panger’s critique and second to decide how to respond from the perspective of your social role. Unlike the last presentation, this time you will prepare two documents, an internal memo and a public statement. The memo (due this week) will role play an internal conversation where you discuss Panger and plan your statement. The deck (due next week) will role play the public response itself.

As a method to help you accurately simulate stakeholder behavior, you must not break character at any time in the memo or the deck. In other words, everything you write or speak should be in the voice of the role you are playing. Both in your memo and your statement, you should maintain a focus on the significance of the Facebook Experiment itself.

Guidelines for preparing your private memo are below, while those for your public statement can be found in next week’s prompt.

How to submit

Prepare your private memo as a Google Doc. One team member should:

  • create it from their bConnected account,
  • set sharing to “Anyone at UC Berkeley with the link can comment”, and
  • move the doc to the submission folder.

Meet once with your team this week. Choose a role, then prepare a draft of the private memo. You can get started on next week’s slide deck if you want to, but your instructor will review your draft memo during a live session work-in-progress consultation with your team.

See below for additional guidance on this week’s checkpoint.

Which role will you play?

Each role can be represented by only one team. As soon as you can, discuss among your team which role is your first and your second choice, in order. A member of your team should respond in the comment thread under your section’s team assignment announcement on our Slack channel. If multiple teams give the same preference, your instructor will decide by a coin flip.

Note:

  • The stakeholder role can only be chosen by one team, regardless of which stakeholder you choose.
Internal memo

Describe a private venue in which your interest group would have a candid conversation with an internal record. Mention or signal the venue at the top of your document. Otherwise any written format is fine, as long as it is a Google Doc and the questions below are addressed. In the course of your preparation, research at least one real source besides the Panger article that refers to the Facebook Experiment, and include a bibliography entry for it at the bottom of your memo.

(A private conversation, submitted but not presented live)

  • Who are we? How or where are we having this conversation?
  • How could Panger affect our interests, for better or worse?
  • How are we being portrayed by Panger?
  • Honestly, what did Panger get right? What did he get wrong?
  • What do others think about Panger?
    • Whose behavior are we most concerned about?
    • What do we think they think about Panger?
    • How might they react to Panger?
    • Why do we care?
  • What is our dilemma/decision?
    • What are the risks of each choice?
    • What are the opportunities for each choice?
  • We have to respond…
    • What if we don’t?
    • What outcome do we want?
    • How do we get there?

Compose

Individual 2: Big Idea 2 (25%)

So you were in love with your first idea? Don’t be too sad to turn away, because it was just one draw from your creative urn. Move forward, don’t look back, and focus on the how of quickly creating a concept from start to finish. As did your first, your second proposal should encapsulate the fundamental elements of problem, solution, and the difference it makes. Call it a seed that you can easily plant in the minds of an audience, and if they are interested they will help you to make it grow. If they don’t like a concept, no big loss, because you can reimagine it or move on to the next big idea.

As with your Week 2 checkpoint, lay the foundation of Big Idea #2 by writing a title, slug, three headings, and a sentence under each heading. Give your first idea a break for now and try not to tinker with it (a rest can actually be productive).


How to [White Fang](https://www.youtube.com/watch?v=VLBM8bEcnYc) your first big idea.

Figure 19: How to White Fang your first big idea.

When you begin, remember how important it is to be clear about who the client is. It is the client’s struggles that allow you to begin to clarify the problem.

Before you write the middle, think carefully about the basics of your research design. The problem is bigger than you can solve in one go; let your design carve off a manageable chunk of the larger strategy. Include a research question at the beginning of your middle, and let the RQ make a more precise, scientific statement about the general problem you introduced at the beginning. Describe the data and methods you would use to create results or findings that would answer the question and thereby contribute to decision making.5

At the end, explain how the solution helps, that is, how the results or findings would be used by clients. Trumpet the practical gains you expect for your client. Be honest about what is left to do, mention risks and pitfalls, but always end by reinforcing the valued added by your solution.


Figure 20: Forward not backward!

How to submit

Because you pushed a live version of your website, you are now dealing with a “development” and a “production” situation. It’s time to start using git branching so you develop new work without interfering with already deployed content. Branching is tricky at first but is one of the most powerful features of git.

If you’re new to git and still a bit confused that’s totally understandable. It will become more familiar with time. Optionally, here is a fun game to play if you want a visual exploration of the concept of a branching workflow. The game exposes you to more features than you need to know right now, but it’s a good brain marinade.

Save these as a commit to your 02.Rmd file and be sure to stay synced with your remote (i.e. GitHub.com) if working locally. Before making a commit, however, start a new branch called idea2, and do all of your 02.Rmd work their and not on master.

Making a new branch is easy. To set one up for the first time:

git checkout -b idea2

Where idea2 can be any branch name not already used. You can switch to that branch to develop a new version. Later when you finish the work you want to do on idea2, you will merge it back to master. We will learn how this can keep your commit history on master much cleaner, so that only substantial and significant changes (drafts) appear in master. But for now, do your incremental work on idea2.

Gitbits: it’s a tidbit of git!

git checkout is your tool to move from branch to branch. So git checkout master would move you to master if you weren’t already there, and git checkout idea2 would move you back to idea2,etc. git branch will show you all the branches and the one you’re currently checking out.

If you try to checkout a different branch when there is new work in your current directory, you’ll get an error that there are unstaged changes. Git only remembers file changes that have been staged or committed. What’s the difference? git stage, a.k.a. git add, adds files and file changes to the index. This basically means that, of all the changes that you might have made to files in your git directory, these are the changes you want to add to the next commit. Again, only staged changes will be committed. Once a file is staged or added for the first time, it is also known as being tracked, which is to say that it is listed in the index. Whew! git status will give you info about what’s currently going on with all of this in your directory!

Practically, to switch branches your changes should at least be staged. Be careful, because forcing a branch change will result in permanently losing all unstaged changes! This possibility is a reason to back up one’s git directory with a service like Dropbox.


I can't commit unless you stage me.

Figure 21: I can’t commit unless you stage me.

Slug Debug

Check the deployed page for Big Idea #1. If you see your slug displayed in curly braces after your title, then you have a syntax error in the slug, which is actually a cross reference in bookdown syntax. The slug can only contain letters, numbers, or hyphens. Switch to your master branch. If in 01.Rmd your title line reads:

# My Long Title {My Short, Slug!}

You should change it to:

# My Long Title {#My-Short-Slug}

Note that a # should also appear first in the brackets. Then rebuild and redeploy locally or from binder. Your slug should no longer appear in your built page after the title, and you should notice that the slug becomes the name of the html page for your Big Idea, so My-Short-Slug.html.

Review

Participation 1: Peer Review 1 (100%, 4 points)

Workshop in small groups by reading and reviewing the work of a close circle of peers. Using the collaboration functions available through GitHub.com, provide feedback about what they should keep, cut, rearrange, or add to strengthen the composition.

Each of your ideas is complex and multidimensional, so it can be difficult to assign it a rating or a score in a fair and accurate way. In education such evaluative decisions are left up to the instructors. While this may be a good way to reflect the desires and intentions of the course, there are limitations.

  • Your performance is based on the opinion of a single person.
  • The instructor’s judgment may be affected by anchoring and recency bias from an arbitrary ordering of items.
  • Fatigue is a factor detracting from the accuracy in judgement of later items.
  • Reducing a complex research design to a single grade or numerical score may not be very meaningful.

As an alternative form of assessment, we will combine crowd sourcing with a data science approach to ranking complex items. The central innovation is to replace the task of cardinal scoring with the cognitively easier task of ordinal ranking. Rather than assign an absolute value, such as a “9 out of 10” or an “A-”, we compare two items at a time and make a judgement about which is higher on a particular criterion. This judgement of relative position tends to be easier and more stable than assigning an absolute position.

Given enough comparisons, however, a solution to the cardinal scoring of each item under the assumption of transitivity can be achieved. Simply, from relative or ordinal comparisons we can derive absolute or cardinal scores.

The wrinkle to this otherwise good idea is that it requires more input data. To make cardinal judgements, however innacurate they may be, requires completing 15 tasks for 15 items. Making ordinal judgments on pairs, however, requires completing \(\frac{n(n-1)}{2}\) or 105 tasks for 15 items. Crowdsourcing is the only feasible solution to this problem. Moreover, the high number of tasks and the use of multiple scorers presents an opportunity to design research in a way that mitigates the biases mentioned above. Compared to the traditional method of evaluation, pairwise scoring:

  • Yields more stable estimates of quality by limiting the effect of anchors.
  • Makes it feasible to directly measure multiple criteria of quality per pair.
  • Simulates the opinion of “the crowd”, which may be more meaningful in a real world scenario.
  • In an educational setting, gives students an opportunity to learn from and contribute to each other’s work.

To learn more about the background of the research design used below, go to peergrading.org. Raman and Joachims (2014) provides a good overview of statistical ranking methods.

How to submit

Your tasks will be due Monday, 04/09/2018 at 11:59 PM PDT, but don’t wait until the last minute! You should aim to complete the tasks in batches, sitting down for two or more separate sessions. You will recieve a ballot as an email to your @ischool.berkeley.edu address from your instructor. Completing the feedback requires that you read every student’s submission. They aren’t long, but you should set aside 15-30 minutes to complete each task. If you go faster great, but plan ahead and try not to leave everything to the last minute.

If a ballot link is broken then the original author made a change after it was prepared. Navigate to their main page and discern which idea corresponded to your original link. So if https://w201rdada.github.io/portfolio-brooksambrose/paul-mccartneys-backward-lentil-soup.html breaks try https://w201rdada.github.io/portfolio-brooksambrose.

As mentioned, to mitigate bias we’re adopting a “paired comparison” design, and the order in which you read the proposals matters. Just follow the instructions on the Qualtrics form, Your lists are different from everyone else in the class. That way, as a group, we can make all discrete comparisons! Your instructor is taking a portion of the load as well.

You could spend a lot of time comparing each pair of proposals, but don’t. Your hunch or “feeling of quality” is great, and in fact, that’s what we’re looking for. The same goes for qualitative feedback; your two cents will be great and don’t write more than a tweet.

Bibliography

Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. “Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks.” Proceedings of the National Academy of Sciences 111 (24): 8788–90. doi:10.1073/pnas.1320040111.

Panger, Galen. 2014. “Why the Facebook Experiment Is Lousy Social Science.” Medium. https://medium.com/@gpanger/why-the-facebook-experiment-is-lousy-social-science-8083cbef3aee.

Raman, Karthik, and Thorsten Joachims. 2014. “Methods for Ordinal Peer Grading.” arXiv:1404.3656 [Cs], April. http://arxiv.org/abs/1404.3656.

  1. Results are information you serve to a client without too much interpretation. Findings are when you take the extra step to interpet and explain the meaning of the results to the client. You should charge more if you are preparing findings!↩