Linkademia

— More evidence that menu positions influence food orders

— Syllabi are important

— The future of teacher education is in the same location as the future of student education: The Internet

— Simply thinking about choice can affect your policy preferences

— Uncommon Scents

Why do MLB Draft Picks Hold Out for More Money?

The August 15th MLB draft signing deadline is fast approaching, and that means it’s time for my yearly gripe about players turning down seven figure bonuses in the hope of excelling for an independent league team and then getting a larger bonus the following year. Not only have the successful implementations of this strategy been moderate when compared to the catastrophic failures, but it is nearly impossible to justify the decision to re-enter the draft from an economic standpoint.

Imagine two players. Player A is a “risk-taker.” He is supremely confident in his ability and sure he will be an above average major league player. Player B is “risk-averse.” He lives in constant fear that an injury or other adverse event will end his career.

For player A the optimal strategy is to sign as quickly as possible in order to minimize the time until he becomes a free agent. When a player chooses to re-enter the draft he delays the start of his MLB career. That means he’ll become eligible for arbitration one year later, hit free agency one year later, and potentially retire having played one fewer year.  By re-entering the draft the player could effectively be giving up a year’s salary in the prime of his career (anywhere from $2m-$15m).

For player B the optimal strategy is also to sign as quickly as possibly. If player B wants guaranteed money because he fears injury, the best thing to do is accept the guaranteed money offered to him and not risk waiting another year.

Given that the dominant strategy is to sign, why do players still choose to re-enter the draft? The simple answer is that a player and his agent don’t have the same incentives.  For a player, the difference between $3.5m and $4m may not be worth the risk of getting nothing, but for an agent attempting to attract new clients and brand himself as “the biggest bonus-getter,” it’s important to make a splash. If the player happens to suffer an injury between drafts it could be catastrophic for his future earnings, but for the agent making 10%, it’s not a huge loss.  In general, holding out for the highest possible bonus has higher rewards and lower risks for the agent than it does for the player.

School Choice is Important Because Every Student is Different

Ask somebody why school choice is important and they’ll cite free market principles and the role of competition in eliminating under-performing schools. What they won’t mention is a much simpler reason for why choice is good: Giving students options allows them to find a better social situation.

Two recent studies help illustrate why this is important. In the first study researchers found that four measures of peer relationships — peer acceptance, presence/absence of a best friend, number of friends, and perceived peer support — were significant predictors of a student’s “liking” of school. Essentially, when students have friends they’re more likely to enjoy school. Because students who enjoy school are more likely to achieve, giving students the opportunity to go to a bunch of different schools allows them to increase achievement through an improved social situation.

The second study looked at how the racial composition of schools influences academic achievement. The researchers found that in diverse schools students performed better on tests when they had more peers of the same race or ethnicity. Once again, students are more likely to have racially similar peers, and thus better test scores, when they have the opportunity to attend different schools.

It’s rare to hear people mention the social impact on individuals when discussing school choice because we tend to focus on the macro-benefits (e.g. driving a bad school of out business) and ignore the micro-benefits (e.g. being able to have friends at school). The consensus seems to be that it’s a travesty if students are forced to attend a school with below average history scores, but irrelevant if a student is forced to attend a school where he has no friends.

The studies above highlight the social benefits that school choice can provide, and it’s important to consider these benefits because without understanding the unique situations of individuals we cannot properly quantify the costs and benefits of school choice.

Are you a “Speculator” or an “Evaluator”?

In the days leading up to last week’s NBA draft my go-to form of procrastination was reading all the draft-related rumors and predictions I could find.  Then the draft happened and I didn’t bother to read a single analysis of the picks. My rationale was that “professional” pundits generally know nothing because it’s impossible to effectively evaluate picks until the players start playing.

Alternatively, one could argue that the pre-draft coverage is completely valueless when compared with the post-draft coverage. Why concern yourself about a myriad of possibilities when you’ll soon find out exactly what happens? In fact, this is my attitude when it comes to political speeches. I have no interest in what people think Obama will or should say, but after he says it I’m usually interested in reading what people think.

The rapid demise of my interest in draft coverage got me thinking about the consequences of having a tendency to prefer pre-event coverage (being a  “speculator”) or post-event coverage (being an “evaluator.”)  It seems that being a speculator is beneficial when reacting to a low probability event that reveals little new information about the world (a single soccer game, a successful high-risk terrorist attack). In these cases the pre-event analysis is still accurate and the post-event analysis is likely to be skewed by over-emphasis on the event.

On the other hand, being an evaluator is likely to be beneficial when reacting to an event that reveals significant information (a seven game NBA series, an IPO, a TV show plot twist). In these cases the pre-event analysis is rendered relatively obsolete and those who eagerly consumed it will likely be left with inaccurate beliefs.

Are you a speculator, an evaluator, or both?

What Does an Equitable Classroom Look Like?

In my previous post I mentioned Brown and Clift’s AERJ paper to point out how easy it is to criticize NCLB. A second thing worth mentioning is that the paper is another example of the difficulties that arise due to our failure to define or openly discuss what makes an equitable or “fair” classroom.

Brown and Clift reference various schools that focus a disproportionate amount of attention on students who are on the cusp of proficiency. This is viewed as unfair, and the implication is that in a “fair” classroom every student receives an equal amount of attention. However, there are people who would call a classroom is which every student receives equal attention unfair because the strongest students receive the same amount of attention as the struggling students. This disagreement cannot be resolved until we make a real effort to reach a consensus on what type of “teacher attention distribution” we aspire to have.

In a 1988 article in Ethics Christopher Jencks conceptualizes teacher attention as a valuable but limited resource and lays out five types of equality (or equity, or fairness, or whatever you want to call it.)

1. Democratic Equality — The teacher gives every student an equal amount of attention.

2. Moralistic Justice — The teacher gives more attention to those who put in the most effort.

3. Weak Humane Justice — The teacher gives more attention to students who have been shortchanged at home or in their earlier schooling.

4. Strong Humane Justice — The teacher gives more attention to students who have been shortchanged in any way (Jencks was mainly referring to genetics, an idea that remains somewhat controversial today.)

5. Utilitarianism — The teacher gives more attention to the students who perform the best.

I think a case can be made for most or all the rules. It’s legitimately hard to decide what’s best.

Any constructive debate on teacher attention needs to start with which of Jencks’ equalities or combination of equalities is the gold standard. Different states, different districts, and different schools can come up with their own answer, but in order for an outcome to be properly labeled as “bad,” those doing the labeling must first define what is “good.”

A major problem with the formulation of public policy (and education policy in particular) is that we rush into fighting over policy specifies before we reach an ideological compromise on which to base the policies. (I’ve touched on this before here and here.) In the case of teacher attention we have people claiming that something is unfair without first establishing the definition of fairness.

NCLB Will Never Silence its Critics

Most policies have ideological opponents who will be critical regardless of outcomes, but No Child Left Behind seems to suffer from a uniquely acute case of this condition.  A recent paper in the American Educational Research Journal illustrates how much ammo NCLB opponents have in their arsenal. The paper uses the physics concept of an attractor basin to qualitatively analyze the differences in incentives and behavior of schools that are far below, near, or far above the AYP mark.

What caught my eye are the reported downsides for schools near the AYP mark, or far above the mark. First, schools that are near AYP or in the “attractor basin”:

We also saw evidence of more strategic responses: shifts in the curriculum toward tested areas and in choices regarding which students receive teachers’ attention. These shifts are ones our attractor basin model predicts: The law encourages extra attention on the topics and students that are weak enough that they may not measure up but strong enough that they can be remediated with reasonable effort.

[…]

For schools in the attractor basin, the increased effort and attention that NCLB has stimulated comes at a cost of a narrower curriculum and uneven attention across students in the classroom

And now the “good” schools far about the AYP cutoff:

There are two areas where NCLB has the potential to adversely affect the incentives of high performers. First, it invites complacency among those who are passing the test. Second, some high-performing school communities are afraid of the negative effects that the choice provision in NCLB will have on their schools’ performance.

See what’s going on here? NCLB is bad for schools near the AYP cutoff because it will lead them to devote resources inequitably, and NCLB is also bad for schools far above the cutoff because they will not be sufficiently incentivized to make changes. Change is bad, but so is not making changes. The paper is laudably neither pro nor anti NCLB, but the types of seemingly mutually exclusive criticisms it finds evidence for hint at the near impossibility of winning over stubborn NCLB critics.

NCLB Will Never Silence its Critics

Most policies have ideological opponents who will be critical regardless of outcomes, but No Child Left Behind seems to suffer from a uniquely acute case of this condition.  A recent paper in the American Educational Research Journal illustrates how much ammo NCLB opponents have in their arsenal. The paper uses the physics concept of an attractor basin to qualitatively analyze the differences in incentives and behavior of schools that are far below, near, or far above the AYP mark.

What caught my eye are the reported downsides for schools near the AYP mark, or far above the mark. First, schools that are near AYP or in the “attractor basin”:

We also saw evidence of more strategic responses: shifts in the curriculum toward tested areas and in choices regarding which students receive teachers’ attention. These shifts are ones our attractor basin model predicts: The law encourages extra attention on the topics and students that are weak enough that they may not measure up but strong enough that they can be remediated with reasonable effort.

[…]

For schools in the attractor basin, the increased effort and attention that NCLB has stimulated comes at a cost of a narrower curriculum and uneven attention across students in the classroom

And now the “good” schools far about the AYP cutoff:

There are two areas where NCLB has the potential to adversely affect the incentives of high performers. First, it invites complacency among those who are passing the test. Second, some high-performing school communities are afraid of the negative effects that the choice provision in NCLB will have on their schools’ performance.

See what’s going on here? NCLB is bad for schools near the AYP cutoff because it will lead them to devote resources inequitably, and NCLB is also bad for schools far above the cutoff because they will not be sufficiently incentivized to make changes. Change is bad, but so is not making changes. The paper is laudably neither pro nor anti NCLB, but the types of seemingly mutually exclusive criticisms it finds evidence for hint at the near impossibility of winning over stubborn NCLB critics.

Improving Example-Based Learning

Anybody who spent time in a generic classroom is familiar with the simple three step process of example-based learning.

1. Teacher instructs students about principles and concepts in a certain domain.
2. Students see worked-examples that demonstrate the principles and concepts.
3. Students attempt problems on their own.

Understanding exactly how students learn from worked-examples is important because worked-examples are a key component of the technology-based cognitive tutors and personalized lessons that could one day be the backbone of our education system. Computers will never be able to mimic everything a teacher can do, but they can come close to mimicking instruction through worked-examples.

For this reason it was nice to see a meta-analysis in a recent Review of Education Psychology that examined how instructional explanations — explanations that clarify problem states or explicate principles — affect the learning outcomes of worked-example lessons. Although the authors found the overall effect of instructional explanations to be small, they did make a few discoveries that point to hidden significance.

First, instructional explanations had greater effects on mathematics knowledge and conceptual knowledge, two types of knowledge that are generally considered to be more difficult to obtain and more important than counterparts such as history or procedural knowledge. The authors also found that providing instructional explanations was helpful only when students were not encouraged to engage in self-explaining.

What’s important here is that nothing about the findings suggests a significant benefit to human instruction.  Although some kind of reflection about the problem solving process is clearly helpful,  it need not be a complex instructional explanation that requires a human teacher. Simply giving the students a prompt that asks them to think about what they’re doing and why they’re doing it may be enough to reap the potential learning gains.

Thou Shalt Be Biased

One roadblock to better public policy is that people tend to weigh evidence in a subjective manner. (Research suggests this due to some combination of motivational and cognitive factors.) A new study by Anthony Bastardi, Eric Uhlmann, and Lee Ross plays up the role of desire in the process. The authors discovered that our desires alone are enough to corrupt how we weigh evidence.

Evaluations of purported scientific evidence were shaped more by what participants desired to be true than by what they had initially believed to be true. Conflicted participants, who planned to use day care for their children but initially believed such care to be markedly inferior to home care, interpreted ambiguous scientific evidence in a manner congruent with their desire to believe that their plans would not be disadvantageous for their children. After they examined mixed scientific evidence, these conflicted participants shifted their beliefs and considered home care to be no better than day care. By contrast, after exposure to the same mixed evidence, unconflicted participants—those who shared the same initial belief in the superiority of home care and intended to use only home care when they became parents—maintained their strong initial belief.

Why does this matter? Because if you want to believe America has the best healthcare system in the world, a climate policy that won’t end the world, and an education system that will make us the first to discover new things about the world,  your actions (i.e. votes) will be based on those beliefs in combination with scientific evidence rather than on the scientific evidence alone.

Hold Teachers Accountable, But Let Them Make Their Case

Tom Vander Ark writes about Summit Prep, A San Francisco high school with an innovative teacher compensation system.

Teachers are charged with gathering and presenting evidence of their performance as demonstrated in student work and achievement.  For example, a teacher wanting to be evaluated as highly proficient on Curriculum/Differentiation would have to present evidence he/she consistently differentiated throughout the course, and that students of all levels of prior knowledge and skill were able to access and demonstrate mastery as a result.

The school essentially says to teachers, “We understand you don’t like your performance being based on test scores. Fine. Now it’s up to you to prove you’re a good teacher. What is your evidence?”

The idea is powerful because it has to potential to be an ideological middle ground between the pro-accountability and anti-accountability factions. Pro-accountability reformers get a real accountability system and in return they simply allow a certain percentage of it to be based on self-evaluations. The anti-accountability people have to swallow a legitimate accountability system, but in the end they get one that is more impartial and gives teachers more control over their ratings.

One thing making education reform difficult is that the debate is still very ideological. Views on whether teachers should be rated by test scores are a lot more intractable than views on how much teachers should be rated by test scores. Giving teachers control over a part of the equation has the potential to move things to a place more conducive to compromise. If both sides can agree that any teacher rating system should be part quantitative-metrics and part self-evaluation, then they can move on to debating the size of the parts.