How Not to Do a Survey
Recently, Trump called the mainstream media - by which he evidently means anything that isn't an alt-right, dog-whistling, fake news machine (we'll get to that later) - the "enemy of the American people". (You can imagine how the press reacted - but you don't have to, because Internet.) Besides being frighteningly autocratic, it's dangerously undermining confidence in the Fourth Estate, which exists to keep the people informed so that they can be responsible, informed, good citizens.You cannot have a strong democracy without good information access, which is why we have a free press. It is a necessary - though not sufficient (we'll get to that later) - component of democratic society. We need a free press to mediate between the government and the populace. The press not only keeps us informed of what government officials say and do, but puts information in context, balancing official statements and rulings against expert opinions and commentary. They expose corruption, so that crooked individuals have a harder time taking and abusing power. Are they perfect? No, of course not. Nothing human is perfect (not even Misha Collins, though he comes awfully close, but we're not getting into that today).
So, yes, media accountability is an important issue, and we should be wise news consumers, and how to do that is something I'll address shortly, I promise. But let's start by saying Trump's "Mainstream Media Accountability Survey" is not how it's done.
First of all, you have to figure out which MMAS you're doing. I've found two, and I'm not even looking.I saw one link on a friend's Facebook post. Later, I went looking for it to show MP, and pulled up the other. "Wow, my memory is going wonky - I thought it was slightly different," I said to myself. My memory is fine; there are multiples. Here's ONE. Here's TWO. How many others are there? I don't know - maybe there's only two, maybe there's lots. Why would there be multiples at all? Maybe I'm too cynical, but I'm guessing it's so they can cherry-pick the results to match what they want to say.
Then there are the questions themselves.Most are ridiculously misleading - "Do you agree with the President’s decision to break with tradition by giving lesser known reporters and bloggers the chance to ask the White House Press Secretary questions?" states one of the surveys. Well, what's wrong with giving "lesser known reporters and bloggers" the same opportunities as the Big Fish? Is anyone honestly going to say, "No! Only the known names should be involved!"
But context is important. Who are these Small Fry Intrepid Journalists? Breitbart and the like. In fact, in the other survey, we're asked:
Then there are the unanswerable questions - or, rather, the answers can be construed to mean whatever the person tabulating the data wants them to mean. For instance, one of the surveys states, "The media has not done its due diligence to expose ObamaCare’s many failures." Your choices in answer are 'yes', 'no', and 'no opinion'. (Unlike the other survey, this one does not let you given your own answer on anything.) So, what do those answers mean? 'Yes, the media hasn't done so' seems obvious - but it could also be taken as 'Yes, it has - the media has actually covered the flaws in the ACA'. 'No' could mean, 'No, the media hasn't done its due diligence!' or 'No, I don't agree with this statement at all! ObamaCare is awesome!' or even 'No, the media has done a fair and exacting job covering flaws in the ACA!'
Then there are statements that are misleading and unanswerable, like "Were you aware that a poll was released revealing that a majority of Americans actually supported President Trump's temporary restriction executive order?" What are the possible answers to that? 'Yes - I was aware of such a poll! It's a disgrace!' or 'Yeah, but there are lots more credible polls that show the majority didn't support it.' Or even, 'Yes - but that doesn't mean it was right or constitutional.' Or there's 'No, I wasn't aware! Let me be outraged!' or 'No, and I don't believe that poll exists or is accurate if it does.'
A couple of the questions on one survey are open-ended, but both are just asking for a list of what sources you use; you can, on this survey (which is apparently intended only for Republicans), input data for the 'other' answers on most of the questions. The other survey has no open-ended questions and does not allow you to answer with clarifications or qualifications.
The questions are all biased and inflammatory, and the surveys are obviously intended to enrage true believers and denigrate the press. One of the surveys is littered with references to "our Party," meaning the survey intentionally singled out Republicans - but would the gathered data have been reported that way? Or will the multiple surveys be conflated, so that the data swings far Right without acknowledgement of bias?
There is zero attempt to make the survey representative of the American people - not just with political demographics, either. Very little demographic data is gathered - only a zip code - so there can be no critical appraisal of how answers (such as they are presented) break down according to age, class, ethnicity, etc. - or how biased the data may unintentionally be due to these and like factors.
Moreover, and a bigger problem in my opinion, is the lack of anonymity. In order for your answers to count, you have to enter your name and email address and click either 'Record My Vote!' on the in-your-face Republican version or 'Submit' for the other. What happens then? Are you sorted out to different lists based on your answers? That seems conspiracy theory-ish. More likely, you're added to a mailing list of some sort, and I don't want to be on a mailing list of some sort. So, this tactic would discourage people who don't particularly want to hear any more propaganda, even if it is disguised as a survey. If you want to read more about this ridiculous survey from a reputable news site (I'm not a news site, and I'm not sure how reputable I am, either), I suggest NPR's coverage: The Trump Media Survey Is Phenomenally Biased. It's Also Useful
We the People, on the other hand, need to learn how to sift out actual news from propaganda, especially when we have an administration so devoted to pushing propaganda. Like I said earlier (see, I get back to everything eventually), a free press, while necessary for democratic society building and maintenance, is not sufficient - the people also have to be good at consuming the news. Even reputable, generally trustworthy sites can sometimes get it wrong. (Have you watched The Newsroom? You should definitely watch The Newsroom. No Misha, but, hey - can't be perfect.) It's our duty as good citizens to try and learn as much as possible, so we can be informed participants in a democratic society. Voting without making sure we have good information is a lot like going to pick up a pet without learning how to take care of it beforehand - it's your fish, now, and if you kill it, you're responsible. (One big difference - generally, a dead fish doesn't take you down with it.) Reading multiple reputable news sources and taking the time to think critically about what they're saying is important - you aren't a good citizen, otherwise. Learning how to tell what is and is not a good source is also important. I'm not going to teach you how to do that, since I've already built a wall of text here, and my fingers are tired of typing, and you can take this whole course from Poynter News University, which is awesome! It's free, it'll only take you an hour or two to go through, and it will help you be a better news consumer and producer. Go forth and do the thing. (For a more advanced course that should be available soon, check this out: In age of alternative facts, a college course on calling out scientific crap.)
Oh, also, here's how you do a survey. Someone pass this on to the White House, yeah?
Meanwhile, as a reward for digging through all my lecture above, here's a pretty picture:
Links!
So, yes, media accountability is an important issue, and we should be wise news consumers, and how to do that is something I'll address shortly, I promise. But let's start by saying Trump's "Mainstream Media Accountability Survey" is not how it's done.
First of all, you have to figure out which MMAS you're doing. I've found two, and I'm not even looking.I saw one link on a friend's Facebook post. Later, I went looking for it to show MP, and pulled up the other. "Wow, my memory is going wonky - I thought it was slightly different," I said to myself. My memory is fine; there are multiples. Here's ONE. Here's TWO. How many others are there? I don't know - maybe there's only two, maybe there's lots. Why would there be multiples at all? Maybe I'm too cynical, but I'm guessing it's so they can cherry-pick the results to match what they want to say.
Then there are the questions themselves.Most are ridiculously misleading - "Do you agree with the President’s decision to break with tradition by giving lesser known reporters and bloggers the chance to ask the White House Press Secretary questions?" states one of the surveys. Well, what's wrong with giving "lesser known reporters and bloggers" the same opportunities as the Big Fish? Is anyone honestly going to say, "No! Only the known names should be involved!"
But context is important. Who are these Small Fry Intrepid Journalists? Breitbart and the like. In fact, in the other survey, we're asked:
Do you know what all those sites have in common? They're all alt-right, discredited, clickbaity, dog-whistling, and/or fake news nonsense. (See, I told you we'd get to it.) Six of the eight listed are on Melissa Zimdars's False, Misleading, Clickbait-y, and/or Satirical “News” Sources. The other two aren't - yet - but both are in the same general cluster, as far as what they produce. Not a single one is known for fair, balanced, in-depth, actual news coverage. Since the rest of the survey goes on to attack reputable news sources, I can only assume this question is actually product placement. Do I think actual journalists from reputable producers should be shushed in order to give hacks from National Review the chance to spread misinformation? Absolutely not!
Then there are the unanswerable questions - or, rather, the answers can be construed to mean whatever the person tabulating the data wants them to mean. For instance, one of the surveys states, "The media has not done its due diligence to expose ObamaCare’s many failures." Your choices in answer are 'yes', 'no', and 'no opinion'. (Unlike the other survey, this one does not let you given your own answer on anything.) So, what do those answers mean? 'Yes, the media hasn't done so' seems obvious - but it could also be taken as 'Yes, it has - the media has actually covered the flaws in the ACA'. 'No' could mean, 'No, the media hasn't done its due diligence!' or 'No, I don't agree with this statement at all! ObamaCare is awesome!' or even 'No, the media has done a fair and exacting job covering flaws in the ACA!'
Then there are statements that are misleading and unanswerable, like "Were you aware that a poll was released revealing that a majority of Americans actually supported President Trump's temporary restriction executive order?" What are the possible answers to that? 'Yes - I was aware of such a poll! It's a disgrace!' or 'Yeah, but there are lots more credible polls that show the majority didn't support it.' Or even, 'Yes - but that doesn't mean it was right or constitutional.' Or there's 'No, I wasn't aware! Let me be outraged!' or 'No, and I don't believe that poll exists or is accurate if it does.'
A couple of the questions on one survey are open-ended, but both are just asking for a list of what sources you use; you can, on this survey (which is apparently intended only for Republicans), input data for the 'other' answers on most of the questions. The other survey has no open-ended questions and does not allow you to answer with clarifications or qualifications.
The questions are all biased and inflammatory, and the surveys are obviously intended to enrage true believers and denigrate the press. One of the surveys is littered with references to "our Party," meaning the survey intentionally singled out Republicans - but would the gathered data have been reported that way? Or will the multiple surveys be conflated, so that the data swings far Right without acknowledgement of bias?
There is zero attempt to make the survey representative of the American people - not just with political demographics, either. Very little demographic data is gathered - only a zip code - so there can be no critical appraisal of how answers (such as they are presented) break down according to age, class, ethnicity, etc. - or how biased the data may unintentionally be due to these and like factors.
Moreover, and a bigger problem in my opinion, is the lack of anonymity. In order for your answers to count, you have to enter your name and email address and click either 'Record My Vote!' on the in-your-face Republican version or 'Submit' for the other. What happens then? Are you sorted out to different lists based on your answers? That seems conspiracy theory-ish. More likely, you're added to a mailing list of some sort, and I don't want to be on a mailing list of some sort. So, this tactic would discourage people who don't particularly want to hear any more propaganda, even if it is disguised as a survey. If you want to read more about this ridiculous survey from a reputable news site (I'm not a news site, and I'm not sure how reputable I am, either), I suggest NPR's coverage: The Trump Media Survey Is Phenomenally Biased. It's Also Useful
We the People, on the other hand, need to learn how to sift out actual news from propaganda, especially when we have an administration so devoted to pushing propaganda. Like I said earlier (see, I get back to everything eventually), a free press, while necessary for democratic society building and maintenance, is not sufficient - the people also have to be good at consuming the news. Even reputable, generally trustworthy sites can sometimes get it wrong. (Have you watched The Newsroom? You should definitely watch The Newsroom. No Misha, but, hey - can't be perfect.) It's our duty as good citizens to try and learn as much as possible, so we can be informed participants in a democratic society. Voting without making sure we have good information is a lot like going to pick up a pet without learning how to take care of it beforehand - it's your fish, now, and if you kill it, you're responsible. (One big difference - generally, a dead fish doesn't take you down with it.) Reading multiple reputable news sources and taking the time to think critically about what they're saying is important - you aren't a good citizen, otherwise. Learning how to tell what is and is not a good source is also important. I'm not going to teach you how to do that, since I've already built a wall of text here, and my fingers are tired of typing, and you can take this whole course from Poynter News University, which is awesome! It's free, it'll only take you an hour or two to go through, and it will help you be a better news consumer and producer. Go forth and do the thing. (For a more advanced course that should be available soon, check this out: In age of alternative facts, a college course on calling out scientific crap.)
Oh, also, here's how you do a survey. Someone pass this on to the White House, yeah?
Meanwhile, as a reward for digging through all my lecture above, here's a pretty picture:
Links!
- Library Land
- Just for fun (and possibly a homeschool geography project?): Mapping “The Hunger Games”: Using location quotients to find the Districts of Panem
- Long Overdue: Why public libraries are finally eliminating the late-return fine. "Late fines and replacement fees can have a huge cost to the communities libraries are meant to serve." Fines are stupid; make them go away.
- Grad Student Discovers A Lost Novel Written By Walt Whitman "The find was announced on Monday as the full book was published online in the literary journal Walt Whitman Quarterly Review; the University of Iowa Press is releasing it in book form."
- Information Access and the 800-Pound Gorilla "Good education happens, and, indeed, democracy itself works, only when students and citizens enjoy unfettered access to good information and good scholarship. In practice this means that neither a homeschooled fifth-grader, nor my 15-year-old son, nor a high school student in rural Arkansas, nor a college student at a state university, nor a scholar in Niger, should be denied free and easy access to nearly all unclassified information held by the Library of Congress. Or, for that matter, from any major library. Current technology makes this somewhat lofty vision eminently achievable. But standing in the way is an 800-pound gorilla: U.S. copyright law."
- Health, Science, & Technology
- NASA to reveal discovery beyond our solar system At this point, I'm really, really hoping for benign alien overlords...
- U.S. Court Ruling Sparks Debate Over Access to Laws in the Digital Space "It seems like a given that the public should have easy access to any laws that govern it. But as digital technology has redefined what constitutes access and ease, the idea has become increasingly complicated.This debate was at the center of a recent court ruling that questioned whether technical standards created by private entities and incorporated into law can be copyrighted. The answer, as passed down by a U.S. District Court, was indeed they can, and anyone who subsequently posts them online is committing a copyright violation."
- International
- Terrorists are building drones. France is destroying them with eagles. Neat! Also, pretty.
- Iceland's president forced to clarify views on pineapple pizza ban What is it like to live in a country where this is the big news of the day?
- British Muslim teacher denied entry to US on school trip
- U.S. News
- House bill would scale back number of free school meals Saving money by stealing our children's health - classy.
- Another Wave Of Bomb Threats Targets Jewish Community Centers "In total, there have been 69 threats at 54 JCCs, in 27 states and a Canadian province, the organization reports — including previous threats on Jan. 9, 18 and 31, as well as 11 threats by telephone on Monday."
- High Court To Hear Case Of Mexican Boy Killed In Cross-Border Shooting
In June 2010, 15-year-old Sergio Hernandez and his friends were playing chicken at the U.S.-Mexico border, daring each other to run up and touch the tall border fence separating Juarez, Mexico, from El Paso, Texas.
At some point during their game, U.S. border patrol agent Jesus Mesa arrived on a bicycle. He detained one of the kids on the U.S. side while the others ran away. Hernandez hid behind a pillar beneath a bridge on the Mexican side of the border. A cellphone video shows the boy peeking out from behind the column, before Mesa shoots and kills him.
The case is not about whether or not the agent committed murder - can you believe that? Nope, the Justice Department decided not to prosecute him. The case is about whether or not the boy's parents can sue their son's killer.
- What’s Going On In The First House Elections Of The Trump Era "Election junkies need not wait until November 2018 to get their next fix. They don’t even need to wait for the gubernatorial elections in New Jersey and Virginia later this year. Five special elections for the House of Representatives are expected to take place over the next few months: California’s 34th congressional district, Georgia 6, Kansas 4, Montana At-large and South Carolina 5....Let’s go through them one by one (in rough order of how competitive the race is likely to be)". Sounds like fun!
- Trump administration forecasts rosier growth than CBO That's a nice way of putting it... Related: The Trump White House is already cooking the books Ah, that's a more accurate way to put it. "As the Wall Street Journal first reported (and as I’ve independently confirmed through my own sources), the Trump transition team instead ordered CEA staffers to predict sustained economic growth of 3 to 3.5 percent. The staffers were then directed to backfill all the other numbers in their models to produce these growth rates."
- Homeland Security unveils sweeping plan to deport undocumented immigrants Related: Trump administration issues new immigration enforcement policies, says goal is not ‘mass deportations’ Mass deportations just happen to be a secondary effect, I guess? But, then, what is the goal?
- How Trump spent his first month in office, by the numbers "From the moment Donald Trump was sworn in as president on the steps of the Capitol until noon Monday, precisely one month had passed. A total of 744 hours. Here’s how he spent each one."
Comments
Post a Comment