comedy of (methodological) errors
After three separate research calamities in three days, my Fulbright project is barely registering a pulse. This is frustrating because, in spite of the long technical name for my study, ("a triangulated QUAL (quan) mixed-methods study"), its research methodology is surprisingly straightforward.
Voilà la recette:
1/2 part survey research + 1/2 part interview research = case study on Sub-Saharan garment manufacture that I can lug to a Masters program or deposit on the doorstep of an academic publication.
So far, I have interviewed general managers from 5 different garment companies and administered surveys to workers from 2 of those 5 factories (60 workers total, 30 from each). According to my research design, I am supposed to survey workers from every factory where I interview management. In addition, I am setting up interviews with government officials, industry people and anyone else who is willing to speak with me.
Which naturally leads to two age-old questions.
Question 1: Why Mauritius?
Answer: Because Mauritius is one of the three largest garment exporters in Sub Saharan Africa-- the others being South Africa and Lesotho.
Question 2: Why QUAL (quan) research?
Answers: Because qualitative research is my labor of love for myriad reasons ... the most important ones being my appreciation for QUAL research's depth, richness and explanatory power, and my interest in discourse, narratives and sensemaking -- which is to say the ways in which people process and perceive certain aspects of the world around them.
The second reason that I am doing a Qual(quan) study is that quantitative research suits my topic of study as well as the amount of time I am spending here (ten months). A 10-month research project based solely on interviews would become unmanageable, FAST. During the 2 months I spent researching my senior thesis in Kenya, I conducted a total of 55 interviews with garment workers. Multiply that by 5 to reflect the Fulbright grant period and you're pushing 300. Can you imagine transcribing, coding and analyzing 300+ interviews?? Not only would a task of such magnitude be near-impossible based on my experiences (i.e. taking the better part of eight months to transcribe all of my interviews from Kenya), my data would be redundant.
Survey research, on the other hand, is great for large n-size studies and allows researchers to speak somewhat authoritatively about patterns and trends... provided that their sample population is representative and their survey method is sound/reliable.
Usher in problem one.. unreliable surveys with unrepresentative samples.
Up until now, I have administered surveys to workers of my choosing over their lunch breaks, with management permission but no interference or oversight. Today, however, when I went to administer surveys to workers at Company 123, the management informed me that they would select my participants, bring them to me and supervise the exercise.
While these stipulations alone were alarming, things actually went South from there. The downward slope began when managers started reading my survey to workers aloud, presumably to help them translate from questions from French to Creole (more widely spoken, understood). With in minutes, this evolved into veritable coaching -- first by gesturing towards answers they thought workers should choose, and second by exclusively reading out loud answer choices that boded well for the company (i.e. the responses "very satisfied" and "rather satisfied" on my question regarding worker's level of satisfaction with the textile industry as a whole).
In addition, over what were easily the worst ten minutes I've endured in my brief-stint as a researcher, I also watched a supervisor more or less answer every single question on a female worker's survey. My Mauritian research assistant told me after the fact that he doubted the woman knew a word of French (perhaps making her an Indian migrant worker?). It's probably no surprise then that her survey was the first survey out of sixty that rated its level of job satisfaction as "trés satisfait."
In an interesting turn, my first-time research assistant ran his own type of interference-- for instance, instructing respondents to answer questions differently than I had advised them in the past (i.e. dependents = # of adults and children not working in your household, verses dependents = # of adults and children who you support financially) and urging workers out of the earshot of management to check "pas satisfait du tout " on their surveys...
Quel catastrophe!
My survey is quickly turning into what every survey researcher fears most -- a never-ending pre-test. With all of the irregularities I observed, I don't know whether I should exclude certain questions from tabulation (job satisfaction, dependents), or exclude the entire pile of surveys.
Chagrin and embarrassment aside, let me take a moment to excoriate the buyers who gauge the social compliance of garment producing firms solely by conducting on site interviews with workers. Seeing how it took me all of one day to see how unreliable the process is, shame on you! Your investigatory methods are negligent and irresponsible...
My other research-related calamities of the week come on the qualitative research front.. which is sad because open-ended interviews are "anything goes" by nature. Screwing up an interview takes a rare kind of talent, and unfortunately it seems to be a talent that I am blessed with.
First off, one of this week's interviewees refused to let me make an audio recording of his statements. Thus, instead of a verbatim transcription, the only document that I will have of our 1 1/2 hour interview are three pages of barely-legible notes in shorthand.
Secondly, said interviewee was not a GM but a human resources manager. Not only did this break with my research design, it prevented me from obtaining basic financial information on the firm... i.e. percentage of export devoted to the US, EU, and Mauritius; changes to production following the
MFA-expiry, etc.
Finally, in the most amusing of my week's research disasters, my interviewee showed up late to our appointment and I was forced to cram an interview designed to take 30-45 minutes into a ten-minute timespan. On top of that, my interviewee commandeered my interview schedule, read the questions to himself, and then answered them aloud. Self-administered surveys are one thing, but self-administered interviews?.. Come on! Then, to make things even more interesting, at the end of our ten-minute session I was offered an internship at his company (!?!?).
And thus ends my comedy of (methodological) errors. If living and working in Mauritius has taught me anything, it’s that laughter is potent medicine. Nonetheless, I'm not sure that even laughter rectifies funky research. My only hope is that after this week's disastrous stint in the field, I'll have serious data-collection-karma in my bank account come February...
Labels: research
