Major Data Collection Methods, Tools and Techniques

In this chapter, we will be looking at the spectrum of data collection methods we've already discussed, paying attention to the tools used to record observations and ways to put those observations down on paper or in electronic files.  The whole point of making systematic, objective observations is to ultimately generate findings that are generalizable back to the theories from which hypotheses were derived.  To do so requires near rigid adherence to the methods of objective science. Useful measures will do the trick nicely.   While the observational methods have already been defined and explained to some extent, the particulars regarding the actual tools and their conversion to data remain.  The big idea is to record observations into a consistent, codified data set. So we begin with the measure (which is an operationalized concept, you will remember).

In the case of quantifiable measures with values, each value will correspond to a numerical value.
Thus:

• 1=yes, 0=no
• 1=small, 2=medium, 3=large
• 1=strongly agree, 2=agree, 3=undecided, 4=disagree, 5=strongly disagree
• 1=0 to 4.9 years, 2=5 to 9.9 years, 3=10 to 14.9 years, 4=over 15 years
• and so on ...

If one were to collect data on several such quantifiable measures, the need would quickly arise for a systematic method for recording numerical values into a data set.  So think of data as a block of rows and columns on a spreadsheet, where each column represents a variable or measure, and each row represents a unit of analysis (i.e., a subject's responses).   Suppose we were to collect information and make observations on respondents (subjects, people) for the following variables:

Age (in years), Educational level (in completed years of formal schooling), Gender (male/female), and the first two items on the Religiosity scale from the previous chapter (Experiential Dimension#1 and Experiential Dimension #2 both measured on a 5 point scale from Strongly disagree to Strongly disagree). 

The first step in creating a record of observations, after deciding how to make them, is to create a codebook on the word processor that will define the numerical value of each category of each measure, and provide the column location of each measure on a spreadsheet.

Our codebook would begin like this:
.
From here we could actually begin to create our spreadsheet of observations:

Respondent I.D. columns 1-3	Age cols 4-5	Education cols 6-7	Gender col 8	Experiential#1 column 9	Experiential#2 column 10
001	23	14	m	5	4
002	18	12	f	3	2
...	...		...	...	...
00n where n is the last respondent	45	08	f	2	1

Obviously the spreadsheet could continue toward the right ---> for a great many columns, accomodating a great many variables. The logic of data set construction is simple (albeit a little tedious).  One row of data per respondent - and one column of data per variable. A data set with 20,000 respondents and 10,000 variables has the same characteristics as the simple 5 by 5 data set used here.
 
Whether making observations with rating scales, questionnaires, interviews, by direct observation, or in experimental situations, when numerical data is required for purposes of statistical summary, the creation of codebooks and electronic data sets is the only way to go.  By  paying close attention to detail during these first steps in data collection, the data will be preserved for repeated analysis.  More than a few of your current instructors have skipped a step or two in this process,  opting to commit variable location and detail to memory.  In almost every case they found that the data they spent countless hours collecting is now a big block of meaningless numbers.   If care is taken to accurately record data this way, statistical analysis will be greatly simplified and greatly enhanced.  We will be returning to the idea of data set construction.  For now, it is important to be aware that observations are going to have to be recorded somehow. Numerical data sets are pretty handy.

In Survey Research, questions can be asked in terms of the self-report method, or in face-to-face interviews. Data recording will be the same in either case, but there are some details to consider.  Self-report questionnaires should be designed to counter respondent fatique, motivate respondents to complete the form, be unambiguous and crystal clear. Because there is no interviewer to coax and encourage, self-report questionnaire design must be more goal oriented than face-to-face interviews -  as brief as possible while including all the measures needed for the project. 
Booklets (whole pages folded in half and inserted within each other), make for an eye-catching format.
Here's an example in booklet form survey .

Survey methods using interviewers requires an additional step which is interviewer training. Student labor, usually performed for course credit or program requirements, is the source of interviewer talent at universities.  Typically, outside the university, people don't work for free, however service oriented communities such as those in Northeast Ohio have a sizeable pool of people from which to draw interviewers.  Regardless of the source, interviewers and raters have to be trained before being unleashed on an unsuspecting population. Here are pages from the training manual for a team of interviewers who were assigned neighborhoods in a West Texas city from which they were to draw a sample of respondents.  The ostensible topic was Quality of Family Life in Amarillo, but the actual purpose of the study was to detect levels of domestic violence.  These were general rules to prepare interviewers for meeting respondents for the first time in the respondent's homes. 

Also, in advance of the interviewers going out into the city, a stratified random sample of households to the block were selected using U.S. Census data.  Every household on each selected residential block was sent a postcard explaining that community organizations were going to be sending interviewers out to canvass their neighborhoods.  Finally, each interviewer was given a letter of introduction that stated the purpose of the study and was signed by the principle investigator (the researcher), the chair of the community organization that was sponsoring the event, and the city manager (liaison from the mayor's office). Phone numbers were listed so that any respondent who questioned the study could call and verify that the project was legitimate.

Here are the training points:


1.    The Role of the Interviewer

Most beginning interviewers feel strain between the way they behave “naturally” and the way they are to behave in interviews. They feel false, awkward and perhaps even manipulative because they cannot make the structured role of interviewer fit with the other styles of their own behavior. The purpose of this training session is to instill in you a sense of “professionalism” and to give you some idea of what to expect when you knock on the door of a rospon-~ dent’s home and are invited inside. Listed below are some qualities that you should try to exhibit as you fill the role of interviewer:

• First of all, you must always keep in mind that you are a guest in another person’s home.
  You should always behave with tact, friendliness, graciousness and courtesy.
• Be cheerful. The questionnaire is designed to give both you and your respondents a little breathing space between very serious questions.
• Suppress your own opinions. You are in the business of extracting the sincere opinions of the people you are interview¬ing. Any comment you make or any emotion you give off may seriously bias your respondent’s answers, Simply record the answers given and move on to the next question. (You may occasionally insert a brief, “That’s interesting” or “I see” in order to koep the flow of the interview.
• Restrain prejudice. All of us have our likes and dislikes. Many of them are not always conscious to us and they may serve to bias the interview. Some of the answers to questionnaire items will cause you to stereotype your respondent, unless you are in touch with those feelings as they occur and can stay on top of them.
• Expect the unexpected. The basic task of the interviewer is to adapt the standardized questionnaire to the unstandardized respondent. Respondents may shock you with their answers. They may show a reluctance to select one of the answers provided in the response set.
  
• In filling the role of professional interviewer, try to see such instances as adventurous and challenging. You are ‘discovering’ information, after all.
• Focus on the respondent. The first couple of times you read through the questionnaire, you will probably be most concerned with getting through the questions correctly. FOR THIS REASON, IT IS IMPORTANT THAT YOU ARE VERY FAMILIAR WITH TILE ENTIRE QUESTIONNAIRE BEFORE INTERVIEWING BEGINS.

As you become more comfortable with the questionnaire contents you will begin to tame your “beginners discomfort” and shape up your own self-restraints. Your focus will shift from a preoccupation with the state of your own mind and move to attending to the respondent’s comfort and the state of her mind. You know you have established “rapport” with your respondent when you find the answers even more interesting than the questions.

2.    Some Cautions on Conversation
Respondent’s reach for information about you.
Sometimes respondents will have never thought enough about the questions you are asking them to feel they have an opinion.  Or they may be so ill at ease with their own opinions that they are reluctant to answer.  As a result, repondents may ask you your opinion.
As part of your role as interviewer, you are a Neutral Observer. As such, you should explain, if the occasion arises, that you are not allowed to express your own opinions until the interview is over. Most respondents will accept this rule of the game.

Respondent's reach for relationship with the interviewer.
Sometimes respondents seem to want to talk, for sheer company and warm personal contact. Respondents may be lonely and see the interview as a chance to break their own isolation. Your role as an objective recorder of information may seem cold and heartless at first, but you must remember the goal of the project. If you can manage to keep to the subject, you will have pushed the entire project toward that goal, which is to make life better for some of the very women whom you are interviewing.

Pushing for an answer
Needless to say, we must have answers to all the questions. In the past, respondents have shown very little reluctance to answer; however, some of the questions you will be asking are extremely intimate and extracting answers will sometimes require diplomacy, compassion and tact.

Isolate the respondent as much as possible. In no case should the interview be conducted in the immediate presence of other adult family members. Ask if there is some room in the house where you could talk more privately, If this is not possible, make an appointment to return at a more convenient time, record the time on the interview schedule and promptly return.  If clear and present distress is caused by any particular item, or if the respondent simply refuses to answer; mark the “no response” category on that question and continue the interview.

3.    Getting the Interview
You have bolstered up your courage. You take a deep breath and knock on the door. It opens and a real, live person is looking at you with suspicion.

• The first step is to determine who to interview in the household. You are looking for women who are married or have been married and who are 18 years old or older. If any other type of person answers the door, ask for “the lady of the house”, “your mother” or some other phrase that will help you to zero in on the respondent.
• If such a person does not live at this household, randomly pick another address and begin again. (Flip a coin, heads — go to the house on the right, tails — go to the house on the left.)
• Once a good prospect has been found, you will need to insure her eligibility for interview. Have a well rehearsed introduction ready!
• “Hello, my name is    . I am part of a research team employed by the Junior League of Amarillo to conduct a study about Family Life in the city and I’d like to ask you some questions.” Show your letter of introduction.

Now you will have to deal with several items to dispell any suspicions that the respondent might have:

• What are you selling? Assure the respondent that you are legitimate and offer to phone a supervisor only if necessary.
• I can’t talk right now. Make that appointment to return at a later time.
• How long will it take? A good line of strategy is: “The interviews have run about 35—40 minutes at the least, but most people talk more because they find it interesting. It will really depend on how interesting you find it and how much you want to say.”
• I really don’t have anything to say! Try to inform the respondent that this is their chance to have their opinions count in a survey. Most of us would liko to be interviewed by Gallup or Harris but never get the oppor¬tunity. Also many people feel that they are not important enough to be respondents. In these cases, you must express the importance of getting this particular interview because this particular respondent is an integral part of the whole research process.

If you are courteously persistant, and convincing; then you will have less trouble getting started with each interview.
Once the respondents fears have been dealt with, you must be certain she fulfills the requirements of the interview.

• Is she married, widowed or divorced/separated?
• s she over the age of seventeen? If so, then you are ready to begin.
• Ask these questions at the outset without recording them.
• All you need at this time are two yeses and an invitation to come inside.

4.    Maneuvers during the course of the interview

Cutting the conversation
Traditional theory and training stress the interviewer’s role in opening up communication, but interviewers, in practice often have to cope with tactfully restricting the flow of reminiscence or irrelevance. Digressions can be shortened by interviewer cues of inattention (acted) or even dexterous interruption, but some digression may be essential for the respondent’s continued concentration, interest, and sense of the interviewer’s acceptance. Use your own judgment and try not to let it get out of hand.

The neutral probe
When the respondent’s answer is so vague or general that a Third person, unaware of the respondent’s personality would find little meaning in it; the interviewer is to “probe” (i.e., “Tell me more about that”, or “Could you give me a little more information?”) For this questionnaire, probing of the “Don’t know’ answers will be especially helpful. Probes such as, ‘Just your best estimate’, or “Are your feelings closer to ‘a’ or bt now?” The point is that interviewers should never take a “Don’t know” answer at face value.

The interview can offer something to the respondent.
This is the payoff for her time spent answering all those darned questions. So what are you giving the respondent in exchange for her answers to your questons?
Things that may be scarce in ordinary conversation:

• attentive listening
• consideration of the personal and experiential
• expression of controversial opinions without risk of argument or disapproval
• intellectual stimulation and insight all of which beginning interviewers note with relief and satisfaction that respondents get something out of the exchange, too.
  

I've given you all my day and night time phone numbers - I will be at the day number from 8:00-12:00 and 1:00-5:00 pm
After 5:30 I'll be at the night number.  As you go out into the city to begin your interviews, keep your courage up and be sure to call any time, even if it is to get a pep talk.  Remember, we are going this project for the good of all the families in the city.
-ddw

In addition to Survey Research, Experimental Observations  result in the same kind of codable data. Here is the the  codebook for the family violence study.  The main difference between coding surveys one respondent at a time and coding experimental data has to do with repeating the measures before and after a treatment. But the codebooks will be remarkably similar.

Hidden/Direct Observations and Field Notes
Observations can be made via unstructured immersion in an ecology (See a set of field notes here) or in a very structured, experimental laboratory environment.  For unstructured interviews, the Anecdotal record consists of brief objective accounts (field notes) of incidences that occur over time.  Specimen Recording are sequential detailed narrative accounts of the action while the Running Record is a sort of combination of the two.  Unstructured observation provides a lot of color to research and can breathe a feeling of reality into otherwise sterile analyses.  Method of recording ranges from pad and pencil to tape or video recording. It should be noted that recording devices may make those observed uncomfortable, so caution to the researcher regarding their open use.

Structured observations will likely be recorded using verbatim recordings, and will employ the use of checklists, rating scales, with Likert- type coding schemes.  This type of research would have observers behind the mirror checking off behaviors as they occur out on the child development center recess room.

A checklist, or rating scale, is often used to insure that specific elements are consistently observed. The observer is constantly scanning the environment for all the behavioral elements at once.  Therefore, the observer must be intimately familiar with all the checklist or rating scale.  For example,

The point is to provide a listing of intended observations.  Similarly, a rating system in the form of a semantic differential scale allows a series of concepts with polar opposite values and a range between them to be applied to a behavioral setting.  Rating scales can be applied to individual measurement, to evlauaton research, teaching effectiveness, socialization of key values and 1000's of other concepts

Provided the proper work on reliability and validity goes into the construction of a semantic differential scale, the product can be an index of behavior. 

By adding all the scores up among the adjective pairs, the respondent is awared a score for the overall measure.  Thus, using a rating scale as an index with cut-off points is an empirical way to begin to see qualitative  differences between workers, teachers, mothers, children.

These are the kind of scales that personnel officers use to make downsizing and other workforce related decisions.

Additional examples of semantic differential scales can be found in the reference section of Bierce Library.

Participant Observation  - The field note discussion hold true for participant observation as a method of data gathering. The main difference between P.O. and other kinds of direct observations is simple.  The participant observer actually becomes one of the group of people he or she is observing.  The idea is that, at least in some sectors of human social life, an outsider will never fully penetrate the true nature of the actors under study.  Only as one of them, can the observer be granted entre and be "shown" the true nature of the group.  It's like going undercover to find the truth.  Obviously there are dangers in such endeavor. Studying dangerous or criminal groups as a participant can easily lead to criminal behavior on the part of the observer, or may lead to harm at the hands of an angry group after exposure as an observer.  If the subject matter is far afield the observer may suffer derision or contempt from colleagues and the community.

Given these shortcomings, participant observation can still be a viable tool for inquiry. Often researchers study that which is already part of their experience - evangelical religion, teaching in difficult circumstances, insider looks into the power structure of universities or the government. For FCS, studies of specific family types, families and children with severe disabilities or challenges, shopper behavior, work inside the textile mill - virtually all the disciplines within FCS can and do take advantage of participant observation's power to reveal otherwise hidden facets of  life.

Some examples in print: An Annotated Bibliography of Qualitative Methodology http://filebox.vt.edu/users/nespor/qualbib.html
and these articles:
    Becker, H. S., Geer, B., Hughes, E. C., & Strauss, A. L. (1961). Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press.
    Anderson, E. (1990). Streetwise: Race, Class, and Change in an Urban Community. Chicago: University of Chicago Press.
    Davis, F. (1980). The Cabdriver and his Fare: Facets of a Fleeting Relationship. In L. Coser (Ed.), The Pleasures of Sociology (pp. 515). New York: New American Library.
    Fine, G. A., & Sandstrom, K. L. (1988). Knowing Children: Participant Observation with Minors. (Vol. 15). Thousand Oaks, CA: Sage.
    Humphreys, L. (1970). Tearoom Trade: Impersonal Sex in Public Places. Chicago: Aldine.
    Stack, C. B. (1974). All Our Kin: Strategies for Survival in a Black Community.
    Whyte, W. F. (1955). Street Corner Society: The Social Structure of an Italian Slum. (2nd ed.). Chicago: University of Chicago Press.
     Kelly D, Pearce S, Mulhall A 'being in the same boat': ethnographic insights into an adolescent cancer unit. INT J NURS STUD 41 (8): 847-857 NOV 2004
    Anderson PKA bird in the house: An anthropological perspective on companion parrots. SOC ANIM 11 (4): 393-418 2003
    McCaughtry N, Rovegno I. JOURNAL OF TEACHING IN PHYSICAL EDUCATION 22 (4): 355-368 JUL 2003
Development of pedagogical content knowledge: Moving from blaming students to predicting skillfulness, recognizing motor development, and understanding emotion
    Dameron S. TRAVAIL HUMAIN 65 (4): 339-361 OCT-DEC 2002. The development process of cooperative relationships within design teams
    Perez RL Fiesta as tradition, fiesta as change: ritual, alcohol and violence in a Mexican community. ADDICTION 95 (3): 365-373 MAR 2000

Return to Syllabus