Text:

McMillan, J.H., & Schumacher, S. (2010). Research in Education: Evidence-Based Inquiry. (7^th ed with MyEducationLab access.) New Jersey: Pearson. ISBN-10:013610133X

Directions: As you both read the text individually and subsequently meet in groups, be prepared to:
1) Define and Discuss each Key Educational Research Term; and
2) Choose the best answer for each Discussion Exercise, citing the page number(s) that support(s) your choice.
e.g. In random sample selection every member of the population has an equal chance of being included in the sample? ___ True ___ False
True, p. 170

PART I: FUNDAMENTAL PRINCIPLES OF EDUCATIONAL RESEARCH

Chapter l: Introduction to the Field of Educational Research

1. All research begins with a problem statement and usually involves a literature review?
___ True ___ False

2. Educational research is...
a. Distinguished by its capacity to prove relationships and identify the truth.
b. Sufficiently complicated that only a small circle of trained professionals can access and use its products.
c. Scientific and disciplined inquiry using quantitative and qualitative approaches.
d. The application of quantitative and statistical procedures.
e. Always regulated by the availability of funds.

3. Research is a relatively new activity in the history of education, just as free public schools for all children is a relatively recent notion in the history of humanity? ___ True ___ False

4. Popper held that a scientist values the empirical approach for “its manner of exposing to confirmation, in every conceivable way, the system to be tested?” ___ True ___ False

5. In research, theories are useful if they...
a. Can be consistent with already body of knowledge.
b. Offer simple observed explanations of relations.
c. Can provide a means for verification and revision.
d. Stimulate further research.
e. All of the above.

6. Research principles are the same in the different disciplines? ___ True ___ False

7. Objectivity refers to data collection and analysis procedures from which a reasonable interpretation can be made? ___ True ___ False

8. “Empirical” means guided by expert opinion from identified authorities?___ True ___ False

9. Reasoning is a thinking process in which one proceeds from a general statement to the specific conclusion (induction) or, the reverse, from specific statements to a generalization (deduction)?
___ True ___ False

10. The inquiry process is essentially one of reflective inquiry? ___ True ___ False

11. The difference between quantitative and qualitative research methodologies is summarized most adequately in which one of the following statements?
a. Quantitative research is more precise than qualitative research.
b. Qualitative research findings are more interesting and relevant than quantitative research findings.
c. Qualitative research is directed to real life issues, and quantitative research is directed to laboratory experiments.
d. Qualitative studies are not pursued by researchers capable of marshaling grant funds for their research pursuits.
e. Quantitative research findings are reported with numbers, and qualitative research findings are reported with narrative.

12. Quantitative and qualitative studies are similar with regard to...
a. Assumptions about how the world operates, and how the mind should be used to learn about the world.
b. The importance of sources, data, and evidence.
c. Research purposes.
d. The importance of context.
e. The role of researchers.

13. Typically, a qualitative researcher uses a pre-established design before data collection, and a quantitative researcher uses an emergent design? ___ True ___ False

14. Most quantitative research is designed to establish universal context-free generalizations; qualitative research is designed to develop context-bound generalizations? ___ True ___ False

15. Generalizability is the extent to which the findings of one study can be applied to other populations and situations? ___ True ___ False

TYPES OF RESEARCH? (answers are repeatable)
a. Evaluation Research b. Applied Research c. Basic Research
16. ___ Tests usefulness of theories in a field.
17. ___ Develops knowledge and research methodology in a specific field of human activity.
18. ___ Tests the merit of a specific program or practice.
19. ___ Tests replicability of specific program or practice.
20. ___ Tests universally applicable theories.
21. ___ Results in an accepted body of scientific knowledge and procedures that is applicable in all fields.

22. Action research is the study of specific classroom or school problems conducted by teachers, with principals possibly involved in the process; collaborative research is conducted by field based practitioners who map out a plan with administrators to further their own careers?
___ True ___ False

23. Education is a unified field of human inquiry? ___ True ___ False

24. Researchers can study almost any single topic with a variety of methods? ___ True ___ False

Chapter 2: Modes of Inquiry, Data Collection Techniques, and Research Reports

25. “Research design” is the methods plan used to collect data and answer research questions?
___ True ___ False

26. The difference between experimental and quasi-experimental designs is usually attributable to...
a. Random assignment.
b. Comparison or control group.
c. Units of measurement.
d. Statistical inference.
e. Items a and d, above.

27. In an experimental design the researcher manipulates what the subjects will experience; in a nonexperimental design there is no manipulation of conditions and no direct suggestion of causality? ___ True ___ False

28. Single subject and case study designs apply the same methods, in different disciplines of education?
___ True ___ False

29. Ex post facto designs may be experimental or nonexperimental? ___ True ___ False

30. Ethnographic research is...
a. Guided by deductive logic.
b. Not pursued by respectable educational researchers.
c. Dominating the fields of psychology and sociology, which are often related to education.
d. Characterized by prolonged field work, employing observations and interviews.
e. The type of research that has been applied most frequently in education during the last 50 years.

31. There are basically seven principal ways to collect data: observations, questionnaires, interviews, documents, tests, supplementary techniques and alternative assessments? ___ True ___ False

32. It is acceptable in in-depth interviews to use a general interview guide but not a set of specific questions worded precisely the same for every interviewee? ___ True ___ False

33. In reading research, it is not important to judge the overall credibility of the study?
___ True ___ False

34. A good research problem states the purpose of the study with clarity and brevity?
___ True ___ False

35. An hypothesis is a prediction of the results of a study? ___ True ___ False

36. The methodology section of a quantitative study describes...
a. The importance of the study to modern life.
b. The methodologies applied in studies that were discussed in
the literature review.
c. The general design of the study.
d. The standards of adequacy that were applied to sample
selection.
e. The standard error of measurement.

37. Until recently there has not been much quantitative educational research reported? ___ True ___ False

Chapter 3: Research Problems: Statements, Questions, and Hypotheses

38. The statement of the research problem reveals...
a. The focus and importance of the problem.
b. The context and scope.
c. The framework for reporting the results.
d. Items b and c, above.
e. All of the above.

39. Quantitative problem statements should reflect...
a. Identification of the independent and dependent variables.
b. Identification of a population and a sample.
c. Application of operational definitions and statements of underlying assumptions.
d. Deductive logic.
e. All of the above.

40. Independent variables may be analogous to a cause, and dependent variables to an effect?
___ True ___ False

41. Researchers can have confidence that a variable identified as either independent or dependent in a study will retain the same role in other studies? ___ True ___ False

42. Which of the following is not operationalized?
a. Three hundred and fifty hours of volunteer tutor services from community organizations during the first quarter of 1992.
b. California Achievement Test scores.
c. Completion of 16 units of graduate study, with no course grades less than "C."
d. Parental involvement.
e. Written approval from the principal.

43. In qualitative research, a case is...
a. A single person, selected for study because of a match with the identified attributes shared with the typical member of a larger population.
b. A collection of primary source literature specimens.
c. A portfolio of artifact descriptions, categorized according to American Psychological Association criteria.
d. A social scene selected because it represents a broader phenomena under study.
e. A methodology or design.

44. Ethnographers select a case to study for an understanding of a broader phenomenon; they do not examine a specific variable? ___ True ___ False

45. Quantitative researchers excel in descriptive narrations, sometimes called “rich” or “thick” descriptions told in “loving detail,” with at least four elements: people, incidents, participants’ language, and participants’ “meanings?” ___ True ___ False

46. Qualitative problem statements should reflect...
a. The case that was selected.
b. Inductive logic.
c. Access to data.
d. The forms of interactive or non-interactive research that will be applied.
e. All of the above.

47. Which of the following would probably not be relevant to the Significance of the Research Problem section of a report?
a. A social or political issue of demonstrated concern to many people.
b. A program or practice that was implemented at five locations in a school district.
c. A personal issue of concern to the researcher.
d. Application of a tested psychological theory to education.
e. All of the above.

Chapter 4: Literature Review

48. Which of the following is true?
a. A literature review is a critique of the status of knowledge on a carefully defined topic.
b. New or little-researched topics usually do not require a literature review.
c. Reference sections of articles generally provide enough information to decide which relevant entries should be read.
d. Items a and c, above.
e. All of the above.

LITERATURE REVIEW TOOLS? (answers are repeatable)
a. Resources in Education b. Current Index to Journals in Education
c. Encyclopedias d. None of the above
49. ___ Research that is planned but not yet implemented
50. ___ Literature reported in ERIC ‘fiche.
51. ___ Articles in professional education journals
52. ___ Biographical information and general knowledge from various fields
53. ___ Legal citations and policy memos

54. Computer searches may be relevant if the researcher...
a. Identifies the data bases and keywords that will be used.
b. Has determined why and what type of literature search is needed.
c. Has directed careful thought to the research problem statement.
d. Items a and c, above.
e. All of the above.

55. In educational research, a thesaurus is a publication that lists and sequences synonyms and antonyms of words in common usage? ___ True ___ False

56. Identify which tends to be most useful when determining the relevance of a particular source.
a. Date of publication, since recent sources are most valuable.
b. An abstract.
c. The suggestion of a reference librarian.
d. A descriptor or keyword summary.
e. Databases recommended by a statistician.

57. A literature review in quantitative research consists of a series of quotes from relevant reports on the topic, supplemented by a series of summarized findings? ___ True ___ False

58. A quantitative research literature review should...
a. Discuss methodologies used in previous studies.
b. Show how relevant studies were reviewed.
c. Reference all materials read by the researcher, and those recommended by leaders in the field.
d. Exhaustively review all relevant contributions from the field.
e. Items a and b, above.

59. Qualitative researchers conduct preliminary literature reviews to propose a study; ongoing literature review is required as the data is collected, because the exact focus of qualitative studies unfolds as the research progresses? ___ True ___ False

60. Meta-analysis...
a. Is a qualitative procedure designed to integrate statistical treatment with narrative description.
b. Has been used by scientists for at least 152 years.
c. Is an excellent procedure to avoid political decisions which may stigmatize the researcher.
d. Is a procedure to statistically summarize the results of selected prior, independently conducted, research studies.
e. All of the above.

Chapter 5: Educational Research on the Internet

61. The Internet is a series of interconnected computers that are able to talk to each other by virtue of a shared language or “protocol”? ___ True ___ False

62. The Internet serves as an exhaustive source for educational research? ___ True ___ False

63. Subject directories are not a primary type of Internet search utilities? ___ True ___ False

64. Search engines offer a quick way to search for specific words that may appear in Web pages?
___ True ___ False

65. Which of the following questions would you not ask when evaluating Internet material?
a. Are the facts or statistics verifiable?
b. Was the search efficient?
c. Is the information current?
d. What is the author’s reputation and qualifications in the subject covered?
e. Is there a bibliography?

PART II: QUANTITATIVE RESEARCH DESIGNS AND METHODS

Chapter 6: Introduction to Designing Quantitative Research

INTERNAL AND EXTERNAL VALIDITY ISSUES?
a. The goal of the study report b. Explanation matches reality
c. Procedure to foster generalizability d. Destroys quality of research product
e. Allows generalizing to related groups
66. ___ Plausible rival hypothesis

67. ___ Internal validity
68. ___ Random sampling
69. ___ Credibility
70. ___ External validity

71. In quantitative research the concept of internal validity describes the efficacy with which extraneous variables (threats) have been controlled? ___ True ___ False

SUBJECTS?
a. A reason for using a small sample b. Subjects for a particular study
c. Group from which sample is drawn d. Convenient for researchers.
e. Proportional or nonproportional
72. ___ Population
73. ___ Stratified random sampling
74. ___ Sample
75. ___ Convenience sampling
76. ___ Financial constraints

77. In random sample selection every member of the population has an equal chance of being included in the sample. ___ True ___ False

78. Probability sampling (which contrasts with availability sampling and purposeful sampling) is a way of drawing subjects from a larger population in such a way that the probability of selecting each member of the population is known? ___ True ___ False

79. Systematic sampling is similar to simple random sampling, except the researcher randomly picks a number and then selects subjects from a list with the assigned chosen number--for example, every fifth subject (5, 10, 15...)?
___ True ___ False

80. Samples consisting of volunteers have been demonstrated to be representative of their respective populations, at least in terms of eight variables studied to date: education, social class, intelligence, sociability, conventionality, ideology, altruism, and proportion of introverts/extroverts? ___ True ___ False

81. The number of subjects in a study is called the sample size, represented by the uppercase letter N (lowercase letter n represents the population)? ___ True ___ False

82. Once the results of a test, questionnaire or other measure have been proven valid, it can be applied in any situation with confidence? ___ True ___ False

83. Researchers should describe their procedures in sufficient detail to permit other researchers to replicate the study? ___ True ___ False

84. The Hawthorne effect is the tendency of people to act differently because they realize they are subjects of research? ___ True ___ False

85. It is possible to be so strict with respect to external validity that practically all research is useful only in specific cases? ___ True ___ False

86. Almost all data-gathering in public schools that applies student participation beyond normal testing requires parental as well as school district and principal permission? ___ True ___ False

Chapter 7: Descriptive Statistics

87. Descriptive statistics are used to make predictions; inferential statistics are used to characterize data?
___ True ___ False

88. The mere presence of statistical procedures helps ensure quality in research? ___ True ___ False

89. Histograms may be distorted by manipulating the spacing of numbers along the vertical dimension of the graph; a crafty researcher can make a very small difference appear great by increasing the space between measurement units? ___ True ___ False

BASIC DATA TREATMENT TOOLS?
a. Graph display b. Mean/median/mode very similar c. Measures of central tendency
d. Measure dispersion/variability e. Shows numbers of observations
90. ___ Mean, median, mode
91. ___ Frequency distribution
92. ___ Histogram
93. ___ Range, standard deviation
94. ___ Normal distribution

95. The mode is the point that divides a rank-ordered distribution into equal halves; the median is the most frequently occurring score? ___ True ___ False

96. The standard deviation indicates the amount, on average, that a set of scores deviates from the mean?
___ True ___ False

97. A scatterplot is a graphic representation of relationship, on a visual matrix formed by scores on two variables?
___ True ___ False

98. Positively skewed scores are located mostly at the low end of the distribution, and negatively skewed scores are mostly located at the high end? ___ True ___ False

Chapter 8: Quantitative Data Collection Techniques

99. Test validity is associated with the appropriateness of a measurement instrument for a particular purpose, and reliability is associated with the instrument's consistency of measurement? ___ True ___ False

100. A test by itself is not valid or invalid because it can be used for different purposes?
___ True ___ False

101. Reliability refers to the appropriateness of measurement, the extent to which the results are consistent with the moral requirements of research in education? ___ True ___ False

102. Norm-referenced tests are based on the scores of a reference group, and criterion-referenced tests are based on established performance standards? ___ True ___ False

103. Likert scales and semantic differentials are open form questionnaire structures that encourage subjects to write in any response they want? ___ True ___ False

104. Interviews in quantitative are essentially vocal questionnaires? ___ True ___ False

105. In quantitative studies, the interviewer should be thought of as a neutral medium through which information is exchanged? ___ True ___ False

106. Interviewers should be as expeditious as possible, to save interviewee time; it is inappropriate to engage in "small talk," even for a few minutes? ___ True ___ False

107. Two appropriate forms of note-taking interviewee responses are: to take notes while the interviewee is talking, or to wait until the interview is over to take notes? ___ True ___ False

SECONDARY READINGS:
Chapter 9: Nonexperimental Research Designs & Surveys
Chapter 10: Experimental and Single-Subject Research Designs
Chapter 11: Inferential Statistics

PART III: QUALITATIVE RESEARCH DESIGNS AND METHODS

Chapter 12: Introduction to Designing Qualitative Research

108. Qualitative researchers emphasize the importance of collecting data using a skilled, prepared person, rather than a single instrument? ___ True ___ False

109. Single subject and case study designs apply the same methods, in different disciplines of education?
___ True ___ False

110. The principle of an emergent design requires simultaneous and interactive data collection and treatment, rather than discrete sequential steps; each incremental decision depends on prior information? ___ True ___ False

111. Purposeful sampling, in contrast to probabilistic sampling, is based on information-rich cases for in-depth study, to increase the utility of information obtained from small samples? ___ True ___ False

112. A qualitative study winds down when the researcher senses that further data collection will not yield any new data relevant to the problem? ___ True ___ False

113. Qualitative researchers commonly use combined strategies to enhance design validity and identify mutual meanings: participant language, multiple researchers, etc? ___ True ___ False

114. Disciplined subjectivity in qualitative research refers to...
a. Consistency of the researcher's self-questioning.
b. Self-awareness of personal preferences and theoretical predispositions.
c. Face-to-face interaction between researcher and participants.
d. The use of personal experiential empathy in data collection.
e. All of the above.

QUALITATIVE RESEARCH--STRATEGIES TO ENHANCE DESIGN VALIDITY?
a. Periodically verify data collected b. Subjects review interview synthesis
c. Findings/participant reality match d. Data agreement from research team
e. Actively search for discrepant data
115. ___ Multiple researchers
116. ___ Prolonged field work
117. ___ Negative cases
118. ___ Participant review
119. ___ Member checking

120. Qualitative researchers rarely face ethical decisions that must be resolved before the study is completed--if this happens a proven strategy is to make secret recordings of improper dialogue and forward them to a judge or grand jury? ___ True ___ False

Chapter 13: Qualitative Strategies
121. Qualitative research rarely depends on multimethod strategies to collect data? ___ True ___ False

122. Quantitative researchers begin with foreshadowed problems that will be reformulated during the data collection phases of study? ___ True ___ False

123. Ethnographers think of observations, interviews, and artifacts as data sources? ___ True ___ False

124. Foreshadowed problems are broadly phrased research questions about...
a. Time, place and logistical concerns.
b. Participants, events, and expected findings.
c. Procedures for organizing and treating numeric data.
d. What happens, why, and how.
e. Items a, b, and d, above.

125. Ethnographers can and must observe and record everything? ___ True ___ False

126. The traditional ethnographic role is that of participant-observer or interviewer?
___ True ___ False

127. Ethnographers are required to “listen with all their senses,” to identify participant perceptions, thoughts, beliefs, feelings, and values--the “because, why, and wherefore” of the case? ___ True ___ False

128. In both field notes and reflexive records, qualitative researchers separate actual observations from tentative interpretations? ___ True ___ False

129. In-depth interviews contain open response questions to obtain data of “participant meanings”?
___ True ___ False

130. In the standardized open-ended interview, participants are asked the same questions in the same order, thus reducing interviewer flexibility? ___ True ___ False

TYPES OF ETHNOGRAPHIC INTERVIEWS?
a. Questions from immediate context b. In-depth interviews of known experts
c. Respond well to broad questions d. Obtain data about a culture
e. Predetermined wording and sequence
131. ___ Elites
132. ___ Informal conversation
133. ___ Standardized open-ended
134. ___ Career/life history
135. ___ Key-informant

136. An interview protocol or schedule is a roster of interview questions, designed to facilitate interview probes and the quantification of resultant data? ___ True ___ False

137. It is not necessary to take notes when using a tape recorder? ___ True ___ False

138. As a rule of thumb, for every hour of interviewing a researcher usually allows two hours of further work to obtain the final record or transcript and the additional elaborations (debriefings and self-reflection)?
___ True ___ False

139. Artifact collections are commonly unusual works of art, or everyday utensils that are highly decorated or finely crafted, which qualitative researchers assemble to demonstrate the best qualities of a culture under study? ___ True ___ False

Chapter 14: Qualitative Data Analysis
140. Qualitative data treatment is characterized by...
a. A process of interim and discovery analysis.
b. Pattern seeking for plausible explanations.
c. Interpretivist / Subjectivist style.
d. Developing coding topics and categories.
e. All of the above.

141. In qualitative research, segmenting and classifying data can be accomplished by “coding” (to identify patterns in a complex array of data) and “chunking” (chunks are meaningful clusters of data grouped for synthesis and interpretation)? ___ True ___ False

142. EMIC categories reflect the outsider’s view of the situation; and ETIC categories represent insider’s views? ___ True ___ False

143. Triangulation is a cross-validation procedure in which data from various sources is used to support the evidence? ___ True ___ False

144. In qualitative research, presentation of participants’ language is neither imperative nor desired?
___ True ___ False

PART IV: ANALYTIC RESEARCH
Chapter 15: Concept Analysis and Historical Research

145. Historical research fits into which one of the following categories from our text?
a. Ex post facto research b. Analytic research, as a form of qualitative inquiry
c. Quantitative research d. Survey research e. All of the above.

146. In contrast to experimental research, analytical research commonly describes and interprets the past or recent past from selected sources?___ True ___ False

147. Historical research may focus on biographies, movements, institutions, and concepts?
___ True ___ False

148. Documents may be letters, diaries, wills, maps, journals, court records, regulations and proclamations, laws, autobiographies, newspapers, receipts, and official minutes of meetings?
___ True ___ False

149. Primary sources are documents or testimonies by eyewitnesses of the event under scrutiny; secondary sources are not by eyewitnesses? ___ True ___ False

150. Selective use of secondary source material is appropriate in historical research?
___ True ___ False

151. In historical research, external criticism determines the credibility of the facts stated by the source (was it written by an idiot?), and internal criticism determines the historical authenticity of the source (was it actually written during the time period under study?)? ___ True ___ False

PART V: EVALUATION AND POLICY RESEARCH DESIGNS AND METHODS

Chapter 16: Evaluation and Policy Research Designs and Methods

152. Evaluation research does not require a formal design and procedures to determine the merit and worth of a practice? ___ True ___ False

EVALUATION RESEARCH TERMS?
a. Collect data to improve a curriculum in a development stage?
b. Conducted once a program is fully developed?
c. Should the model be adopted?
153. ___ Worth
154. ___ Summative evaluation
155. ___ Formative evaluation

156. Objectives-oriented evaluation is based on the degree to which the objectives of a practice are attained by the target group? ___ True ___ False

157. A mixed-methods study combines qualitative and quantitative techniques and/or data analysis?
___ True ___ False

158. Content-related evidence for validity can be determined by a panel of local experts who logically compare the curriculum content with the test items? ___ True ___ False

159. Cost-effectiveness analysis compares program outcomes with the costs of alternative programs when the objectives of different programs are similar and when common measures of effectiveness are used?
___ True ___ False

160. Most evaluation and policy studies are not intended to be used, but to inform interested audiences?
___ True ___ False