design methods research paper

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

How to Write a Methods Section for a Psychology Paper

Tips and Examples of an APA Methods Section

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

Verywell / Brianna Gilmartin 

The methods section of an APA format psychology paper provides the methods and procedures used in a research study or experiment . This part of an APA paper is critical because it allows other researchers to see exactly how you conducted your research.

Method refers to the procedure that was used in a research study. It included a precise description of how the experiments were performed and why particular procedures were selected. While the APA technically refers to this section as the 'method section,' it is also often known as a 'methods section.'

The methods section ensures the experiment's reproducibility and the assessment of alternative methods that might produce different results. It also allows researchers to replicate the experiment and judge the study's validity.

This article discusses how to write a methods section for a psychology paper, including important elements to include and tips that can help.

What to Include in a Method Section

So what exactly do you need to include when writing your method section? You should provide detailed information on the following:

• Research design
• Participants
• Participant behavior

The method section should provide enough information to allow other researchers to replicate your experiment or study.

Components of a Method Section

The method section should utilize subheadings to divide up different subsections. These subsections typically include participants, materials, design, and procedure.

Participants 

In this part of the method section, you should describe the participants in your experiment, including who they were (and any unique features that set them apart from the general population), how many there were, and how they were selected. If you utilized random selection to choose your participants, it should be noted here.

For example: "We randomly selected 100 children from elementary schools near the University of Arizona."

At the very minimum, this part of your method section must convey:

• Basic demographic characteristics of your participants (such as sex, age, ethnicity, or religion)
• The population from which your participants were drawn
• Any restrictions on your pool of participants
• How many participants were assigned to each condition and how they were assigned to each group (i.e., randomly assignment , another selection method, etc.)
• Why participants took part in your research (i.e., the study was advertised at a college or hospital, they received some type of incentive, etc.)

Information about participants helps other researchers understand how your study was performed, how generalizable the result might be, and allows other researchers to replicate the experiment with other populations to see if they might obtain the same results.

In this part of the method section, you should describe the materials, measures, equipment, or stimuli used in the experiment. This may include:

• Testing instruments
• Technical equipment
• Any psychological assessments that were used
• Any special equipment that was used

For example: "Two stories from Sullivan et al.'s (1994) second-order false belief attribution tasks were used to assess children's understanding of second-order beliefs."

For standard equipment such as computers, televisions, and videos, you can simply name the device and not provide further explanation.

Specialized equipment should be given greater detail, especially if it is complex or created for a niche purpose. In some instances, such as if you created a special material or apparatus for your study, you might need to include an illustration of the item in the appendix of your paper.

In this part of your method section, describe the type of design used in the experiment. Specify the variables as well as the levels of these variables. Identify:

• The independent variables
• Dependent variables
• Control variables
• Any extraneous variables that might influence your results.

Also, explain whether your experiment uses a  within-groups  or between-groups design.

For example: "The experiment used a 3x2 between-subjects design. The independent variables were age and understanding of second-order beliefs."

The next part of your method section should detail the procedures used in your experiment. Your procedures should explain:

• What the participants did
• How data was collected
• The order in which steps occurred

For example: "An examiner interviewed children individually at their school in one session that lasted 20 minutes on average. The examiner explained to each child that he or she would be told two short stories and that some questions would be asked after each story. All sessions were videotaped so the data could later be coded."

Keep this subsection concise yet detailed. Explain what you did and how you did it, but do not overwhelm your readers with too much information.

Tips for How to Write a Methods Section

In addition to following the basic structure of an APA method section, there are also certain things you should remember when writing this section of your paper. Consider the following tips when writing this section:

• Use the past tense : Always write the method section in the past tense.
• Be descriptive : Provide enough detail that another researcher could replicate your experiment, but focus on brevity. Avoid unnecessary detail that is not relevant to the outcome of the experiment.
• Use an academic tone : Use formal language and avoid slang or colloquial expressions. Word choice is also important. Refer to the people in your experiment or study as "participants" rather than "subjects."
• Use APA format : Keep a style guide on hand as you write your method section. The Publication Manual of the American Psychological Association is the official source for APA style.
• Make connections : Read through each section of your paper for agreement with other sections. If you mention procedures in the method section, these elements should be discussed in the results and discussion sections.
• Proofread : Check your paper for grammar, spelling, and punctuation errors.. typos, grammar problems, and spelling errors. Although a spell checker is a handy tool, there are some errors only you can catch.

After writing a draft of your method section, be sure to get a second opinion. You can often become too close to your work to see errors or lack of clarity. Take a rough draft of your method section to your university's writing lab for additional assistance.

A Word From Verywell

The method section is one of the most important components of your APA format paper. The goal of your paper should be to clearly detail what you did in your experiment. Provide enough detail that another researcher could replicate your study if they wanted.

Finally, if you are writing your paper for a class or for a specific publication, be sure to keep in mind any specific instructions provided by your instructor or by the journal editor. Your instructor may have certain requirements that you need to follow while writing your method section.

Frequently Asked Questions

While the subsections can vary, the three components that should be included are sections on the participants, the materials, and the procedures.

• Describe who the participants were in the study and how they were selected.
• Define and describe the materials that were used including any equipment, tests, or assessments
• Describe how the data was collected

To write your methods section in APA format, describe your participants, materials, study design, and procedures. Keep this section succinct, and always write in the past tense. The main heading of this section should be labeled "Method" and it should be centered, bolded, and capitalized. Each subheading within this section should be bolded, left-aligned and in title case.

The purpose of the methods section is to describe what you did in your experiment. It should be brief, but include enough detail that someone could replicate your experiment based on this information. Your methods section should detail what you did to answer your research question. Describe how the study was conducted, the study design that was used and why it was chosen, and how you collected the data and analyzed the results.

Erdemir F. How to write a materials and methods section of a scientific article ? Turk J Urol . 2013;39(Suppl 1):10-5. doi:10.5152/tud.2013.047

Kallet RH. How to write the methods section of a research paper . Respir Care . 2004;49(10):1229-32. PMID: 15447808.

American Psychological Association.  Publication Manual of the American Psychological Association  (7th ed.). Washington DC: The American Psychological Association; 2019.

American Psychological Association. APA Style Journal Article Reporting Standards . Published 2020.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Types of Research Designs
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Introduction

Before beginning your paper, you need to decide how you plan to design the study .

The research design refers to the overall strategy and analytical approach that you have chosen in order to integrate, in a coherent and logical way, the different components of the study, thus ensuring that the research problem will be thoroughly investigated. It constitutes the blueprint for the collection, measurement, and interpretation of information and data. Note that the research problem determines the type of design you choose, not the other way around!

De Vaus, D. A. Research Design in Social Research . London: SAGE, 2001; Trochim, William M.K. Research Methods Knowledge Base. 2006.

General Structure and Writing Style

The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem logically and as unambiguously as possible . In social sciences research, obtaining information relevant to the research problem generally entails specifying the type of evidence needed to test the underlying assumptions of a theory, to evaluate a program, or to accurately describe and assess meaning related to an observable phenomenon.

With this in mind, a common mistake made by researchers is that they begin their investigations before they have thought critically about what information is required to address the research problem. Without attending to these design issues beforehand, the overall research problem will not be adequately addressed and any conclusions drawn will run the risk of being weak and unconvincing. As a consequence, the overall validity of the study will be undermined.

The length and complexity of describing the research design in your paper can vary considerably, but any well-developed description will achieve the following :

• Identify the research problem clearly and justify its selection, particularly in relation to any valid alternative designs that could have been used,
• Review and synthesize previously published literature associated with the research problem,
• Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem,
• Effectively describe the information and/or data which will be necessary for an adequate testing of the hypotheses and explain how such information and/or data will be obtained, and
• Describe the methods of analysis to be applied to the data in determining whether or not the hypotheses are true or false.

The research design is usually incorporated into the introduction of your paper . You can obtain an overall sense of what to do by reviewing studies that have utilized the same research design [e.g., using a case study approach]. This can help you develop an outline to follow for your own paper.

NOTE : Use the SAGE Research Methods Online and Cases and the SAGE Research Methods Videos databases to search for scholarly resources on how to apply specific research designs and methods . The Research Methods Online database contains links to more than 175,000 pages of SAGE publisher's book, journal, and reference content on quantitative, qualitative, and mixed research methodologies. Also included is a collection of case studies of social research projects that can be used to help you better understand abstract or complex methodological concepts. The Research Methods Videos database contains hours of tutorials, interviews, video case studies, and mini-documentaries covering the entire research process.

Creswell, John W. and J. David Creswell. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . 5th edition. Thousand Oaks, CA: Sage, 2018; De Vaus, D. A. Research Design in Social Research . London: SAGE, 2001; Gorard, Stephen. Research Design: Creating Robust Approaches for the Social Sciences . Thousand Oaks, CA: Sage, 2013; Leedy, Paul D. and Jeanne Ellis Ormrod. Practical Research: Planning and Design . Tenth edition. Boston, MA: Pearson, 2013; Vogt, W. Paul, Dianna C. Gardner, and Lynne M. Haeffele. When to Use What Research Design . New York: Guilford, 2012.

Action Research Design

Definition and Purpose

The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. Then the intervention is carried out [the "action" in action research] during which time, pertinent observations are collected in various forms. The new interventional strategies are carried out, and this cyclic process repeats, continuing until a sufficient understanding of [or a valid implementation solution for] the problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper understanding of a given situation, starting with conceptualizing and particularizing the problem and moving through several interventions and evaluations.

What do these studies tell you ?

• This is a collaborative and adaptive research design that lends itself to use in work or community situations.
• Design focuses on pragmatic and solution-driven research outcomes rather than testing theories.
• When practitioners use action research, it has the potential to increase the amount they learn consciously from their experience; the action research cycle can be regarded as a learning cycle.
• Action research studies often have direct and obvious relevance to improving practice and advocating for change.
• There are no hidden controls or preemption of direction by the researcher.

What these studies don't tell you ?

• It is harder to do than conducting conventional research because the researcher takes on responsibilities of advocating for change as well as for researching the topic.
• Action research is much harder to write up because it is less likely that you can use a standard format to report your findings effectively [i.e., data is often in the form of stories or observation].
• Personal over-involvement of the researcher may bias research results.
• The cyclic nature of action research to achieve its twin outcomes of action [e.g. change] and research [e.g. understanding] is time-consuming and complex to conduct.
• Advocating for change usually requires buy-in from study participants.

Coghlan, David and Mary Brydon-Miller. The Sage Encyclopedia of Action Research . Thousand Oaks, CA:  Sage, 2014; Efron, Sara Efrat and Ruth Ravid. Action Research in Education: A Practical Guide . New York: Guilford, 2013; Gall, Meredith. Educational Research: An Introduction . Chapter 18, Action Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Gorard, Stephen. Research Design: Creating Robust Approaches for the Social Sciences . Thousand Oaks, CA: Sage, 2013; Kemmis, Stephen and Robin McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research . Norman Denzin and Yvonna S. Lincoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605; McNiff, Jean. Writing and Doing Action Research . London: Sage, 2014; Reason, Peter and Hilary Bradbury. Handbook of Action Research: Participative Inquiry and Practice . Thousand Oaks, CA: SAGE, 2001.

Case Study Design

A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey or comprehensive comparative inquiry. It is often used to narrow down a very broad field of research into one or a few easily researchable examples. The case study research design is also useful for testing whether a specific theory and model actually applies to phenomena in the real world. It is a useful design when not much is known about an issue or phenomenon.

• Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
• A researcher using a case study design can apply a variety of methodologies and rely on a variety of sources to investigate a research problem.
• Design can extend experience or add strength to what is already known through previous research.
• Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and the extension of methodologies.
• The design can provide detailed descriptions of specific and rare cases.
• A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things.
• Intense exposure to the study of a case may bias a researcher's interpretation of the findings.
• Design does not facilitate assessment of cause and effect relationships.
• Vital information may be missing, making the case hard to interpret.
• The case may not be representative or typical of the larger problem being investigated.
• If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your interpretation of the findings can only apply to that particular case.

Case Studies. Writing@CSU. Colorado State University; Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 4, Flexible Methods: Case Study Design. 2nd ed. New York: Columbia University Press, 1999; Gerring, John. “What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Greenhalgh, Trisha, editor. Case Study Evaluation: Past, Present and Future Challenges . Bingley, UK: Emerald Group Publishing, 2015; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Stake, Robert E. The Art of Case Study Research . Thousand Oaks, CA: SAGE, 1995; Yin, Robert K. Case Study Research: Design and Theory . Applied Social Research Methods Series, no. 5. 3rd ed. Thousand Oaks, CA: SAGE, 2003.

Causal Design

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

• Empirical association -- a valid conclusion is based on finding an association between the independent variable and the dependent variable.
• Appropriate time order -- to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable.
• Nonspuriousness -- a relationship between two variables that is not due to variation in a third variable.
• Causality research designs assist researchers in understanding why the world works the way it does through the process of proving a causal link between variables and by the process of eliminating other possibilities.
• Replication is possible.
• There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared.
• Not all relationships are causal! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent].
• Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven.
• If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and, therefore, to establish which variable is the actual cause and which is the  actual effect.

Beach, Derek and Rasmus Brun Pedersen. Causal Case Study Methods: Foundations and Guidelines for Comparing, Matching, and Tracing . Ann Arbor, MI: University of Michigan Press, 2016; Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice . Chapter 5, Causation and Research Designs. 3rd ed. Thousand Oaks, CA: Pine Forge Press, 2007; Brewer, Ernest W. and Jennifer Kubn. “Causal-Comparative Design.” In Encyclopedia of Research Design . Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 125-132; Causal Research Design: Experimentation. Anonymous SlideShare Presentation; Gall, Meredith. Educational Research: An Introduction . Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base. 2006.

Cohort Design

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, rather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed."

• Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof.
• Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease).
• The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors often relies upon cohort designs.
• Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect.
• Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.].
• Either original data or secondary data can be used in this design.
• In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables.
• Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
• Due to the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants.

Healy P, Devane D. “Methodological Considerations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36; Glenn, Norval D, editor. Cohort Analysis . 2nd edition. Thousand Oaks, CA: Sage, 2005; Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Payne, Geoff. “Cohort Study.” In The SAGE Dictionary of Social Research Methods . Victor Jupp, editor. (Thousand Oaks, CA: Sage, 2006), pp. 31-33; Study Design 101. Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study. Wikipedia.

Cross-Sectional Design

Cross-sectional research designs have three distinctive features: no time dimension; a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure differences between or from among a variety of people, subjects, or phenomena rather than a process of change. As such, researchers using this design can only employ a relatively passive approach to making causal inferences based on findings.

• Cross-sectional studies provide a clear 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time.
• Unlike an experimental design, where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena.
• Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
• Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling.
• Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound.
• Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population.
• Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
• Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult.
• Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical or temporal contexts.
• Studies cannot be utilized to establish cause and effect relationships.
• This design only provides a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen.
• There is no follow up to the findings.

Bethlehem, Jelke. "7: Cross-sectional Research." In Research Methodology in the Social, Behavioural and Life Sciences . Herman J Adèr and Gideon J Mellenbergh, editors. (London, England: Sage, 1999), pp. 110-43; Bourque, Linda B. “Cross-Sectional Design.” In  The SAGE Encyclopedia of Social Science Research Methods . Michael S. Lewis-Beck, Alan Bryman, and Tim Futing Liao. (Thousand Oaks, CA: 2004), pp. 230-231; Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design Application, Strengths and Weaknesses of Cross-Sectional Studies. Healthknowledge, 2009. Cross-Sectional Study. Wikipedia.

Descriptive Design

Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation.

• The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject [a.k.a., the Heisenberg effect whereby measurements of certain systems cannot be made without affecting the systems].
• Descriptive research is often used as a pre-cursor to more quantitative research designs with the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
• If the limitations are understood, they can be a useful tool in developing a more focused study.
• Descriptive studies can yield rich data that lead to important recommendations in practice.
• Appoach collects a large amount of data for detailed analysis.
• The results from a descriptive research cannot be used to discover a definitive answer or to disprove a hypothesis.
• Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
• The descriptive function of research is heavily dependent on instrumentation for measurement and observation.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999; Given, Lisa M. "Descriptive Research." In Encyclopedia of Measurement and Statistics . Neil J. Salkind and Kristin Rasmussen, editors. (Thousand Oaks, CA: Sage, 2007), pp. 251-254; McNabb, Connie. Descriptive Research Methodologies. Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design, September 26, 2008; Erickson, G. Scott. "Descriptive Research Design." In New Methods of Market Research and Analysis . (Northampton, MA: Edward Elgar Publishing, 2017), pp. 51-77; Sahin, Sagufta, and Jayanta Mete. "A Brief Study on Descriptive Research: Its Nature and Application in Social Science." International Journal of Research and Analysis in Humanities 1 (2021): 11; K. Swatzell and P. Jennings. “Descriptive Research: The Nuts and Bolts.” Journal of the American Academy of Physician Assistants 20 (2007), pp. 55-56; Kane, E. Doing Your Own Research: Basic Descriptive Research in the Social Sciences and Humanities . London: Marion Boyars, 1985.

Experimental Design

A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation.

• Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “What causes something to occur?”
• Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects.
• Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
• Approach provides the highest level of evidence for single studies.
• The design is artificial, and results may not generalize well to the real world.
• The artificial settings of experiments may alter the behaviors or responses of participants.
• Experimental designs can be costly if special equipment or facilities are needed.
• Some research problems cannot be studied using an experiment because of ethical or technical reasons.
• Difficult to apply ethnographic and other qualitative methods to experimentally designed studies.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs. School of Psychology, University of New England, 2000; Chow, Siu L. "Experimental Design." In Encyclopedia of Research Design . Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 448-453; "Experimental Design." In Social Research Methods . Nicholas Walliman, editor. (London, England: Sage, 2006), pp, 101-110; Experimental Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Kirk, Roger E. Experimental Design: Procedures for the Behavioral Sciences . 4th edition. Thousand Oaks, CA: Sage, 2013; Trochim, William M.K. Experimental Design. Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research. Slideshare presentation.

Exploratory Design

An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to or rely upon to predict an outcome . The focus is on gaining insights and familiarity for later investigation or undertaken when research problems are in a preliminary stage of investigation. Exploratory designs are often used to establish an understanding of how best to proceed in studying an issue or what methodology would effectively apply to gathering information about the issue.

The goals of exploratory research are intended to produce the following possible insights:

• Familiarity with basic details, settings, and concerns.
• Well grounded picture of the situation being developed.
• Generation of new ideas and assumptions.
• Development of tentative theories or hypotheses.
• Determination about whether a study is feasible in the future.
• Issues get refined for more systematic investigation and formulation of new research questions.
• Direction for future research and techniques get developed.
• Design is a useful approach for gaining background information on a particular topic.
• Exploratory research is flexible and can address research questions of all types (what, why, how).
• Provides an opportunity to define new terms and clarify existing concepts.
• Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
• In the policy arena or applied to practice, exploratory studies help establish research priorities and where resources should be allocated.
• Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large.
• The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings. They provide insight but not definitive conclusions.
• The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value to decision-makers.
• Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem.

Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Streb, Christoph K. "Exploratory Case Study." In Encyclopedia of Case Study Research . Albert J. Mills, Gabrielle Durepos and Eiden Wiebe, editors. (Thousand Oaks, CA: Sage, 2010), pp. 372-374; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research. Wikipedia.

Field Research Design

Sometimes referred to as ethnography or participant observation, designs around field research encompass a variety of interpretative procedures [e.g., observation and interviews] rooted in qualitative approaches to studying people individually or in groups while inhabiting their natural environment as opposed to using survey instruments or other forms of impersonal methods of data gathering. Information acquired from observational research takes the form of “ field notes ” that involves documenting what the researcher actually sees and hears while in the field. Findings do not consist of conclusive statements derived from numbers and statistics because field research involves analysis of words and observations of behavior. Conclusions, therefore, are developed from an interpretation of findings that reveal overriding themes, concepts, and ideas. More information can be found HERE .

• Field research is often necessary to fill gaps in understanding the research problem applied to local conditions or to specific groups of people that cannot be ascertained from existing data.
• The research helps contextualize already known information about a research problem, thereby facilitating ways to assess the origins, scope, and scale of a problem and to gage the causes, consequences, and means to resolve an issue based on deliberate interaction with people in their natural inhabited spaces.
• Enables the researcher to corroborate or confirm data by gathering additional information that supports or refutes findings reported in prior studies of the topic.
• Because the researcher in embedded in the field, they are better able to make observations or ask questions that reflect the specific cultural context of the setting being investigated.
• Observing the local reality offers the opportunity to gain new perspectives or obtain unique data that challenges existing theoretical propositions or long-standing assumptions found in the literature.

What these studies don't tell you

• A field research study requires extensive time and resources to carry out the multiple steps involved with preparing for the gathering of information, including for example, examining background information about the study site, obtaining permission to access the study site, and building trust and rapport with subjects.
• Requires a commitment to staying engaged in the field to ensure that you can adequately document events and behaviors as they unfold.
• The unpredictable nature of fieldwork means that researchers can never fully control the process of data gathering. They must maintain a flexible approach to studying the setting because events and circumstances can change quickly or unexpectedly.
• Findings can be difficult to interpret and verify without access to documents and other source materials that help to enhance the credibility of information obtained from the field  [i.e., the act of triangulating the data].
• Linking the research problem to the selection of study participants inhabiting their natural environment is critical. However, this specificity limits the ability to generalize findings to different situations or in other contexts or to infer courses of action applied to other settings or groups of people.
• The reporting of findings must take into account how the researcher themselves may have inadvertently affected respondents and their behaviors.

Historical Design

The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute a hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid.

• The historical research design is unobtrusive; the act of research does not affect the results of the study.
• The historical approach is well suited for trend analysis.
• Historical records can add important contextual background required to more fully understand and interpret a research problem.
• There is often no possibility of researcher-subject interaction that could affect the findings.
• Historical sources can be used over and over to study different research problems or to replicate a previous study.
• The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem.
• Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts.
• Interpreting historical sources can be very time consuming.
• The sources of historical materials must be archived consistently to ensure access. This may especially challenging for digital or online-only sources.
• Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
• Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity.
• It is rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged.

Howell, Martha C. and Walter Prevenier. From Reliable Sources: An Introduction to Historical Methods . Ithaca, NY: Cornell University Press, 2001; Lundy, Karen Saucier. "Historical Research." In The Sage Encyclopedia of Qualitative Research Methods . Lisa M. Given, editor. (Thousand Oaks, CA: Sage, 2008), pp. 396-400; Marius, Richard. and Melvin E. Page. A Short Guide to Writing about History . 9th edition. Boston, MA: Pearson, 2015; Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58;  Gall, Meredith. Educational Research: An Introduction . Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007.

Longitudinal Design

A longitudinal study follows the same sample over time and makes repeated observations. For example, with longitudinal surveys, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study sometimes referred to as a panel study.

• Longitudinal data facilitate the analysis of the duration of a particular phenomenon.
• Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments.
• The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time].
• Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.
• The data collection method may change over time.
• Maintaining the integrity of the original sample can be difficult over an extended period of time.
• It can be difficult to show more than one variable at a time.
• This design often needs qualitative research data to explain fluctuations in the results.
• A longitudinal research design assumes present trends will continue unchanged.
• It can take a long period of time to gather results.
• There is a need to have a large sample size and accurate sampling to reach representativness.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Forgues, Bernard, and Isabelle Vandangeon-Derumez. "Longitudinal Analyses." In Doing Management Research . Raymond-Alain Thiétart and Samantha Wauchope, editors. (London, England: Sage, 2001), pp. 332-351; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Menard, Scott, editor. Longitudinal Research . Thousand Oaks, CA: Sage, 2002; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-120; Longitudinal Study. Wikipedia.

Meta-Analysis Design

Meta-analysis is an analytical methodology designed to systematically evaluate and summarize the results from a number of individual studies, thereby, increasing the overall sample size and the ability of the researcher to study effects of interest. The purpose is to not simply summarize existing knowledge, but to develop a new understanding of a research problem using synoptic reasoning. The main objectives of meta-analysis include analyzing differences in the results among studies and increasing the precision by which effects are estimated. A well-designed meta-analysis depends upon strict adherence to the criteria used for selecting studies and the availability of information in each study to properly analyze their findings. Lack of information can severely limit the type of analyzes and conclusions that can be reached. In addition, the more dissimilarity there is in the results among individual studies [heterogeneity], the more difficult it is to justify interpretations that govern a valid synopsis of results. A meta-analysis needs to fulfill the following requirements to ensure the validity of your findings:

• Clearly defined description of objectives, including precise definitions of the variables and outcomes that are being evaluated;
• A well-reasoned and well-documented justification for identification and selection of the studies;
• Assessment and explicit acknowledgment of any researcher bias in the identification and selection of those studies;
• Description and evaluation of the degree of heterogeneity among the sample size of studies reviewed; and,
• Justification of the techniques used to evaluate the studies.
• Can be an effective strategy for determining gaps in the literature.
• Provides a means of reviewing research published about a particular topic over an extended period of time and from a variety of sources.
• Is useful in clarifying what policy or programmatic actions can be justified on the basis of analyzing research results from multiple studies.
• Provides a method for overcoming small sample sizes in individual studies that previously may have had little relationship to each other.
• Can be used to generate new hypotheses or highlight research problems for future studies.
• Small violations in defining the criteria used for content analysis can lead to difficult to interpret and/or meaningless findings.
• A large sample size can yield reliable, but not necessarily valid, results.
• A lack of uniformity regarding, for example, the type of literature reviewed, how methods are applied, and how findings are measured within the sample of studies you are analyzing, can make the process of synthesis difficult to perform.
• Depending on the sample size, the process of reviewing and synthesizing multiple studies can be very time consuming.

Beck, Lewis W. "The Synoptic Method." The Journal of Philosophy 36 (1939): 337-345; Cooper, Harris, Larry V. Hedges, and Jeffrey C. Valentine, eds. The Handbook of Research Synthesis and Meta-Analysis . 2nd edition. New York: Russell Sage Foundation, 2009; Guzzo, Richard A., Susan E. Jackson and Raymond A. Katzell. “Meta-Analysis Analysis.” In Research in Organizational Behavior , Volume 9. (Greenwich, CT: JAI Press, 1987), pp 407-442; Lipsey, Mark W. and David B. Wilson. Practical Meta-Analysis . Thousand Oaks, CA: Sage Publications, 2001; Study Design 101. Meta-Analysis. The Himmelfarb Health Sciences Library, George Washington University; Timulak, Ladislav. “Qualitative Meta-Analysis.” In The SAGE Handbook of Qualitative Data Analysis . Uwe Flick, editor. (Los Angeles, CA: Sage, 2013), pp. 481-495; Walker, Esteban, Adrian V. Hernandez, and Micheal W. Kattan. "Meta-Analysis: It's Strengths and Limitations." Cleveland Clinic Journal of Medicine 75 (June 2008): 431-439.

Mixed-Method Design

• Narrative and non-textual information can add meaning to numeric data, while numeric data can add precision to narrative and non-textual information.
• Can utilize existing data while at the same time generating and testing a grounded theory approach to describe and explain the phenomenon under study.
• A broader, more complex research problem can be investigated because the researcher is not constrained by using only one method.
• The strengths of one method can be used to overcome the inherent weaknesses of another method.
• Can provide stronger, more robust evidence to support a conclusion or set of recommendations.
• May generate new knowledge new insights or uncover hidden insights, patterns, or relationships that a single methodological approach might not reveal.
• Produces more complete knowledge and understanding of the research problem that can be used to increase the generalizability of findings applied to theory or practice.
• A researcher must be proficient in understanding how to apply multiple methods to investigating a research problem as well as be proficient in optimizing how to design a study that coherently melds them together.
• Can increase the likelihood of conflicting results or ambiguous findings that inhibit drawing a valid conclusion or setting forth a recommended course of action [e.g., sample interview responses do not support existing statistical data].
• Because the research design can be very complex, reporting the findings requires a well-organized narrative, clear writing style, and precise word choice.
• Design invites collaboration among experts. However, merging different investigative approaches and writing styles requires more attention to the overall research process than studies conducted using only one methodological paradigm.
• Concurrent merging of quantitative and qualitative research requires greater attention to having adequate sample sizes, using comparable samples, and applying a consistent unit of analysis. For sequential designs where one phase of qualitative research builds on the quantitative phase or vice versa, decisions about what results from the first phase to use in the next phase, the choice of samples and estimating reasonable sample sizes for both phases, and the interpretation of results from both phases can be difficult.
• Due to multiple forms of data being collected and analyzed, this design requires extensive time and resources to carry out the multiple steps involved in data gathering and interpretation.

Burch, Patricia and Carolyn J. Heinrich. Mixed Methods for Policy Research and Program Evaluation . Thousand Oaks, CA: Sage, 2016; Creswell, John w. et al. Best Practices for Mixed Methods Research in the Health Sciences . Bethesda, MD: Office of Behavioral and Social Sciences Research, National Institutes of Health, 2010Creswell, John W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . 4th edition. Thousand Oaks, CA: Sage Publications, 2014; Domínguez, Silvia, editor. Mixed Methods Social Networks Research . Cambridge, UK: Cambridge University Press, 2014; Hesse-Biber, Sharlene Nagy. Mixed Methods Research: Merging Theory with Practice . New York: Guilford Press, 2010; Niglas, Katrin. “How the Novice Researcher Can Make Sense of Mixed Methods Designs.” International Journal of Multiple Research Approaches 3 (2009): 34-46; Onwuegbuzie, Anthony J. and Nancy L. Leech. “Linking Research Questions to Mixed Methods Data Analysis Procedures.” The Qualitative Report 11 (September 2006): 474-498; Tashakorri, Abbas and John W. Creswell. “The New Era of Mixed Methods.” Journal of Mixed Methods Research 1 (January 2007): 3-7; Zhanga, Wanqing. “Mixed Methods Application in Health Intervention Research: A Multiple Case Study.” International Journal of Multiple Research Approaches 8 (2014): 24-35 .

Observational Design

This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project.

• Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe [data is emergent rather than pre-existing].
• The researcher is able to collect in-depth information about a particular behavior.
• Can reveal interrelationships among multifaceted dimensions of group interactions.
• You can generalize your results to real life situations.
• Observational research is useful for discovering what variables may be important before applying other methods like experiments.
• Observation research designs account for the complexity of group behaviors.
• Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and are difficult to replicate.
• In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups.
• There can be problems with bias as the researcher may only "see what they want to see."
• There is no possibility to determine "cause and effect" relationships since nothing is manipulated.
• Sources or subjects may not all be equally credible.
• Any group that is knowingly studied is altered to some degree by the presence of the researcher, therefore, potentially skewing any data collected.

Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods . Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Payne, Geoff and Judy Payne. "Observation." In Key Concepts in Social Research . The SAGE Key Concepts series. (London, England: Sage, 2004), pp. 158-162; Rosenbaum, Paul R. Design of Observational Studies . New York: Springer, 2010;Williams, J. Patrick. "Nonparticipant Observation." In The Sage Encyclopedia of Qualitative Research Methods . Lisa M. Given, editor.(Thousand Oaks, CA: Sage, 2008), pp. 562-563.

Philosophical Design

Understood more as an broad approach to examining a research problem than a methodological design, philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable, assumptions underpinning an area of study. This approach uses the tools of argumentation derived from philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the relevance of logic and evidence in academic debates, to analyze arguments about fundamental issues, or to discuss the root of existing discourse about a research problem. These overarching tools of analysis can be framed in three ways:

• Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what is fundamental and what is derivative?
• Epistemology -- the study that explores the nature of knowledge; for example, by what means does knowledge and understanding depend upon and how can we be certain of what we know?
• Axiology -- the study of values; for example, what values does an individual or group hold and why? How are values related to interest, desire, will, experience, and means-to-end? And, what is the difference between a matter of fact and a matter of value?
• Can provide a basis for applying ethical decision-making to practice.
• Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of research.
• Brings clarity to general guiding practices and principles of an individual or group.
• Philosophy informs methodology.
• Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse.
• Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of reality (metaphysics).
• Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas.
• Limited application to specific research problems [answering the "So What?" question in social science research].
• Analysis can be abstract, argumentative, and limited in its practical application to real-life issues.
• While a philosophical analysis may render problematic that which was once simple or taken-for-granted, the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation and documentation.
• There are limitations in the use of metaphor as a vehicle of philosophical analysis.
• There can be analytical difficulties in moving from philosophy to advocacy and between abstract thought and application to the phenomenal world.

Burton, Dawn. "Part I, Philosophy of the Social Sciences." In Research Training for Social Scientists . (London, England: Sage, 2000), pp. 1-5; Chapter 4, Research Methodology and Design. Unisa Institutional Repository (UnisaIR), University of South Africa; Jarvie, Ian C., and Jesús Zamora-Bonilla, editors. The SAGE Handbook of the Philosophy of Social Sciences . London: Sage, 2011; Labaree, Robert V. and Ross Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008): 43-70; Maykut, Pamela S. Beginning Qualitative Research: A Philosophic and Practical Guide . Washington, DC: Falmer Press, 1994; McLaughlin, Hugh. "The Philosophy of Social Research." In Understanding Social Work Research . 2nd edition. (London: SAGE Publications Ltd., 2012), pp. 24-47; Stanford Encyclopedia of Philosophy . Metaphysics Research Lab, CSLI, Stanford University, 2013.

Sequential Design

• The researcher has a limitless option when it comes to sample size and the sampling schedule.
• Due to the repetitive nature of this research design, minor changes and adjustments can be done during the initial parts of the study to correct and hone the research method.
• This is a useful design for exploratory studies.
• There is very little effort on the part of the researcher when performing this technique. It is generally not expensive, time consuming, or workforce intensive.
• Because the study is conducted serially, the results of one sample are known before the next sample is taken and analyzed. This provides opportunities for continuous improvement of sampling and methods of analysis.
• The sampling method is not representative of the entire population. The only possibility of approaching representativeness is when the researcher chooses to use a very large sample size significant enough to represent a significant portion of the entire population. In this case, moving on to study a second or more specific sample can be difficult.
• The design cannot be used to create conclusions and interpretations that pertain to an entire population because the sampling technique is not randomized. Generalizability from findings is, therefore, limited.
• Difficult to account for and interpret variation from one sample to another over time, particularly when using qualitative methods of data collection.

Betensky, Rebecca. Harvard University, Course Lecture Note slides; Bovaird, James A. and Kevin A. Kupzyk. "Sequential Design." In Encyclopedia of Research Design . Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 1347-1352; Cresswell, John W. Et al. “Advanced Mixed-Methods Research Designs.” In Handbook of Mixed Methods in Social and Behavioral Research . Abbas Tashakkori and Charles Teddle, eds. (Thousand Oaks, CA: Sage, 2003), pp. 209-240; Henry, Gary T. "Sequential Sampling." In The SAGE Encyclopedia of Social Science Research Methods . Michael S. Lewis-Beck, Alan Bryman and Tim Futing Liao, editors. (Thousand Oaks, CA: Sage, 2004), pp. 1027-1028; Nataliya V. Ivankova. “Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice.” Field Methods 18 (February 2006): 3-20; Bovaird, James A. and Kevin A. Kupzyk. “Sequential Design.” In Encyclopedia of Research Design . Neil J. Salkind, ed. Thousand Oaks, CA: Sage, 2010; Sequential Analysis. Wikipedia.

Systematic Review

• A systematic review synthesizes the findings of multiple studies related to each other by incorporating strategies of analysis and interpretation intended to reduce biases and random errors.
• The application of critical exploration, evaluation, and synthesis methods separates insignificant, unsound, or redundant research from the most salient and relevant studies worthy of reflection.
• They can be use to identify, justify, and refine hypotheses, recognize and avoid hidden problems in prior studies, and explain data inconsistencies and conflicts in data.
• Systematic reviews can be used to help policy makers formulate evidence-based guidelines and regulations.
• The use of strict, explicit, and pre-determined methods of synthesis, when applied appropriately, provide reliable estimates about the effects of interventions, evaluations, and effects related to the overarching research problem investigated by each study under review.
• Systematic reviews illuminate where knowledge or thorough understanding of a research problem is lacking and, therefore, can then be used to guide future research.
• The accepted inclusion of unpublished studies [i.e., grey literature] ensures the broadest possible way to analyze and interpret research on a topic.
• Results of the synthesis can be generalized and the findings extrapolated into the general population with more validity than most other types of studies .
• Systematic reviews do not create new knowledge per se; they are a method for synthesizing existing studies about a research problem in order to gain new insights and determine gaps in the literature.
• The way researchers have carried out their investigations [e.g., the period of time covered, number of participants, sources of data analyzed, etc.] can make it difficult to effectively synthesize studies.
• The inclusion of unpublished studies can introduce bias into the review because they may not have undergone a rigorous peer-review process prior to publication. Examples may include conference presentations or proceedings, publications from government agencies, white papers, working papers, and internal documents from organizations, and doctoral dissertations and Master's theses.

Denyer, David and David Tranfield. "Producing a Systematic Review." In The Sage Handbook of Organizational Research Methods .  David A. Buchanan and Alan Bryman, editors. ( Thousand Oaks, CA: Sage Publications, 2009), pp. 671-689; Foster, Margaret J. and Sarah T. Jewell, editors. Assembling the Pieces of a Systematic Review: A Guide for Librarians . Lanham, MD: Rowman and Littlefield, 2017; Gough, David, Sandy Oliver, James Thomas, editors. Introduction to Systematic Reviews . 2nd edition. Los Angeles, CA: Sage Publications, 2017; Gopalakrishnan, S. and P. Ganeshkumar. “Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare.” Journal of Family Medicine and Primary Care 2 (2013): 9-14; Gough, David, James Thomas, and Sandy Oliver. "Clarifying Differences between Review Designs and Methods." Systematic Reviews 1 (2012): 1-9; Khan, Khalid S., Regina Kunz, Jos Kleijnen, and Gerd Antes. “Five Steps to Conducting a Systematic Review.” Journal of the Royal Society of Medicine 96 (2003): 118-121; Mulrow, C. D. “Systematic Reviews: Rationale for Systematic Reviews.” BMJ 309:597 (September 1994); O'Dwyer, Linda C., and Q. Eileen Wafford. "Addressing Challenges with Systematic Review Teams through Effective Communication: A Case Report." Journal of the Medical Library Association 109 (October 2021): 643-647; Okoli, Chitu, and Kira Schabram. "A Guide to Conducting a Systematic Literature Review of Information Systems Research."  Sprouts: Working Papers on Information Systems 10 (2010); Siddaway, Andy P., Alex M. Wood, and Larry V. Hedges. "How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-analyses, and Meta-syntheses." Annual Review of Psychology 70 (2019): 747-770; Torgerson, Carole J. “Publication Bias: The Achilles’ Heel of Systematic Reviews?” British Journal of Educational Studies 54 (March 2006): 89-102; Torgerson, Carole. Systematic Reviews . New York: Continuum, 2003.

• << Previous: Purpose of Guide
• Next: Design Flaws to Avoid >>
• Last Updated: Apr 5, 2024 1:38 PM
• URL: https://libguides.usc.edu/writingguide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology

Research Design | Step-by-Step Guide with Examples

Published on 5 May 2022 by Shona McCombes . Revised on 20 March 2023.

A research design is a strategy for answering your research question  using empirical data. Creating a research design means making decisions about:

• Your overall aims and approach
• The type of research design you’ll use
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research aims and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, frequently asked questions.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities – start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Prevent plagiarism, run a free check.

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types. Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships, while descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends, and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analysing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study – plants, animals, organisations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region, or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalise your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study, your aim is to deeply understand a specific context, not to generalise to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question.

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviours, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews.

Observation methods

Observations allow you to collect data unobtrusively, observing characteristics, behaviours, or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected – for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are reliable and valid.

Operationalisation

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalisation means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in – for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced , while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method, you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample – by mail, online, by phone, or in person?

If you’re using a probability sampling method, it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method, how will you avoid bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organising and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymise and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well organised will save time when it comes to analysing them. It can also help other researchers validate and add to your findings.

On their own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyse the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarise your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarise your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analysing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

Operationalisation means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioural avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalise the variables that you want to measure.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, March 20). Research Design | Step-by-Step Guide with Examples. Scribbr. Retrieved 2 April 2024, from https://www.scribbr.co.uk/research-methods/research-design/

Is this article helpful?

Shona McCombes

• How it works

How to Write a Research Design – Guide with Examples

Published by Alaxendra Bets at August 14th, 2021 , Revised On October 3, 2023

A research design is a structure that combines different components of research. It involves the use of different data collection and data analysis techniques logically to answer the  research questions .

It would be best to make some decisions about addressing the research questions adequately before starting the research process, which is achieved with the help of the research design.

Below are the key aspects of the decision-making process:

• Data type required for research
• Research resources
• Participants required for research
• Hypothesis based upon research question(s)
• Data analysis  methodologies
• Variables (Independent, dependent, and confounding)
• The location and timescale for conducting the data
• The time period required for research

The research design provides the strategy of investigation for your project. Furthermore, it defines the parameters and criteria to compile the data to evaluate results and conclude.

Your project’s validity depends on the data collection and  interpretation techniques.  A strong research design reflects a strong  dissertation , scientific paper, or research proposal .

Step 1: Establish Priorities for Research Design

Before conducting any research study, you must address an important question: “how to create a research design.”

The research design depends on the researcher’s priorities and choices because every research has different priorities. For a complex research study involving multiple methods, you may choose to have more than one research design.

Multimethodology or multimethod research includes using more than one data collection method or research in a research study or set of related studies.

If one research design is weak in one area, then another research design can cover that weakness. For instance, a  dissertation analyzing different situations or cases will have more than one research design.

For example:

• Experimental research involves experimental investigation and laboratory experience, but it does not accurately investigate the real world.
• Quantitative research is good for the  statistical part of the project, but it may not provide an in-depth understanding of the  topic .
• Also, correlational research will not provide experimental results because it is a technique that assesses the statistical relationship between two variables.

While scientific considerations are a fundamental aspect of the research design, It is equally important that the researcher think practically before deciding on its structure. Here are some questions that you should think of;

• Do you have enough time to gather data and complete the write-up?
• Will you be able to collect the necessary data by interviewing a specific person or visiting a specific location?
• Do you have in-depth knowledge about the  different statistical analysis and data collection techniques to address the research questions  or test the  hypothesis ?

If you think that the chosen research design cannot answer the research questions properly, you can refine your research questions to gain better insight.

Step 2: Data Type you Need for Research

Decide on the type of data you need for your research. The type of data you need to collect depends on your research questions or research hypothesis. Two types of research data can be used to answer the research questions:

Primary Data Vs. Secondary Data

Qualitative vs. quantitative data.

Also, see; Research methods, design, and analysis .

Need help with a thesis chapter?

• Hire an expert from ResearchProspect today!
• Statistical analysis, research methodology, discussion of the results or conclusion – our experts can help you no matter how complex the requirements are.

Step 3: Data Collection Techniques

Once you have selected the type of research to answer your research question, you need to decide where and how to collect the data.

It is time to determine your research method to address the  research problem . Research methods involve procedures, techniques, materials, and tools used for the study.

For instance, a dissertation research design includes the different resources and data collection techniques and helps establish your  dissertation’s structure .

The following table shows the characteristics of the most popularly employed research methods.

Research Methods

Step 4: Procedure of Data Analysis

Use of the  correct data and statistical analysis technique is necessary for the validity of your research. Therefore, you need to be certain about the data type that would best address the research problem. Choosing an appropriate analysis method is the final step for the research design. It can be split into two main categories;

Quantitative Data Analysis

The quantitative data analysis technique involves analyzing the numerical data with the help of different applications such as; SPSS, STATA, Excel, origin lab, etc.

This data analysis strategy tests different variables such as spectrum, frequencies, averages, and more. The research question and the hypothesis must be established to identify the variables for testing.

Qualitative Data Analysis

Qualitative data analysis of figures, themes, and words allows for flexibility and the researcher’s subjective opinions. This means that the researcher’s primary focus will be interpreting patterns, tendencies, and accounts and understanding the implications and social framework.

You should be clear about your research objectives before starting to analyze the data. For example, you should ask yourself whether you need to explain respondents’ experiences and insights or do you also need to evaluate their responses with reference to a certain social framework.

Step 5: Write your Research Proposal

The research design is an important component of a research proposal because it plans the project’s execution. You can share it with the supervisor, who would evaluate the feasibility and capacity of the results  and  conclusion .

Read our guidelines to write a research proposal  if you have already formulated your research design. The research proposal is written in the future tense because you are writing your proposal before conducting research.

The  research methodology  or research design, on the other hand, is generally written in the past tense.

How to Write a Research Design – Conclusion

A research design is the plan, structure, strategy of investigation conceived to answer the research question and test the hypothesis. The dissertation research design can be classified based on the type of data and the type of analysis.

Above mentioned five steps are the answer to how to write a research design. So, follow these steps to  formulate the perfect research design for your dissertation .

ResearchProspect writers have years of experience creating research designs that align with the dissertation’s aim and objectives. If you are struggling with your dissertation methodology chapter, you might want to look at our dissertation part-writing service.

Our dissertation writers can also help you with the full dissertation paper . No matter how urgent or complex your need may be, ResearchProspect can help. We also offer PhD level research paper writing services.

Frequently Asked Questions

What is research design.

Research design is a systematic plan that guides the research process, outlining the methodology and procedures for collecting and analysing data. It determines the structure of the study, ensuring the research question is answered effectively, reliably, and validly. It serves as the blueprint for the entire research project.

How to write a research design?

To write a research design, define your research question, identify the research method (qualitative, quantitative, or mixed), choose data collection techniques (e.g., surveys, interviews), determine the sample size and sampling method, outline data analysis procedures, and highlight potential limitations and ethical considerations for the study.

How to write the design section of a research paper?

In the design section of a research paper, describe the research methodology chosen and justify its selection. Outline the data collection methods, participants or samples, instruments used, and procedures followed. Detail any experimental controls, if applicable. Ensure clarity and precision to enable replication of the study by other researchers.

How to write a research design in methodology?

To write a research design in methodology, clearly outline the research strategy (e.g., experimental, survey, case study). Describe the sampling technique, participants, and data collection methods. Detail the procedures for data collection and analysis. Justify choices by linking them to research objectives, addressing reliability and validity.

You May Also Like

Find how to write research questions with the mentioned steps required for a perfect research question. Choose an interesting topic and begin your research.

How to write a hypothesis for dissertation,? A hypothesis is a statement that can be tested with the help of experimental or theoretical research.

Not sure how to approach a company for your primary research study? Don’t worry. Here we have some tips for you to successfully gather primary study.

USEFUL LINKS

LEARNING RESOURCES

COMPANY DETAILS

• How It Works

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• How to Write Your Methods

Ensure understanding, reproducibility and replicability

What should you include in your methods section, and how much detail is appropriate?

Why Methods Matter

The methods section was once the most likely part of a paper to be unfairly abbreviated, overly summarized, or even relegated to hard-to-find sections of a publisher’s website. While some journals may responsibly include more detailed elements of methods in supplementary sections, the movement for increased reproducibility and rigor in science has reinstated the importance of the methods section. Methods are now viewed as a key element in establishing the credibility of the research being reported, alongside the open availability of data and results.

A clear methods section impacts editorial evaluation and readers’ understanding, and is also the backbone of transparency and replicability.

For example, the Reproducibility Project: Cancer Biology project set out in 2013 to replicate experiments from 50 high profile cancer papers, but revised their target to 18 papers once they understood how much methodological detail was not contained in the original papers.

What to include in your methods section

What you include in your methods sections depends on what field you are in and what experiments you are performing. However, the general principle in place at the majority of journals is summarized well by the guidelines at PLOS ONE : “The Materials and Methods section should provide enough detail to allow suitably skilled investigators to fully replicate your study. ” The emphases here are deliberate: the methods should enable readers to understand your paper, and replicate your study. However, there is no need to go into the level of detail that a lay-person would require—the focus is on the reader who is also trained in your field, with the suitable skills and knowledge to attempt a replication.

A constant principle of rigorous science

A methods section that enables other researchers to understand and replicate your results is a constant principle of rigorous, transparent, and Open Science. Aim to be thorough, even if a particular journal doesn’t require the same level of detail . Reproducibility is all of our responsibility. You cannot create any problems by exceeding a minimum standard of information. If a journal still has word-limits—either for the overall article or specific sections—and requires some methodological details to be in a supplemental section, that is OK as long as the extra details are searchable and findable .

Imagine replicating your own work, years in the future

As part of PLOS’ presentation on Reproducibility and Open Publishing (part of UCSF’s Reproducibility Series ) we recommend planning the level of detail in your methods section by imagining you are writing for your future self, replicating your own work. When you consider that you might be at a different institution, with different account logins, applications, resources, and access levels—you can help yourself imagine the level of specificity that you yourself would require to redo the exact experiment. Consider:

• Which details would you need to be reminded of? 
• Which cell line, or antibody, or software, or reagent did you use, and does it have a Research Resource ID (RRID) that you can cite?
• Which version of a questionnaire did you use in your survey? 
• Exactly which visual stimulus did you show participants, and is it publicly available? 
• What participants did you decide to exclude? 
• What process did you adjust, during your work? 

Tip: Be sure to capture any changes to your protocols

You yourself would want to know about any adjustments, if you ever replicate the work, so you can surmise that anyone else would want to as well. Even if a necessary adjustment you made was not ideal, transparency is the key to ensuring this is not regarded as an issue in the future. It is far better to transparently convey any non-optimal methods, or methodological constraints, than to conceal them, which could result in reproducibility or ethical issues downstream.

Visual aids for methods help when reading the whole paper

Consider whether a visual representation of your methods could be appropriate or aid understanding your process. A visual reference readers can easily return to, like a flow-diagram, decision-tree, or checklist, can help readers to better understand the complete article, not just the methods section.

Ethical Considerations

In addition to describing what you did, it is just as important to assure readers that you also followed all relevant ethical guidelines when conducting your research. While ethical standards and reporting guidelines are often presented in a separate section of a paper, ensure that your methods and protocols actually follow these guidelines. Read more about ethics .

Existing standards, checklists, guidelines, partners

While the level of detail contained in a methods section should be guided by the universal principles of rigorous science outlined above, various disciplines, fields, and projects have worked hard to design and develop consistent standards, guidelines, and tools to help with reporting all types of experiment. Below, you’ll find some of the key initiatives. Ensure you read the submission guidelines for the specific journal you are submitting to, in order to discover any further journal- or field-specific policies to follow, or initiatives/tools to utilize.

Tip: Keep your paper moving forward by providing the proper paperwork up front

Be sure to check the journal guidelines and provide the necessary documents with your manuscript submission. Collecting the necessary documentation can greatly slow the first round of peer review, or cause delays when you submit your revision.

Randomized Controlled Trials – CONSORT The Consolidated Standards of Reporting Trials (CONSORT) project covers various initiatives intended to prevent the problems of  inadequate reporting of randomized controlled trials. The primary initiative is an evidence-based minimum set of recommendations for reporting randomized trials known as the CONSORT Statement . 

Systematic Reviews and Meta-Analyses – PRISMA The Preferred Reporting Items for Systematic Reviews and Meta-Analyses ( PRISMA ) is an evidence-based minimum set of items focusing  on the reporting of  reviews evaluating randomized trials and other types of research.

Research using Animals – ARRIVE The Animal Research: Reporting of In Vivo Experiments ( ARRIVE ) guidelines encourage maximizing the information reported in research using animals thereby minimizing unnecessary studies. (Original study and proposal , and updated guidelines , in PLOS Biology .) 

Laboratory Protocols Protocols.io has developed a platform specifically for the sharing and updating of laboratory protocols , which are assigned their own DOI and can be linked from methods sections of papers to enhance reproducibility. Contextualize your protocol and improve discovery with an accompanying Lab Protocol article in PLOS ONE .

Consistent reporting of Materials, Design, and Analysis – the MDAR checklist A cross-publisher group of editors and experts have developed, tested, and rolled out a checklist to help establish and harmonize reporting standards in the Life Sciences . The checklist , which is available for use by authors to compile their methods, and editors/reviewers to check methods, establishes a minimum set of requirements in transparent reporting and is adaptable to any discipline within the Life Sciences, by covering a breadth of potentially relevant methodological items and considerations. If you are in the Life Sciences and writing up your methods section, try working through the MDAR checklist and see whether it helps you include all relevant details into your methods, and whether it reminded you of anything you might have missed otherwise.

Summary Writing tips

The main challenge you may find when writing your methods is keeping it readable AND covering all the details needed for reproducibility and replicability. While this is difficult, do not compromise on rigorous standards for credibility!

• Keep in mind future replicability, alongside understanding and readability.
• Follow checklists, and field- and journal-specific guidelines.
• Consider a commitment to rigorous and transparent science a personal responsibility, and not just adhering to journal guidelines.
• Establish whether there are persistent identifiers for any research resources you use that can be specifically cited in your methods section.
• Deposit your laboratory protocols in Protocols.io, establishing a permanent link to them. You can update your protocols later if you improve on them, as can future scientists who follow your protocols.
• Consider visual aids like flow-diagrams, lists, to help with reading other sections of the paper.
• Be specific about all decisions made during the experiments that someone reproducing your work would need to know.

Don’t

• Summarize or abbreviate methods without giving full details in a discoverable supplemental section.
• Presume you will always be able to remember how you performed the experiments, or have access to private or institutional notebooks and resources.
• Attempt to hide constraints or non-optimal decisions you had to make–transparency is the key to ensuring the credibility of your research.
• How to Write a Great Title
• How to Write an Abstract
• How to Report Statistics
• How to Write Discussions and Conclusions
• How to Edit Your Work

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

• University Libraries
• Research Guides
• Topic Guides
• Research Methods Guide
• Research Design & Method

Research Methods Guide: Research Design & Method

• Introduction
• Survey Research
• Interview Research
• Data Analysis
• Resources & Consultation

Tutorial Videos: Research Design & Method

Research Methods (sociology-focused)

Qualitative vs. Quantitative Methods (intro)

Qualitative vs. Quantitative Methods (advanced)

FAQ: Research Design & Method

What is the difference between Research Design and Research Method?

Research design is a plan to answer your research question.  A research method is a strategy used to implement that plan.  Research design and methods are different but closely related, because good research design ensures that the data you obtain will help you answer your research question more effectively.

Which research method should I choose ?

It depends on your research goal.  It depends on what subjects (and who) you want to study.  Let's say you are interested in studying what makes people happy, or why some students are more conscious about recycling on campus.  To answer these questions, you need to make a decision about how to collect your data.  Most frequently used methods include:

• Observation / Participant Observation
• Focus Groups
• Experiments
• Secondary Data Analysis / Archival Study
• Mixed Methods (combination of some of the above)

One particular method could be better suited to your research goal than others, because the data you collect from different methods will be different in quality and quantity.   For instance, surveys are usually designed to produce relatively short answers, rather than the extensive responses expected in qualitative interviews.

What other factors should I consider when choosing one method over another?

Time for data collection and analysis is something you want to consider.  An observation or interview method, so-called qualitative approach, helps you collect richer information, but it takes time.  Using a survey helps you collect more data quickly, yet it may lack details.  So, you will need to consider the time you have for research and the balance between strengths and weaknesses associated with each method (e.g., qualitative vs. quantitative).

• << Previous: Introduction
• Next: Survey Research >>
• Last Updated: Aug 21, 2023 10:42 AM

• Privacy Policy

Buy Me a Coffee

Home » Research Methodology – Types, Examples and writing Guide

Research Methodology – Types, Examples and writing Guide

Table of Contents

Research Methodology

Definition:

Research Methodology refers to the systematic and scientific approach used to conduct research, investigate problems, and gather data and information for a specific purpose. It involves the techniques and procedures used to identify, collect , analyze , and interpret data to answer research questions or solve research problems . Moreover, They are philosophical and theoretical frameworks that guide the research process.

Structure of Research Methodology

Research methodology formats can vary depending on the specific requirements of the research project, but the following is a basic example of a structure for a research methodology section:

I. Introduction

• Provide an overview of the research problem and the need for a research methodology section
• Outline the main research questions and objectives

II. Research Design

• Explain the research design chosen and why it is appropriate for the research question(s) and objectives
• Discuss any alternative research designs considered and why they were not chosen
• Describe the research setting and participants (if applicable)

III. Data Collection Methods

• Describe the methods used to collect data (e.g., surveys, interviews, observations)
• Explain how the data collection methods were chosen and why they are appropriate for the research question(s) and objectives
• Detail any procedures or instruments used for data collection

IV. Data Analysis Methods

• Describe the methods used to analyze the data (e.g., statistical analysis, content analysis )
• Explain how the data analysis methods were chosen and why they are appropriate for the research question(s) and objectives
• Detail any procedures or software used for data analysis

V. Ethical Considerations

• Discuss any ethical issues that may arise from the research and how they were addressed
• Explain how informed consent was obtained (if applicable)
• Detail any measures taken to ensure confidentiality and anonymity

VI. Limitations

• Identify any potential limitations of the research methodology and how they may impact the results and conclusions

VII. Conclusion

• Summarize the key aspects of the research methodology section
• Explain how the research methodology addresses the research question(s) and objectives

Research Methodology Types

Types of Research Methodology are as follows:

Quantitative Research Methodology

This is a research methodology that involves the collection and analysis of numerical data using statistical methods. This type of research is often used to study cause-and-effect relationships and to make predictions.

Qualitative Research Methodology

This is a research methodology that involves the collection and analysis of non-numerical data such as words, images, and observations. This type of research is often used to explore complex phenomena, to gain an in-depth understanding of a particular topic, and to generate hypotheses.

Mixed-Methods Research Methodology

This is a research methodology that combines elements of both quantitative and qualitative research. This approach can be particularly useful for studies that aim to explore complex phenomena and to provide a more comprehensive understanding of a particular topic.

Case Study Research Methodology

This is a research methodology that involves in-depth examination of a single case or a small number of cases. Case studies are often used in psychology, sociology, and anthropology to gain a detailed understanding of a particular individual or group.

Action Research Methodology

This is a research methodology that involves a collaborative process between researchers and practitioners to identify and solve real-world problems. Action research is often used in education, healthcare, and social work.

Experimental Research Methodology

This is a research methodology that involves the manipulation of one or more independent variables to observe their effects on a dependent variable. Experimental research is often used to study cause-and-effect relationships and to make predictions.

Survey Research Methodology

This is a research methodology that involves the collection of data from a sample of individuals using questionnaires or interviews. Survey research is often used to study attitudes, opinions, and behaviors.

Grounded Theory Research Methodology

This is a research methodology that involves the development of theories based on the data collected during the research process. Grounded theory is often used in sociology and anthropology to generate theories about social phenomena.

Research Methodology Example

An Example of Research Methodology could be the following:

Research Methodology for Investigating the Effectiveness of Cognitive Behavioral Therapy in Reducing Symptoms of Depression in Adults

Introduction:

The aim of this research is to investigate the effectiveness of cognitive-behavioral therapy (CBT) in reducing symptoms of depression in adults. To achieve this objective, a randomized controlled trial (RCT) will be conducted using a mixed-methods approach.

Research Design:

The study will follow a pre-test and post-test design with two groups: an experimental group receiving CBT and a control group receiving no intervention. The study will also include a qualitative component, in which semi-structured interviews will be conducted with a subset of participants to explore their experiences of receiving CBT.

Participants:

Participants will be recruited from community mental health clinics in the local area. The sample will consist of 100 adults aged 18-65 years old who meet the diagnostic criteria for major depressive disorder. Participants will be randomly assigned to either the experimental group or the control group.

Intervention :

The experimental group will receive 12 weekly sessions of CBT, each lasting 60 minutes. The intervention will be delivered by licensed mental health professionals who have been trained in CBT. The control group will receive no intervention during the study period.

Data Collection:

Quantitative data will be collected through the use of standardized measures such as the Beck Depression Inventory-II (BDI-II) and the Generalized Anxiety Disorder-7 (GAD-7). Data will be collected at baseline, immediately after the intervention, and at a 3-month follow-up. Qualitative data will be collected through semi-structured interviews with a subset of participants from the experimental group. The interviews will be conducted at the end of the intervention period, and will explore participants’ experiences of receiving CBT.

Data Analysis:

Quantitative data will be analyzed using descriptive statistics, t-tests, and mixed-model analyses of variance (ANOVA) to assess the effectiveness of the intervention. Qualitative data will be analyzed using thematic analysis to identify common themes and patterns in participants’ experiences of receiving CBT.

Ethical Considerations:

This study will comply with ethical guidelines for research involving human subjects. Participants will provide informed consent before participating in the study, and their privacy and confidentiality will be protected throughout the study. Any adverse events or reactions will be reported and managed appropriately.

Data Management:

All data collected will be kept confidential and stored securely using password-protected databases. Identifying information will be removed from qualitative data transcripts to ensure participants’ anonymity.

Limitations:

One potential limitation of this study is that it only focuses on one type of psychotherapy, CBT, and may not generalize to other types of therapy or interventions. Another limitation is that the study will only include participants from community mental health clinics, which may not be representative of the general population.

Conclusion:

This research aims to investigate the effectiveness of CBT in reducing symptoms of depression in adults. By using a randomized controlled trial and a mixed-methods approach, the study will provide valuable insights into the mechanisms underlying the relationship between CBT and depression. The results of this study will have important implications for the development of effective treatments for depression in clinical settings.

How to Write Research Methodology

Writing a research methodology involves explaining the methods and techniques you used to conduct research, collect data, and analyze results. It’s an essential section of any research paper or thesis, as it helps readers understand the validity and reliability of your findings. Here are the steps to write a research methodology:

• Start by explaining your research question: Begin the methodology section by restating your research question and explaining why it’s important. This helps readers understand the purpose of your research and the rationale behind your methods.
• Describe your research design: Explain the overall approach you used to conduct research. This could be a qualitative or quantitative research design, experimental or non-experimental, case study or survey, etc. Discuss the advantages and limitations of the chosen design.
• Discuss your sample: Describe the participants or subjects you included in your study. Include details such as their demographics, sampling method, sample size, and any exclusion criteria used.
• Describe your data collection methods : Explain how you collected data from your participants. This could include surveys, interviews, observations, questionnaires, or experiments. Include details on how you obtained informed consent, how you administered the tools, and how you minimized the risk of bias.
• Explain your data analysis techniques: Describe the methods you used to analyze the data you collected. This could include statistical analysis, content analysis, thematic analysis, or discourse analysis. Explain how you dealt with missing data, outliers, and any other issues that arose during the analysis.
• Discuss the validity and reliability of your research : Explain how you ensured the validity and reliability of your study. This could include measures such as triangulation, member checking, peer review, or inter-coder reliability.
• Acknowledge any limitations of your research: Discuss any limitations of your study, including any potential threats to validity or generalizability. This helps readers understand the scope of your findings and how they might apply to other contexts.
• Provide a summary: End the methodology section by summarizing the methods and techniques you used to conduct your research. This provides a clear overview of your research methodology and helps readers understand the process you followed to arrive at your findings.

When to Write Research Methodology

Research methodology is typically written after the research proposal has been approved and before the actual research is conducted. It should be written prior to data collection and analysis, as it provides a clear roadmap for the research project.

The research methodology is an important section of any research paper or thesis, as it describes the methods and procedures that will be used to conduct the research. It should include details about the research design, data collection methods, data analysis techniques, and any ethical considerations.

The methodology should be written in a clear and concise manner, and it should be based on established research practices and standards. It is important to provide enough detail so that the reader can understand how the research was conducted and evaluate the validity of the results.

Applications of Research Methodology

Here are some of the applications of research methodology:

• To identify the research problem: Research methodology is used to identify the research problem, which is the first step in conducting any research.
• To design the research: Research methodology helps in designing the research by selecting the appropriate research method, research design, and sampling technique.
• To collect data: Research methodology provides a systematic approach to collect data from primary and secondary sources.
• To analyze data: Research methodology helps in analyzing the collected data using various statistical and non-statistical techniques.
• To test hypotheses: Research methodology provides a framework for testing hypotheses and drawing conclusions based on the analysis of data.
• To generalize findings: Research methodology helps in generalizing the findings of the research to the target population.
• To develop theories : Research methodology is used to develop new theories and modify existing theories based on the findings of the research.
• To evaluate programs and policies : Research methodology is used to evaluate the effectiveness of programs and policies by collecting data and analyzing it.
• To improve decision-making: Research methodology helps in making informed decisions by providing reliable and valid data.

Purpose of Research Methodology

Research methodology serves several important purposes, including:

• To guide the research process: Research methodology provides a systematic framework for conducting research. It helps researchers to plan their research, define their research questions, and select appropriate methods and techniques for collecting and analyzing data.
• To ensure research quality: Research methodology helps researchers to ensure that their research is rigorous, reliable, and valid. It provides guidelines for minimizing bias and error in data collection and analysis, and for ensuring that research findings are accurate and trustworthy.
• To replicate research: Research methodology provides a clear and detailed account of the research process, making it possible for other researchers to replicate the study and verify its findings.
• To advance knowledge: Research methodology enables researchers to generate new knowledge and to contribute to the body of knowledge in their field. It provides a means for testing hypotheses, exploring new ideas, and discovering new insights.
• To inform decision-making: Research methodology provides evidence-based information that can inform policy and decision-making in a variety of fields, including medicine, public health, education, and business.

Advantages of Research Methodology

Research methodology has several advantages that make it a valuable tool for conducting research in various fields. Here are some of the key advantages of research methodology:

• Systematic and structured approach : Research methodology provides a systematic and structured approach to conducting research, which ensures that the research is conducted in a rigorous and comprehensive manner.
• Objectivity : Research methodology aims to ensure objectivity in the research process, which means that the research findings are based on evidence and not influenced by personal bias or subjective opinions.
• Replicability : Research methodology ensures that research can be replicated by other researchers, which is essential for validating research findings and ensuring their accuracy.
• Reliability : Research methodology aims to ensure that the research findings are reliable, which means that they are consistent and can be depended upon.
• Validity : Research methodology ensures that the research findings are valid, which means that they accurately reflect the research question or hypothesis being tested.
• Efficiency : Research methodology provides a structured and efficient way of conducting research, which helps to save time and resources.
• Flexibility : Research methodology allows researchers to choose the most appropriate research methods and techniques based on the research question, data availability, and other relevant factors.
• Scope for innovation: Research methodology provides scope for innovation and creativity in designing research studies and developing new research techniques.

Research Methodology Vs Research Methods

About the author.

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Organizing Academic Research Papers: Types of Research Designs

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

Introduction

Before beginning your paper, you need to decide how you plan to design the study .

The research design refers to the overall strategy that you choose to integrate the different components of the study in a coherent and logical way, thereby, ensuring you will effectively address the research problem; it constitutes the blueprint for the collection, measurement, and analysis of data. Note that your research problem determines the type of design you can use, not the other way around!

General Structure and Writing Style

Action research design, case study design, causal design, cohort design, cross-sectional design, descriptive design, experimental design, exploratory design, historical design, longitudinal design, observational design, philosophical design, sequential design.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New York University, Spring 2006; Trochim, William M.K. Research Methods Knowledge Base . 2006.

The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem as unambiguously as possible. In social sciences research, obtaining evidence relevant to the research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a program, or to accurately describe a phenomenon. However, researchers can often begin their investigations far too early, before they have thought critically about about what information is required to answer the study's research questions. Without attending to these design issues beforehand, the conclusions drawn risk being weak and unconvincing and, consequently, will fail to adequate address the overall research problem.

 Given this, the length and complexity of research designs can vary considerably, but any sound design will do the following things:

• Identify the research problem clearly and justify its selection,
• Review previously published literature associated with the problem area,
• Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem selected,
• Effectively describe the data which will be necessary for an adequate test of the hypotheses and explain how such data will be obtained, and
• Describe the methods of analysis which will be applied to the data in determining whether or not the hypotheses are true or false.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New Yortk University, Spring 2006.

Definition and Purpose

The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. Then the intervention is carried out (the action in Action Research) during which time, pertinent observations are collected in various forms. The new interventional strategies are carried out, and the cyclic process repeats, continuing until a sufficient understanding of (or implement able solution for) the problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper understanding of a given situation, starting with conceptualizing and particularizing the problem and moving through several interventions and evaluations.

What do these studies tell you?

• A collaborative and adaptive research design that lends itself to use in work or community situations.
• Design focuses on pragmatic and solution-driven research rather than testing theories.
• When practitioners use action research it has the potential to increase the amount they learn consciously from their experience. The action research cycle can also be regarded as a learning cycle.
• Action search studies often have direct and obvious relevance to practice.
• There are no hidden controls or preemption of direction by the researcher.

What these studies don't tell you?

• It is harder to do than conducting conventional studies because the researcher takes on responsibilities for encouraging change as well as for research.
• Action research is much harder to write up because you probably can’t use a standard format to report your findings effectively.
• Personal over-involvement of the researcher may bias research results.
• The cyclic nature of action research to achieve its twin outcomes of action (e.g. change) and research (e.g. understanding) is time-consuming and complex to conduct.

Gall, Meredith. Educational Research: An Introduction . Chapter 18, Action Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Kemmis, Stephen and Robin McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research . Norman Denzin and Yvonna S. Locoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605.; Reason, Peter and Hilary Bradbury. Handbook of Action Research: Participative Inquiry and Practice . Thousand Oaks, CA: SAGE, 2001.

A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey. It is often used to narrow down a very broad field of research into one or a few easily researchable examples. The case study research design is also useful for testing whether a specific theory and model actually applies to phenomena in the real world. It is a useful design when not much is known about a phenomenon.

• Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
• A researcher using a case study design can apply a vaiety of methodologies and rely on a variety of sources to investigate a research problem.
• Design can extend experience or add strength to what is already known through previous research.
• Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and extension of methods.
• The design can provide detailed descriptions of specific and rare cases.
• A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things.
• The intense exposure to study of the case may bias a researcher's interpretation of the findings.
• Design does not facilitate assessment of cause and effect relationships.
• Vital information may be missing, making the case hard to interpret.
• The case may not be representative or typical of the larger problem being investigated.
• If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your intepretation of the findings can only apply to that particular case.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 4, Flexible Methods: Case Study Design. 2nd ed. New York: Columbia University Press, 1999; Stake, Robert E. The Art of Case Study Research . Thousand Oaks, CA: SAGE, 1995; Yin, Robert K. Case Study Research: Design and Theory . Applied Social Research Methods Series, no. 5. 3rd ed. Thousand Oaks, CA: SAGE, 2003.

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

• Empirical association--a valid conclusion is based on finding an association between the independent variable and the dependent variable.
• Appropriate time order--to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable.
• Nonspuriousness--a relationship between two variables that is not due to variation in a third variable.
• Causality research designs helps researchers understand why the world works the way it does through the process of proving a causal link between variables and eliminating other possibilities.
• Replication is possible.
• There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared.
• Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent].
• Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven.
• If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and therefore to establish which variable is the actual cause and which is the  actual effect.

Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice . Chapter 5, Causation and Research Designs. 3rd ed.  Thousand Oaks, CA: Pine Forge Press, 2007; Causal Research Design: Experimentation. Anonymous SlideShare Presentation ; Gall, Meredith. Educational Research: An Introduction . Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base . 2006.

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, r ather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed."

• Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof.
• Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease).
• The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors  often relies on cohort designs.
• Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect.
• Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.].
• Either original data or secondary data can be used in this design.
• In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables.
• Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
• Because of the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants.

Healy P, Devane D. “Methodological Considerations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36;  Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Study Design 101 . Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study . Wikipedia.

Cross-sectional research designs have three distinctive features: no time dimension, a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure diffrerences between or from among a variety of people, subjects, or phenomena rather than change. As such, researchers using this design can only employ a relative passive approach to making causal inferences based on findings.

• Cross-sectional studies provide a 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time.
• Unlike the experimental design where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena.
• Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
• Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling.
• Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound.
• Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population.
• Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
• Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult.
• Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical contexts.
• Studies cannot be utilized to establish cause and effect relationships.
• Provide only a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen.
• There is no follow up to the findings.

Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods. Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design, Application, Strengths and Weaknesses of Cross-Sectional Studies . Healthknowledge, 2009. Cross-Sectional Study . Wikipedia.

Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation.

• The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject.
• Descriptive research is often used as a pre-cursor to more quantitatively research designs, the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
• If the limitations are understood, they can be a useful tool in developing a more focused study.
• Descriptive studies can yield rich data that lead to important recommendations.
• Appoach collects a large amount of data for detailed analysis.
• The results from a descriptive research can not be used to discover a definitive answer or to disprove a hypothesis.
• Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
• The descriptive function of research is heavily dependent on instrumentation for measurement and observation.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999;  McNabb, Connie. Descriptive Research Methodologies . Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design , September 26, 2008. Explorable.com website.

A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental Research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation.

• Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “what causes something to occur?”
• Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects.
• Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
• Approach provides the highest level of evidence for single studies.
• The design is artificial, and results may not generalize well to the real world.
• The artificial settings of experiments may alter subject behaviors or responses.
• Experimental designs can be costly if special equipment or facilities are needed.
• Some research problems cannot be studied using an experiment because of ethical or technical reasons.
• Difficult to apply ethnographic and other qualitative methods to  experimental designed research studies.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs . School of Psychology, University of New England, 2000; Experimental Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Trochim, William M.K. Experimental Design . Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research . Slideshare presentation.

An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to. The focus is on gaining insights and familiarity for later investigation or undertaken when problems are in a preliminary stage of investigation.

The goals of exploratory research are intended to produce the following possible insights:

• Familiarity with basic details, settings and concerns.
• Well grounded picture of the situation being developed.
• Generation of new ideas and assumption, development of tentative theories or hypotheses.
• Determination about whether a study is feasible in the future.
• Issues get refined for more systematic investigation and formulation of new research questions.
• Direction for future research and techniques get developed.
• Design is a useful approach for gaining background information on a particular topic.
• Exploratory research is flexible and can address research questions of all types (what, why, how).
• Provides an opportunity to define new terms and clarify existing concepts.
• Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
• Exploratory studies help establish research priorities.
• Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large.
• The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings.
• The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value in decision-making.
• Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem.

Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research . Wikipedia.

The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute your hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, logs, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid.

• The historical research design is unobtrusive; the act of research does not affect the results of the study.
• The historical approach is well suited for trend analysis.
• Historical records can add important contextual background required to more fully understand and interpret a research problem.
• There is no possibility of researcher-subject interaction that could affect the findings.
• Historical sources can be used over and over to study different research problems or to replicate a previous study.
• The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem.
• Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts.
• Interpreting historical sources can be very time consuming.
• The sources of historical materials must be archived consistentally to ensure access.
• Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
• Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity.
• It rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged.

Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58;  Gall, Meredith. Educational Research: An Introduction . Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007.

A longitudinal study follows the same sample over time and makes repeated observations. With longitudinal surveys, for example, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study and is sometimes referred to as a panel study.

• Longitudinal data allow the analysis of duration of a particular phenomenon.
• Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments.
• The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time].
• Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.
• The data collection method may change over time.
• Maintaining the integrity of the original sample can be difficult over an extended period of time.
• It can be difficult to show more than one variable at a time.
• This design often needs qualitative research to explain fluctuations in the data.
• A longitudinal research design assumes present trends will continue unchanged.
• It can take a long period of time to gather results.
• There is a need to have a large sample size and accurate sampling to reach representativness.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-120; Longitudinal Study . Wikipedia.

This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project.

• Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe (data is emergent rather than pre-existing).
• The researcher is able to collect a depth of information about a particular behavior.
• Can reveal interrelationships among multifaceted dimensions of group interactions.
• You can generalize your results to real life situations.
• Observational research is useful for discovering what variables may be important before applying other methods like experiments.
• Observation researchd esigns account for the complexity of group behaviors.
• Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and difficult to replicate.
• In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups.
• There can be problems with bias as the researcher may only "see what they want to see."
• There is no possiblility to determine "cause and effect" relationships since nothing is manipulated.
• Sources or subjects may not all be equally credible.
• Any group that is studied is altered to some degree by the very presence of the researcher, therefore, skewing to some degree any data collected (the Heisenburg Uncertainty Principle).

Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods . Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Rosenbaum, Paul R. Design of Observational Studies . New York: Springer, 2010.

Understood more as an broad approach to examining a research problem than a methodological design, philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable, assumptions underpinning an area of study. This approach uses the tools of argumentation derived from philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the relevance of logic and evidence in academic debates, to analyze arguments about fundamental issues, or to discuss the root of existing discourse about a research problem. These overarching tools of analysis can be framed in three ways:

• Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what is fundamental and what is derivative?
• Epistemology -- the study that explores the nature of knowledge; for example, on what does knowledge and understanding depend upon and how can we be certain of what we know?
• Axiology -- the study of values; for example, what values does an individual or group hold and why? How are values related to interest, desire, will, experience, and means-to-end? And, what is the difference between a matter of fact and a matter of value?
• Can provide a basis for applying ethical decision-making to practice.
• Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of research.
• Brings clarity to general guiding practices and principles of an individual or group.
• Philosophy informs methodology.
• Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse.
• Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of reality (metaphysics).
• Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas.
• Limited application to specific research problems [answering the "So What?" question in social science research].
• Analysis can be abstract, argumentative, and limited in its practical application to real-life issues.
• While a philosophical analysis may render problematic that which was once simple or taken-for-granted, the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation and documentation.
• There are limitations in the use of metaphor as a vehicle of philosophical analysis.
• There can be analytical difficulties in moving from philosophy to advocacy and between abstract thought and application to the phenomenal world.

Chapter 4, Research Methodology and Design . Unisa Institutional Repository (UnisaIR), University of South Africa;  Labaree, Robert V. and Ross Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008): 43-70; Maykut, Pamela S. Beginning Qualitative Research: A Philosophic and Practical Guide . Washington, D.C.: Falmer Press, 1994; Stanford Encyclopedia of Philosophy . Metaphysics Research Lab, CSLI, Stanford University, 2013.

• The researcher has a limitless option when it comes to sample size and the sampling schedule.
• Due to the repetitive nature of this research design, minor changes and adjustments can be done during the initial parts of the study to correct and hone the research method. Useful design for exploratory studies.
• There is very little effort on the part of the researcher when performing this technique. It is generally not expensive, time consuming, or workforce extensive.
• Because the study is conducted serially, the results of one sample are known before the next sample is taken and analyzed.
• The sampling method is not representative of the entire population. The only possibility of approaching representativeness is when the researcher chooses to use a very large sample size significant enough to represent a significant portion of the entire population. In this case, moving on to study a second or more sample can be difficult.
• Because the sampling technique is not randomized, the design cannot be used to create conclusions and interpretations that pertain to an entire population. Generalizability from findings is limited.
• Difficult to account for and interpret variation from one sample to another over time, particularly when using qualitative methods of data collection.

Rebecca Betensky, Harvard University, Course Lecture Note slides ; Cresswell, John W. Et al. “Advanced Mixed-Methods Research Designs.” In Handbook of Mixed Methods in Social and Behavioral Research . Abbas Tashakkori and Charles Teddle, eds. (Thousand Oaks, CA: Sage, 2003), pp. 209-240; Nataliya V. Ivankova. “Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice.” Field Methods 18 (February 2006): 3-20; Bovaird, James A. and Kevin A. Kupzyk. “Sequential Design.” In Encyclopedia of Research Design . Neil J. Salkind, ed. Thousand Oaks, CA: Sage, 2010; Sequential Analysis . Wikipedia.  

• << Previous: Purpose of Guide
• Next: Design Flaws to Avoid >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

Title: Review of models and frameworks for set-based design

Authors : Mayara Silvestre de Oliveira; Fernando Antônio Forcellini; Jaime Andrés Lozano; Jader Riso Barbosa

Addresses : Polo - Research Laboratories for Emerging Technologies in Cooling and Thermophysics, Federal University of Santa Catarina (UFSC), Florianópolis, Santa Catarina, Brazil ' Mechanical Engineering Department, Federal University of Santa Catarina (UFSC), Florianópolis, Santa Catarina, Brazil ' Polo - Research Laboratories for Emerging Technologies in Cooling and Thermophysics, Federal University of Santa Catarina (UFSC), Florianópolis, Santa Catarina, Brazil ' Polo - Research Laboratories for Emerging Technologies in Cooling and Thermophysics, Federal University of Santa Catarina (UFSC), Florianópolis, Santa Catarina, Brazil

Abstract : This paper aims to investigate the state-of-the-art in models and frameworks for set-based design and identify the main gaps and contributions in the literature. As a result, 121 models were analysed. Most models are quantitative, computational, engineering design-oriented and focus on early stages. Although the narrowing down process plays a central role in the set-based design, very little is addressed regarding its management processes. No model was found describing the inputs and outputs of the set-based design and the narrowing down process simultaneously. Thus, knowledge is dispersed and focused on specific parts. The relevance of this study relies on providing a comprehensive investigation of the state-of-the-art, identifying opportunities to advance in this study field and providing recommendations for future works seeking to support the development of new methods for implementing and managing set-based design, enabling and encouraging its adoption.

Keywords : set-based design; lean product development; literature review; narrowing down process; concurrent engineering; design space; value; trade-off; quality function deployment; integrated product teams.

DOI : 10.1504/IJPD.2024.137766

International Journal of Product Development, 2024 Vol.28 No.1/2, pp.73 - 103

Received: 31 Jan 2022 Received in revised form: 08 Jun 2022 Accepted: 02 Sep 2022 Published online: 05 Apr 2024 *

Keep up-to-date

• Our Newsletter ( subscribe for free )
• New issue alerts
• Inderscience is a member of publishing organisations including:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

Journal of Materials Chemistry A

Inverse design of experimentally synthesizable crystal structures by leveraging computational and experimental data †.

* Corresponding authors

a Institute of Atomic and Molecular Physics, Sichuan University, Chengdu 610065, China E-mail: [email protected]

b College of Physics, Sichuan University, Chengdu 610064, China

Crystal structure prediction (CSP) drives the discovery and design of innovative materials. However, existing CSP methods rely heavily on formation enthalpies calculated by density functional theory (DFT) and ignore the differences between DFT and experimental values, resulting in predicted structures that may be limited in experimental synthesis. To overcome these limitations, a novel collaborative approach was proposed for CSP that utilizes advanced deep learning models and optimization algorithms combined with experimental data. The formation enthalpies obtained from our approach are highly consistent with the actual experimental observations by transfer learning from the experimental data. By incorporating the experimentally synthesizable information of crystals, our approach is able to inverse design crystal structures that can be synthesized experimentally. Applying the model to 17 representative compounds, the results indicate that the approach can accurately identify experimentally synthesized structures with high precision. Moreover, the obtained formation enthalpies and lattice constants closely align with experimental values, underscoring the approach's effectiveness. The synergistic approach bridges the longstanding disparities between theoretical predictions and experimental results, providing guidance for experimentally synthesizing new materials and alleviating the demand for extensive and costly experimental trials.

• This article is part of the themed collection: Journal of Materials Chemistry A HOT Papers

Supplementary files

• Supplementary information PDF (2311K)

Article information

Download citation, permissions.

Inverse design of experimentally synthesizable crystal structures by leveraging computational and experimental data

C. Qin, J. Liu, S. Ma, J. Du, G. Jiang and L. Zhao, J. Mater. Chem. A , 2024, Advance Article , DOI: 10.1039/D4TA00725E

To request permission to reproduce material from this article, please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements

• Reference Manager
• Simple TEXT file

People also looked at

Original research article, impact of industrial policy on urban green innovation: empirical evidence of china’s national high-tech zones based on double machine learning.

• College of Economics and Management, Taiyuan University of Technology, Taiyuan, China

Effective industrial policies need to be implemented, particularly aligning with environmental protection goals to drive the high-quality growth of China’s economy in the new era. Setting up national high-tech zones falls under the purview of both regional and industrial policies. Using panel data from 163 prefecture-level cities in China from 2007 to 2019, this paper empirically analyzes the impact of national high-tech zones on the level of urban green innovation and its underlying mechanisms. It utilizes the national high-tech zones as a quasi-natural experiment and employs a double machine learning model. The study findings reveal that the policy for national high-tech zones greatly enhances urban green innovation. This conclusion remains consistent even after adjusting the measurement method, empirical samples, and controlling for other policy interferences. The findings from the heterogeneity analysis reveal that the impact of the national high-tech zone policy on green innovation exhibits significant regional heterogeneity, with a particularly significant effect in the central and western regions. Among cities, there is a notable push for green innovation levels in second-tier, third-tier, and fourth-tier cities. The moderating effect results indicate that, at the current stage of development, transportation infrastructure primarily exerts a negative moderating effect on how the national high-tech zone policy impacts the level of urban green innovation. This research provides robust empirical evidence for informing the optimization of the industrial policy of China and the establishment of a future ecological civilization system.

1 Introduction

The Chinese economy currently focuses on high-quality development rather than quick growth. The traditional demographic and resource advantages gradually diminish, making the earlier crude development model reliant on excessive resource input and consumption unsustainable. Simultaneously, resource impoverishment, environmental pollution, and carbon emissions are growing more severe ( Wang F. et al., 2022 ). Consequently, pursuing a mutually beneficial equilibrium between the economy and the environment has emerged as a critical concern in China’s economic growth. Green innovation, the integration of innovation with sustainability development ideas, is progressively gaining significance within the framework of reshaping China’s economic development strategy and addressing the challenges associated with resource and environmental limitations. In light of the present circumstances, and with the objectives outlined in the “3060 Plan” for carbon peak and carbon neutral, the pursuit of a green and innovative development trajectory, emphasizing heightened innovation alongside environ-mental preservation, has emerged as a pivotal concern within the context of China’s contemporary economic progress.

Industrial policy is pivotal in government intervention within market-driven resource allocation and correcting structural disparities. The government orchestrates this initiative to bolster industrial expansion and operational effectiveness. In contrast to Western industrial policies, those in China are predominantly crafted within the administrative framework and promulgated through administrative regulations. Over an extended period, numerous industrial policies have been devised in response to regional disparities in industrial development. These policies aim to identify new growth opportunities in diverse regions, focusing on optimizing and upgrading industrial structures. These strategies have been implemented at various administrative levels, from the central government to local authorities ( Sun and Sun, 2015 ). As a distinctive regional economic policy in China, the national high-tech zone represents one of the foremost supportive measures a city can acquire at the national level. Its crucial role involves facilitating the dissemination and advancement of regional economic growth. Over more than three decades, it has evolved into the primary platform through which China executes its strategy of concentrating on high-tech industries and fostering development driven by innovation. Concurrently, the national high-tech zone, operating as a geographically focused policy customized for a specific region ( Cao, 2019 ), enhances the precision of policy support for the industries under its purview, covering a more limited range of municipalities, counties, and regions. Contrasting with conventional regional industrial policies, the industry-focused policy within national high-tech zones prioritizes comprehensive resource allocation advice and economic foundations to maximize synergy and promote the long-term sustainable growth of the regional economy, and this represents a significant paradigm shift in location-based policies within the framework of carrying out the new development idea. Its inception embodies a combination of central authorization, high-level strategic planning, local grassroots decision-making, and innovative system development. In recent years, driven by the objective of dual carbon, national high-tech have proactively promoted environmentally friendly innovation. Nevertheless, given the proliferation of new industrial policies and the escalating complexity of the policy framework, has the setting up of national high-tech zones genuinely elevated the level of urban green innovation in contrast to conventional regional industrial policies? What are the underlying mechanisms? Simultaneously, concerning the variations among different cities, have the industrial policy tools within the national high-tech zones been employed judiciously and adaptable? What are the concrete practical outcomes? Investigating these matters has emerged as a significant subject requiring resolution by government, industry and academia.

2 Literature review and research hypothesis

2.1 literature review.

When considering industrial policy, the setting up national high-tech zones embodies the intersection of regional and industrial policies. Domestic and international academic research concerning setting up national high-tech zones primarily centers on economic activities and innovation. Notably, the economic impact of national high-tech zones encompasses a wide range of factors, including their influence on total factor productivity ( Tan and Zhang, 2018 ; Wang and Liu, 2023 ), foreign trade ( Alder et al., 2016 ), industrial structure upgrades ( Yuan and Zhu, 2018 ), and economic growth ( Liu and Zhao, 2015 ; Huang and Fernández-Maldonado, 2016 ; Wang Z. et al., 2022 ). Regarding innovation, numerous researchers have confirmed the positive effects of national high-tech zones on company innovation ( Vásquez-Urriago et al., 2014 ; Díez-Vial and Fernández-Olmos, 2017 ; Wang and Xu, 2020 ); Nevertheless, a few scholars have disagreed on this matter ( Hong et al., 2016 ; Sosnovskikh, 2017 ). In general, the consensus among scholars is that setting up high-tech national zones fosters regional innovation significantly. This consensus is supported by various aspects of innovation, including innovation efficiency ( Park and Lee, 2004 ; Chandrashekar and Bala Subrahmanya, 2017 ), agglomeration effect ( De Beule and Van Beveren, 2012 ), innovation capability ( Yang and Guo, 2020 ), among other relevant dimensions. The existing literature predominantly delves into the correlation between the setting up of national high-tech zones, innovation, and economic significance. However, the rise of digital economic developments, notably industrial digitization, has accentuated the limitations of the traditional innovation paradigm. These shortcomings, such as the inadequate exploration of the social importance and sustainability of innovation, have become apparent in recent years. As the primary driver of sustainable development, green innovation represents a potent avenue for achieving economic benefits and environmental value ( Weber et al., 2014 ). Its distinctiveness from other innovation forms lies in its potential to facilitate the transformation of development modes, reshape economic structures, and address pollution prevention and control challenges. However, in the context of green innovation, based on the double-difference approach, Wang et al. (2020) has pointed out that national high-tech zones enhance the effectiveness of urban green innovation, but this is only significant in the eastern region.

Furthermore, scholars have also explored the mechanisms underlying the innovation effects of national high-tech. For example, Cattapan et al. (2012) focused on science parks in Italy. They found that green innovation represents a potent avenue for achieving economic benefits as the primary driver of sustainable development, and environmental value technology transfer services positively influence product innovation. Albahari et al. (2017) confirmed that higher education institutions’ involvement in advancing corporate innovation within technology and science parks has a beneficial moderating effect. Using the moderating effect of spatial agglomeration as a basis, Li WH. et al. (2022) found that industrial agglomeration has a significantly unfavorable moderating influence on the effectiveness of performance transformation in national high-tech zones. Multiple studies have examined the national high-tech zone industrial policy’s regulatory framework and urban innovation. However, in the age of rapidly expanding new infrastructure, infrastructure construction is concentrated on information technologies like blockchain, big data, cloud computing, artificial intelligence, and the Internet; Further research is needed to explore whether traditional infrastructure, particularly transportation infrastructure, can promote urban green innovation. Transportation infrastructure has consistently been vital in fostering economic expansion, integrating regional resources, and facilitating coordinated development ( Behrens et al., 2007 ; Zhang et al., 2018 ; Pokharel et al., 2021 ). Therefore, it is necessary to investigate whether transportation infrastructure can continue encouraging innovative urban green practices in the digital economy.

In summary, the existing literature has extensively examined the influence of national high-tech zones on economic growth and innovation from various levels and perspectives, establishing a solid foundation and offering valuable research insights for this study. Nonetheless, previous studies frequently overlooked the impact of national high-tech zones on urban green innovation levels, and a subsequent series of work in this paper aims to address this issue. Further exploration and expansion are needed to understand the industrial policy framework’s strategy for relating national high-tech zones to urban green innovation. Furthermore, there is a need for further improvement and refinement of the research model and methodology. Based on these, this paper aims to discuss the industrial policy effects of national high-tech zones from the perspective of urban green innovation to enrich and expand the existing research.

In contrast to earlier research, the marginal contribution of this paper is organized into three dimensions: 1) Most scholars have primarily focused on the effects of national high-tech zones on economic activity and innovation, with less emphasis on green innovation and rare studies according to the level of green innovation perspective. The study on national high-tech zones as an industrial policy that has already been done is enhanced by this work. 2) Regarding the research methodology, the Double Machine Learning (DML) approach is used to evaluate the policy effects of national high-tech zones, leveraging the advantages of machine learning algorithms for high-dimensional and non-parametric prediction. This approach circumvents the problems of model setting bias and the “curse of dimensionality” encountered in traditional econometric models ( Chernozhukov et al., 2018 ), enhancing the credibility of the research findings. 3) By introducing transportation infrastructure as a moderator variable, this study investigates the underlying mechanism of national high-tech zones on urban green innovation, offering suggestions for maximizing the influence of these zones on policy.

2.2 Theoretical analysis and hypotheses

2.2.1 national high-tech zones’ industrial policies and urban green innovation.

As one of the ways to land industrial policies at the national level, national high-tech zones serve as effective driving forces for enhancing China’s ability to innovate regionally and its contribution to economic growth ( Xu et al., 2022 ). Green innovation is a novel form of innovation activity that harmoniously balances the competing goals of environmental preservation and technological advancement, facilitating the superior expansion of the economy by alleviating the strain on resources and the environment ( Li, 2015 ). National high-tech zones mainly impact urban green innovation through three main aspects. Firstly, based on innovation compensation effects, national high-tech zones, established based on the government’s strategic planning, receive special treatment in areas such as land, taxation, financing, credit, and more, serving as pioneering special zones and experimental fields established by the government to promote high-quality regional development. When the government offers R&D subsidies to enterprises engaged in green innovation activities within the zones, enterprises are inclined to respond positively to the government’s policy support and enhance their level of green innovation as a means of seeking external legitimacy ( Fang et al., 2021 ), thereby contributing to the advancement of urban green innovation. Secondly, based on the industrial restructuring effect, strict regulation of businesses with high emissions, high energy consumption, and high pollution levels is another aspect of implementing the national high-tech zone program. Consequently, businesses with significant emissions and energy consumption are required to optimize their industrial structure to access various benefits within the park, resulting in the gradual transformation and upgrading of high-energy-consumption industries towards green practices, thereby further contributing to regional green innovation. Based on Porter’s hypothesis, the green and low-carbon requirements of the park policy increase the production costs for polluting industries, prompting polluting enterprises to upgrade their existing technology and adopt green innovation practices. Lastly, based on the theory of industrial agglomeration, the national high-tech zones’ industrial policy facilitates the concentration of innovative talents to a certain extent, resulting in intensified competition in the green innovation market. Increased competition fosters the sharing of knowledge, technology, and talent, stimulating a market environment where the survival of the fittest prevails ( Melitz and Ottaviano, 2008 ). These increase the effectiveness of urban green innovation, helping to propel urban green innovation forward. Furthermore, the infrastructure development within the national high-tech zones establishes a favorable physical environment for enterprises to engage in creative endeavors. Also, it enables the influx of high-quality innovation capital from foreign sources, complementing the inherent characteristics of national high-tech zones that attract such capital and concentrate green innovation resources, ultimately resulting in both environmental and economic benefits. Based on the above analysis, Hypothesis 1 is proposed:

Hypothesis 1. Implementing industrial policies in national high-tech zones enhances levels of urban green innovation.

2.2.2 Heterogeneity analysis

Given the variations in economic foundations, industrial statuses, and population distributions across different regions, development strategies in different regions are also influenced by these variations ( Chen and Zheng, 2008 ). Theoretically, when using administrative boundaries or geographic locations as benchmarks, the impact of national high-tech zone industrial policy on urban green innovation should be achieved through strategies like aligning with the region’s existing industrial structure. Compared to the western and central regions, the eastern region exhibits more incredible innovation and dynamism due to advantages such as a developed economy, good infrastructure, advanced management concepts, and technologies, combined with a relatively high initial level of green innovation factor endowment. Considering the diminishing marginal effect principle of green innovation, the industrial policy implementation in national high-tech zones favors an “icing on the cake” approach in the eastern region, contrasting with a “send carbon in the snow” approach in the central and western regions. In other words, the economic benefits of national high-tech zones for promoting urban green innovation may need to be more robust than their impact on the central and western regions. Literature confirms that establishing national high-tech zones yields a more beneficial technology agglomeration effect in the less developed central and western regions ( Liu and Zhao, 2015 ), leading to a more substantial impact on enhancing the level of urban green innovation.

Moreover, local governments consider economic development, industrial structure, and infrastructure levels when establishing national high-tech zones. These factors serve as the foundation for regional classification to address variations in regional quality and to compensate for gaps in theoretical research on the link between national high-tech zone industrial policy implementation and urban green innovation. Consequently, the execution of industrial policies in national high-tech zones relies on other vital factors influencing urban green innovation. Significant variations exist in economic development and infrastructure levels among cities of different grades ( Luo and Wang, 2023 ). Generally, cities with higher rankings exhibit strong economic growth and infrastructure, contrasting those with lower rankings. Consequently, the effect of establishing a national high-tech zone on green innovation may vary across different city grades. Thus, considering the disparities across city rankings, we delve deeper into identifying the underlying reasons for regional diversity in the green innovation outcomes of industrial policies implemented in national high-tech zones based on city grades. Based on the above analysis, Hypothesis 2 is proposed:

Hypothesis 2. There is regional heterogeneity and city-level heterogeneity in the impact of national high-tech zone policies on the level of urban green innovation.

2.2.3 The moderating effect of transportation infrastructure

Implementing industrial policies and facilitating the flow of innovation factors are closely intertwined with the role of transport infrastructure as carriers and linkages. Generally, enhanced transportation infrastructure facilitates the absorption of local factors and improves resource allocation efficiency, thereby influencing the spatial redistribution of production factors like labor, resources, and technology across cities. Enhanced transportation infrastructure fosters the development of more robust and advanced innovation networks ( Fritsch and Slavtchev, 2011 ). Banister and Berechman (2001) highlighted that transportation infrastructure exhibits network properties that are fundamental to its agglomeration or diffusion effects. From this perspective, robust infrastructure impacts various economic activities, including interregional labor mobility, factor agglomeration, and knowledge exchange among firms, thereby expediting the spillover effects of green technological innovations ( Yu et al., 2013 ). In turn, this could positively moderate the influence of national hi-tech zone policies on green innovation. On the other hand, while transportation infrastructure facilitates the growth of national high-tech zone policies, it also brings negative impacts, including high pollution, emissions, and ecological landscape fragmentation. Improving transportation infrastructure can also lead to the “relative congestion effect” in national high-tech zones. This phenomenon, observed in specific regions, refers to the excessive concentration of similar enterprises across different links of the same industrial chain, which exacerbates the competition for innovation resources among enterprises, making it challenging for enterprises in the region to allocate their limited innovation resources to technological research and development activities ( Li et al., 2015 ). As a result, there needs to be a higher green innovation level. Therefore, the impact of transportation infrastructure in the current stage of development will be more complex. When the level of transport infrastructure is moderate, adequate transport infrastructure supports the promotion of urban green innovation through national high-tech zone policies. However, the impact of transport infrastructure regulation may be harmful. Based on the above analysis, Hypothesis 3 is proposed:

Hypothesis 3. Transportation infrastructure moderates the relationship between national high-tech zones and levels of urban green invention.

3 Research design

3.1 model setting.

This research explores the impact of industrial policies of national high-tech zones on the level of urban green innovation. Many related studies utilize traditional causal inference models to assess the impact of these policies. However, these models have several limitations in their application. For instance, the commonly used double-difference model in the parallel trend test has stringent requirements for the sample data. Although the synthetic control approach can create a virtual control group that meets parallel trends’ needs, it is limited to addressing the ‘one-to-many’ problem and requires excluding groups with extreme values. The selection of matching variables in propensity score matching is subjective, among other limitations ( Zhang and Li, 2023 ). To address the limitations of conventional causal inference models, scholars have started to explore applying machine learning to infer causality ( Chernozhukov et al., 2018 ; Knittel and Stolper, 2021 ). Machine learning algorithms excel at an impartial assessment of the effect on the intended target variable for making accurate predictions.

In contrast to traditional machine learning algorithms, the formal proposal of DML was made in 2018 ( Chernozhukov et al., 2018 ). This approach offers a more robust approach to causal inference by mitigating bias through the incorporation of residual modeling. Currently, some scholars utilize DML to assess causality in economic phenomena. For instance, Hull and Grodecka-Messi (2022) examined the effects of local taxation, crime, education, and public services on migration using DML in the context of Swedish cities between 2010 and 2016. These existing research findings serve as valuable references for this study. Compared to traditional causal inference models, DML offers distinct advantages in variable selection and model estimation ( Zhang and Li, 2023 ). However, in promoting urban green innovation in China, there is a high probability of non-linear relationships between variables, and the traditional linear regression model may lead to bias and errors. Moreover, the double machine learning model can effectively avoid problems such as setting bias. Based on this, the present study employs a DML model to evaluate the policy implications of establishing a national high-tech zone.

3.1.1 Double machine learning framework

Prior to applying the DML algorithm, this paper refers to the practice of Chernozhukov et al. (2018) to construct a partially linear DML model, as depicted in Eq. 1 below:

where i represents the city, t represents the year, and l n G I i t represents the explained variable, which in this paper is the green innovation level of the city. Z o n e i t represents the disposition variable, which in this case is a national high-tech zone’s policy variable. It takes a value of 1 after the implementation of the pilot and 0 otherwise. θ 0 is the disposal factor that is the focus of this paper. X i t represents the set of high-dimensional control variables. Machine learning algorithms are utilized to estimate the specific form of g ^ X i t , whereas U i t , which has a conditional mean of 0, stands for the error term. n represents the sample size. Direct estimation of Eq. 1 provides an estimate for the coefficient of dispositions.

We can further explore the estimation bias by combining Eqs 1 , 2 as depicted in Eq. ( 3 ) below:

where a = 1 n ∑ i ∈ I , t ∈ T   Z o n e i t 2 − 1 1 n ∑ i ∈ I , t ∈ T   Z o n e i t U i t , by a normal distribution having 0 as the mean, b = 1 n ∑ i ∈ I , t ∈ T   Z o n e i t 2 − 1 1 n ∑ i ∈ I , t ∈ T   Z o n e i t g X i t − g ^ X i t . It is important to note that DML utilizes machine learning and a regularization algorithm to estimate a specific functional form g ^ X i t . The introduction of “canonical bias” is inevitable as it prevents the estimates from having excessive variance while maintaining their unbiasedness. Specifically, the convergence of g ^ X i t to g X i t , n −φg > n −1/2 , as n tends to infinity, b also tends to infinity, θ ^ 0 is difficult to converge to θ 0 . To expedite convergence and ensure unbiasedness of the disposal coefficient estimates with small samples, an auxiliary regression is constructed as follows:

where m X i t represents the disposition variable’s regression function on the high-dimensional control variable, this function also requires estimation using a machine learning algorithm in the specific form of m ^ X i t . Additionally, V i t represents the error term with a 0 conditional mean.

3.1.2 The test of the mediating effect within the DML framework

This study investigates how the national high-tech zone industrial policy influences the urban green innovation. It incorporates moderating variables within the DML framework, drawing on the testing procedure outlined by Jiang (2022) , and integrates it with the practice of He et al. (2022) , as outlined below:

Equation 5 is based on Eq. 1 with the addition of variables l n t r a i t and Z o n e i t * l n t r a i t .Where l n t r a i t represents the moderating variable, which in this paper is the transportation infrastructure. Z o n e i t * l n t r a i t represents the interaction term of the moderating variable and the disposition variable. The variables l n t r a i t and Z o n e i t are added to the high-dimensional control variables X i t , and the rest of the variables in Eq. 5 are identical to Eq. 1 . θ 1 represents the disposal factor to focus on.

3.2 Variable selection

3.2.1 dependent variable: level of urban green innovation (lngi).

Nowadays, many academics use indicators like the number of applications for patents or authorizations to assess the degree of urban innovation. To be more precise, the quantity of patent applications is a measure of technological innovation effort, while the number of patents authorized undergoes strict auditing and can provide a more direct reflection of the achievements and capacity of scientific and technological innovation. Thus, this paper refers to the studies of Zhou and Shen (2020) and Li X. et al. (2022) to utilize the count of authorized green invention patents in each prefecture-level city to indicate the level of green innovation. For the empirical study, the count of authorized green patents plus 1 is transformed using logarithm.

3.2.2 Disposal variable: dummy variables for national high-tech zones (Zone)

The national high-tech zone dummy variable’s value correlates with the city in which it is located and the list of national high-tech zones released by China’s Ministry of Science and Technology. If a national high-tech zone was established in the city by 2017, the value is set to 1 for the year the high-tech zone is established and subsequent years. Otherwise, it is set to 0.

3.2.3 Moderating variable: transportation infrastructure (lntra)

Previous studies have shown that China’s highway freight transport comprises 75% of the total freight transport ( Li and Tang, 2015 ). Highway transportation infrastructure has a significant influence on the evolution of the Chinese economy. The development and improvement of highway infrastructure are crucial for modern transportation. This paper uses the research methods of Wu (2019) and uses the roadway mileage (measured in kilometers) to population as a measure of the quality of the transportation system.

3.2.4 Control variables

(1) Foreign direct investment (lnfdi): There is general agreement among academics that foreign direct investment (FDI) significantly influences urban green innovation, as FDI provides expertise in management, human resources, and cutting-edge industrial technology ( Luo et al., 2021 ). Thus, it is necessary to consider and control the level of FDI. This paper uses the ratio of foreign investment to the local GDP in a million yuan.

(2) Financial development level (lnfd): Innovation in science and technology is greatly aided by finance. For the green innovation-driven strategy to advance, it is imperative that funding for science and technology innovation be strengthened. The amount of capital raised for innovation is strongly impacted by the state of urban financial development ( Zhou and Du, 2021 ). Thus, this paper uses the loan balance to GDP ratio as an indicator.

(3) Human capital (lnhum): Highly skilled human capital is essential for cities to drive green innovation. Generally, highly qualified human capital significantly boosts green innovation ( Ansaris et al., 2016 ). Therefore, a measure was employed: the proportion of people in the city who had completed their bachelor’s degree or above.

(4) Industrial structure (lnind): Generally, the secondary industry in China is the primary source of pollution, and there is a significant impact of industrial structure on green innovation ( Qiu et al., 2023 ). The metric used in this paper is the secondary industry-to-GDP ratio for the area.

(5) Regional economic development level (lnagdp): A region’s level of economic growth is indicative of the material foundation for urban green innovation and in-fluences the growth of green innovation in the region ( Bo et al., 2020 ). This research uses the annual gross domestic product per capita as a measurement.

3.3 Data source

By 2017, China had developed 157 national high-tech zones in total. In conjunction with the study’s objectives, this study performs sample adjustments and a screening process. The study’s sample period spans from 2007 to 2019. 57 national high-tech zones that were created prior to 2000 are omitted to lessen the impact on the test results of towns having high-tech zones founded before 2007. Due to the limitations of high-tech areas in cities at the county level in promoting urban green innovation, 8 high-tech zones located in county-level cities are excluded. And 4 high-tech zones with missing severe data are excluded. Among the list of established national high-tech zones, 88 high-tech zones are distributed across 83 prefecture-level cities due to multiple districts within a single city. As a result, 83 cities are selected as the experimental group for this study. Additionally, a control group of 80 cities was selected from among those that did not have high-tech zones by the end of 2019, resulting in a final sample size of 163 cities. This paper collects green patent data for each city from the China Green Patent Statistical Report published by the State Intellectual Property Office. The author compiled the list of national high-tech zones and the starting year of their establishment on the official government website. In addition, the remaining data in this paper primarily originated from the China Urban Statistical Yearbook (2007–2019), the EPS database, and the official websites of the respective city’s Bureau of Statistics. Missing values were addressed through linear interpolation. To address heteroskedasticity in the model, the study logarithmically transforms the variables, excluding the disposal variable. Table 1 shows the descriptive analysis of the variables.

Table 1 . Descriptive analysis.

4 Empirical analysis

4.1 national high-tech zones’ policy effects on urban green innovation.

This study utilizes the DML model to estimate the impact of industrial policies implemented in national high-tech zones at the level of urban green innovation. Following the approach of Zhang and Li (2023) , the sample is split in a ratio of 1:4, and the random forest algorithm is used to perform predictions and combine Eq. ( 1 ) with Eq. ( 4 ) for the regression. Table 2 presents the results with and without controlling for time and city effects. The results indicate that the treatment effect sizes for these four columns are 0.376, 0.293, 0.396, and 0.268, correspondingly, each of which was significant at a 1% level. Thus, Hypothesis 1 is supported.

Table 2 . Benchmark regression results.

4.2 Robustness tests

4.2.1 eliminate the influence of extreme values.

To reduce the impact of extreme values on the estimation outcomes, all variables on the benchmark regression, excluding the disposal variable, undergo a shrinkage process based on the upper and lower 1% and 5% quantiles. Values lower than the lowest and higher than the highest quantile are replaced accordingly. Regression analyses are conducted. Table 3 demonstrates that removing outliers did not substantially alter the findings of this study.

Table 3 . Extreme values removal results.

4.2.2 Considering province-time interaction fixed effects

Since provinces are critical administrative units in the governance system of the Chinese government, cities within the same province often share similarities in policy environment and location characteristics. Therefore, to account for the influence of temporal changes across different provinces, this study incorporates province-time interaction fixed effects based on the benchmark regression. Table 4 presents the individual regression results. Based on the regression results, after accounting for the correlation between different city characteristics within the same province, national high-tech zone policies continue to significantly influence urban green innovation, even at the 1% level.

Table 4 . The addition of province and time fixed effects interaction terms.

4.2.3 Excluding other policy disturbances

When analyzing how national high-tech zones affect strategy for urban green innovation, it is susceptible to the influence of concurrent policies. This study accounts for other comparable policies during the same period to ensure an accurate estimation of the policy effect. Since 2007, national high-tech zone policies have been successively implemented, including the development of “smart cities.” Therefore, this study incorporates a policy dummy variable for “smart cities” in the benchmark regression. The specific regression findings are shown in Table 5 . After controlling for the impact of concurrent policies, the importance of national high-tech zones’ policy impact remains consistent.

Table 5 . Results of removing the impact of parallel policies.

4.2.4 Resetting the DML model

To mitigate the potential bias introduced by the settings in the DML model on the conclusions, the purpose of this study is to assess the conclusions’ robustness using the following methods. First, the sample split ratio of the DML model is adjusted from 1:4 to 1:2 to examine the potential impact of the sample split ratio on the conclusions of this study. Second, the machine learning algorithm is substituted, replacing the random forest algorithm, which has been utilized as a prediction algorithm, with lasso regression, gradient boosting, and neural networks to investigate the potential influence of prediction algorithms on the conclusions of this study. Third, regarding benchmark regression, additional linear models were constructed and analyzed using DML, which involves subjective decisions regarding model form selection. Therefore, DML was employed to construct more comprehensive interactive models, aiming to assess the influence of model settings on the conclusions of this study. The main and auxiliary regressions utilized for the analysis were modified as follows:

Combining Eqs ( 7 ), ( 8 ) for the regression, the interactive model yielded estimated coefficients for the disposition effect:

The results of Eq. ( 9 ) are shown in column (5) of Table 6 . And all the regression results obtained from the modified DML model are presented in Table 6 .

Table 6 . Results of resetting the DML model.

The findings indicate that the sample split ratio in the DML model, the prediction algorithm used, or the model estimation approach does not impact the conclusion that the national high-tech zone policy raises urban areas’ level of green innovation. These factors only modify the magnitude of the policy effect to some degree.

4.3 Heterogeneity analysis

4.3.1 regional heterogeneity.

The sample cities were further divided into the east, central, and west regions based on the three major economic subregions to examine regional variations in national high-tech zone policies ' effects on urban green innovation, with the results presented in Table 7 . National high-tech zone policies do not statistically significantly affect urban green innovation in the eastern region. However, they have a considerable beneficial influence in the central and western areas. The lack of statistical significance may be explained by the possibility that the setting up of national high-tech zones in the eastern region will provide obstacles to the growth of urban green innovation, such as resource strain and environmental pollution. Given the central and western regions’ relatively underdeveloped economic status and industrial structure, coupled with the preceding theoretical analysis, establishing national high-tech zones is a crucial catalyst, significantly boosting urban green innovation levels. Furthermore, the central government emphasizes that setting high-tech national zones should consider regional resource endowments and local conditions, implementing tailored policies. The central and western regions possess unique geographic locations and natural conditions that make them well-suited for developing solar energy, wind energy, and other forms of green energy. Compared to the central region, the national high-tech zone initiative has a more pronounced impact on promoting urban green innovation in the western region. While further optimization is needed for the western region’s urban innovation environment, the policy on national high-tech zones has a more substantial incentive effect in this region due to its more significant development potential, positive transformation of industrial structure, and increased policy support from the state, including the development strategy for the western region.

Table 7 . Heterogeneity test results for different regions.

4.3.2 Urban hierarchical heterogeneity

The New Tier 1 Cities Institute’s ‘2020 City Business Charm Ranking’ is the basis for this study, with the sample cities categorized into Tier 1 (New Tier 1), Tier 2, Tier 3, Tier 4, and Tier 5. Table 8 presents the regression findings for each of the groups.

Table 8 . Heterogeneity test results for different classes of cities.

The results in Table 8 reveal significant heterogeneity at the city level regarding national high-tech zones’ effects on urban green innovation, confirming Hypothesis 2 . In particular, the coefficients for the first-tier cities are not statistically significant due to the small sample size, and the same applies to the fifth-tier cities. This could be attributed to the relatively weak economy and infrastructure development issues in the fifth-tier cities. Additionally, due to their limited level of development, the fifth-tier cities may have a relatively homogeneous industrial structure, with a dominance of traditional industries or agriculture and a need for a more diversified industrial layout. National high-tech zones have not greatly aided the development of green innovation in these cities. In contrast, national high-tech zone policies in second-tier, third-tier, and fourth-tier cities have a noteworthy favorable impact on green innovation, indicating their favorable influence on enhancing green innovation in these cities. Despite the lower level of economic development in fourth-tier cities compared to second-tier and third-tier cities, the fourth-tier cities’ national high-tech zones have the most pronounced impact on promoting green innovation. This could be attributed to the ongoing transformation of industries in fourth-tier cities, which are still in the technology diffusion and imitation stage, allowing these cities’ national high-tech zones to maintain a high marginal effect. Thus, Hypothesis 2 is supported.

5 Further analysis

According to the empirical findings, setting high-tech national zones significantly raises the bar for urban green innovation. Therefore, it is essential to understand the underlying factors and mechanisms that contribute to the positive correlation. This paper constructs a moderating effect test model using Eqs 5 , 6 and provides a detailed discussion by introducing transportation infrastructure as a moderating variable.

The empirical finding of the moderating impact of transportation infrastructure is shown in Table 9 . The dichotomous interaction term Zone*lntra is significantly negative at the 5% level, suggesting that the impact of national high-tech zone policies on the level of urban green innovation is negatively moderated by transportation infrastructure. This result deviates from the general expectation, but it aligns with the complexity of the role played by transportation infrastructure in the context of modern economic development, as discussed in the previous theoretical analysis. This could be attributed to the insufficient green innovation benefits generated by the policy on national high-tech zones at the current stage, which fails to compensate for the adverse effects of excessive resource consumption and environmental pollution caused by the construction of the zone. Furthermore, transportation infrastructure can lead to an excessive concentration of similar enterprises in the high-tech zones. This excessive concentration creates a relative crowding effect, intensifying competition among enterprises. It diminishes their inclination to engage in green innovation collaboration and investment and hinders their effective implementation of technological research and development activities. Moreover, the excessive clustering of similar enterprises implies a need for more diversity in green innovation activities among businesses located in national high-tech zones. This results in duplicated green innovation outputs and hinders the advancement of green innovation. Thus, Hypothesis 3 is supported.

Table 9 . Empirical results of moderating effects.

6 Conclusion and policy recommendations

6.1 conclusion.

Based on panel data from 163 prefecture-level cities in China from 2007 to 2019, the net effect of setting national high-tech zones on urban green innovation was analyzed using the double machine learning model. The results found that: firstly, the national high-tech zone policy significantly raises the degree of local green innovation, and these results remain robust even after accounting for various factors that could affect the estimation results. Secondly, in the central and western regions, the level of urban green innovation is positively impacted by the national high-tech zone policy; However, this impact is less significant in the eastern region. In the western region compared to the central region, the national high-tech zone initiative has a stronger impact on increasing the level of urban green innovation. Across different city levels, compared to second-tier and third-tier cities, the high-tech zone policy has a more substantial impact on increasing the level of green innovation in fourth-tier cities. Thirdly, based on the moderating effect mechanism test, the construction of transportation infrastructure weakens the promotional effect of national high-tech zones on urban green innovation.

6.2 Policy recommendations

In order that national high-tech zones can better promote China’s high-quality development, this paper proposes the following policy recommendations:

(1) Urban green innovation in China depends on accelerating the setting up of national high-tech zones and creating an atmosphere that supports innovation. Establishing national high-tech zones as testbeds for high-quality development and green innovation has significantly elevated urban green innovation. Thus, cities can efficiently foster urban green innovation by supporting the development of national high-tech zones. Cities that have already established national high-tech zones should further encourage enterprises within these zones to increase their investment in research and development. They should also proceed to foster the leadership of national high-tech zones for urban green innovation, assuming the role of pilot cities as models and leaders. Additionally, it is essential to establish mechanisms for cooperation and synergy between the pilot cities and their neighboring cities to promote collective green development in the region.

(2) Expanding the pilot program and implementing tailored policies based on local conditions are essential. Industrial policies about national high-tech zones have differing effects on urban green innovation. Regions should leverage their comparative advantages, consider urban development’s commonalities and unique aspects, and foster a stable and sustainable green innovation ecosystem. The western and central regions should prioritize constructing and enhancing new infrastructure and bolster support for the high-tech green industry. The western region should seize the opportunity presented by national policies that prioritize support, quicken the rate of environmental innovation, and progressively bridge the gap with the eastern and central regions in various aspects. Furthermore, second-tier, third-tier, and fourth-tier cities should enhance the advantages of national high-tech zone policies, further maintaining the high standard of green innovation and keeping green innovation at an elevated level. Regions facing challenges in green innovation, particularly fifth-tier cities, should learn from the development experiences of advanced regions with national high-tech zones to compensate for their deficiencies in green innovation.

(3) Highlighting the importance of transportation regulation and enhancing collaboration in green innovation is crucial. Firstly, transportation infrastructure should be maximized to strengthen coordination and cooperation among regions, facilitate the smooth movement of innovative talents across regions, and facilitate the rational sharing of innovative resources, collectively enhancing green innovation. Additionally, attention ought to be given to the industrial clustering effect of parks to prevent the wastage of resources and inefficiencies resulting from the excessive clustering of similar industries. Efforts should be focused on effectively harnessing the latent potential of crucial transportation infrastructure areas as long-term drivers of development, promptly mitigating the negative impact of transportation infrastructure construction, and gradually achieving the synergistic promotion of the setting up of national high-tech zones and the raising of urban levels of green innovation, among other overarching objectives.

6.3 Limitations and future research

Our study has some limitations because the research in this paper is conducted in the institutional context of China. For example, not all countries are suitable for implementing similar industrial policies to develop the economy while focusing on environmental protection. However, we recognize that this study is interesting and relevant, and it encourages us to focus more intensely on environmental protection from an industrial policy perspective. Moreover, this paper exhibits certain limitations in the research process. Firstly, the urban green innovation measurement index was developed using the quantity of green patent authorizations. Future studies could focus on green innovation processes, such as the quality of green patents granted. Secondly, the paper employs machine learning techniques for causal inference. Subsequent investigations could delve further into the potential applications of machine learning algorithms in environmental sciences to maximize the benefits of innovative research methodologies.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author contributions

WC: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing–review and editing. YJ: Conceptualization, Data curation, Formal Analysis, Investigation, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing–original draft, Writing–review and editing. BT: Investigation, Project administration, Writing–review and editing.

The authors declare that financial support was received for the research, authorship, and/or publication of this article. This research was supported by the Youth Fund for Humanities and Social Science research of Ministry of Education (20YJC790004).

Acknowledgments

The authors are grateful to the editors and the reviewers for their insightful comments.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Albahari, A., Pérez-Canto, S., Barge-Gil, A., and Modrego, A. (2017). Technology parks versus science parks: does the university make the difference? Technol. Forecast. Soc. Change 116, 13–28. doi:10.1016/j.techfore.2016.11.012

CrossRef Full Text | Google Scholar

Alder, S., Shao, L., and Zilibotti, F. (2016). Economic reforms and industrial policy in a panel of Chinese cities. J. Econ. Growth 21, 305–349. doi:10.1007/s10887-016-9131-x

Ansaris, M., Ashrafi, S., and Jebellie, H. (2016). The impact of human capital on green innovation. Industrial Manag. J. 8 (2), 141–162. doi:10.22059/imj.2016.60653

Banister, D., and Berechman, Y. (2001). Transport investment and the promotion of economic growth. J. Transp. Geogr. 9 (3), 209–218. doi:10.1016/s0966-6923(01)00013-8

Behrens, K., Lamorgese, A. R., Ottaviano, G. I., and Tabuchi, T. (2007). Changes in transport and non-transport costs: local vs global impacts in a spatial network. Regional Sci. Urban Econ. 37 (6), 625–648. doi:10.1016/j.regsciurbeco.2007.08.003

Bo, W., Yongzhong, Z., Lingshan, C., and Xing, Y. (2020). Urban green innovation level and decomposition of its determinants in China. Sci. Res. Manag. 41 (8), 123. doi:10.19571/j.cnki.1000-2995.2020.08.013

Cao, Q. F. (2019). The latest researches on place based policy and its implications for the construction of xiong’an national new district. Sci. Technol. Prog. Policy 36 (2), 36–43. (in Chinese).

Google Scholar

Cattapan, P., Passarelli, M., and Petrone, M. (2012). Brokerage and SME innovation: an analysis of the technology transfer service at area science park, Italy. Industry High. Educ. 26 (5), 381–391. doi:10.5367/ihe.2012.0119

Chandrashekar, D., and Bala Subrahmanya, M. H. (2017). Absorptive capacity as a determinant of innovation in SMEs: a study of Bengaluru high-tech manufacturing cluster. Small Enterp. Res. 24 (3), 290–315. doi:10.1080/13215906.2017.1396491

Chen, M., and Zheng, Y. (2008). China's regional disparity and its policy responses. China & World Econ. 16 (4), 16–32. doi:10.1111/j.1749-124x.2008.00119.x

Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., et al. (2018). Double/debiased machine learning for treatment and structural parameters. Econ. J. 21 (1), C1–C68. doi:10.1111/ectj.12097

De Beule, F., and Van Beveren, I. (2012). Does firm agglomeration drive product innovation and renewal? An application for Belgium. Tijdschr. Econ. Soc. Geogr. 103 (4), 457–472. doi:10.1111/j.1467-9663.2012.00715.x

Díez-Vial, I., and Fernández-Olmos, M. (2017). The effect of science and technology parks on firms’ performance: how can firms benefit most under economic downturns? Technol. Analysis Strategic Manag. 29 (10), 1153–1166. doi:10.1080/09537325.2016.1274390

Fang, Z., Kong, X., Sensoy, A., Cui, X., and Cheng, F. (2021). Government’s awareness of environmental protection and corporate green innovation: a natural experiment from the new environmental protection law in China. Econ. Analysis Policy 70, 294–312. doi:10.1016/j.eap.2021.03.003

Fritsch, M., and Slavtchev, V. (2011). Determinants of the efficiency of regional innovation systems. Reg. Stud. 45 (7), 905–918. doi:10.1080/00343400802251494

He, J. A., Peng, F. P., and Xie, X. Y. (2022). Mixed-ownership reform, political connection and enterprise innovation: based on the double/unbiased machine learning method. Sci. Technol. Manag. Res. 42 (11), 116–126. (in Chinese).

Hong, J., Feng, B., Wu, Y., and Wang, L. (2016). Do government grants promote innovation efficiency in China's high-tech industries? Technovation 57, 4–13. doi:10.1016/j.technovation.2016.06.001

Huang, W. J., and Fernández-Maldonado, A. M. (2016). High-tech development and spatial planning: comparing The Netherlands and Taiwan from an institutional perspective. Eur. Plan. Stud. 24 (9), 1662–1683. doi:10.1080/09654313.2016.1187717

Hull, I., and Grodecka-Messi, A. (2022). Measuring the impact of taxes and public services on property values: a double machine learning approach . arXiv preprint arXiv:2203.14751.

Jiang, T. (2022). Mediating effects and moderating effects in causal inference. China Ind. Econ. 5, 100–120. doi:10.19581/j.cnki.ciejournal.2022.05.005

Knittel, C. R., and Stolper, S. (2021). Machine learning about treatment effect heterogeneity: the case of household energy use. Nashv. TN 37203, 440–444. doi:10.1257/pandp.20211090

Li, H., and Tang, L. (2015). Transportation infrastructure investment, spatial spillover effect and enterprise inventory. Manag. World 4, 126–136. doi:10.19744/j.cnki.11-1235/f.2015.04.012

Li, W. H., Liu, F., and Liu, T. S. (2022a). Can national high-tech zones improve the urban innovation efficiency? an empirical test based on the effect of spatial agglomeration regulation. Manag. Rev. 34 (5), 93. doi:10.14120/j.cnki.cn11-5057/f.2022.05.007

Li, X. (2015). Analysis and outlook of the related researches on green innovation. R&D Manag. 27 (2), 1–11. doi:10.13581/j.cnki.rdm.2015.02.001

Li, X., Shao, X., Chang, T., and Albu, L. L. (2022b). Does digital finance promote the green innovation of China's listed companies? Energy Econ. 114, 106254. doi:10.1016/j.eneco.2022.106254

Li, X. P., Li, P., Lu, D. G., and Jiang, F. T. (2015). Economic agglomeration, selection effects and firm productivity. J. Manag. World 4, 25–37+51. (in Chinese). doi:10.19744/j.cnki.11-1235/f.2015.04.004

Liu, R. M., and Zhao, R. J. (2015). Does the national high-tech zone promote regional economic development? A verification based on differences-in-differences method. J. Manag. World 8, 30–38. doi:10.19744/j.cnki.11-1235/f.2015.08.005

Luo, R., and Wang, Q. M. (2023). Does the construction of national demonstration logistics park produce economic growth effect? Econ. Surv. 40 (1), 47–56. doi:10.15931/j.cnki.1006-1096.2023.01.015

Luo, Y., Salman, M., and Lu, Z. (2021). Heterogeneous impacts of environmental regulations and foreign direct investment on green innovation across different regions in China. Sci. total Environ. 759, 143744. doi:10.1016/j.scitotenv.2020.143744

PubMed Abstract | CrossRef Full Text | Google Scholar

Melitz, M. J., and Ottaviano, G. I. (2008). Market size, trade, and productivity. Rev. Econ. Stud. 75 (1), 295–316. doi:10.1111/j.1467-937x.2007.00463.x

Park, S. C., and Lee, S. K. (2004). The regional innovation system in Sweden: a study of regional clusters for the development of high technology. Ai Soc. 18 (3), 276–292. doi:10.1007/s00146-003-0277-7

Pokharel, R., Bertolini, L., Te Brömmelstroet, M., and Acharya, S. R. (2021). Spatio-temporal evolution of cities and regional economic development in Nepal: does transport infrastructure matter? J. Transp. Geogr. 90, 102904. doi:10.1016/j.jtrangeo.2020.102904

Qiu, Y., Wang, H., and Wu, J. (2023). Impact of industrial structure upgrading on green innovation: evidence from Chinese cities. Environ. Sci. Pollut. Res. 30 (2), 3887–3900. doi:10.1007/s11356-022-22162-1

Sosnovskikh, S. (2017). Industrial clusters in Russia: the development of special economic zones and industrial parks. Russ. J. Econ. 3 (2), 174–199. doi:10.1016/j.ruje.2017.06.004

Sun, Z., and Sun, J. C. (2015). The effect of Chinese industrial policy: industrial upgrading or short-term economic growth. China Ind. Econ. 7, 52–67. (in Chinese). doi:10.19581/j.cnki.ciejournal.2015.07.004

Tan, J., and Zhang, J. (2018). Does national high-tech development zones promote the growth of urban total factor productivity? —based on" quasi-natural experiments" of 277 cities. Res. Econ. Manag. 39 (9), 75–90. doi:10.13502/j.cnki.issn1000-7636.2018.09.007

Vásquez-Urriago, Á. R., Barge-Gil, A., Rico, A. M., and Paraskevopoulou, E. (2014). The impact of science and technology parks on firms’ product innovation: empirical evidence from Spain. J. Evol. Econ. 24, 835–873. doi:10.1007/s00191-013-0337-1

Wang, F., Dong, M., Ren, J., Luo, S., Zhao, H., and Liu, J. (2022a). The impact of urban spatial structure on air pollution: empirical evidence from China. Environ. Dev. Sustain. 24, 5531–5550. doi:10.1007/s10668-021-01670-z

Wang, M., and Liu, X. (2023). The impact of the establishment of national high-tech zones on total factor productivity of Chinese enterprises. China Econ. 18 (3), 68–93. doi:10.19602/j.chinaeconomist.2023.05.04

Wang, Q., She, S., and Zeng, J. (2020). The mechanism and effect identification of the impact of National High-tech Zones on urban green innovation: based on a DID test. China Popul. Resour. Environ. 30 (02), 129–137.

Wang, W. S., and Xu, T. S. (2020). A research on the impact of national high-teach zone establishment on enterprise innovation performance. Econ. Surv. 37 (6), 76–87. doi:10.15931/j.cnki.1006-1096.20201010.001

Wang, Z., Yang, Y., and Wei, Y. (2022b). Has the construction of national high-tech zones promoted regional economic growth? empirical research from prefecture-level cities in China. Sustainability 14 (10), 6349. doi:10.3390/su14106349

Weber, M., Driessen, P. P., and Runhaar, H. A. (2014). Evaluating environmental policy instruments mixes; a methodology illustrated by noise policy in The Netherlands. J. Environ. Plan. Manag. 57 (9), 1381–1397. doi:10.1080/09640568.2013.808609

Wu, Y. B. (2019). Does fiscal decentralization promote technological innovation. Mod. Econ. Sci. 41, 13–25.

Xu, S. D., Jiang, J., and Zheng, J. (2022). Has the establishment of national high-tech zones promoted industrial Co-Agglomeration? an empirical test based on difference in difference method. Inq. into Econ. Issues 11, 113–127. (in Chinese).

Yang, F., and Guo, G. (2020). Fuzzy comprehensive evaluation of innovation capability of Chinese national high-tech zone based on entropy weight—taking the northern coastal comprehensive economic zone as an example. J. Intelligent Fuzzy Syst. 38 (6), 7857–7864. doi:10.3233/jifs-179855

Yu, N., De Jong, M., Storm, S., and Mi, J. (2013). Spatial spillover effects of transport infrastructure: evidence from Chinese regions. J. Transp. Geogr. 28, 56–66. doi:10.1016/j.jtrangeo.2012.10.009

Yuan, H., and Zhu, C. L. (2018). Do national high-tech zones promote the transformation and upgrading of China’s industrial structure. China Ind. Econ. 8, 60–77. doi:10.19581/j.cnki.ciejournal.2018.08.004

Zhang, T., Chen, L., and Dong, Z. (2018). Highway construction, firm dynamics and regional economic efficiency. China Ind. Econ. 1, 79–99. doi:10.19581/j.cnki.ciejournal.20180115.003

Zhang, T., and Li, J. C. (2023). Network infrastructure, inclusive green growth, and regional inequality: from causal inference based on double machine learning. J. Quantitative Technol. Econ. 40 (4), 113–135. doi:10.13653/j.cnki.jqte.20230310.005

Zhou, L., and Shen, K. (2020). National city group construction and green innovation. China Popul. Resour. Environ. 30 (8), 92–99.

Zhou, X., and Du, J. (2021). Does environmental regulation induce improved financial development for green technological innovation in China? J. Environ. Manag. 300, 113685. doi:10.1016/j.jenvman.2021.113685

Keywords: national high-tech zone, industrial policy, green innovation, heterogeneity analysis, moderating effect, double machine learning

Citation: Cao W, Jia Y and Tan B (2024) Impact of industrial policy on urban green innovation: empirical evidence of China’s national high-tech zones based on double machine learning. Front. Environ. Sci. 12:1369433. doi: 10.3389/fenvs.2024.1369433

Received: 12 January 2024; Accepted: 15 March 2024; Published: 04 April 2024.

Reviewed by:

Copyright © 2024 Cao, Jia and Tan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Yu Jia, [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Guide to Experimental Design | Overview, Steps, & Examples

Guide to Experimental Design | Overview, 5 steps & Examples

Published on December 3, 2019 by Rebecca Bevans . Revised on June 21, 2023.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design create a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying.

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead. This minimizes several types of research bias, particularly sampling bias , survivorship bias , and attrition bias as time passes.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, other interesting articles, frequently asked questions about experiments.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalized and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomized design vs a randomized block design .
• A between-subjects design vs a within-subjects design .

Randomization

An experiment can be completely randomized or randomized within blocks (aka strata):

• In a completely randomized design , every subject is assigned to a treatment group at random.
• In a randomized block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomization isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs. within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomizing or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Prevent plagiarism. Run a free check.

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimize research bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalized to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word “between” means that you’re comparing different conditions between groups, while the word “within” means you’re comparing different conditions within the same group.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Guide to Experimental Design | Overview, 5 steps & Examples. Scribbr. Retrieved April 5, 2024, from https://www.scribbr.com/methodology/experimental-design/

Is this article helpful?

Rebecca Bevans

Other students also liked, random assignment in experiments | introduction & examples, quasi-experimental design | definition, types & examples, how to write a lab report, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Advertisement

A global analysis of habitat fragmentation research in reptiles and amphibians: what have we done so far?

• Review Paper
• Open access
• Published: 08 January 2023
• Volume 32 , pages 439–468, ( 2023 )

Cite this article

You have full access to this open access article

• W. C. Tan   ORCID: orcid.org/0000-0002-6067-3528 1 ,
• A. Herrel   ORCID: orcid.org/0000-0003-0991-4434 2 , 3 , 4 &
• D. Rödder   ORCID: orcid.org/0000-0002-6108-1639 1  

7445 Accesses

10 Citations

27 Altmetric

Explore all metrics

Habitat change and fragmentation are the primary causes of biodiversity loss worldwide. Recent decades have seen a surge of funding, published papers and citations in the field as these threats to biodiversity continue to rise. However, how research directions and agenda are evolving in this field remains poorly understood. In this study, we examined the current state of research on habitat fragmentation (due to agriculture, logging, fragmentation, urbanisation and roads) pertaining to two of the most threatened vertebrate groups, reptiles and amphibians. We did so by conducting a global scale review of geographical and taxonomical trends on the habitat fragmentation types, associated sampling methods and response variables. Our analyses revealed a number of biases with existing research efforts being focused on three continents (e.g., North America, Europe and Australia) and a surplus of studies measuring species richness and abundance. However, we saw a shift in research agenda towards studies utilising technological advancements including genetic and spatial data analyses. Our findings suggest important associations between sampling methods and prevalent response variables but not with the types of habitat fragmentation. These research agendas are found homogeneously distributed across all continents. Increased research investment with appropriate sampling techniques is crucial in biodiversity hotpots such as the tropics where unprecedented threats to herpetofauna exist.

Similar content being viewed by others

Habitat conservation research for amphibians: methodological improvements and thematic shifts.

Gentile Francesco Ficetola

Effect of landscape composition and configuration on biodiversity at multiple scales: a case study with amphibians from Sierra Madre del Sur, Oaxaca, Mexico

Daniel G. Ramírez-Arce, Leticia M. Ochoa-Ochoa & Andrés Lira-Noriega

A dry future for the Everglades favors invasive herpetofauna

Hunter J. Howell, Giacomo L. Delgado, … Christopher A. Searcy

Avoid common mistakes on your manuscript.

Introduction

Habitat loss and fragmentation are the predominant causes underlying widespread biodiversity changes in terrestrial ecosystems (Fahrig 2003 ; Newbold et al. 2015 ). These processes may cause population declines by disrupting processes such as dispersal, gene flow, and survival. Over the past 30 years habitat loss and fragmentation have been suggested to have reduced biodiversity by up to 75% in different biomes around the world (Haddad et al. 2015 ). This is mainly due to the clearing of tropical forests, the expansion of agricultural landscapes, the intensification of farmland production, and the expansion of urban areas (FAO and UNEP 2020 ). The rate of deforestation and corresponding land conversions of natural habitats are happening rapidly and will continue to increase in the future at an accelerated rate, particularly in biodiversity hotspots (Deikumah et al. 2014 ; Habel et al. 2019 ; FAO and UNEP 2020 ).

For this reason, habitat fragmentation has been a central research focus for ecologists and conservationists over the past two decades (Fardila et al. 2017 ). However, habitat fragmentation consists of two different processes: loss of habitat and fragmentation of existing habitat (Fahrig 2003 ). The former simply means the removal of habitat, and latter is the transformation of continuous areas into discontinuous patches of a given habitat. In a radical review, Fahrig ( 2003 ) suggested that fragmentation per se, i.e., the breaking up of habitat after controlling for habitat loss, has a weaker or even no effect on biodiversity compared to habitat loss. She further recommended that the effects of these two components should be measured independently (Fahrig 2017 ). Despite being recognised as two different processes, researchers tend not to distinguish between their effects and commonly lump the combined consequences under a single umbrella term “habitat fragmentation” (Fahrig 2003 , 2017 ; Lindenmayer and Fischer 2007 ; Riva and Fahrig 2022 ). Nonetheless, fragmentation has been widely recognised in the literature and describes changes that occur in landscapes, including the loss of habitat (Hadley and Betts 2016 ). Hence, to avoid imprecise or inconsistent use of terminology and provide a holistic view of the effect of modified landscapes, we suggest the term “habitat fragmentation” to indicate any type of landscape change, both habitat loss and fragmentation throughout the current paper.

One main conundrum is that biodiversity decline does not occur homogeneously everywhere nor among all species (Blowes et al. 2019 ). Moreover, we should expect a global disparity in biodiversity responses to habitat fragmentation across different biomes (Newbold et al. 2020 ; Cordier et al. 2021 ). For example, tropical regions are predicted to have higher negative effects of habitat fragmentation than temperate regions. There are two possible reasons: a) higher intensification of land use change in the tropics (Barlow et al. 2018 ), and b) forest animals in the tropics are less likely to cross open areas (Lindell et al. 2007 ). Furthermore, individual species respond to landscape modification differently; some thrive whereas others decline (Fahrig 2003 ). Habitat specialists with broader habitat tolerance and wide-ranging distributions are most likely to benefit from increase landscape heterogeneity and more open and edge habitat (Hamer and McDonnell 2008 ; Newbold et al. 2014 ; Palmeirim et al. 2017 ). Therefore, appropriate response metrics should be used in measuring the effect of habitat fragmentation on biodiversity depending on the taxa group, biome and scale of study as patterns of richness can sometimes be masked by the abundance of generalist species (Riemann et al. 2015 ; Palmeirim et al. 2017 ).

Previous reviews have identified general patterns and responses of reptile and amphibian populations to habitat modification. They have been largely centred around specific types of habitat fragmentation: land use change (Newbold et al. 2020 ), logging (Sodhi et al. 2004 ), fragmentation per se (Fahrig 2017 ), urbanisation (Hamer and McDonnell 2008 ; McDonald et al. 2013 ), fire (Driscoll et al. 2021 ), and roads (Rytwinski and Fahrig 2012 ). Few reviews have, however, attempted a global synthesis of all types of land use changes and even fewer have addressed biases in geographical regions and taxonomical groups (but see Gardner et al. ( 2007 ) and Cordier et al. ( 2021 )). Gardner et al. ( 2007 ) synthesised the extant literature and focused on 112 papers on the consequences of habitat fragmentation on reptiles and amphibians published between 1945 and 2006. They found substantial biases across geographic regions, biomes, types of data collected as well as sampling design and effort. However, failure to report basic statistics by many studies prevented them from performing meta-analyses on research conclusions. More recently, a review by Cordier et al. ( 2021 ) conducted a meta-analysis based on 94 primary studies on the overall effects of land use changes through time and across the globe. Yet, there has been no comprehensive synthesis on the research patterns and agenda of published literature on habitat fragmentation associated with the recent advances of novel research tools and techniques. Therefore, our review may provide new insights of the evolution and biases in the field over the last decades and provide a basis for future research directions. Knowledge gaps caused by these biases could hamper the development of habitat fragmentation research and the implementation of effective strategies for conservation.

We aim to remedy this by examining research patterns for the two vertebrate classes Amphibia and Reptilia, at a global scale. We chose amphibians and reptiles for several reasons. First, habitat fragmentation research has been dominated by birds and mammals (Fardila et al. 2017 ). Reptiles and amphibians, on the other hand, are under-represented; together, they constitute only 10% of the studies (Fardila et al. 2017 ). Second, high proportions of amphibian and reptile species are threatened globally. To date, more than one third of amphibian (40%) and one in five reptile species (21%) are threatened with extinction (Stuart et al. 2004 ; Cox et al. 2022 ). Amphibians are known to be susceptible to land transformation as a result of their cryptic nature, habitat requirements, and reduced dispersal ability (Green 2003 ; Sodhi et al. 2008 ; Ofori‐Boateng et al. 2013 ; Nowakowski et al. 2017 ). Although poorly studied (with one in five species classified as data deficient) (Böhm et al. 2013 ), reptiles face the same threats as those impacting amphibians (Gibbons et al. 2000 ; Todd et al. 2010 ; Cox et al. 2022 ). Reptiles have small distributional ranges with high endemism compared to other vertebrates and as such are likely vulnerable to habitat fragmentation (Todd et al. 2010 ; Meiri et al. 2018 ). Third, both these groups are poikilotherms whose physiology makes them highly dependent on temperature and precipitation levels. Hence, they are very sensitive to changing thermal landscapes (Nowakowski et al. 2017 ).

Here, we first ask how is the published literature distributed across geographic regions and taxa? Is there a bias in the geographic distribution of species studied compared to known species? It is well known that conservation and research efforts are often concentrated in wealthy and English-speaking countries (Fazey et al. 2005 ), but has this bias improved over the years? Second, how are researchers conducting these studies? We assess whether certain sampling methods and response variables are associated to specific types of habitat fragmentation. Over the past decades new tools and techniques are constantly being discovered or developed. Combinations of methodologies are now shedding new light on biodiversity responses and consequences of habitat fragmentation. In particular, genetic techniques are useful in detecting changes in population structure, identifying isolated genetic clusters, and in estimating dispersal (Smith et al. 2016 ). Similarly, habitat occupancy and modelling can also provide powerful insights into dispersal (Driscoll et al. 2014 ). Remote sensing data are now used in analysing effects of area, edge, and isolation (Ray et al. 2002 ). Finally, how are these associations or research agendas distributed across space? We expect to find geographic structure of emerging agendas across the globe. For instance, we predict genetic studies to be located in North America and Europe but also in East Asian countries such as China and Japan as a result of their advancement in genetics (Forero et al. 2016 ). On the other hand, simple biodiversity response indicators which do not require extensive capacity building and application of advanced technologies are likely more used in developing regions of the world (Barber et al. 2014 ). These findings are valuable to evaluate and update the global status of our research on the effects of habitat fragmentation on herpetofauna and to suggest recommendations for conservation plans.

Materials and methods

Data collection.

We conducted the review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Fig.  1 ) (Moher et al. 2009 ). We conducted a comprehensive and exhaustive search using Web of Science to review published studies reporting the consequences of habitat fragmentation on amphibians and reptiles. We consulted the database in November 2019 by using two general search strings: (1) Habitat fragmentation AND (frog* OR amphib* OR salamander* OR tadpole*) (2) Habitat fragmentation AND (reptil* OR snake* OR lizard* OR turtle* OR crocodile*). This returned a total of 869 records from search (1) and 795 from search (2), with 1421 unique records remaining after duplicates were removed. We did not include “habitat loss” in our search term as it would only introduce unrelated articles focusing on biodiversity and conservation management instead of methodology and mechanistic approaches.

PRISMA flow-diagram of the study selection process

Throughout, we use the term papers to refer to individual journal article records. Out of the 1421 papers, we were unfortunately not able to locate seven papers from Acta Oecologica, Zoology: Analysis of Complex Systems, Israel Journal of Ecology and Evolution, Western North American Naturalist, Natural Areas Journal, Ecology, and the Herpetological Journal. We screened all articles from the title through the full text to determine whether they met our criteria for inclusion. To be included, studies needed to fulfil several criteria. First, papers needed to be peer-reviewed journal articles containing data collected on reptiles and/or amphibians at the species level (224 articles rejected because no species-specific data was available). Reviews and metastudies (n = 102) were excluded from the data analysis as they may represent duplicates as they are mainly based on data sets from other papers, but these form an integral part of our discussion. Furthermore, papers which do not provide data on contemporary time scales such as long-term (> 10, 000 years ago) changes on the paleo-spatial patterns (n = 59) were excluded. Because the effects of fragmentation per se have been measured inconsistently by many authors and have not been differentiated from habitat removal (Fahrig 2003 ), we consider any recent anthropogenic habitat degradation, and modification at patch or/and landscape scales during the Holocene as an effect of habitat fragmentation. Only papers which examined direct or indirect effects of habitat fragmentation were included in our analysis, regardless of the magnitude and direction. Papers which did not mention specific types of habitat fragmentation as the focus of their study (n = 338) were excluded.

Geographical and taxonomical distribution

Using the selected papers, we compiled a taxonomic and geographical database for each paper: (a) GPS or georeferenced location of the study site; (b) the focal group investigated (amphibian and/or reptile); (c) taxonomic groups (order, family, genus).

We listed the overall number of species studied covered by selected papers in each continent and compared them to the total number of currently described species. We obtained total described species of both reptiles and amphibians from the following sources: ReptileDatabase (Uetz et al. 2021 ) and AmphibiaWeb (AmphibiaWeb 2021 ). Then, we calculated the proportions of species covered by the selected papers compared to total number of described species for each continent. We did not update species nomenclature from selected papers as the mismatches from these potentially outdated taxonomies would be insignificant in our analyses.

Categorisation of papers

Each paper is classified into three main types of data collected: forms of habitat fragmentation, sampling methods, and response variables (Online Appendix 1). A paper can be classified into one or multiple categories in each type of data. The types of data and their following categories were:

Forms of habitat fragmentation

We recorded different types of habitat fragmentation from the selection of studies: (1) “Fragmentation” (includes patch isolation, edge and area effects); (2) “Agriculture” (includes any form of commercial and subsistence farming such as monocultures, plantations, and livestock farming); (3) “Logging” (e.g., agroforestry and silviculture); (4) “Mining” (presence of mining activities); (5) “Urbanisation” (includes presence of cities, towns or villages and parks created for recreational purposes); (6) “Road” (includes any vehicle roadway such as railways and highways) and (7) “Other types of habitat fragmentation” (e.g., fire, river dams, ditches, diseases, desertification etc.). Many studies deal with more than one type of habitat fragmentation. However, we made sure the selection for fragmentation forms is mainly based on the focus and wordings in the methodology section.

Sampling methods

We report trends in the design and sampling methods among the compiled studies over the last three decades. Due to the substantial variability in the level of sample design information reported by different studies, we narrowed them down into six general categories representing common sampling methods. Common methods used in estimating herpetofauna diversity (e.g., visual transect surveys, acoustic monitoring and trapping methods) were not included in the analyses due to their omnipresence in the data. The categories are:

(1) “Genetics” studies documented any use of codominant markers (i.e., allozymes and microsatellites), dominant markers [i.e., DNA sequences, random amplified polymorphic DNA (RAPDs) and amplified fragment length polymorphism (AFLPs)] to analyse genetic variability and gene diversity respectively. (2) “Direct tracking methods” studies measured potential dispersal distances or species movement patterns by means of radio telemetry, mark-recapture methods, or fluorescent powder tracking. (3) “Aerial photographs” studies reported the use of aerial photographs while (4) “GIS/Satellite image” studies described the use of satellite imagery and land cover data (i.e., Landsat) and GIS programs (e.g., QGIS and ArcGIS, etc.) in analysing spatial variables. (5) “Experimental” studies involved predictions tested through empirical studies, regardless if they occur naturally or artificially; in a natural or a captive environment. (6) “Prediction/simulation models” studies made use of techniques such as ecological niche models, habitat suitability (i.e., occurrence and occupancy models) and simulations for probability of survival and population connectivity.

Response variables

To further conceptualise how the effects of habitat fragmentation are measured, we assigned 12 biodiversity or ecological response variables. We recorded the type of data that was used in all selected studies: (1) “Species richness or diversity” which are measures of species richness, evenness or diversity (such as the Shannon–Wiener index) (Colwell 2009 ); (2) “Functional richness or species guilds” describes diversity indices based on functional traits (such as body size, reproductive modes, microhabitat association or taxonomic groups); (3) “Presence/absence” or species occupancy; (4) “Population” includes an estimation of population size or density (only when measured specifically in the paper). It includes genetic variation and divergence within and between populations; (5) “Abundance” or counts of individuals for comparison between habitat fragmentation type or species; (6) “Dispersal” takes into account any displacement or movement and can include indirect measurements of dispersal using genetic techniques; (7) “Breeding sites” which measures available breeding or reproduction sites; (8) “Fitness measure” are records of any physiological, ecological or behavioural changes; (9) “Interspecific interaction” depicts any interaction between species including competition and predation; (10) “Extinction or colonisation rate” counts the number of population extinctions or colonisations within a time period; (11) “Microhabitat preference” includes any direct observation made on an individual’s surrounding environmental features (substrate type, perch height, vegetation type, distance to cover etc.); (12) “Generalist or specialist comparison” involves any comparison made between generalist and specialist species. Generalists are able to thrive in various environments whereas specialists occupy a much narrower niche; (13) “Other response variables” can include road kill mortality counts, infection rate of diseases, injury, or any effect from introduced animals and a variety of other responses.

Data analysis

All statistical analyses were conducted in the open source statistical software package R 4.1.0 (R Core Team 2021 ). To gain a broad insight into our understanding of the complexity of habitat fragmentation we applied a Multiple Correspondence Analysis (MCA) (Roux and Rouanet 2004 ) and Hierarchical Clustering on Principle Components (HCPC) (Ward 1963 ) to investigate potential interactions between forms of habitat fragmentation, sampling methods and response variables. MCA is ideal for investigation of datasets with multiple categorical variables and exploration of unbiased relationships between these variables.

We first separate the dataset into papers concerning amphibians or reptiles. The MCA was performed using the MCA function from FactoMineR package of R version 3.1 (Lê et al. 2008 ). To identify subgroups (cluster) of similar papers within our dataset, we performed cluster analysis on our MCA results using HCPC. The cluster results are then visualised in factor map and dendrogram for easier interpretation using factoextra package. This allows us to identify categorical variables which have the highest effect within each cluster. Statistical analyses were considered significant at α = 0.05, while a p between 0.10 and 0.05 was considered as a tendency. The p-value is less than 5% when one category is significantly linked to other categories. The V tests show whether the category is over-expressed (positive values) or under-expressed (negative values) in the cluster (Lebart et al. 1995 ).

Results from the literature review were also analysed with VOSviewer, freeware for constructing and visualising bibliometric networks ( http://www.vosviewer.com/ ). The program uses clustering techniques to analyse co-authors, co-occurrence of keywords, citations, or co-citations (van Eck and Waltman 2014 ). First, we analyse co-authorships of countries to provide a geographical representation of groups of authors in various countries over the past 30 years. Each circle represents an author’s country and the size represents the collaboration frequency with other countries. The lines between the nodes represent the collaboration networks between the countries while the thickness of the lines indicates the collaboration intensities between them. Lastly, to complement the MCA and HCPC, we used VOSviewer to analyse a clustering solution of categories at an aggregate level. Aggregate clustering is a meta-clustering method to improve the robustness of clustering and does not require a priori information about the number of clusters. In this case, instead of author’s keywords, we used the co-occurrence of categories associated to each selected paper as input to run the software.

We identified a total of 698 papers published between January 1991 and November 2019 reporting consequences of habitat fragmentations corresponding to our selection criteria (Fig.  1 ). The complete list of studies included (hereafter termed “selected papers”) is available in Online Appendix 2. The distribution of these selected papers between focal groups and among continents was non-homogeneous (Fig.  2 ). Selected papers reviewed were predominantly studies which were conducted in North America 310 (44%) and Europe 131 (19%), but also from Oceania 104 (15%), South America 85 (12%), Asia 37 (5%) and Africa 31 (5%). For co-authorships between countries based on VOSviewer, the minimum document number of a country was set as 5 and a total of 21 and 14 countries met the threshold for amphibians and reptiles respectively (Fig.  3 ). For amphibians, countries in the American continent such as United States of America or USA (178 articles), Brazil (38 articles) and Canada (35 articles) have the largest research weight (Fig.  3 a). Authors from the USA have the largest international cooperation network, followed by Brazil. Australia and other European countries such as Germany, France and England also have high collaboration relationships with other countries. In contrast, reptile studies were mainly concentrated around two countries: the USA (139 articles) and Australia (86 articles) (Fig.  3 b). No other country from the rest of the world has more than 20 articles. While both the USA and Australia have the largest collaboration networks, Canada, Spain and Mexico are also highly cooperative with authors from other countries.

Map of study locations for a amphibians and b reptiles with each circle representing the study location of papers included in the review. The colour scale of the continents ranging from 0 – 0.9 indicates the proportions of amphibian and reptile species represented in the reviewed papers when compared to known species in the world (obtained from AmphibiaWeb and ReptileDatabase): a Europe (0.73), Africa (0.23), North America (0.23), South America (0.18), Oceania (0.07) and Asia (0.06) and b Europe (0.27), Oceania (0.18), Africa (0.12), North America (0.11), South America (0.09) and Asia (0.02)

Co-authorship map of countries involved in habitat fragmentation research in a amphibians and b reptiles. The colours represent the continents countries belong to. Each circle represents an author’s country and the size represents the collaboration frequency with other countries. The lines between the nodes represent the collaboration networks between the countries while the thickness of the lines indicates the collaboration intensities between them. Category co-occurrence network maps for c amphibians and d reptiles. The colour represents the different cluster groups each category belongs to. Abbreviations for the categories in forms of habitat change: fragmentation (FGM), agriculture (AGR), Logging (LOG), Mining (MIN), Urbanisation (URB), road (RD), other habitat fragmentation (OHC); sampling methods: genetics (GEN), direct tracking method (DTM), aerial photographs (APT), GIS/ Satellite images (GIS), experimental (EXP), prediction/ simulation models (PSM) and response variables: species richness/ diversity (SPR), functional richness/ species guild (FCR), presence/ absence (PAS), population (POP), abundance (ABD), dispersal (DSP), breeding sites (BRD), fitness measure (FIT), interspecific interaction (INT),extinction/ colonisation rate (ECR), microhabitat preference (MHP), comparison between generalist and specialist (CGS), other response varialbes (ORV) (see also Online Appendix 1). Maps are created in VOSviewer

Overall, over half of all selected papers included only amphibians (376 papers; 54%), whilst 276 papers (39%) included only reptiles and 46 papers (7%) assessed both reptiles and amphibians. In relation to species richness, we identified 1490 amphibian species and 1199 reptile species across all papers; among which 141 taxa were not identified to species level but were still included in our analyses as taxonomic units analogous to species (Online Appendix 2). Among these species, more than half of the studied amphibians were found in South America (537; 38%) and North America (328; 23%), followed by Africa (297; 21%), Asia (137; 10%), Europe (77; 5%), and Oceania (51; 3%). Half of the reptile species studied were from North America (302; 25%) and Africa (278; 23%), with the other half consisting of species from Oceania (276; 23%), South America (200; 17%), Europe (76; 6%), and Asia (67; 6%).

When compared to the known species richness in the world, large portions of European species are studied while species from other continents were severely under-represented (Fig.  2 ). The proportions of amphibian species represented in papers were the highest in Europe (73%), while the proportions are much lower for Africa (23%), North America (23%), South America (18%), Oceania (7%) and Asia (6%) (Fig.  2 a). Among reptiles, Europe represents again the highest proportion of studied species (27%), followed by Oceania (18%), Africa (12%), North America (11%) and South America (8.9%) (Fig.  2 b). In contrast, of all Asian reptile species, only a mere 1.73% were included in the selected papers. The species coverage in our selected papers does not seem optimistic. Amphibians and reptiles each have only six families with more than half of the species covered (including three reptilian families containing one species in total). Meanwhile, 23 and 25 families remain fully neglected for amphibians and reptiles respectively (Figs.  4 – 5 ).

Species coverage for each taxonomic family in selected papers of amphibians. The numbers on each row indicate the total number of species known in its respective family (obtained from AmphibiaWeb 2021 )

Species coverage for each taxonomic family in selected papers of reptiles. The numbers on each row indicate the total number of species known in its respective family (obtained from ReptileDatabase)

Multiple correspondence analysis provided important insights into underlying patterns in our data allowing us to visualise the relationship between forms of habitat fragmentation (Median = 1 [1–4]), sampling methods (Median = 1 [0–5]) and response variables (Median = 2 [1–6]). Percentage of variance (or eigenvalues) from MCA output represents the contribution of each dimension in explaining the observed patterns. The top ten new dimensions identified by MCA explained a total of 61.64% and 61.16% of the total variance for amphibians and reptiles respectively. The two dimensions with the highest variance percentages explained were found in the first (Dim1, 12.55%) and second (Dim2, 9.13%) dimensions in amphibians (Online Appendix 3–4). Genetics (sampling method; 13.73%) and population (response variable; 12.39%) contributed the most to Dim1, together with species richness (response variable;10.41%) and dispersal (response variable; 9.20%). For Dim2, experimental (sampling method; 14.38%) was the dominant variable, the rest was determined by GIS/Satellite images (sampling method; 9.71%), fitness measure (response variable; 9.12%) and urbanisation (form of fragmentation; 8.94%). For reptiles, the two dimensions explaining the most variation were the first (Dim1, 11.34%) and second (Dim2, 8.28%) dimensions (Online Appendix 3–4). The variables contributing the most to Dim1 were species richness (response variable; 15.51%), abundance (response variable; 10.11%), presence/absence (response variable; 6.97%) and genetics (sampling method; 6.39%). On the other hand, Dim2 was determined by interspecific interaction (response variable; 13.49%), genetics (12.79%), experimental methods (sampling method; 11.21%) and fitness measure (response variable; 10.94%). The contribution of each category to the definition of the dimensions is reported in Online Appendix 3. The categories identified in the MCA dimensions are subsequently used for building the distance matrix in the clustering analysis.

The HCPC analysis identified three clusters of variables for amphibians and reptiles (Online Appendix 5–6). The output of the HCPC analysis is reported in Online Appendix 7. V test represent the influence of variables in the cluster composition. In general, three clusters for both amphibians and reptiles appeared to be uniquely similar by definition of categories (Fig.  6 ). For amphibians, cluster 1 was defined by studies on species richness (p < 0.05, V test = 14.30) and presence/absence (p < 0.05, V test = 13.42), while cluster 2 was determined by experimental studies (p < 0.05, V test = 10.95) and fitness measures (p < 0.05, V test = 9.77). Cluster 3 was defined by genetics (p < 0.05, V test = 18.44) and population studies (p < 0.05, V test = 17.73) (Online Appendix 7). Abundance and functional richness were also unique to cluster 1; other response variables and direct tracking methods were important to cluster 2 and dispersal was present in cluster 3 even though these variables are less expressed (Fig.  6 a).

Percentage contribution of the categories contributing to the uniqueness of each cluster in amphibians (Dark green = 1, Bright green = 2, Bright yellow = 3) and reptiles (Dark red = 1, Orange = 2, Dark yellow = 3) based on the Cla/Mod results of HCPC (see Online Appendix 7). Abbreviations for the categories can be found in Fig.  3 and in Online Appendix 1

For reptiles, cluster 1 was represented by species richness (p < 0.05, V test = 14.26), abundance (p < 0.05, V test = 11.22) and presence absence (p < 0.05, V test = 8.55) papers, whereas cluster 2 was determined by papers on fitness measures (p < 0.05, V test = 10.99), direct tracking methods (p < 0.05, V test = 8.64) and interspecific interaction (p < 0.05, V test = 7.86), and cluster 3 was defined by genetics (p < 0.05, V test = 12.79), population (p < 0.05, V test = 9.95) and prediction/simulation models (p < 0.05, V test = 7.68) papers (Online Appendix 7). Although slightly less expressed in the clusters, papers using comparisons between generalist and specialist species and papers on functional richness were also unique to cluster 1; experimental methods and other response variables were heavily present in cluster 2, while dispersal studies were distinct to cluster 3 (Fig.  6 b).

Results from VOSviewer categories of both amphibians and reptiles appear to be similar to each other (Fig.  3 c, d). The clustering of the categories in the co-occurrence network maps confirms what we observed in the HCPC results (Fig.  6 ). In addition to geographical representation of study locations in (1), the corresponding clusters of selected papers are also mapped in Figs.  7 and 8 to investigate the spatial grouping patterns for the three clusters (see Online Appendix 8–9 for geographical representation for each category). We also plotted the temporal trend in Online Appendix 10 and 11. Overall, the three clusters are distributed homogeneously across the globe, but concentrated in the USA, Europe and south eastern Australia. Cluster 1 papers were found to be the most predominant cluster in amphibians (57% papers) across all continents (see Online Appendix 12; Fig.  7 ). When compared to other clusters, studies from this cluster are often conducted in Afrotropics, particularly Madagascar (100% papers), central (Costa Rica (60% papers) and Mexico (92% papers) and south America (80% papers) (Online Appendix 12, Figs.  7 , 8 ). On the other hand, cluster 2 papers appear to be more prevalent for reptile studies compared to amphibian studies, with a higher number of studies conducted across North America (65 to 51) and Australia (22 to 2) (Figs.  7 , 8 ). Lastly, a vast majority of cluster 3 papers were located in North America and Europe (both contributing to 79% of the papers) for amphibians and North America and Australia (both contributing to 84% of the papers) for reptiles (Online Appendix 12, Figs.  7 , 8 ). Publications from this cluster started to gain popularity from 2005 onwards, following similar increasing trends as cluster 2 (Online Appendix 10–11). Overall, except for cluster 1 in South America, most of the clusters in Asia and Africa appear to experience very little or no increase in publications over the years (Online Appendix 10–11).

Map of the individual selected papers belonging to each cluster groups (Dark green = 1, Bright green = 2, Bright yellow = 3) for amphibians, with each circle representing the study location. The colour scale of the continents ranging from 0 to 0.9 indicates the proportions of amphibian species represented in the reviewed papers when compared to known species in the world (obtained from AmphibiaWeb)

Map of the individual selected papers belonging to each cluster groups (Dark red=1, Orange=2, Dark yellow=3) for reptiles, with each circle representing the study location. The colour scale of the continents ranging from 0.0 – 0.9 indicates the proportions of reptile species represented in the reviewed papers when compared to known species in the world (obtained from ReptileDatabase).

Our review found no improvement in the geographical and taxonomic bias in habitat fragmentation studies for both reptiles and amphibians compared to earlier studies (Fardila et al. 2017 ). Yet, our study has made an effective contribution towards identifying major spatial gaps in habitat fragmentation studies over the past three decades (updating reviews such as Cushman 2006 ; Gardner et al. 2007 )). In particular, we found an overall increase in the number of studies measuring species richness and abundance throughout the years while population-level and genetics studies are still lacking in developing countries. Here, we discuss the issues of (1) biogeographical bias, (2) the extent and focus of habitat fragmentation research and (3) the limitations and knowledge gaps in habitat fragmentation research in herpetology and provide recommendations for future research.

Biogeographical bias

Geographic bias in research papers.

Given the research effort in relatively wealthy countries (Holmgren and Schnitzer 2004 ; Fazey et al. 2005 ) it is not surprising that more than half the papers concern North America and Europe, where there is strong prevalence of herpetological research. This pattern is also evident in other taxonomic groups and biological areas including invasion biology (Pyšek et al. 2008 ), biodiversity conservation (Trimble and Aarde 2012 ; Christie et al. 2020 ), and habitat fragmentation (Fardila et al. 2017 ). The USA alone contributed more than a third of the publications in terms of both authors and location of study (Fazey et al. 2005 ; Melles et al. 2019 ). English speaking countries including the USA, the United Kingdom, and Australia have dominated research output over the last 30 years (Melles et al. 2019 ). These patterns were reflected in the collaboration network maps generated by VOSviewer (Fig.  3 ). Similar hotspots found between who does the research (Fig.  3 ) and the study locations (Fig.  2 ) suggest that authors tend not to move much and only to study ecosystems near to where they are based (Meyer et al. 2015 ). One reason for this bias is the distance to field sites accentuated by the costs and time of travelling.

However, the near absence of studies from many parts of the world that are currently under extreme pressures of habitat loss and degradation are of great concern (Habel et al. 2019 ). We feel that the level of threat associated with habitat fragmentation in these continents is not proportional to the level of research attention required. Naturally biodiverse but less economically developed Southeast Asian and sub-Saharan countries will suffer greatest diversity losses in the next century (Newbold et al. 2015 ). If this persists at the current rate, biodiverse areas will likely disappear before new discoveries in those hotspots are made (Moura and Jetz 2021 ). Although conversely our study found that among other developing countries Brazil is currently conducting relatively more in-country amphibian studies and collaboration with other countries. However, how much of this information reaches decision makers and practitioners remains unknown. This is largely due to the lack of intermediary evidence bridges (Kadykalo et al. 2021 ). These intermediaries identify evidence summaries based on research and priorities and distribute them to practitioners, facilitating exchange of knowledge between and among researchers and practitioners (Holderegger et al. 2019 ; Kadykalo et al. 2021 ).

Geographic bias in focal groups

Congruent to results reported in Gardner et al. ( 2007 ), studies on amphibians were more abundant than studies on reptiles. Over the past years, there has been a strong focus on amphibian population declines. This was catalysed by the emergence of chytridiomycosis and global decline of amphibians (Fisher and Garner 2020 ). Amphibians, and predominantly frogs, are the principal focus of herpetological research, with the highest allocation of resources and the highest publication rates (Ferronato 2019 ). Another reason for this bias may be that amphibians serve as valuable indicators of environmental stress and degradation owing to their aquatic and terrestrial lifestyle and permeable skin (Green 2003 ). These attributes make them extremely sensitive to changes in temperature and precipitation as well as pollution (Sodhi et al. 2008 ). Lizards, also susceptible to temperature changes, however, are characterised by a high degree of endemism, restricted geographic ranges, late maturity, a long life-span and are thus very susceptible to population declines (Todd et al. 2010 ; Meiri et al. 2018 ). Certain groups of reptiles, such as worm lizards and blind snakes lead cryptic and solitary lives in contrast to the large breeding aggregations and choruses of, for example, frogs. Such characteristics make them difficult for researchers to study as they require large amount of search effort for little data (Thompson 2013 ).

• Taxonomic bias

We found a heightened geographical bias in the taxonomic coverage of studies. Given the sheer number of selected papers investigated, it is not surprising that the continents of North and South America cover more than half of the amphibian species studied whereas North America and Africa cover almost half of the reptile species studied. This trend broadly mirrors the geographic distribution pattern of the global described species in both these taxa (AmphibiaWeb 2021 ; Uetz et al. 2021 ). While a large proportion of the known European and North American families such as Alytidae and Ambystomatidae have been investigated (Fig.  4 ), species from other continents remain severely under-represented. Yet, the European continent represents only 2% of the described species globally. This high research intensity bias in low biodiverse regions of the world has been noted previously (Fazey et al. 2005 ). In general, reptiles and amphibians have been disproportionately poorly studied in the tropics and in developing areas despite that these areas show among the highest rates of deforestation and a corresponding rise in the number of threatened species (Böhm et al. 2013 ; Deikumah et al. 2014 ). These biodiverse areas largely consist of threatened species having restricted home ranges (Meiri et al. 2018 ). Even though we observed a great fraction of the species investigated in the Afrotropics (Vallan 2002 ; Hillers et al. 2008 ; Ofori‐Boateng et al. 2013 ; Riemann et al. 2015 ; Blumgart et al. 2017 ), especially Madagascar (see Mantellidae and Opluridae in Fig.  4 ), it seems insufficient when considering that an estimated 3.94 million hectares of forest area of the continent was cleared yearly over the last century (FAO and UNEP 2020 ). Further, biodiverse hotspots such as the neotropical regions and Indo-Malayan tropics have the highest chances of new species of amphibians and reptiles being discovered (Moura and Jetz 2021 ).

Being herpetofauna diversity hotspots, countries in South America and Asia are indeed understudied. Although Brazil has a high number of amphibian studies, less than one percent of known reptile species was studied in both continents (Fig.  2 ). A number of factors contribute to this lack of representation. First, there is an overwhelming number of new species being discovered every year in these hotspots (Moura et al. 2018 ; Moura and Jetz 2021 ). Furthermore, newly discovered species tend to belong to more secretive groups such as burrowing snakes, worm lizards and caecilians (Colli et al. 2016 ). Yet, these fossorial organisms are clearly neglected in fragmentation studies (see Fig.  4 – 5 ) with researchers focusing on well-known taxonomic groups (Böhm et al. 2013 ). On a positive note, despite having the country (Australia) with the highest reptile diversity (Uetz et al. 2021 ), Oceania represented a fair coverage of reptile diversity compared to other continents. Since 2001, there has been an increase of fragmentation studies in Australia (e.g., Brown 2001 ; Mac Nally and Brown 2001 ; Hazell et al. 2001 ) and there is a continuing increase in research output (Melles et al. 2019 ), contributing 85 out of 89 reviewed studies in Oceania over the last 30 years.

Extent and focus of research

Our findings showed important associations between methods and response metrics but not different forms of habitat fragmentation. This either suggests that researchers were not favouring any sampling method and response variable for evaluating the effects of certain habitat fragmentation or this pattern may occur due to a relatively even split of papers dealing with different forms or combinations of habitat fragmentation in the clusters. In general, species richness or diversity appears to explain most of the variation in our data ( see Online Appendix 4 ). While species richness remains a popular diversity metric employed in conservation biology (Online Appendix 12; also see Gardner et al. 2007 ), we also found an increasing trend in the use of genetic techniques for habitat fragmentation studies. More specifically in recent years, molecular genetics have become popular and are often studied together with population connectivity to capture species responses to habitat fragmentation ( see Online Appendix 4 ) (Keyghobadi 2007 ). The HCPC approach identified three main clusters of research fields which will be referred to as research agendas from here onwards. Contrary to our expectation, we did not find a global spatial pattern of research agendas, but instead a rather homogeneous distribution of papers, possibly due to the lack of selected studies which are found in developing countries outside USA, Europe and Australia (Figs.  7 , 8 ). This nevertheless indicates that different sampling methods are shared and used between leading herpetological experts from different countries and that there are continuing collaborations between countries, particularly in North America and Europe.

Below, we describe the research agendas and their corresponding categories (Fig.  6 ) that have contributed significantly to the study of habitat fragmentation for the past 30 years: (a) Agenda 1: Measures of direct individual species responses, (b) Agenda 2: Physiological and movement ecology, and (c) Agenda 3: Technology advancement in conservation research.

Agenda 1: Measures of direct individual species responses

We found that the majority of studies around the globe evaluated patterns of assemblage richness, species presence/absence, and abundance (Figs.  7 , 8 ). These simple patterns of richness, diversity and abundance are the most common responses measured because they provide a good indication of species response to habitat fragmentation and are easy to calculate (Colwell 2009 ). Although species richness does not consider abundance or biomass but treats each species as of equal importance to diversity, species evenness weighs each species by its relative abundance (Hill 1972). Further, composite measures like species diversity indices (e.g., Simpson’s 1/D or Shannon’s H) combine both richness and evenness in a single measure (Colwell 2009 ), preventing biases in results. However, directly measuring these species responses might not be ecologically relevant as they fail to account for patterns in species assemblage turnover. In fact, few selected papers (38 out of 697) in our study have attempted to categorise species into meaningful functional groups or guilds, despite that the categorisation of ecological functions such as habitat preference, taxonomic family, reproductive mode, and body size can be easily done (but see Knutson et al. 1999 ; Peltzer et al. 2006 ; Moreira and Maltchik 2014 ). Knutson et al.( 1999 ) was the first in our selected papers to group species with similar life-history characteristics into guilds and to examine their responses to landscape features. They observed negative associations between urban land use and anuran guilds. Analyses of guilds or functional groups can reveal contradictory results (but not always, see Moreira and Maltchik 2014 ). For example, the species richness of anurans in logged areas of West Africa is found to be as high as in primary habitat (Ernst et al. 2006 ). Yet, analyses of functional groups indicated significantly higher diversity in primary forest communities (Ernst et al. 2006 ). Similar differences were also observed for species with varying degrees of niche overlaps, habitat specialists, and for different continents (Ernst et al. 2006 ; Seshadri 2014 ). These results underline that species richness alone is a poor indicator of the functional value of species in the ecosystem as the relationships between functional diversity and species richness are inconsistent and can sometimes be redundant (functional diversity remains constant if assemblages are functionally similar; Riemann et al. 2017 ; Palmeirim et al. 2017 ; Silva et al. 2022 ). The results of some species richness studies may consequently provide misleading inferences regarding consequences of habitat fragmentation and conservation management (Gardner et al. 2007 ).

Although not substantially greater than the agendas 2 and 3, the measure of individual species responses has always been popular across the globe but also increasingly popular in the tropical and subtropical regions (e.g., South America and Africa; Online Appendix 10–11). For example, a research team led by Mark-Oliver Roedel from Germany has conducted numerous studies on Afrotropical amphibian communities (Hillers et al. 2008 ; Ofori‐Boateng et al. 2013 ; Riemann et al. 2017 ). Due to the higher biodiversity and species rarity in these regions compared to temperate areas, it is reasonable to expect a greater level of sampling effort in patterns of species richness, abundance, and guild assemblage to obtain comparisons of diversity with sufficient statistical power across different land use changes (Gardner et al. 2007 ). Access to highly specific expertise and most up to date methods and technology may not be available in these regions, and as such, study designs are limited to multispecies survey addressing simple patterns of diversity and species assemblages (Hetu et al. 2019 ). Unfortunately at the same time, these forest biomes holding the highest richness and abundance of amphibians and reptiles have showed consistent negative responses to land use changes (Cordier et al. 2021 ).

Agenda 2: physiological and movement ecology

We did not observe a strong association between occupancy and dispersal in our study. Perhaps this is because only a few papers investigated dispersal via habitat occupancy compared to the overwhelming proportions of papers examining the presence of species in response to habitat fragmentation in research agenda 1. Similarly, few studies measure dispersal with direct tracking methods, with the majority that discussed dispersal being based on indirect inferences, such as genetic divergence (see Fig.  3 c, d; Driscoll et al. 2014 ). Genetic approaches can be effective in situations where more direct approaches are not possible (Lowe and Allendorf 2010 ). For instance, using microsatellites and mitochondrial DNA, Buckland et al. ( 2014 ) found no migration occurring between isolated subpopulations of a forest day gecko ( Phelsuma guimbeaui ) in a fragmented forest and predicted a dramatic decrease in survival and allelic diversity in the next 50 years if no migration occurs (Buckland et al. 2014 ). In some cases, molecular markers also allow direct dispersal studies by assigning individuals to their parents or population of origin (Manel et al. 2005 ). However, there are limitations on when these techniques can be applied. Assignment tests require appropriate choices of molecular markers and sampling design to permit quantification of indices of dispersal (Broquet and Petit 2009 ; Lowe and Allendorf 2010 ). Parent–offspring analysis is constrained by the uncertainty in assessing whether offspring dispersal is completed at the time of sampling and sample size (Broquet and Petit 2009 ). Genetic tools may thus be best applied in combination with direct approaches because they contain complementary information (Lowe and Allendorf 2010 ; Safner et al. 2011 ; Smith et al. 2016 ).

Traditional approaches in habitat fragmentation research like radiotracking or capture-mark-recapture of animals can be effective in evaluating dispersal and ecological connectivity between populations. For example, based on mark-recapture data over a nine year period, facultative dispersal rates in an endangered amphibian ( Bombina variegata ) were found to be sex biased and relatively low from resulting patch loss (Cayuela et al. 2018 ). In our case, direct tracking methods are more commonly and effectively used in examining the impacts of habitat modification on changes in ecology directly relating to fitness (Fig.  6 ): home ranges (Price-Rees et al. 2013 ), foraging grounds (MacDonald et al. 2012 ) and survival rates (Breininger et al. 2012 ). Yet, such routine movements associated with resource exploitation do not reflect the biological reality and evolutionary consequences of how organisms change as landscape changes (Van Dyck and Baguette 2005 ). Instead, directed behavioural movements affecting dispersal processes (emigration, displacement or immigration) are crucial in determining the functional connectivity between populations in a fragmented landscape (Bonte et al. 2012 ). In one study, spotted salamanders Ambystoma maculatum tracked with fluorescent powder exhibited strong edge mediated behaviour when dispersing across borders between forest and field habitats and can perceive forest habitats from some distance (Pittman and Semlitsch 2013 ). Knowing such behaviour rules can improve predictions of the effects of habitat configuration on survival and dispersal. However, ongoing conversion of natural ecosystems to human modified land cover increases the need to consider various cover types that may be permeable to animal movements. As such, experimental approaches can be effective in examining the effect of matrix type on species movements as seen in our results (Fig.  6 ) (Rothermel and Semlitsch 2002 ; Mazerolle and Desrochers 2005 ). For example, researchers conducted experimental releases of post-metamorphic individuals of forest amphibians into different substrates and mapped the movements of paths and performance (Cline and Hunter Jr 2016 ). They showed that non-forest matrices with lower structural complexity influence the ability of frogs to travel across open cover and to orient themselves towards the forest from distances greater than 40–55 m. Therefore, it is inaccurate to assume matrix permeability to be uniform across all open-matrix types, particularly in amphibians (Cline and Hunter 2014 , 2016 ).

In addition, the ability to move and disperse is highly dependent on the range of external environments and internal physiological limits (Bonte et al. 2012 ), especially in reptiles and amphibians (Nowakowski et al. 2017 ). The study of physiological effects on movement was seen throughout our selected studies (Fig.  6 ). For example, higher temperatures and lower soil moisture in open habitats could increase evaporative water loss in salamanders (Rothermel and Semlitsch 2002 ). Other tests including interaction effects between landscape configuration and physiological constraints (e.g., dehydration rate Rothermel and Semlitsch 2002 ; Watling and Braga 2015 ); body size (Doherty et al. 2019 ) can be useful to better understand fitness and population persistence. We argue here that multidisciplinary projects examining movement physiology, behaviour and environmental constraints in addition to measuring distance moved are needed to progress this field.

Our results indicate a high bias of agenda 2 papers represented among developed countries, with a strong focus on reptiles compared to amphibians (Price-Rees et al. 2013 ; Doherty et al. 2019 ) (Online Appendix 12, Figs.  7 , 8 ). The adoption of direct tracking as well as genetic methods can be cost prohibitive in developing and poorer regions. However, cheaper and simpler methods to track individuals are increasing (Mennill et al. 2012 ; Cline and Hunter 2014 , 2016 ). Although existing application might not be ideal for reptiles and amphibians, new technologies for tagging and tracking small vertebrates are being developed including acoustic surveys and improved genetic methods (Broquet and Petit 2009 ; Mennill et al. 2012 ; Marques et al. 2013 ). While there are many improvements needed to obtain better quality dispersal data studies on movement ecology, reptiles and amphibians still only account for a mere 2.2% of the studies on dispersal when compared to plants and invertebrates which comprised over half of the studies based on a systematic review (Driscoll et al. 2014 ). Thus, we urge more studies to be conducted on these lesser-known taxa, especially in biodiverse regions. Given the limited dispersal in amphibians and reptiles, having a deeper understanding on their dispersal can be critical for the effective management and conservation of populations and metapopulations (Smith and Green 2005 ).

Agenda 3: technology advancement in conservation research

While community level approaches such as responses in species richness, occupancy, and abundance measure biodiversity response to habitat fragmentation, they are limited in inference because they do not reflect patterns of fitness across environmental gradients and landscape patterns. Instead, genetic structure at the population level can offer a higher resolution of species responses (Manel and Holderegger 2013 ). For instance, genetic erosion heavily affects the rate of species loss in many amphibian species (Allentoft and O’Brien 2010 ; Rivera‐Ortíz et al. 2015 ). Over the past decades we have seen a rapid increase in studies applying genetic analysis to assess the effects of habitat fragmentation (Keyghobadi 2007 ), reflecting the strength of these approaches. This growth is mostly evident in North America and Europe (but also Oceania for reptiles) ( Online Appendix 10–11). The availability of different genetic markers has been increasing, from microsatellites in the 1990s then shifting towards genotyping by sequencing (NGS) technologies that enable rapid genome-wide development (Allendorf et al. 2010 ; Monteiro et al. 2019 ). However, the study of population structure alone can lead to misleading results as environmental changes to species dynamics are not considered. The resistance imposed by landscape features on the dispersal of animals can ultimately shape gene flow and genetic structure (Bani et al. 2015 ; Pilliod et al. 2015 ; Monteiro et al. 2019 ).

To understand this, researchers combine genetic, land cover and climate variables to study the gene flow patterns across heterogeneous and fragmented landscapes (Manel and Holderegger 2013 ). Spatial analyses can be a powerful tool for monitoring biodiversity by quantifying environmental and landscape parameters. The growing interest in both landcover data and the rapid development of computer processing power prompted the development of new prediction methods, primarily in spatial models (Ray et al. 2002 ), ecological niche modelling (Urbina-Cardona and Loyola 2008 ; Tan et al. 2021 ), and landscape connectivity (Cushman et al. 2013 ; Ashrafzadeh et al. 2019 ). In some cases, niche models are useful in assessing the effectiveness of protected areas for endangered species (Urbina-Cardona and Loyola 2008 ; Tan et al. 2021 ).

The integration of genetic data in ecological niche models for recognising possible dispersal movements between populations were observed in our study (Fig.  3 c, d), especially in reptiles (Fig.  6b ). The hallmark of landscape genetics is the ability to estimate functional connectivity among populations and offer empirical approach of adaptive genetic variation in real landscapes to detect environmental factors driving evolutionary adaptation. The most common approach of landscape genetics is determining whether effective distances as determined by the presence of suitable habitat between populations, better predict genetic distances than do Euclidean distances (assuming spatially homogeneous landscape). However, straight-line geographic distance does not normally reflect true patterns of dispersal as landscape barriers or facilitators in a heterogeneous landscape could strongly affect gene flow (Emel and Storfer 2012 ; Fenderson et al. 2020 ). Therefore, in these cases, ecological distances or landscape resistance can often explain a greater deal of genetic variation between fragmented populations (Cushman 2006 ; Bani et al. 2015 ). Using a combination of habitat suitability modelling (e.g., Maxent, Phillips et al. 2017 ), multiple least-cost paths (LCPs) (Adriaensen et al. 2003 ) and the more recent circuit theory analysis (McRae et al. 2008 ) to investigate landscape resistance can be highly effective predicting potential pathways along which dispersal may occur, hence informing conservation management (Emel and Storfer 2012 ; Bani et al. 2015 ; Pilliod et al. 2015 ). To date, landscape genetics has been shown to be particularly useful in studying organisms with complex life histories (Emel and Storfer 2012 ; Shaffer et al. 2015 ). Yet, the applications of landscape genetics have been limited to contemporary patterns using modern genetic data. Few studies have benefitted from the inclusion of temporal genetic data (Fenderson et al. 2020 ). For example, historical DNA samples and heterochronous analyses could allow us to explore how anthropogenic impacts have affected past genetic diversity and population dynamics (Pacioni et al. 2015 ) and identify areas of future suitability of endangered animals in face of climate change (Nogués-Bravo et al. 2016 ). The possibility to investigate migration through spatiotemporal population connectivity can greatly improve the prediction of species responses under future landscape and climate change scenarios (Fenderson et al. 2020 ).

Population genetic and niche modelling studies for both taxa are rarely found in developing regions of the world, especially in Asia and Africa (Figs.  7 , 8 ). Even though conservation priorities are concentrated in these biodiverse regions, invaluable highly specific expertise such as conservation genetics and other contemporary methodologies might not be readily available due to lack of funding and infrastructure (Hetu et al. 2019 ). Thus, we encourage collaborations with the poorer countries initiated by foreign service providers from developed countries. Contrary to expectations, very few studies on conservation genetics were found in China and Japan despite their vast advances in genetic techniques. Fortunately, China has made substantial progress in the last 20 years in understanding human genetic history and interpreting genetic studies of human diseases (Forero et al. 2016 ) as well as biodiversity conservation (Wang et al. 2020 ), yet the same cannot be said for conservation genetics on reptiles and amphibians (Figs.  7 , 8 ), but see Fan et al. ( 2018 ) and Hu et al. ( 2021 ).

Limitations and knowledge gaps

The forms of habitat fragmentation which we categorised may not reflect the ecological impact in the real world as interactions between different habitat fragmentation forms were not accounted for. Although each of these forms of habitat fragmentation possesses serious environmental consequences, their combination could have severe synergistic impacts (Blaustein and Kiesecker 2002 ). For example, a fragmented landscape is not just reduced and isolated, but subject to other anthropogenic disturbances such as hunting, fire, invasive species, and pollution (Laurance and Useche 2009 ; Lazzari et al. 2022 ). Altered climatic conditions and emerging pathogens such as batrachochytrids can also interact with each other, and other threats (Fisher and Garner 2020 ). The use of habitat suitability models based on climatic scenarios, combined with hydrological and urbanisation models, are effective in detecting best to worst case scenarios and local extinctions, as shown for the spotted marsh frog ( Limnodynastes tasmaniensis ) (Wilson et al. 2013 ).

We acknowledge the bias of scientific research introduced from the limitation of search term to English-speaking literature on the geographic distribution of the papers we sampled (Konno et al. 2020 ; Angulo et al. 2021 ). In Latin American journals for example, we found a number of papers published in Spanish, but unfortunately, they did not fit the criteria of our selection (see Online Appendix 2). Conservation studies written in languages other than English are often published in local journals which do not normally go through international peer review.

The homogeneous distribution of the research agendas across geographical regions in our study may be explained by the lack of studies found in South America, Asia and Africa, preventing us to see a potentially dichotomous spatial pattern among the clusters. However, this reflects the current state of research and the challenges faced in less developed countries.

(4) Our study did not investigate whether habitat fragmentation has led to an improved or decreased biotic response. Predicting species response to habitat modification has been reviewed countless times (Rytwinski and Fahrig 2012 ; Driscoll et al. 2014 ; Doherty et al. 2020 ; Newbold et al. 2020 ; Cordier et al. 2021 ). Yet, these reviews often yield little or no general patterns (Doherty et al. 2020 ; Cordier et al. 2021 ). Response variables or traits measured are often found to be poor predictors of the impacts of habitat fragmentation. There are two possible explanations for this discrepancy. First, the strength and direction of the responses differs between species, ecophysiological groups (Rothermel and Semlitsch 2002 ), and phylogenetic or functional groups (Mazerolle and Desrochers 2005 ; Nowakowski et al. 2017 ). Second, responses in animals to different types of disturbance may be specific to the ecosystem where they live. Different biogeographic regions or biomes have different characteristics affecting local species (Lindell et al. 2007 ; Blowes et al. 2019 ; Newbold et al. 2020 ; Cordier et al. 2021 ).

Conclusions and recommendations

Our results underline promising research fields and geographic areas and may serve as a guideline or starting point for future habitat fragmentation studies. We suspect similar paradigms of geographic and thematic patterns to occur in other taxonomic groups.

Although studies dealing with habitat fragmentation impacts on mammals and birds are already widely recognised (Fardila et al. 2017 ), research on reptiles and amphibians has been lacking. We argue that amphibians and reptiles need more attention as they are equally or more threatened but highly neglected (Rytwinski and Fahrig 2012 ; Ferronato 2019 ; Cox et al. 2022 ).

Greater investment is required for studies in tropical and subtropical areas (Segovia et al. 2020 ), especially within the Asian continent. These areas are currently experiencing the highest rates of habitat loss (McDonald et al. 2013 ). Tropical specialists are further restricted to smaller geographic range sizes according to Rapoport’s rule which states that there is a positive latitudinal correlation with range size (Stevens 1989 ) (at least for amphibians in the Northern hemisphere where there is higher temperature and precipitation seasonality; Whitton et al. 2012 ). Having a small range size is often associated with negative responses to habitat modification (Doherty et al. 2020 ). Thus, more effort is needed in developing countries where the crisis is greatest and there is lack of funding and strong language barriers (Fazey et al. 2005 ). There is an urgent need to better integrate studies published in languages other than English with the broader international literature. Useful integration actions include training of local conservation biologists and promoting partnerships and research visits in these regions may have greater conservation consequences to understand global patterns of habitat modification (Meyer et al. 2015 ). Doing so will help remediate the sampling bias towards temperate generalists and will shed light on the fate of tropical specialists.

We encourage improved access to intermediary evidence-based conservation data (Kadykalo et al. 2021 ). Even when well-established genetic and genomic analyses have been proven to be promising area in herpetological conservation (Shaffer et al. 2015 ), there is a general lack of the transfer of knowledge between scientists and practitioners (Holderegger et al. 2019 ). As practitioners are generally interested in species monitoring and the evaluation of success of connectivity measures, an establishment of scientist-practitioner community to facilitate a platform for international exchange would help tremendously in future conservation planning and management (Holderegger et al. 2019 ).

Although different study designs and landscape measures have different strengths and limitations depending on the study objectives, we suggest reporting basic data to describe the effect of habitat fragmentation using standardised sampling methods, indices, and design (Holderegger et al. 2019 ). The results will allow future meta-analyses to be performed.

Incorporate remote sensing data, whenever possible, in studies involving habitat change and fragmentation. The use of niche modelling techniques combined with high resolution remote sensing has been instrumental in detecting potentially fragmented populations. With advances in landscape genomics, we are now able to examine the correlation between environmental factors and genomic data in natural populations (Manel and Holderegger 2013 ; Shaffer et al. 2015 ). Adopting such tools would be valuable in understanding how habitat amounts and configurations affect dispersal, survival, and population dynamics as well as the impacts of anthropogenic changes such as climate change (Shaffer et al. 2015 ).

Data availability

The datasets generated during the current study are available in Online Appendix 1. Codes used in the analyses are available from corresponding author on request.

Adriaensen F, Chardon JP, De Blust G et al (2003) The application of ‘least-cost’ modelling as a functional landscape model. Landsc Urban Plan 64:233–247. https://doi.org/10.1016/S0169-2046(02)00242-6

Article   Google Scholar  

Allendorf FW, Hohenlohe PA, Luikart G (2010) Genomics and the future of conservation genetics. Nat Rev Genet 11:697–709. https://doi.org/10.1038/nrg2844

Article   CAS   Google Scholar  

Allentoft ME, O’Brien J (2010) Global amphibian declines, loss of genetic diversity and fitness: a review. Diversity 2:47–71. https://doi.org/10.3390/d2010047

AmphibiaWeb (2021) AmphibiaWeb. https://amphibiaweb.org/ . Accessed 22 Feb 2021

Angulo E, Diagne C, Ballesteros-Mejia L et al (2021) Non-English languages enrich scientific knowledge: the example of economic costs of biological invasions. Sci Total Environ 775:144441. https://doi.org/10.1016/j.scitotenv.2020.144441

Ashrafzadeh MR, Naghipour AA, Haidarian M et al (2019) Effects of climate change on habitat and connectivity for populations of a vulnerable, endemic salamander in Iran. Glob Ecol Conserv 19:e00637. https://doi.org/10.1016/j.gecco.2019.e00637

Bani L, Pisa G, Luppi M et al (2015) Ecological connectivity assessment in a strongly structured fire salamander ( Salamandra salamandra ) population. Ecol Evol 5:3472–3485. https://doi.org/10.1002/ece3.1617

Barber PH, Ablan-Lagman MCA, Ambariyanto, et al (2014) Advancing biodiversity research in developing countries: the need for changing paradigms. Bull Mar Sci 90:187–210. https://doi.org/10.5343/bms.2012.1108

Barlow J, França F, Gardner TA et al (2018) The future of hyperdiverse tropical ecosystems. Nature 559:517–526. https://doi.org/10.1038/s41586-018-0301-1

Blaustein AR, Kiesecker JM (2002) Complexity in conservation: lessons from the global decline of amphibian populations. Ecol Lett 5:597–608. https://doi.org/10.1046/j.1461-0248.2002.00352.x

Blowes SA, Supp SR, Antão LH et al (2019) The geography of biodiversity change in marine and terrestrial assemblages. Science 366:339–345. https://doi.org/10.1126/science.aaw1620

Blumgart D, Dolhem J, Raxworthy CJ (2017) Herpetological diversity across intact and modified habitats of Nosy Komba Island, Madagascar. J Nat Hist 51:625–642. https://doi.org/10.1080/00222933.2017.1287312

Böhm M, Collen B, Baillie JEM et al (2013) The conservation status of the world’s reptiles. Biol Conserv 157:372–385. https://doi.org/10.1016/j.biocon.2012.07.015

Bonte D, Van Dyck H, Bullock JM et al (2012) Costs of dispersal. Biol Rev 87:290–312. https://doi.org/10.1111/j.1469-185X.2011.00201.x

Breininger DR, Mazerolle MJ, Bolt MR et al (2012) Habitat fragmentation effects on annual survival of the federally protected eastern indigo snake: indigo snake survival. Anim Conserv 15:361–368. https://doi.org/10.1111/j.1469-1795.2012.00524.x

Broquet T, Petit EJ (2009) Molecular estimation of dispersal for ecology and population genetics. Annu Rev Ecol Evol Syst 40:193–216. https://doi.org/10.1146/annurev.ecolsys.110308.120324

Brown GW (2001) The influence of habitat disturbance on reptiles in a Box-Ironbark eucalypt forest of south-eastern Australia. Biodivers Conserv 10:161–176. https://doi.org/10.1023/A:1008919521638

Buckland S, Cole NC, Groombridge JJ et al (2014) High risks of losing genetic diversity in an endemic Mauritian gecko : implications for conservation. PLoS ONE 9:e93387. https://doi.org/10.1371/journal.pone.0093387

Cayuela H, Besnard A, Quay L et al (2018) Demographic response to patch destruction in a spatially structured amphibian population. J Appl Ecol 55:2204–2215. https://doi.org/10.1111/1365-2664.13198

Christie AP, Amano T, Martin PA et al (2020) The challenge of biased evidence in conservation. Conserv Biol 35:249–262. https://doi.org/10.1111/cobi.13577

Cline BB, Hunter ML Jr (2014) Different open-canopy vegetation types affect matrix permeability for a dispersing forest amphibian. J Appl Ecol 51:319–329. https://doi.org/10.1111/1365-2664.12197

Cline BB, Hunter ML Jr (2016) Movement in the matrix: substrates and distance-to-forest edge affect postmetamorphic movements of a forest amphibian. Ecosphere 7:e01202. https://doi.org/10.1002/ecs2.1202

Colli GR, Fenker J, Tedeschi LG et al (2016) In the depths of obscurity: Knowledge gaps and extinction risk of Brazilian worm lizards (Squamata, Amphisbaenidae). Biol Conserv 204:51–62. https://doi.org/10.1016/j.biocon.2016.07.033

Colwell R (2009) Biodiversity: concepts, patterns, and measurement. The Princeton guide to ecology. Princeton, Princeton University Press, pp 257–263

Chapter   Google Scholar  

Cordier JM, Aguilar R, Lescano JN et al (2021) A global assessment of amphibian and reptile responses to land-use changes. Biol Conserv 253:108863. https://doi.org/10.1016/j.biocon.2020.108863

Cox N, Young BE, Bowles P et al (2022) A global reptile assessment highlights shared conservation needs of tetrapods. Nature 605:285–290. https://doi.org/10.1038/s41586-022-04664-7

Cushman SA (2006) Effects of habitat loss and fragmentation on amphibians: a review and prospectus. Biol Conserv 128:231–240. https://doi.org/10.1016/j.biocon.2005.09.031

Cushman SA, Shirk AJ, Landguth EL (2013) Landscape genetics and limiting factors. Conserv Genet 14:263–274. https://doi.org/10.1007/s10592-012-0396-0

Deikumah JP, McAlpine CA, Maron M (2014) Biogeographical and taxonomic biases in tropical forest fragmentation research. Conserv Biol J Soc Conserv Biol 28:1522–1531. https://doi.org/10.1111/cobi.12348

Doherty TS, Fist CN, Driscoll DA (2019) Animal movement varies with resource availability, landscape configuration and body size: a conceptual model and empirical example. Landsc Ecol 34:603–614. https://doi.org/10.1007/s10980-019-00795-x

Doherty TS, Balouch S, Bell K et al (2020) Reptile responses to anthropogenic habitat modification: a global meta-analysis. Glob Ecol Biogeogr 29:1265–1279. https://doi.org/10.1111/geb.13091

Driscoll DA, Banks SC, Barton PS et al (2014) The trajectory of dispersal research in conservation biology. Syst Rev PLOS ONE 9:e95053. https://doi.org/10.1371/journal.pone.0095053

Driscoll DA, Armenteras D, Bennett AF et al (2021) How fire interacts with habitat loss and fragmentation. Biol Rev 96:976–998. https://doi.org/10.1111/brv.12687

Emel SL, Storfer A (2012) A decade of amphibian population genetic studies: synthesis and recommendations. Conserv Genet 13:1685–1689. https://doi.org/10.1007/s10592-012-0407-1

Ernst R, Linsenmair KE, Rödel M-O (2006) Diversity erosion beyond the species level: dramatic loss of functional diversity after selective logging in two tropical amphibian communities. Biol Conserv 133:143–155. https://doi.org/10.1016/j.biocon.2006.05.028

Fahrig L (2003) Effects of habitat fragmentation on biodiversity. Annu Rev Ecol Evol Syst 34:487–515. https://doi.org/10.1146/annurev.ecolsys.34.011802.132419

Fahrig L (2017) Ecological responses to habitat fragmentation per se. Annu Rev Ecol Evol Syst 48:1–23. https://doi.org/10.1146/annurev-ecolsys-110316-022612

Fan H, Hu Y, Wu Q et al (2018) Conservation genetics and genomics of threatened vertebrates in China. J Genet Genomics 45:593–601. https://doi.org/10.1016/j.jgg.2018.09.005

FAO and UNEP (2020) The State of the World’s Forests 2020: Forests, biodiversity and people. FAO and UNEP,

Fardila D, Kelly LT, Moore JL, McCarthy MA (2017) A systematic review reveals changes in where and how we have studied habitat loss and fragmentation over 20years. Biol Conserv 212:130–138. https://doi.org/10.1016/j.biocon.2017.04.031

Fazey I, Fischer J, Lindenmayer DB (2005) Who does all the research in conservation biology? Biodivers Conserv 14:917–934. https://doi.org/10.1007/s10531-004-7849-9

Fenderson LE, Kovach AI, Llamas B (2020) Spatiotemporal landscape genetics: investigating ecology and evolution through space and time. Mol Ecol 29:218–246. https://doi.org/10.1111/mec.15315

Ferronato B (2019) An assessment of funding and publication rates in herpetology. Herpetol J. https://doi.org/10.33256/hj29.4.264273

Fisher MC, Garner TWJ (2020) Chytrid fungi and global amphibian declines. Nat Rev Microbiol 18:332–343. https://doi.org/10.1038/s41579-020-0335-x

Forero DA, Wonkam A, Wang W et al (2016) Current needs for human and medical genomics research infrastructure in low and middle income countries. J Med Genet 53:438–440. https://doi.org/10.1136/jmedgenet-2015-103631

Gardner TA, Barlow J, Peres CA (2007) Paradox, presumption and pitfalls in conservation biology: the importance of habitat change for amphibians and reptiles. Biol Conserv 138:166–179. https://doi.org/10.1016/j.biocon.2007.04.017

Gibbons JW, Scott DE, Ryan TJ et al (2000) The Global Decline of Reptiles, Déjà Vu Amphibians: reptile species are declining on a global scale. Six significant threats to reptile populations are habitat loss and degradation, introduced invasive species, environmental pollution, disease, unsustainable use, and global climate change. Bioscience 50:653–666. https://doi.org/10.1641/0006-3568(2000)050[0653:TGDORD]2.0.CO;2

Green DM (2003) The ecology of extinction: population fluctuation and decline in amphibians. Biol Conserv 111:331–343. https://doi.org/10.1016/S0006-3207(02)00302-6

Habel JC, Rasche L, Schneider UA et al (2019) Final countdown for biodiversity hotspots. Conserv Lett 12:e12668. https://doi.org/10.1111/conl.12668

Haddad NM, Brudvig LA, Clobert J et al (2015) Habitat fragmentation and its lasting impact on Earth’s ecosystems. Sci Adv 1:e1500052. https://doi.org/10.1126/sciadv.1500052

Hadley AS, Betts MG (2016) Refocusing habitat fragmentation research using lessons from the last decade. Curr Landsc Ecol Rep 1:55–66. https://doi.org/10.1007/s40823-016-0007-8

Hamer AJ, McDonnell MJ (2008) Amphibian ecology and conservation in the urbanising world: a review. Biol Conserv 141:2432–2449. https://doi.org/10.1016/j.biocon.2008.07.020

Hazell D, Cunnningham R, Lindenmayer D et al (2001) Use of farm dams as frog habitat in an Australian agricultural landscape: factors affecting species richness and distribution. Biol Conserv 102:155–169. https://doi.org/10.1016/S0006-3207(01)00096-9

Hetu M, Koutouki K, Joly Y (2019) Genomics for All: International Open Science Genomics Projects and Capacity Building in the Developing World. Front Genet. https://doi.org/10.3389/fgene.2019.00095

Hillers A, Veith M, Rödel M-O (2008) Effects of forest fragmentation and habitat degradation on West African leaf-litter frogs. Conserv Biol 22:762–772. https://doi.org/10.1111/j.1523-1739.2008.00920.x

Holderegger R, Balkenhol N, Bolliger J et al (2019) Conservation genetics: linking science with practice. Mol Ecol 28:3848–3856. https://doi.org/10.1111/mec.15202

Holmgren M, Schnitzer SA (2004) Science on the rise in developing countries. PLOS Biol 2:e1. https://doi.org/10.1371/journal.pbio.0020001

Hu Y, Fan H, Chen Y et al (2021) Spatial patterns and conservation of genetic and phylogenetic diversity of wildlife in China. Sci Adv 7:eabd5725. https://doi.org/10.1126/sciadv.abd5725

Kadykalo AN, Buxton RT, Morrison P et al (2021) Bridging research and practice in conservation. Conserv Biol 35:1725–1737. https://doi.org/10.1111/cobi.13732

Keyghobadi NK (2007) The genetic implications of habitat fragmentation for animals. Can J Zool. https://doi.org/10.1139/Z07-095

Knutson MG, Sauer JR, Olsen DA et al (1999) Effects of Landscape Composition and Wetland Fragmentation on Frog and Toad Abundance and Species Richness in Iowa and Wisconsin, U.S.A. Conserv Biol 13:1437–1446. https://doi.org/10.1046/j.1523-1739.1999.98445.x

Konno K, Akasaka M, Koshida C et al (2020) Ignoring non-English-language studies may bias ecological meta-analyses. Ecol Evol 10:6373–6384. https://doi.org/10.1002/ece3.6368

Laurance WF, Useche DC (2009) Environmental synergisms and extinctions of tropical species. Conserv Biol 23:1427–1437. https://doi.org/10.1111/j.1523-1739.2009.01336.x

Lazzari J, Sato CF, Driscoll DA (2022) Traits influence reptile responses to fire in a fragmented agricultural landscape. Landsc Ecol 37:2363–2382. https://doi.org/10.1007/s10980-022-01417-9

Lê S, Josse J, Husson F (2008) FactoMineR: an R package for multivariate analysis. J Stat Softw 25:1–18. https://doi.org/10.18637/jss.v025.i01

Lebart L, Morineau A, Piron M (1995) Statistique exploratoire multidimensionnelle. Dunod Paris

Lindell CA, Riffell SK, Kaiser SA et al (2007) Edge responses of tropical and temperate birds. Wilson J Ornithol 119:205–220. https://doi.org/10.1676/05-133.1

Lindenmayer DB, Fischer J (2007) Tackling the habitat fragmentation panchreston. Trends Ecol Evol 22:127–132. https://doi.org/10.1016/j.tree.2006.11.006

Lowe WH, Allendorf FW (2010) What can genetics tell us about population connectivity? Mol Ecol 19:3038–3051. https://doi.org/10.1111/j.1365-294X.2010.04688.x

Mac Nally R, Brown GW (2001) Reptiles and habitat fragmentation in the box-ironbark forests of central Victoria, Australia: predictions, compositional change and faunal nestedness. Oecologia 128:116–125. https://doi.org/10.1007/s004420100632

MacDonald B, Lewison R, Madrak S et al (2012) Home ranges of East Pacific green turtles Chelonia mydas in a highly urbanized temperate foraging ground. Mar Ecol Prog Ser 461:211–221. https://doi.org/10.3354/meps09820

Manel S, Gaggiotti OE, Waples RS (2005) Assignment methods: matching biological questions with appropriate techniques. Trends Ecol Evol 20:136–142. https://doi.org/10.1016/j.tree.2004.12.004

Manel S, Holderegger R (2013) Ten years of landscape genetics. Trends Ecol Evol 28:614–621. https://doi.org/10.1016/j.tree.2013.05.012

Marques TA, Thomas L, Martin SW et al (2013) Estimating animal population density using passive acoustics. Biol Rev 88:287–309. https://doi.org/10.1111/brv.12001

Mazerolle MJ, Desrochers A (2005) Landscape resistance to frog movements. Can J Zool. https://doi.org/10.1139/z05-032

McDonald RI, Marcotullio PJ, Güneralp B (2013) Urbanization and Global Trends in Biodiversity and Ecosystem Services. In: Elmqvist T, Fragkias M, Goodness J et al (eds) Urbanization, Biodiversity and Ecosystem Services: Challenges and Opportunities: A Global Assessment. Springer, Dordrecht, pp 31–52

Google Scholar  

McRae BH, Dickson BG, Keitt TH, Shah VB (2008) Using circuit theory to model connectivity in ecology, evolution, and conservation. Ecology 89:2712–2724. https://doi.org/10.1890/07-1861.1

Meiri S, Bauer AM, Allison A et al (2018) Extinct, obscure or imaginary: the lizard species with the smallest ranges. Divers Distrib 24:262–273. https://doi.org/10.1111/ddi.12678

Melles SJ, Scarpone C, Julien A et al (2019) Diversity of practitioners publishing in five leading international journals of applied ecology and conservation biology, 1987–2015 relative to global biodiversity hotspots. Écoscience 26:323–340. https://doi.org/10.1080/11956860.2019.1645565

Mennill DJ, Battiston M, Wilson DR et al (2012) Field test of an affordable, portable, wireless microphone array for spatial monitoring of animal ecology and behaviour. Methods Ecol Evol 3:704–712. https://doi.org/10.1111/j.2041-210X.2012.00209.x

Meyer C, Kreft H, Guralnick R, Jetz W (2015) Global priorities for an effective information basis of biodiversity distributions. Nat Commun 6:8221. https://doi.org/10.1038/ncomms9221

Moher D, Liberati A, Tetzlaff J et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6:e1000097. https://doi.org/10.1371/journal.pmed.1000097

Monteiro WP, Veiga JC, Silva AR et al (2019) Everything you always wanted to know about gene flow in tropical landscapes (but were afraid to ask). PeerJ 7:e6446. https://doi.org/10.7717/peerj.6446

Moreira LFB, Maltchik L (2014) Does organic agriculture benefit anuran diversity in rice fields? Wetlands 34:725–733. https://doi.org/10.1007/s13157-014-0537-y

Moura MR, Jetz W (2021) Shortfalls and opportunities in terrestrial vertebrate species discovery. Nat Ecol Evol 5:631–639. https://doi.org/10.1038/s41559-021-01411-5

Moura MR, Costa HC, Peixoto MA et al (2018) Geographical and socioeconomic determinants of species discovery trends in a biodiversity hotspot. Biol Conserv 220:237–244. https://doi.org/10.1016/j.biocon.2018.01.024

Newbold T, Hudson LN, Phillips HRP et al (2014) A global model of the response of tropical and sub-tropical forest biodiversity to anthropogenic pressures. Proc R Soc B Biol Sci 281:20141371. https://doi.org/10.1098/rspb.2014.1371

Newbold T, Hudson LN, Hill SLL et al (2015) Global effects of land use on local terrestrial biodiversity. Nature 520:45–50. https://doi.org/10.1038/nature14324

Newbold T, Oppenheimer P, Etard A, Williams JJ (2020) Tropical and Mediterranean biodiversity is disproportionately sensitive to land-use and climate change. Nat Ecol Evol 4:1630–1638. https://doi.org/10.1038/s41559-020-01303-0

Nogués-Bravo D, Veloz S, Holt BG et al (2016) Amplified plant turnover in response to climate change forecast by Late Quaternary records. Nat Clim Change 6:1115–1119. https://doi.org/10.1038/nclimate3146

Nowakowski AJ, Watling JI, Whitfield SM et al (2017) Tropical amphibians in shifting thermal landscapes under land-use and climate change. Conserv Biol 31:96–105. https://doi.org/10.1111/cobi.12769

Ofori-Boateng C, Oduro W, Hillers A et al (2013) Differences in the Effects of Selective Logging on Amphibian Assemblages in Three West African Forest Types. Biotropica 45:94–101. https://doi.org/10.1111/j.1744-7429.2012.00887.x

Pacioni C, Hunt H, Allentoft ME et al (2015) Genetic diversity loss in a biodiversity hotspot: ancient DNA quantifies genetic decline and former connectivity in a critically endangered marsupial. Mol Ecol 24:5813–5828. https://doi.org/10.1111/mec.13430

Palmeirim AF, Vieira MV, Peres CA (2017) Herpetofaunal responses to anthropogenic forest habitat modification across the neotropics: insights from partitioning β-diversity. Biodivers Conserv 26:2877–2891. https://doi.org/10.1007/s10531-017-1394-9

Peltzer PM, Lajmanovich RC, Attademo AM, Beltzer AH (2006) Diversity of anurans across agricultural ponds in Argentina. In: Hawksworth DL, Bull AT (eds) Marine, Freshwater, and Wetlands Biodiversity Conservation. Springer, Dordrecht, pp 131–145

Phillips SJ, Anderson RP, Dudík M et al (2017) Opening the black box: an open-source release of Maxent. Ecography 40:887–893. https://doi.org/10.1111/ecog.03049

Pilliod DS, Arkle RS, Robertson JM et al (2015) Effects of changing climate on aquatic habitat and connectivity for remnant populations of a wide-ranging frog species in an arid landscape. Ecol Evol 5:3979–3994. https://doi.org/10.1002/ece3.1634

Pittman SE, Semlitsch RD (2013) Habitat type and distance to edge affect movement behavior of juvenile pond-breeding salamanders. J Zool 291:154–162. https://doi.org/10.1111/jzo.12055

Price-Rees SJ, Brown GP, Shine R (2013) Spatial ecology of bluetongue lizards ( Tiliqua spp.) in the Australian wet–dry tropics. Austral Ecol 38:493–503. https://doi.org/10.1111/j.1442-9993.2012.02439.x

Pyšek P, Richardson DM, Pergl J et al (2008) Geographical and taxonomic biases in invasion ecology. Trends Ecol Evol 23:237–244. https://doi.org/10.1016/j.tree.2008.02.002

R Core Team (2021) R: A language and environment for statistical computing

Ray N, Lehmann A, Joly P (2002) Modeling spatial distribution of amphibian populations: a GIS approach based on habitat matrix permeability. Biodivers Conserv 11:2143–2165. https://doi.org/10.1023/A:1021390527698

Riemann JC, Ndriantsoa SH, Raminosoa NR et al (2015) The value of forest fragments for maintaining amphibian diversity in Madagascar. Biol Conserv 191:707–715. https://doi.org/10.1016/j.biocon.2015.08.020

Riemann JC, Ndriantsoa SH, Rödel M-O, Glos J (2017) Functional diversity in a fragmented landscape — Habitat alterations affect functional trait composition of frog assemblages in Madagascar. Glob Ecol Conserv 10:173–183. https://doi.org/10.1016/j.gecco.2017.03.005

Riva F, Fahrig L (2022) Protecting many small patches will maximize biodiversity conservation for most taxa: the SS > SL principle. Preprints

Rivera-Ortíz FA, Aguilar R, Arizmendi MDC et al (2015) Habitat fragmentation and genetic variability of tetrapod populations. Anim Conserv 18:249–258. https://doi.org/10.1111/acv.12165

Rothermel BB, Semlitsch RD (2002) An Experimental investigation of landscape resistance of forest versus old-field habitats to emigrating juvenile amphibians. Conserv Biol 16:1324–1332. https://doi.org/10.1046/j.1523-1739.2002.01085.x

Roux BL, Rouanet H (2004) Geometric Data Analysis: From Correspondence Analysis to Structured Data Analysis. Springer Science, Dordrecht

Rytwinski T, Fahrig L (2012) Do species life history traits explain population responses to roads? A meta-analysis. Biol Conserv 147:87–98. https://doi.org/10.1016/j.biocon.2011.11.023

Safner T, Miaud C, Gaggiotti O et al (2011) Combining demography and genetic analysis to assess the population structure of an amphibian in a human-dominated landscape. Conserv Genet 12:161–173. https://doi.org/10.1007/s10592-010-0129-1

Segovia ALR, Romano D, Armsworth PR (2020) Who studies where? Boosting tropical conservation research where it is most needed. Front Ecol Environ 18:159–166. https://doi.org/10.1002/fee.2146

Seshadri KS (2014) Effects of Historical Selective Logging on Anuran Communities in a Wet Evergreen Forest, South India. Biotropica 46:615–623. https://doi.org/10.1111/btp.12141

Shaffer HB, Gidiş M, McCartney-Melstad E et al (2015) Conservation genetics and genomics of amphibians and reptiles. Annu Rev Anim Biosci 3:113–138. https://doi.org/10.1146/annurev-animal-022114-110920

Silva DJ, Palmeirim AF, Santos-Filho M et al (2022) Habitat Quality, Not Patch Size, Modulates Lizard Responses to Habitat Loss and Fragmentation in the Southwestern Amazon. J Herpetol 56:75–83. https://doi.org/10.1670/20-145

Smith MA, Green DM (2005) Dispersal and the metapopulation paradigm in amphibian ecology and conservation: are all amphibian populations metapopulations? Ecography 28:110–128. https://doi.org/10.1111/j.0906-7590.2005.04042.x

Smith AL, Landguth EL, Bull CM et al (2016) Dispersal responses override density effects on genetic diversity during post-disturbance succession. Proc R Soc B Biol Sci 283:20152934. https://doi.org/10.1098/rspb.2015.2934

Sodhi NS, Koh LP, Brook BW, Ng PKL (2004) Southeast Asian biodiversity: an impending disaster. Trends Ecol Evol 19:654–660. https://doi.org/10.1016/j.tree.2004.09.006

Sodhi NS, Bickford D, Diesmos AC et al (2008) Measuring the meltdown: drivers of global amphibian extinction and decline. PLoS ONE 3:e1636. https://doi.org/10.1371/journal.pone.0001636

Stevens GC (1989) The latitudinal gradient in geographical range: how so many species coexist in the tropics. Am Nat 133:240–256

Stuart SN, Chanson JS, Cox NA et al (2004) Status and trends of amphibian declines and extinctions worldwide. Science 306:1783–1786. https://doi.org/10.1126/science.1103538

Tan WC, Ginal P, Rhodin AGJ et al (2021) A present and future assessment of the effectiveness of existing reserves in preserving three critically endangered freshwater turtles in Southeast Asia and South Asia. Front Biogeogr. https://doi.org/10.21425/F5FBG50928

Thompson W (2013) Sampling Rare or Elusive Species: Concepts, Designs, and Techniques for Estimating Population Parameters. Island Press, Washington

Todd B, Willson J, Gibbons J (2010) The Global Status of Reptiles and Causes of Their Decline. Ecotoxicology of Amphibians and Reptiles. CRC Press, Boca Raton, pp 47–67

Trimble MJ, van Aarde RJ (2012) Geographical and taxonomic biases in research on biodiversity in human-modified landscapes. Ecosphere 3:art119. https://doi.org/10.1890/ES12-00299.1

Uetz P, Freed P, Aguilar R, Hošek J (2021) The Reptile Database. http://www.reptile-database.org/ . Accessed 6 Mar 2021

Urbina-Cardona JN, Loyola RD (2008) Applying niche-based models to predict endangered-hylid potential distributions: are neotropical protected areas effective enough? Trop Conserv Sci 1:417–445. https://doi.org/10.1177/194008290800100408

Vallan D (2002) Effects of anthropogenic environmental changes on amphibian diversity in the rain forests of Eastern Madagascar. J Trop Ecol 18:725–742

Van Dyck H, Baguette M (2005) Dispersal behaviour in fragmented landscapes: routine or special movements? Basic Appl Ecol 6:535–545. https://doi.org/10.1016/j.baae.2005.03.005

van Eck NJ, Waltman L (2014) Visualizing Bibliometric Networks. In: Ding Y, Rousseau R, Wolfram D (eds) Measuring Scholarly Impact: Methods and Practice. Springer International Publishing, Cham, pp 285–320

Wang W, Feng C, Liu F, Li J (2020) Biodiversity conservation in China: a review of recent studies and practices. Environ Sci Ecotechnology 2:100025. https://doi.org/10.1016/j.ese.2020.100025

Ward JH (1963) Hierarchical grouping to optimize an objective function. J Am Stat Assoc 58:236–244. https://doi.org/10.1080/01621459.1963.10500845

Watling JI, Braga L (2015) Desiccation resistance explains amphibian distributions in a fragmented tropical forest landscape. Landsc Ecol 30:1449–1459. https://doi.org/10.1007/s10980-015-0198-0

Whitton FJS, Purvis A, Orme CDL, Olalla-Tárraga MÁ (2012) Understanding global patterns in amphibian geographic range size: does Rapoport rule? Glob Ecol Biogeogr 21:179–190. https://doi.org/10.1111/j.1466-8238.2011.00660.x

Wilson JN, Bekessy S, Parris KM et al (2013) Impacts of climate change and urban development on the spotted marsh frog ( Limnodynastes tasmaniensis ). Austral Ecol 38:11–22. https://doi.org/10.1111/j.1442-9993.2012.02365.x

Download references

Acknowledgements

W.C. Tan was supported financially through a scholarship by the German Academic Exchange Service (DAAD). This work would not be possible without M. Flecks for his invaluable technical assistance with the figures.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Herpetology Section, LIB, Museum Koenig, Bonn, Leibniz Institute for the Analysis of Biodiversity Change, Adenauerallee 127, 53113, Bonn, Germany

W. C. Tan & D. Rödder

UMR 7179 C.N.R.S/M.N.H.N., Département Adaptations du Vivant, Bâtiment d’Anatomie Comparée, 55 Rue Buffon, 75005, Paris, France

Department of Biology, Evolutionary Morphology of Vertebrates, Ghent University, K.L. Ledeganckstraat 35, 9000, Gent, Belgium

Department of Biology, University of Antwerp, Universiteitsplein 1, B-2610, Antwerpen, Belgium

You can also search for this author in PubMed   Google Scholar

Contributions

WCT, AH, and DR contributed to the study idea and conception. Literature search and data collection were performed by WCT and data analysis by DR and WCT. The first draft of the manuscript was written by WCT and all authors critically revised on later versions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to W. C. Tan .

Ethics declarations

Competing interests.

The authors declare no conflicts of interest.

Additional information

Communicated by Ricardo Correia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Online appendices

Below is the link to the electronic supplementary material.

Captions for appendices (PDF 288 kb)

Appendix 1(pdf 138 kb), appendix 2 (csv 3608 kb), appendix 3 (xlsx 47 kb), appendix 4 (pdf 113 kb), appendix 5 (pdf 293 kb), appendix 6 (pdf 80 kb), appendix 7 (xlsx 18 kb), appendix 8 (pdf 55343 kb), appendix 9 (pdf 55290 kb), appendix 10 (eps 5675 kb), appendix 11 (eps 5665 kb), appendix 12 (xlsx 13 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Tan, W.C., Herrel, A. & Rödder, D. A global analysis of habitat fragmentation research in reptiles and amphibians: what have we done so far?. Biodivers Conserv 32 , 439–468 (2023). https://doi.org/10.1007/s10531-022-02530-6

Download citation

Received : 18 August 2022

Revised : 02 December 2022

Accepted : 09 December 2022

Published : 08 January 2023

Issue Date : February 2023

DOI : https://doi.org/10.1007/s10531-022-02530-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Habitat change
• Herpetofauna
• Geographical bias
• Research agendas
• Systematic review
• Find a journal
• Publish with us
• Track your research