what is table of contents research paper

Link to facebook
Link to linkedin
Link to twitter
Link to youtube
Writing Tips

How to Structure the Table of Contents for a Research Paper

4-minute read

16th July 2023

So you’ve made it to the important step of writing the table of contents for your paper. Congratulations on making it this far! Whether you’re writing a research paper or a dissertation , the table of contents not only provides the reader with guidance on where to find the sections of your paper, but it also signals that a quality piece of research is to follow. Here, we will provide detailed instructions on how to structure the table of contents for your research paper.

Steps to Create a Table of Contents

Insert the table of contents after the title page.

Within the structure of your research paper , you should place the table of contents after the title page but before the introduction or the beginning of the content. If your research paper includes an abstract or an acknowledgements section , place the table of contents after it.

List all the paper’s sections and subsections in chronological order.

Depending on the complexity of your paper, this list will include chapters (first-level headings), chapter sections (second-level headings), and perhaps subsections (third-level headings). If you have a chapter outline , it will come in handy during this step. You should include the bibliography and all appendices in your table of contents. If you have more than a few charts and figures (more often the case in a dissertation than in a research paper), you should add them to a separate list of charts and figures that immediately follows the table of contents. (Check out our FAQs below for additional guidance on items that should not be in your table of contents.)

Paginate each section.

Label each section and subsection with the page number it begins on. Be sure to do a check after you’ve made your final edits to ensure that you don’t need to update the page numbers.

Format your table of contents.

The way you format your table of contents will depend on the style guide you use for the rest of your paper. For example, there are table of contents formatting guidelines for Turabian/Chicago and MLA styles, and although the APA recommends checking with your instructor for formatting instructions (always a good rule of thumb), you can also create a table of contents for a research paper that follows APA style .

Add hyperlinks if you like.

Depending on the word processing software you’re using, you may also be able to hyperlink the sections of your table of contents for easier navigation through your paper. (Instructions for this feature are available for both Microsoft Word and Google Docs .)

To summarize, the following steps will help you create a clear and concise table of contents to guide readers through your research paper:

1. Insert the table of contents after the title page.

2. List all the sections and subsections in chronological order.

3. Paginate each section.

4. Format the table of contents according to your style guide.

5. Add optional hyperlinks.

If you’d like help formatting and proofreading your research paper , check out some of our services. You can even submit a sample for free . Best of luck writing your research paper table of contents!

What is a table of contents?

A table of contents is a listing of each section of a document in chronological order, accompanied by the page number where the section begins. A table of contents gives the reader an overview of the contents of a document, as well as providing guidance on where to find each section.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

What should I include in my table of contents?

If your paper contains any of the following sections, they should be included in your table of contents:

● Chapters, chapter sections, and subsections

● Introduction

● Conclusion

● Appendices

● Bibliography

Although recommendations may differ among institutions, you generally should not include the following in your table of contents:

● Title page

● Abstract

● Acknowledgements

● Forward or preface

If you have several charts, figures, or tables, consider creating a separate list for them that will immediately follow the table of contents. Also, you don’t need to include the table of contents itself in your table of contents.

Is there more than one way to format a table of contents?

Yes! In addition to following any recommendations from your instructor or institution, you should follow the stipulations of your style guide .

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

2-minute read

How to Cite the CDC in APA

If you’re writing about health issues, you might need to reference the Centers for Disease...

5-minute read

Six Product Description Generator Tools for Your Product Copy

Introduction If you’re involved with ecommerce, you’re likely familiar with the often painstaking process of...

3-minute read

What Is a Content Editor?

Are you interested in learning more about the role of a content editor and the...

The Benefits of Using an Online Proofreading Service

Proofreading is important to ensure your writing is clear and concise for your readers. Whether...

6 Online AI Presentation Maker Tools

Creating presentations can be time-consuming and frustrating. Trying to construct a visually appealing and informative...

What Is Market Research?

No matter your industry, conducting market research helps you keep up to date with shifting...

Make sure your writing is the best it can be with our expert English proofreading and editing.

Bipolar Disorder
Therapy Center
When To See a Therapist
Types of Therapy
Best Online Therapy
Best Couples Therapy
Best Family Therapy
Managing Stress
Sleep and Dreaming
Understanding Emotions
Self-Improvement
Healthy Relationships
Student Resources
Personality Types
Guided Meditations
Verywell Mind Insights
2023 Verywell Mind 25
Mental Health in the Classroom
Editorial Process
Meet Our Review Board
Crisis Support

A Table of Contents in APA Format

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

what is table of contents research paper

Adah Chung is a fact checker, writer, researcher, and occupational therapist.

General Guidelines

Table of Contents

APA style does not require a table of contents, but there are cases where you may need to include one. For example, your instructor may specify that your paper must be submitted with a table of contents. A table of contents can be particularly helpful in cases where your paper is lengthy or covers a lot of material, such as a thesis paper or dissertation. Research papers, in particular, may benefit from the addition of a table of contents.

APA style is the official publication style of the American Psychological Association. APA style is used in psychology courses as well as other social science classes including those in social science, behavioral sciences, and education.

The table of contents serves as a basic roadmap of your paper. It should list all of the major headings and subheadings within the body of your paper. For a standard psychology paper, it might include listings for the introduction, method, results, and discussion sections of your paper.

While the APA may not specify guidelines for a table of contents, you should use the basic APA format for formatting your table of contents:

Use one-inch margins on all sides
Use 12-point Times New Roman font
Double-space

Since APA does not require a table of contents, you should always refer to your instructor’s guidelines when deciding whether or not to include one.

It is also important to note that the 7th edition of the Publication Manual of the American Psychological Association was published in 2020, and included updated guidelines on many topics.

For example, while the previous edition of the style manual required a running head on each page of a paper, the 7th edition has eliminated that requirement on student papers unless your instructor specifies to include it. Always ask first.

If you are using a standard APA paper format, your table of contents should include the following sections:

Introduction

The above format may work well for a standard lab report or research paper. However, your table of contents will look much different if you are writing something such as a critique, essay, or case study.

Notice, that the table of contents does not include the abstract or acknowledgments pages. When applicable, it should list the appendices and the lists of tables and figures.

The exact order of your paper depends largely on the type of paper you are writing. In general, your paper should be presented in the following order:

Main Body of Paper

Table of Contents Format

Because there is no standard format for a table of contents in APA style, you should always defer to the provided guidelines for your assignment.

If your instructor does not have a preferred format, consider using the following:

Title the page “Table of Contents” and center the title at the top of the page.
Most papers should include at least two levels of headings, up to five levels.
Level one headings will be for main topics, such as chapter titles like "Chapter One; Name of Chapter," or research sections like "Method," "Results," and "Discussion."
All level-one headings should be flush-left and sub-headings should be indented five spaces deeper than the last.
All heading levels should be in title case, capitalizing the first letter of each word. The font type, style, and size stay the same for each level.
The page number for each heading is formatted flush-right. Include dot leaders between the headings and the page number to improve readability.

While you might not think that following APA format is important, it is one of those areas where students can lose points for making small errors. It pays to spend a little extra time and attention making sure that your paper is formatted in proper APA style.

If you need help, you can get assistance from your school's writing lab.
Getting your own copy of the latest edition of the APA publication manual can be very helpful.
Always refer to any instructions or guidelines that were provided by your course instructor.
There is a helpful feature in most word processors that you can use to pre-format your paper in APA style. It takes a little effort to set it up, but well worth it in the end, especially for longer documents. You can save the style to apply to your future papers saving you the effort next time.

For those writing a paper to submit for publication, check with the publisher for any specific formatting requirements that they may have.

American Psychological Association. Publication Manual of the American Psychological Association (7th ed.) ; 2020.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

charlesstreetresearch.com

academic paper writing tips

Search for:

How to Write a Table of Contents for Research Paper

Are you struggling to keep your research paper organised and easy to navigate? Then you’ve finally stumbled on the right post! A well-crafted table of contents is the key to a seamless and stress-free reading experience for your audience.

It’s common knowledge that writing a table of contents can be tricky, especially when unsure where to start. That’s why this comprehensive guide on writing a table of contents in a research paper will take the guesswork out of the process and help you elevate your paper to new heights. From using headings and subheadings to using consistent formatting, we’ve got you covered.

So don’t wait any longer; let’s dive in and create a table of contents paper that will make your overall research work shine!

What is a table of contents?

A table of contents (TOC) is a list of a research paper’s main sections and subsections, along with their page numbers. It serves as a roadmap for the reader, allowing them to find the information they need within the paper quickly.

The table of contents for a research paper is usually placed at the beginning of the paper, after the title page and abstract, but before the introduction. It typically includes the main sections of the paper, such as the introduction, literature review, methods, results, and conclusion. It may also include sub-sections, such as specific chapters or sections within each main section.

So, “what does a table of contents look like for a research paper? ” If you are, take a look at a table of contents example for research paper we have provided below.

Introduction
Background of the Study
Statement of problem
Research Question
Significance of the Study
Literature Review
Theoretical Framework
Previous Research
Gaps in the Literature
Study Population
Study Design
Sample Size
Data Collection
Data Analysis
Ethical Considerations
Descriptive Statistics
Inferential Statistics
Interpretation of Results
Implications for Future Research
Limitations of the Study
Summary of Findings
Implications for Practice
Suggestions for Future Research

It is important to note that the table of contents format may vary depending on the specific guidelines of the work or department where the paper is being submitted. Additionally, the sections and subsections may vary depending on the nature and scope of the research. For another example of table of contents in research paper, you can look online to see which best suits your department.

Practical tips on how to write a table of contents for a research paper

When writing a table of contents for a research paper, it is essential to follow a few fundamental guidelines to ensure that it is straightforward to use. Here are some tips for writing a table of contents that will help your readers find the information they need:

Use headings and subheadings

Use headings and subheadings to organise your paper into sections and subsections. This will ensure your readers find the information they need quickly and easily.

Use consistent formatting

Use the same formatting for all the headings and subheadings in your paper. This will make it easy for your readers to identify the different sections of the paper.

Be specific

Be specific when listing the sections and subsections in your table of contents. Use descriptive titles that accurately reflect the content of each section.

Use page numbers

Include the page numbers for each section and subsection in your table of contents. This will make it easy for your readers to find the information they need by quickly navigating to the appropriate page.

Review and proofread

Review your table of contents for accuracy and completeness, and proofread it for any errors. This will ensure that your table of contents is easy to use and free of errors.

A table of contents is an essential component of a research paper as it helps the reader to navigate the paper easily and quickly find the information they need. Following these guidelines ensures that your table of contents is straightforward, easy to use, and accurate.

Your Step to Success

Plagiarism Check within 10min

Printing & Binding with 3D Live Preview

How do you like this article cancel reply.

Save my name, email, and website in this browser for the next time I comment.

A guide to the table of contents page

Inhaltsverzeichnis

1 Definition: Table of Contents
3 Everything for Your Thesis
5 Create in Microsoft Word
6 In a Nutshell

Definition: Table of Contents

The table of contents is an organized listing of your document’s chapters, sections and, often, figures, clearly labelled by page number. Readers should be able to look at your table of contents page and understand immediately how your paper is organized, enabling them to skip to any relevant section or sub-section. The table of contents should list all front matter, main content and back matter, including the headings and page numbers of all chapters and the bibliography . A good table of contents should be easy to read, accurately formatted and completed last so that it is 100% accurate. Although you can complete a table of contents manually, many word processing tools like Microsoft Word enable you to format your table of contents automatically.

When adding the finishing touches to your dissertation, the table of contents is one of the most crucial elements. It helps the reader navigate (like a map) through your argument and topic points. Adding a table of contents is simple and it can be inserted easily after you have finished writing your paper. In this guide, we look at the do’s and don’ts of a table of contents; this will help you process and format your dissertation in a professional way.

When adding the finishing touches to your dissertation, the table of contents is one of the most crucial elements. It helps the reader navigate (like a map) through your argument and topic points. Adding a table of contents is simple and can be inserted easily after you have finished writing your paper. In this guide, we look at the do’s and don’ts of a table of contents; this will help you process and format your dissertation in a professional way.

✓ Post a picture on Instagram
✓ Get the most likes on your picture
✓ Receive up to $300 cash back

What is a table of contents?

A table of contents is a list, usually on a page at the beginning of a piece of academic writing , which outlines the chapters or sections names with their corresponding page numbers. In addition to chapter names, it includes bullet points of the sub-chapter headings or subsection headings. It usually comes right after the title page of a research paper.

How do you write a table of contents

To write a table of contents, you first write the title or chapter names of your research paper in chronological order. Secondly, you write the subheadings or subtitles, if you have them in your paper. After that, you write the page numbers for the corresponding headings and subheadings. You can also very easily set up a table of contents in Microsoft Word.

Where do you put a table of contents?

The table of contents is found on a page right at the beginning of an academic writing project. It comes specifically after the title page and acknowledgements, but before the introductory page of a writing project. This position at the beginning of an academic piece of writing is universal for all academic projects.

What to include in a table of contents?

A sample table of contents includes the title of the paper at the very top, followed by the chapter names and subtitles in chronological order. At the end of each line, is the page number of the corresponding headings. Examples of chapter names can be: executive summary, introduction, project description, marketing plan, summary and conclusion. The abstract and acknowledgments are usually not included in the table of contents, however this could depend on the formatting that is required by your institution. Scroll down to see some examples.

How important is a table of contents?

A table of contents is very important at the beginning of a writing project for two important reasons. Firstly, it helps the reader easily locate contents of particular topics itemized as chapters or subtitles. Secondly, it helps the writer arrange their work and organize their thoughts so that important sections of an academic project are not left out. This has the extra effect of helping to manage the reader’s expectation of any academic essay or thesis right from the beginning.

Everything for Your Thesis

A table of contents is a crucial component of an academic thesis. Whether you’re completing a Bachelor’s or a postgraduate degree, the table of contents is a requirement for dissertation submissions. As a rule of thumb, your table of contents will usually come after your title page , abstract, acknowledgement or preface. Although it’s not necessary to include a reference to this front matter in your table of contents, different universities have different policies and guidelines.

Although the table of contents is best completed after you have finished your thesis, it’s a good idea to draw up a mock table of contents in the early stages of writing. This allows you to formulate a structure and think through your topic and how you are going to research, answer and make your argument. Think of this as a form of “reverse engineering”. Knowing how your chapters are going to be ordered and what topics or research questions are included in each will help immensely when it comes to your writing.

The table of contents is not just an academic formality, it allows your examiner to quickly get a feel for your topic and understand how your dissertation will be presented. An unclear or sloppy table of contents may even have an adverse effect on your grade because the dissertation is difficult to follow.

Examiners are readers, after all, and a dissertation is an exercise in producing an argument. A clear table of contents will give both a good impression and provide an accurate roadmap to make the examiner’s job easier and your argument more persuasive.

Your table of contents section will come after your acknowledgements and before your introduction. It includes a list of all your headers and their respective pages and will also contain a sub-section listing your tables, figures or illustrations (if you are using them). In general, your thesis can be ordered like this:

1. Title Page 2. Copyright / Statement of Originality 3. Abstract 4. Acknowledgement, Dedication and Preface (optional) 5. Table of Contents 6. List of Figures/Tables/Illustrations 7. Chapters 8. Appendices 9. Endnotes (depending on your formatting) 10. Bibliography / References

The formatting of your table of contents will depend on your academic field and thesis length. Some disciplines, like the sciences, have a methodical structure which includes recommended subheadings on methodology, data results, discussion and conclusion. Humanities subjects, on the other hand, are far more varied. Whichever discipline you are working in, you need to create an organized list of all chapters in their order of appearance, with chapter subheadings clearly labelled.

Sample table of contents for a short dissertation:

Abstract ………………………………………………………………………………………………….. ii Acknowledgements ………………………………………………………………………………………………….. iii Dedication ………………………………………………………………………………………………….. iv List of Tables ………………………………………………………………………………………………….. x List of Figures ………………………………………………………………………………………………….. xi Chapter 1: Introduction ………………………………………………………………………………………………….. 1 Chapter 2: Literature Survey ………………………………………………………………………………………………….. 13 Chapter 3: Methodology ………………………………………………………………………………………………….. 42 Chapter 4: Analysis ………………………………………………………………………………………………….. 100 Chapter 5: Conclusion ………………………………………………………………………………………………….. 129 Appendices ………………………………………………………………………………………………….. 169 References ………………………………………………………………………………………………….. 172

When producing a more significant and longer dissertation, say for a Master’s degree or even a PhD, your chapter descriptions should contain all subheadings. These are listed with the chapter number, followed by a decimal point and the subheading number.

Sample table of contents for a PhD dissertation:

Chapter 1 1.1 Introduction 1.2 Literature Review 1.3 Data 1.4 Findings 1.5 Conclusion

Chapter 2, and so on.

The key to writing a good table of contents is consistency and accuracy. You cannot list subheadings for one chapter and forget them for another. Subheadings are not always required but they can be very helpful if you are dealing with a detailed topic. The page numbers in the table of contents must match with the respective pages in your thesis or manuscript.

What’s more, chapter titles and subheading titles must match their corresponding pages. If your first chapter is called “Chapter 1: The Beginning”, it must be written as such on both the table of contents and first chapter page. So long as you remain both accurate and consistent, your table of contents will be perfect.

Create in Microsoft Word

Fortunately, the days of manually writing a contents page are over. You can still produce a contents page manually with Microsoft Word, but consider using their automatic feature to guarantee accuracy and save time.

To produce an automatically-generated table of contents, you must first work with heading styles. These can be found in the home tab under “Styles”. Select top-level headings (your chapter titles) and apply the Heading 1 style. This ensures that they will be formatted as main headings. Second-level headings (subheadings) can be applied with the Heading 2 style. This will place them underneath and within each main heading.

Once you have worked with heading styles, simply click on the “References” tab and select “Table of Contents”. This option will allow you to automatically produce a page with accurate page links to your document. To customize the format and style applied to your table of contents, select “Custom Table of Contents” at the bottom of the tab. Remember to update your table of contents by selecting the table and choosing “Update” from the drop-down menu. This will ensure that your headings, sub-headings and page numbers all add up.

Thesis Printing & Binding

You are already done writing your thesis and need a high quality printing & binding service? Then you are right to choose BachelorPrint! Check out our 24-hour online printing service. For more information click the button below :

In a Nutshell

The table of contents is a vital part of any academic thesis or extensive paper.
It is an accurate map of your manuscript’s content – its headings, sub-headings and page numbers.
It shows how you have divided your thesis into more manageable chunks through the use of chapters.
By breaking apart your thesis into discrete sections, you make your argument both more persuasive and easier to follow.
What’s more, your contents page should produce an accurate map of your thesis’ references, bibliography, illustrations and figures.
It is an accurate map of the chapters, references, bibliography, illustrations and figures in your thesis.

We use cookies on our website. Some of them are essential, while others help us to improve this website and your experience.

External Media

Individual Privacy Preferences

Cookie Details Privacy Policy Imprint

Here you will find an overview of all cookies used. You can give your consent to whole categories or display further information and select certain cookies.

Accept all Save

Essential cookies enable basic functions and are necessary for the proper function of the website.

Show Cookie Information Hide Cookie Information

Statistics cookies collect information anonymously. This information helps us to understand how our visitors use our website.

Content from video platforms and social media platforms is blocked by default. If External Media cookies are accepted, access to those contents no longer requires manual consent.

How to Effectively Create a Table of Contents for Your Research Paper?

Creating a table of contents (TOC) for a research paper might seem straightforward, but it’s a crucial part of your document that requires careful consideration. A well-organized TOC not only guides your readers through your paper but also reflects the depth and thoroughness of your research. So, how do you craft a TOC that enhances the readability and professionalism of your research paper?

Understanding the Basics of a Table of Contents

Before diving into the specifics, let’s understand what a TOC is. Essentially, it’s an organized listing of the chapters and major sections of your document. It’s like a roadmap, allowing readers to quickly navigate to sections that are most relevant to them. A clear, concise, and well-formatted TOC is more than just a list; it’s an integral part of your paper’s structure.

Step-by-Step Guide to Crafting Your TOC

Identify the Key Sections : Start by listing down all the chapters, sections, and significant sub-sections of your paper. This includes your introduction, literature review, methodology, results, discussion, and conclusion.
Use Consistent Formatting : Consistency is key in a TOC. Use the same font style and size as your main text, but you can bold or italicize the headings for emphasis. Remember, your TOC should be easy to read and navigate.
Leverage Word Processing Tools : Most word processors, like Microsoft Word, have built-in features to create a TOC automatically. Utilize these tools to ensure accuracy and ease of updating your TOC as your paper evolves.
Page Numbering : Each entry in your TOC should have a corresponding page number. This is crucial for guiding your readers directly to the content they’re interested in.
Update Regularly : As your research paper develops, so should your TOC. Regular updates ensure that the TOC accurately reflects the content of your paper.

While the basics of creating a TOC are universal, different research fields and styles might have specific requirements or preferences. For instance, APA style papers have particular guidelines for TOC formatting, including how to handle different levels of headings and page numbers. Here are some insights gathered from various online sources to help you tailor your TOC to your paper’s needs:

APA Style Specifics : If you’re writing in APA style, pay attention to how you format the different levels of headings in your TOC. The APA guidelines provide clear instructions on this, ensuring that your TOC aligns with the rest of your document’s formatting.
The Role of a TOC in Navigation : A TOC is more than just a list; it’s a navigational tool. It should provide a clear and concise overview of your paper’s structure, allowing readers to easily locate specific sections or topics.
Customizing Your TOC : Depending on your research paper’s complexity, you might want to customize your TOC. This could involve including or excluding certain elements, such as figures or tables, based on their relevance to your paper’s overall structure.
Automating Page Numbers : Modern word processors can automatically update page numbers in your TOC. This feature is incredibly useful, especially when you’re making significant edits to your paper that might affect its pagination.
Clarity and Readability : Above all, your TOC should be clear and easy to read. Avoid cluttering it with unnecessary details. Stick to the main sections and headings that provide a straightforward overview of your paper’s content.

Remember, your TOC is often one of the first elements your readers will interact with. Make it engaging and reflective of the depth of your research. Use it to showcase the organization and thoroughness of your work. A well-crafted TOC not only aids in navigation but also sets the tone for the rest of your paper. It’s an opportunity to make a strong first impression, indicating to your readers that your research is well-structured and thoughtfully presented.

To illustrate the importance of a well-structured TOC, consider the case of a complex research paper covering multiple interconnected topics. A clear TOC allows readers to easily navigate between these topics, understanding how they interrelate and contribute to the overall thesis of the paper. This not only enhances the reader’s experience but also underscores the researcher’s ability to organize complex information effectively.

Actionable Steps

For those looking to create their own TOC, here are some actionable steps:

Review Your Document’s Structure : Before creating your TOC, ensure that your document is well-organized with clearly defined sections and headings.
Utilize Built-in TOC Features : Use the TOC features of your word processor to automatically generate and update your TOC.
Customize According to Style Guides : If your paper must adhere to a specific style guide (like APA), customize your TOC to meet these requirements.
Regularly Update Your TOC : As your paper evolves, regularly update your TOC to reflect any changes in structure or page numbers.
Keep It Simple and Clear : Your TOC should be easy to navigate. Avoid overcomplicating it with too many sub-levels or unnecessary information.

Book Editing

Manuscript Editing Services

Medical Editing

Bioscience Editing
Physical Science Editing
PhD Thesis Editing Services
PhD Editing
Master’s Proofreading
Bachelor’s Editing
Dissertation Proofreading Services
Best Dissertation Proofreaders
Masters Dissertation Proofreading
PhD Proofreaders
Proofreading PhD Thesis Price
Journal Article Editing
Book Editing Service
Editing and Proofreading Services
Research Paper Editing
Medical Manuscript Editing
Academic Editing
Social Sciences Editing
Academic Proofreading
PhD Theses Editing
Dissertation Proofreading
Proofreading Rates UK
Medical Proofreading
PhD Proofreading Services UK
Academic Proofreading Services UK

Medical Editing Services

Life Science Editing
Biomedical Editing
Environmental Science Editing
Pharmaceutical Science Editing
Economics Editing
Psychology Editing
Sociology Editing
Archaeology Editing
History Paper Editing
Anthropology Editing
Law Paper Editing
Engineering Paper Editing
Technical Paper Editing
Philosophy Editing
PhD Dissertation Proofreading
Lektorat Englisch
Akademisches Lektorat
Lektorat Englisch Preise
Wissenschaftliches Lektorat
Lektorat Doktorarbeit

PhD Thesis Editing

Thesis Proofreading Services
PhD Thesis Proofreading
Proofreading Thesis Cost
Proofreading Thesis
Thesis Editing Services
Professional Thesis Editing
Thesis Editing Cost
Proofreading Dissertation
Dissertation Proofreading Cost
Dissertation Proofreader
Correção de Artigos Científicos
Correção de Trabalhos Academicos
Serviços de Correção de Inglês
Correção de Dissertação
Correção de Textos Precos
定額ネイティブチェック
Copy Editing
FREE Courses
Revision en Ingles
Revision de Textos en Ingles
Revision de Tesis
Revision Medica en Ingles
Revision de Tesis Precio
Revisão de Artigos Científicos
Revisão de Trabalhos Academicos
Serviços de Revisão de Inglês
Revisão de Dissertação
Revisão de Textos Precos
Corrección de Textos en Ingles
Corrección de Tesis
Corrección de Tesis Precio
Corrección Medica en Ingles
Corrector ingles

Select Page

How To Write a Table of Contents for Academic Papers

Posted by Rene Tetzner | Mar 17, 2021 | How To Get Published | 0 |

How To Write a Table of Contents for Academic Papers Although every author begins a writing project with the best of intentions and an ideal outline in mind, it often proves difficult to stick to one’s initial plans as the text begins to unfold and its complexities grow in number and depth. Sometimes a document quickly exceeds the word limits for a project, and at others certain important aspects are neglected or turn out a good deal shorter than intended. Drafting a working table of contents for your writing project can provide an excellent tool for keeping your discussion on track and your text within length requirements as you write.

A working table of contents should begin with a title. This title may change as you draft your text, but a working title will help you focus your thoughts as you devise the headings and plan the content for the main parts, chapters, sections and subsections that should be added beneath it. All headings, whether numbered or not, should be formatted in effective and consistent ways that distinguish section levels and clearly indicate the overall structure of the text. These headings can also be altered as your writing advances, but they will provide an effective outline of what you need to discuss and the order in which you think the main topics should be presented. At this initial stage, it is also a good idea to write under each heading a brief summary of or rough notes about what you hope to include in that part of the document, and you can continue to add, adjust and move material around within and among the sections as your table of contents and ultimately your text progresses. Reminders of how long (measured in words, paragraphs or pages) the entire text and each of its parts should ideally be may also prove helpful.

Once you have your annotated table of contents drafted, it will serve as an informative list of both content and order that can be productively consulted as you write. Assuming you construct your working table of contents as a computer file in the same program you intend to use for writing the entire document, you can also use the table of contents as a template for composing the text as a whole, replacing your rough notes under each heading with the formal text as you draft it. This practice lends an immediate physical presence to the guidance provided by your table of contents because you are literally working within that outline, which can be especially wise if you tend to run on or become distracted by new ideas as you write.

Finally, your working table of contents can become your final table of contents if one is required for your project. If you would like to use the working table of contents in this way and are also using it as a template, be sure to rename the file and save a separate copy before you begin adding the formal text of your document. Then you can simply delete your summaries and rough notes from the original table of contents to make your final one, leaving only the headings, to which you can add relevant page numbers as required.

You might be interested in Services offered by Proof-Reading-Service.com

Journal editing.

Journal article editing services

PhD thesis editing services

Scientific Editing

Manuscript editing.

Manuscript editing services

Expert Editing

Expert editing for all papers

Research Editing

Research paper editing services

Professional book editing services

Choosing the Right Journal

September 10, 2021

Example of a Quantitative Research Paper

September 4, 2021

What Is a Good H-Index Required for an Academic Position?

September 3, 2021

Acknowledgements Example for an Academic Research Paper

September 1, 2021

Our Recent Posts

Examples of Research Paper Topics in Different Study Areas

Our review ratings

Examples of Research Paper Topics in Different Study Areas Score: 98%
Dealing with Language Problems – Journal Editor’s Feedback Score: 95%
Making Good Use of a Professional Proofreader Score: 92%
How To Format Your Journal Paper Using Published Articles Score: 95%
Journal Rejection as Inspiration for a New Perspective Score: 95%

University of Michigan Library
Research Guides

Microsoft Word for Dissertations

Table of Contents
Introduction, Template, & Resources
Formatting for All Readers
Applying a Style
Modifying a Style
Setting up a Heading 1 Example
Images, Charts, Other Objects
Footnotes, Endnotes, & Citations
Cross-References
Appendix Figures & Tables
List of Figures/Tables
Chapter and Section Numbering
Page Numbers
Landscape Pages
Combining Chapter Files
Commenting and Reviewing
The Two-inch Top Margin
Troubleshooting
Finalizing Without Styles
Preparing Your Final Document

Automatic Table of Contents

An automatic Table of Contents relies on Styles to keep track of page numbers and section titles for you automatically. Microsoft Word can scan your document and find everything in the Heading 1 style and put that on the first level of your table of contents, put any Heading 2’s on the second level of your table of contents, and so on.

If you want an automatic table of contents you need to apply the Heading 1 style to all of your chapter titles and front matter headings (like “Dedication” and “Acknowledgements”). All section headings within your chapters should use the Heading 2 style. All sub-section headings should use Heading 3 , etc....

If you have used Heading styles in your document, creating an automatic table of contents is easy.

Place your cursor where you want your table of contents to be.
On the References Ribbon, in the Table of Contents Group , click on the arrow next to the Table of Contents icon, and select Custom Table of Contents .
We suggest that you set each level (Chapters, sections, sub-sections, aka TOC 1, TOC 2, TOC 3) to be single-spaced, with 12 points of space afterwards. This makes each item in your ToC clump together if they're long enough to wrap to a second line, with the equivalent of a double space between each item, and makes the ToC easier to read and understand than if every line were double-spaced. See the video below for details.
If you want to change which headings appear in your Table of Contents, you can do so by changing the number in the Show levels: field. Select "1" to just include the major sections (Acknowledgements, List of Figures, Chapters, etc...). Select "4" to include Chapters, sections, sub-sections, and sub-sub-sections.
Click OK to insert your table of contents.

The table of contents is a snapshot of the headings and page numbers in your document, and does not automatically update itself as you make changes. At any time, you can update it by right-clicking on it and selecting Update field . Notice that once the table of contents is in your document, it will turn gray if you click on it. This just reminds you that it is a special field managed by Word, and is getting information from somewhere else.

Modifying the format of your Table of Contents

The video below shows how to make your Table of Contents a little easier to read by formatting the spacing between items in your Table of Contents. You may recognize the "Modify Style" window that appears, which can serve as a reminder that you can use this window to modify more than just paragraph settings. You can modify the indent distance, or font, or tab settings for your ToC, just the same as you may have modified it for Styles.

an image of the Modify Table of Contents window, where you can set Show Levels

By default, the Table of Contents tool creates the ToC by pulling in Headings 1 through 3. If you'd like to modify that -- to only show H1's, or to show Headings 1 through 4 -- then go to the References tab and select Custom Table of Contents . In the window that appears, set Show Levels to "1" to only show Heading 1's in the Table of Contents, or set it to "4" to show Headings 1 through 4.

Bonus tip for updating fields like the Table of Contents

You'll quickly realize that all of the automatic Lists and Tables need to be updated occasionally to reflect any changes you've made elsewhere in the document -- they do not dynamically update by themselves. Normally, this means going to each field, right-clicking on it and selecting "Update Field".

Alternatively, to update all fields throughout your document (Figure/Table numbers & Lists, cross-references, Table of Contents, etc...), just select "Print". This will cause Word to update everything in anticipation of printing. Once the print preview window appears, just cancel.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

Knowledge Base
Dissertation
Dissertation Table of Contents in Word | Instructions & Examples

Dissertation Table of Contents in Word | Instructions & Examples

Published on 15 May 2022 by Tegan George .

The table of contents is where you list the chapters and major sections of your thesis, dissertation, or research paper, alongside their page numbers. A clear and well-formatted table of contents is essential, as it demonstrates to your reader that a quality paper will follow.

The table of contents (TOC) should be placed between the abstract and the introduction. The maximum length should be two pages. Depending on the nature of your thesis, dissertation, or paper, there are a few formatting options you can choose from.

Download Word doc Download Google doc

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

What to include in your table of contents, what not to include in your table of contents, creating a table of contents in microsoft word, table of contents examples, updating a table of contents in microsoft word, other lists in your thesis, dissertation, or research paper, frequently asked questions about the table of contents.

Depending on the length of your document, you can choose between a single-level, subdivided, or multi-level table of contents.

A single-level table of contents only includes ‘level 1’ headings, or chapters. This is the simplest option, but it may be too broad for a long document like a dissertation.
A subdivided table of contents includes chapters as well as ‘level 2’ headings, or sections. These show your reader what each chapter contains.
A multi-level table of contents also further divides sections into ‘level 3’ headings. This option can get messy quickly, so proceed with caution. Remember your table of contents should not be longer than 2 pages. A multi-level table is often a good choice for a shorter document like a research paper.

Examples of level 1 headings are Introduction, Literature Review, Methodology, and Bibliography. Subsections of each of these would be level 2 headings, further describing the contents of each chapter or large section. Any further subsections would be level 3.

In these introductory sections, less is often more. As you decide which sections to include, narrow it down to only the most essential.

Including appendices and tables

You should include all appendices in your table of contents. Whether or not you include tables and figures depends largely on how many there are in your document.

If there are more than three figures and tables, you might consider listing them on a separate page. Otherwise, you can include each one in the table of contents.

Theses and dissertations often have a separate list of figures and tables.
Research papers generally don’t have a separate list of figures and tables.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

All level 1 and level 2 headings should be included in your table of contents, with level 3 headings used very sparingly.

The following things should never be included in a table of contents:

Your acknowledgements page
Your abstract
The table of contents itself

The acknowledgements and abstract always precede the table of contents, so there’s no need to include them. This goes for any sections that precede the table of contents.

To automatically insert a table of contents in Microsoft Word, be sure to first apply the correct heading styles throughout the document, as shown below.

Choose which headings are heading 1 and which are heading 2 (or 3!
For example, if all level 1 headings should be Times New Roman, 12-point font, and bold, add this formatting to the first level 1 heading.
Highlight the level 1 heading.
Right-click the style that says ‘Heading 1’.
Select ‘Update Heading 1 to Match Selection’.
Allocate the formatting for each heading throughout your document by highlighting the heading in question and clicking the style you wish to apply.

Once that’s all set, follow these steps:

Add a title to your table of contents. Be sure to check if your citation style or university has guidelines for this.
Place your cursor where you would like your table of contents to go.
In the ‘References’ section at the top, locate the Table of Contents group.
Here, you can select which levels of headings you would like to include. You can also make manual adjustments to each level by clicking the Modify button.
When you are ready to insert the table of contents, click ‘OK’ and it will be automatically generated, as shown below.

The key features of a table of contents are:

Clear headings and subheadings
Corresponding page numbers

Check with your educational institution to see if they have any specific formatting or design requirements.

Prevent plagiarism, run a free check.

Write yourself a reminder to update your table of contents as one of your final tasks before submitting your dissertation or paper. It’s normal for your text to shift a bit as you input your final edits, and it’s crucial that your page numbers correspond correctly.

It’s easy to update your page numbers automatically in Microsoft Word. Simply right-click the table of contents and select ‘Update Field’. You can choose either to update page numbers only or to update all information in your table of contents.

In addition to a table of contents, you might also want to include a list of figures and tables, a list of abbreviations and a glossary in your thesis or dissertation. You can use the following guides to do so:

List of figures and tables
List of abbreviations

It is less common to include these lists in a research paper.

All level 1 and 2 headings should be included in your table of contents . That means the titles of your chapters and the main sections within them.

The contents should also include all appendices and the lists of tables and figures, if applicable, as well as your reference list .

Do not include the acknowledgements or abstract in the table of contents.

To automatically insert a table of contents in Microsoft Word, follow these steps:

Apply heading styles throughout the document.
In the references section in the ribbon, locate the Table of Contents group.
Click the arrow next to the Table of Contents icon and select Custom Table of Contents.
Select which levels of headings you would like to include in the table of contents.

Make sure to update your table of contents if you move text or change headings. To update, simply right click and select Update Field.

The table of contents in a thesis or dissertation always goes between your abstract and your introduction.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, May 15). Dissertation Table of Contents in Word | Instructions & Examples. Scribbr. Retrieved 15 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/contents-page/

Is this article helpful?

Tegan George

Other students also liked, dissertation title page, how to write an abstract | steps & examples, thesis & dissertation acknowledgements | tips & examples.

Get science-backed answers as you write with Paperpal's Research feature

What is Academic Writing: Tips for Students

Students and early career researchers need to distinguish clearly between academic and general or non-academic writing. Both types of writing have their respective styles, structures, and basic guidelines that writers use depending on the matter and the audience they are intended for. It’s crucial to understand the implications of mixing these two styles, as it can lead to confusion and a lack of clarity in your writing. In this article, we will explain the difference between general and academic writing, explain the nuances of academic writing and share some valuable tips for students and early career researchers to follow while undertaking formal academic writing assignments.

Critical characteristics of academic writing

Research paper

Literature review, lab reports.

Do’s
Don’ts

Critical characteristics of academic writing

Academic writing can be defined as a formal style of writing which is typically bound by strict scholarly conventions and is based on extensive research and analysis to support arguments. The main aim of academic writing is to contribute to new knowledge or insights within a specific field of study. Academic writing has its distinct characteristics and purpose. Some key characteristics of academic writing are:

It takes a formal approach and an impersonal tone.
It is usually written in the third person.
The sentences are well-structured and consist of arguments and evidence.
It is well-referenced, citing scholarly sources.
It is evidence-based.
The focus will be on a well-formulated and apparent research problem.
There is an appropriate choice of words.
The terminology used will be academic and discipline specific.
It is usually intended for a scholarly audience.

Hence, academic writing is not:

Informal
Personal
Conversational
Long-winded
Emotional

Types of academic writing

There are different types and categories of academic writing. Some key ones include:

On a topic or question provided by the instructor, students are expected to write a short and focused text consisting of a central idea supporting evidence that is properly cited and share researched analysis and interpretation. Sometimes, students are allowed to write an essay on a topic of their choice.

Unlike essays, research papers are more detailed, scholarly, and based on independent research. They should demonstrate their knowledge of the topic, employ analytical rigour, and demonstrate strength in dealing with a variety of sources. Your study and investigation should be able to make an original contribution to the existing body of knowledge in a specific discipline. This means that your research should add something new to the field, whether it’s a new perspective, a new method, or new data. It’s not enough to summarize what others have said; you need to bring something new to the table.

A literature review is a comprehensive and critical review of published literature on a particular topic. It provides an overview of the current knowledge in the field and helps situate your work within the larger body of knowledge. A literature review is not just a summary of what others have said about your topic; it’s a critical analysis that identifies gaps in the existing literature and provides context for your study. It’s an essential part of academic writing, as it shows that you’re familiar with the current state of research in your field and that your work is building on that knowledge.

A lab report communicates the aim, method, results, and conclusions of a scientific experiment. It is usually used in the fields of science, technology, engineering, and mathematics.

Dos and don’ts for students who are starting with academic writing

For practical academic writing skills, focus on developing a habit of writing clearly. Spend sufficient time planning your work, creating an outline, organizing your thoughts, and managing your time.
Depending upon your institutional or discipline-specific guidelines, you have the power to choose one style manual, such as the MLA, APA, or the Chicago Manual of Style. This choice is yours, and it’s essential to use the chosen style consistently. Each manual style has its own rules regarding how to write numbers, footnotes or endnotes, citations, and references. The choice of words is essential for clear and effective writing. Be concise and avoid confusing language.
While the use of discipline-specific terminology is essential, make sure to use it appropriately and accurately.
Structure your writing well so that the information is presented logically and straightforwardly so that the reader can quickly understand it. Make sure that your text is divided into chapters or sections. Structure your paragraphs so that each one presents a central point or idea and there is a clear link between the paragraphs. The thread of your research paper’s central argument should run throughout the text.
Citing sources: Follow academic conventions and accurately cite any quoted text, data, findings, or arguments from other studies used in your text.
Always make sure to proofread and edit your work before submitting it.

Don’ts

As mentioned earlier, academic writing adopts a formal tone. Therefore, it should altogether avoid conversational language and the use of regional dialects or slang.
Avoid overusing complex jargon just for the sake of using it.
Vague terms or statements that can confuse the reader should not be used.
Avoid plagiarism by following proper citation styles.

Academic writing requires a formal approach, an impersonal tone, and evidence-based arguments to support a well-formulated research problem. It is a distinct type of writing with its characteristics, structure, and purpose. To excel in academic writing, students need to understand the nuances of academic writing and differentiate it from general writing. They also need to structure their writing well, use appropriate terminology, and follow academic conventions. By following these tips, students can produce transparent, concise, and well-referenced academic papers that contribute to new knowledge and insights within their respective fields.

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.

Experience the future of academic writing – Sign up to Paperpal and start writing for free!

Do Plagiarism Checkers Detect AI Content?

Phd qualifying exam: tips for success , you may also like, how to use ai to enhance your college..., how to use paperpal to generate emails &..., ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., word choice problems: how to use the right..., how to avoid plagiarism when using generative ai..., what are journal guidelines on using generative ai..., types of plagiarism and 6 tips to avoid....

Dissertation & Thesis Guides
Basics of Dissertation & Thesis Writing
How to Create a Table of Contents for Dissertation, Thesis or Paper & Examples
Speech Topics
Basics of Essay Writing
Essay Topics
Other Essays
Main Academic Essays
Research Paper Topics
Basics of Research Paper Writing
Miscellaneous
Chicago/ Turabian
Data & Statistics
Methodology
Admission Writing Tips
Admission Advice
Other Guides
Student Life
Studying Tips
Understanding Plagiarism
Academic Writing Tips

Essay Guides
Research Paper Guides
Formatting Guides
Basics of Research Process
Admission Guides

How to Create a Table of Contents for Dissertation, Thesis or Paper & Examples

Table of contents

Use our free Readability checker

A dissertation table of contents is a list of the chapters and sections included in a dissertation or thesis, along with their page numbers. It helps to navigate the document easily and locate specific information. Each chapter or section should be listed with its corresponding page number. The table of contents should be formatted according to the guidelines of the specific style guide being used, such as APA or MLA.

We would guess that students usually start working on the table of contents at the last minute. It is quite apparent and makes sense, as this is the list of chapters and sections with page locations. Do you think it's easy?

From our experience, it can be quite tricky to organize everything according to APA, Chicago, or any other academic writing style. In this blog, we will discuss how to write a table of contents for a research paper , thesis or dissertation in Microsoft Word. We will create it together to guide students through the process.

Also, here you will find examples of table of contents created by thesis writers at StudyCrumb . Let’s go!

What Is a Table of Contents: Definition

It is obvious that the table of contents (TOC) is an essential manuscript part you can’t skip. If you are dealing with a dissertation, thesis or research paper, you need to know how to build it in accordance with academic guidance. This is a detailed roadmap for your work and outlined structure you can follow for a research presentation.

In case you are working on an essay or report, you may not include the table of contents, as it is a short academic text. But for the research paper, thesis or dissertation, table of contents is essential and required. It is possible to say the same about any Master’s project. It should be located between the dissertation abstract and introduction chapter. In most cases, it is about 2-3 pages long.

Our expert dissertation writing service prepared a great template that can be used for your work. Make your research formatting easy with ready solutions!

Types of Table of Contents

How to choose which table of contents will fit your research paper, thesis, dissertation, or report best? Make a decision based on your work length. Some academic writing styles, such as APA paper format or MLA style , have specific formatting for this list.

However, we will outline the most commonly used typology:

Single-level table of contents. At this type, we use only chapters. For instance, you will have an Introduction, Literature Review, methodology, and other chapters with page numbers. It can be used for shorter research work. For long writing forms like manuscripts, it can be too broad, and you will need to go into details.
Subdivided table of contents. The most frequently used form to organize the contents table. It will include not only chapters but also sections — a level 2 subheading for each part. It will help to be more specific about what to expect in each part of your research work.
Table of contents with multiple levels. This is a more divided structure, including subheadings with a level 3 for each section. Quite often, those subheadings can be rewritten or deleted during the last editing. It is essential to keep them in the right order.

Before you decide which type will work best for you, let us share with you some examples of each formatting style.

Example of Table of Contents With a Single Level

Introduction: The Misinformation Roots ………..…… 3 Literature Review .....................................….....………… 10 Research Methodology and Design ……................. 24 Results.............................................................................. 28 Discussion ....................................................................... 32

Sometimes, you will need to put an extra emphasis on subsections. Check this layout to see how your subheadings can be organized.

Example of Table of Contents Page with Subdivided Levels

Introduction: Information War ............……………….. 3 Background…………………………………….………..…… 4 Current State ……………………………………...…...…… 5 Defining Research Questions………………………. 9 Literature Review………………………...……………..……... 11 The Roots of Information Warfare ………....… 11 Information Wars …………………………….………..… 14 Cyber Wars Research ........................................ 17

If you are working on a lengthy, complex paper, this outline will suit your project most. It will help readers navigate through your document by breaking it down into smaller, more manageable sections.

Multi-Level Table of Contents Page Example

Introduction……………………………………………….......……….… 3 Emergence of Climate Change ………..……....….….. 3 Key Activist Groups in Climate Change .............. 5 Greenpeace International ………..…………......... 9 European Climate Foundation …….……………. 10 WWF ……………………………………….……….............. 11 Significant Movements ……………….………....……… 13 Literature Review ……………………………………......…………. 15

What Sections Should Be Included in a Table of Contents?

To start with, the scientific table of contents should include all chapters and its subheading. It is important to choose the formatting that will give your readers a full overview of your work from the very beginning. However, there are other chapters that you may miss constructing the 2-pager table. So, let's look at all you need to include:

Dissertation introduction
Literature review
Research methodology
Results section
Dissertation discussion
Conclusion of a thesis
Reference list. Mention a number of a page where you start listing your sources.
Appendices. For instance, if you have a data set, table or figure, include it in your research appendix .

This is how the ideal structured dissertation or research paper table of contents will look like. Remember that it still should take 2 pages. You need to choose the best formatting style to manage its length.

Tables, Figures, and Appendices in TOC

While creating a table of contents in a research paper, thesis or dissertation, you will need to include appendices in each case you have them. However, the formatting and adding tables and figures can vary based on the number and citation style. If you have more than 3 tables or figures, you may decide to have all of them at the end of your project. So, add them to the table of contents.

Figures, graphics, and diagrams in research papers, dissertations and theses should be numbered. If you use them from another source, ensure that you make a proper citation based on the chosen style guide.

Appendix in Table of Contents Example

Appendix A. Row Data Set…………………………………… 41 Appendix B. IBR Data………………………………………….… 43 Appendix C. SPSS Data………………………………………… 44

What Shouldn't Be Included in a Table of Contents?

When creating a dissertation table of contents, students want to include everything they have in a document. However, some components should not be on this page. Here is what we are talking about:

Thesis acknowledgement
Paper abstract
The content list itself

Acknowledgement and abstract should be located before the content list, so there is no need to add them. You need to present a clear structure that will help your readers to navigate through the work and quickly find any requested information.

How to Create a Table of Contents for a Research Paper or Dissertation In Word?

It may look like working with this list can take a long. But we have one proposal for our users. Instead of writing a table of contents manually, create it automatically in Microsoft Word. You do not need any specific tech knowledge to do this. Let’s go through this process step-by-step and explain how to make a table of contents for a research paper or dissertation in a few clicks.

Open Home tab and choose the style for your table of contents (ToC next).
Apply heading 1 to your chapters, heading 2 to the subheading, and if needed heading 3 to the level 3 heading.
Next, you are going to create a research paper or PhD dissertation table of contents. Open References and choose ToC.
Choose the citation style for your work. For example, let’s choose APL for now. Meeting all style requirements (bold font, title formatting, numbers) is essential.
Define the number of levels for your dissertation or thesis table of contents. In case you want to have 3 levels, choose Automatic Table 2.
You are done! Click ok, and here is your page with listed chapters!

You see how easy it can be! Every time you make changes to your text or headings, it will be automatic.

Updating Your Table of Contents in MS Word

Table of contents of a research paper or dissertation is created, and you continue to edit your work until submission. It is common practice, and with MS Word, you can automate all the updates.

Let’s outline this process in our step-by-step guide!

Right-click on your ToC in a document.
Update field section is next.
Choose “update ToC."
Here, you can update your entire ToC — choose an option that works the best for you!

As you may see, working with automated solutions is much easier when you write a dissertation which has manifold subsections. That is why it is better to learn how to work on MS Word with the content list meaning be able to manage it effectively.

Table of Contents Examples

From our experience, students used to think that the content list was quite a complicated part of the work. Even with automated solutions, you must be clear about what to include and how to organize formatting. To solve the problem and answer all your questions, use our research paper or dissertation contents page example. Our paper writers designed a sample table of contents to illustrate the best practices and various styles in formatting the work.

Check our samples to find advanced options for organizing your own list.

Example of Table of Contents in Research Paper

Research Paper Table of Contents Example

As you can see, this contents page includes sections with different levels.

Thesis/Dissertation Table of Contents Example

Have a question about your specific case? Check samples first, as we are sure you can get almost all the answers in our guides and sample sets.

>> Read more: APA Format Table of Contents

Tips on Creating a Table of Contents

To finalize all that we shared on creating the table of contents page, let’s go through our tips list. We outline the best advice to help you with a dissertation table of contents.

Use automated solutions for creating a list of chapters for your report, research papers, or dissertations — it will save you time in the future.
Be clear with the formatting style you use for the research.
Choose the best level type of list based on the paper length.
Update a list after making changes to the text.
Check the page list before submitting the work.

Bottom Line on Making Table of Contents for Dissertations/ Papers

To summarize, working with a research paper, thesis or dissertation table of contents can be challenging. This article outlines how to create a table of contents in Word and how to update it appropriately. You can learn what to include in the content list, how long it can be, and where to locate it. Write your work using more than one table of contents sample we prepared for students. It is often easy to check how the same list was made for other dissertations before finalizing yours. We encourage you to learn how to create a list with pages automatically and update it. It will definitely make your academic life easier.

Not sure if your work 's quality level is enough for getting a top-notch result? We’ve got you covered! Our team of skilled academic writers is always ready to help once you ask “ write my dissertation for me !" Just select your writer, send them your requirements and get a custom study tailored to your instructions.

Joe Eckel is an expert on Dissertations writing. He makes sure that each student gets precious insights on composing A-grade academic writing.

Protein domains of low sequence complexity—dark matter of the proteome

Steven L. McKnight
Full Text (PDF)
Supplemental Material

In this Perspective, McKnight discusses the evolution of proteins and the biological relevance of proteins with low-complexity domains (LCDs). McKnight delves into the biophysical and biochemical properties of LCDs that underlie their function and make them challenging to study, further highlighting the outstanding questions currently plaguing the field.

RESEARCH PAPERS

WRN exonuclease imparts high fidelity on translesion synthesis by Y family DNA polymerases

Jung-Hoon Yoon ,
Karthi Sellamuthu ,
Louise Prakash ,
and Satya Prakash

In this study, Yoon et al. discover that Werner syndrome protein (WRN) and WRN-interacting protein 1 (WRNIP1) assemble with Rev1 and Y family DNA polymerases (Polη, Polι, or Polκ) upon DNA damage to facilitate error-free translesion DNA synthesis. This role requires WRN's exonuclease function to afford higher fidelity during replication through DNA lesions, which is an essential mechanism that protects against genomic instability.

NFATC2IP is a mediator of SUMO-dependent genome integrity

Tiffany Cho ,
Lisa Hoeg ,
Dheva Setiaputra ,
and Daniel Durocher

In this study, Cho et al. show that NFATC2IP plays a key role in maintaining genome stability and cell survival when protein SUMOylation is suppressed. NFATC2IP interacts with SUMO machinery (namely, SMC5/6 and UBC9) via its SUMO-like domains to control formation of chromosomal aberrations such as micronuclei and chromatin bridges and to promote SUMOylation of chromatin-associated proteins.

A germline point mutation in the MYC-FBW7 phosphodegron initiates hematopoietic malignancies

Brian Freie ,
Patrick A. Carroll ,
Barbara J. Varnum-Finney ,
Erin L. Ramsey ,
Vijay Ramani ,
Irwin Bernstein ,
and Robert N. Eisenman

In this study, Freie et al. demonstrate that a single point mutation (T58A) in MYC's phosphodegron domain stabilizes MYC and deregulates the expression of its target genes, resulting in aberrant self-renewal and apoptotic resistance in hematopoietic progenitors. The authors demonstrate that MYC-T58A multipotential hematopoietic progenitors alter MYC genomic occupancy, target gene transcription, and cell metabolism and are associated with the initiation of hematopoietic malignancies.

Haploinsufficiency of phosphodiesterase 10A activates PI3K/AKT signaling independent of PTEN to induce an aggressive glioma phenotype

Nicholas Nuechterlein ,
Allison Shelbourn ,
Frank Szulzewsky ,
Sonali Arora ,
Michelle Casad ,
Siobhan Pattwell ,
Leyre Merino-Galan ,
Erik Sulman ,
Sumaita Arowa ,
Neriah Alvinez ,
Miyeon Jung ,
Desmond Brown ,
Kayen Tang ,
Sadhana Jackson ,
Stefan Stoica ,
Prashant Chittaboina ,
Yeshavanth K. Banasavadi-Siddegowda ,
Hans-Georg Wirsching ,
Nephi Stella ,
Linda Shapiro ,
Patrick Paddison ,
Anoop P. Patel ,
Mark R. Gilbert ,
Zied Abdullaev ,
Kenneth Aldape ,
Drew Pratt ,
Eric C. Holland ,
and Patrick J. Cimino

In this study, Nuechterlein et al. show that the deficiency of phosphodiesterase PDE10A activates PI3K/AKT/mTOR signaling independent of PTEN and promotes proneural-to-mesenchymal transition and gliomagenesis. Their work highlights that PDE10A may function as a potential tumor suppressor and that PDE10A-deficient glioblastomas may be therapeutically vulnerable to PI3K inhibition.

CORRIGENDUM

Corrigendium: Effects of RAS on the genesis of embryonal rhabdomyosarcoma

David M. Langenau ,
Matthew D. Keefe ,
Narie Y. Storer ,
Jeffrey R. Guyon ,
Jeffery L. Kutok ,
Xiuning Le ,
Wolfram Goessling ,
Donna S. Neuberg ,
Louis M. Kunkel ,
and Leonard I. Zon
Index By Author
About the Cover
Front Matter (PDF)
Advertising (PDF)

Find articles in this issue containing these words:

Current Issue

March 1, 2024, 38 (5-6)

In This Issue

SUMO-dependent genome maintenance promoted by NFATC2IP
Gain-of-function MYC phosphodegron mutation initiates lymphoma and leukemia
Low-complexity domains: properties, functions, and evolution
Alert me to new issues of G&D
Advance Online Articles
Submit a Manuscript
G&D in the News
Editorial Board
Author Information
E-mail Alerts & RSS Feeds
Recommend to Your Library
Job Opportunities

Author Info

Print ISSN: 0890-9369
Online ISSN: 1549-5477

This paper is in the following e-collection/theme issue:

Published on 19.4.2024 in Vol 26 (2024)

Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study

Authors of this article:

Original Paper

Josephine McMurray 1, 2 * , MBA, PhD ;
AnneMarie Levy 1 * , MSc, PhD ;
Wei Pang 1, 3 * , BTM ;
Paul Holyoke 4 , PhD

1 Lazaridis School of Business & Economics, Wilfrid Laurier University, Brantford, ON, Canada

2 Health Studies, Faculty of Human and Social Sciences, Wilfrid Laurier University, Brantford, ON, Canada

3 Biomedical Informatics & Data Science, Yale University, New Haven, CT, United States

4 SE Research Centre, Markham, ON, Canada

*these authors contributed equally

Corresponding Author:

Josephine McMurray, MBA, PhD

Lazaridis School of Business & Economics

Wilfrid Laurier University

73 George St

Brantford, ON, N3T3Y3

Phone: 1 548 889 4492

Email: [email protected]

Background: With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions.

Objective: The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen.

Methods: A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN’s scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments.

Results: The study found that the SCREEN’s longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing.

Conclusions: The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population.

International Registered Report Identifier (IRRID): RR2-10.2196/25520

Introduction

Mild cognitive impairment (MCI) is a syndrome characterized by a slight but noticeable and measurable deterioration in cognitive abilities, predominantly memory and thinking skills, that is greater than expected for an individual’s age and educational level [ 1 , 2 ]. The functional impairments associated with MCI are subtle and often impair instrumental activities of daily living (ADL). Instrumental ADL include everyday tasks such as managing finances, cooking, shopping, or taking regularly prescribed medications and are considered more complex than ADL such as bathing, dressing, and toileting [ 3 , 4 ]. In cases in which memory impairment is the primary indicator of the disease, MCI is classified as amnesic MCI and when significant impairment of non–memory-related cognitive domains such as visual-spatial or executive functioning is dominant, MCI is classified as nonamnesic [ 5 ].

Cognitive decline, more so than cancer and cardiovascular disease, poses a substantial threat to an individual’s ability to live independently or at home with family caregivers [ 6 ]. The Centers for Disease Control and Prevention reports that 1 in 8 adults aged ≥60 years experiences memory loss and confusion, with 35% reporting functional difficulties with basic ADL [ 7 ]. The American Academy of Neurology estimates that the prevalence of MCI ranges from 13.4% to 42% in people aged ≥65 years [ 8 ], and a 2023 meta-analysis that included 233 studies and 676,974 participants aged ≥50 years estimated that the overall global prevalence of MCI is 19.7% [ 9 ]. Once diagnosed, the prognosis for MCI is variable, whereby the impairment may be reversible; the rate of decline may plateau; or it may progressively worsen and, in some cases, may be a prodromal stage to dementia [ 10 - 12 ]. While estimates vary based on sample (community vs clinical), annual rates of conversion from MCI to dementia range from 5% to 24% [ 11 , 12 ], and those who present with multiple domains of cognitive impairment are at higher risk of conversion [ 5 ].

The risk of developing MCI rises with age, and while there are no drug treatments for MCI, nonpharmacologic interventions may improve cognitive function, alleviate the burden on caregivers, and potentially delay institutionalization should MCI progress to dementia [ 13 ]. To overcome the challenges of early diagnosis, which currently depends on self-detection, family observation, or health care provider (HCP) recognition of symptoms, screening high-risk groups for MCI or dementia is suggested as a solution [ 13 ]. However, the Canadian Task Force on Preventive Health Care recommends against screening adults aged ≥65 years due to a lack of meaningful evidence from randomized controlled trials and the high false-positive rate [ 14 - 16 ]. The main objective of a screening test is to reduce morbidity or mortality in at-risk populations through early detection and intervention, with the anticipated benefits outweighing potential harms. Using brief screening tools in primary care might improve MCI case detection, allowing patients and families to address reversible causes, make lifestyle changes, and access disease-modifying treatments [ 17 ].

There is no agreement among experts as to which tests or groups of tests are most predictive of MCI [ 16 ], and the gold standard approach uses a combination of positive results from neuropsychological assessments, laboratory tests, and neuroimaging to infer a diagnosis [ 8 , 18 ]. The clinical heterogeneity of MCI complicates its diagnosis because it influences not only memory and thinking abilities but also mood, behavior, emotional regulation, and sensorimotor abilities, and patients may present with any combination of symptoms with varying rates of onset and decline [ 4 , 8 ]. For this reason, a collaborative approach between general practitioners and specialists (eg, geriatricians and neurologists) is often required to be confident in the diagnosis of MCI [ 8 , 19 , 20 ].

In Canada, diagnosis often begins with screening for cognitive impairment followed by referral for additional testing; this process takes, on average, 5 months [ 20 ]. The current usual practice screening tools for MCI are the Mini-Mental State Examination (MMSE) [ 21 , 22 ] and the Montreal Cognitive Assessment (MoCA) 8.1 [ 3 ]. Both are paper-and-pencil screens administered in 10 to 15 minutes, scored out of 30, and validated as MCI screening tools across diverse clinical samples [ 23 , 24 ]. Universally, the MMSE is most often used to screen for MCI [ 20 , 25 ] and consists of 20 items that measure orientation, immediate and delayed recall, attention and calculation, visual-spatial skills, verbal fluency, and writing. The MoCA 8.1 was developed to improve on the MMSE’s ability to detect early signs of MCI, placing greater emphasis on evaluating executive function as well as language, memory, visual-spatial skills, abstraction, attention, concentration, and orientation across 30 items [ 24 , 26 ]. Scores of <24 on the MMSE or ≤25 on the MoCA 8.1 signal probable MCI [ 21 , 27 ]. Lower cutoff scores for both screens have been recommended to address evidence that they lack specificity to detect mild and early cases of MCI [ 4 , 28 - 31 ]. The clinical efficacy of both screens for tracking change in cognition over time is limited as they are also subject to practice effects with repeated administration [ 32 ].

Novel screening tools, including the Quick Mild Cognitive Impairment (Qmci) screen, have been developed with the goal of improving the accuracy of detecting MCI [ 33 , 34 ]. The Qmci is a sensitive and specific tool that differentiates normal cognition from MCI and dementia and is more accurate at differentiating MCI from controls than either the MoCA 8.1 (Qmci area under the curve=0.97 vs MoCA 8.1 area under the curve=0.92) [ 25 , 35 ] or the Short MMSE [ 33 , 36 ]. It also demonstrates high test-retest reliability (intraclass correlation coefficient [ICC]=0.88) [ 37 ] and is clinically useful as a rapid screen for MCI as the Qmci mean is 4.5 (SD 1.3) minutes versus 9.5 (SD 2.8) minutes for the MoCA 8.1 [ 25 ].

The COVID-19 pandemic and the necessary shift to virtual health care accelerated the use of digital assessment tools, including MCI screening tools such as the electronic MoCA 8.1 [ 38 , 39 ], and the increased use and adoption of technology (eg, smartphones and tablets) by older adults suggests that a lack of proficiency with technology may not be a barrier to the use of such assessment tools [ 40 , 41 ]. BrainFx is a for-profit firm that creates proprietary software designed to assess cognition and changes in neurofunction that may be caused by neurodegenerative diseases (eg, MCI or dementia), stroke, concussions, or mental illness using ecologically relevant tasks (eg, prioritizing daily schedules and route finding on a map) [ 42 ]. Their assessments are administered via a tablet and stylus. The BrainFx 360 performance assessment (referred to hereafter as the 360) is a 90-minute digitally administered test that was designed to assess cognitive, physical, and psychosocial areas of neurofunction across 26 cognitive domains using 49 tasks that are timed and scored [ 42 ]. The BrainFx SCREEN (referred to hereafter as the SCREEN) is a short digital version of the 360 that includes 7 of the cognitive domains included in the 360, is estimated to take approximately 10 to 15 minutes to complete, and was designed to screen for early detection of cognitive impairment [ 43 , 44 ]. Upon completion of any BrainFx assessment, the results of the 360 or SCREEN are added to the BrainFx Living Brain Bank (LBB), which is an electronic database that stores all completed 360 and SCREEN assessments and is maintained by BrainFx. An electronic report is generated by BrainFx comparing an individual’s results to those of others collected and stored in the LBB. Normative data from the LBB are used to evaluate and compare an individual’s results.

The 360 has been used in clinical settings to assess neurofunction among youth [ 45 ] and anecdotally in other rehabilitation settings (T Milner, personal communication, May 2018). To date, research on the 360 indicates that it has been validated in healthy young adults (mean age 22.9, SD 2.4 years) and that the overall test-retest reliability of the tool is high (ICC=0.85) [ 42 ]. However, only 2 of the 7 tasks selected to be included in the SCREEN produced reliability coefficients of >0.70 (visual-spatial and problem-solving abilities) [ 42 ]. Jones et al [ 43 ] explored the acceptability and perceived usability of the SCREEN with a small sample (N=21) of Canadian Armed Forces veterans living with posttraumatic stress disorder. A structural equation model based on the Unified Theory of Acceptance and Use of Technology suggested that behavioral intent to use the SCREEN was predicted by facilitating conditions such as guidance during the test and appropriate resources to complete the test [ 43 ]. However, the validity, reliability, and sensitivity of the SCREEN for detecting cognitive impairment have not been tested.

McMurray et al [ 44 ] designed a protocol to assess the validity, reliability, and sensitivity of the SCREEN for detecting early signs of MCI in asymptomatic adults aged ≥55 years in a primary care setting (5 Family Health Teams [FHTs]). The protocol also used a series of semistructured interviews and surveys guided by the fit between individuals, task, technology, and environment framework [ 46 ], a health-specific model derived from the Task-Technology Fit model by Goodhue and Thompson [ 47 ], to explore the SCREEN’s acceptability and use by HCPs and patients in primary care settings (manuscript in preparation). This study is a psychometric evaluation of the SCREEN’s validity, reliability, and sensitivity for detecting MCI in asymptomatic adults aged ≥55 years in primary care settings.

Study Location, Design, and Data Collection

This was a concurrent, mixed methods, prospective study using a quasi-experimental design. Participants were recruited from 5 primary care FHTs (characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. FHTs that used a registered occupational therapist on staff were eligible to participate in the study, and participating FHTs received a nominal compensatory payment for the time the HCPs spent in training; collecting data for the study; administering the SCREEN, Qmci, and Geriatric Anxiety Scale–10 (GAS-10); and communicating with the research team. A multipronged recruitment approach was used [ 44 ]. A designated occupational therapist at each location was provided with training and equipment to recruit participants, administer assessment tools, and submit collected data to the research team.

The research protocol describing the methods of both the quantitative and qualitative arms of the study is published elsewhere [ 44 ].

Ethical Considerations

This study was approved by the Wilfrid Laurier University Research Ethics Board (ORE 5820) and was reviewed and approved by each FHT. Participants (HCPs, patients, and administrative executives) read and signed an information and informed consent package in advance of taking part in the study. We complied with recommendations for obtaining informed consent and conducting qualitative interviews with persons with dementia when recruiting patients who may be affected by neurocognitive diseases [ 48 - 50 ]. In addition, at the end of each SCREEN assessment, patients were required to provide their consent (electronic signature) to contribute their anonymized scores to the database of SCREEN results maintained by BrainFx. Upon enrolling in the study, participants were assigned a unique identification number that was used in place of their name on all study documentation to anonymize the data and preserve their confidentiality. A master list matching participant names with their unique identification number was stored in a password-protected file by the administering HCP and principal investigator on the research team. The FHTs received a nominal compensatory payment to account for their HCPs’ time spent administering the SCREEN, collecting data for the study, and communicating with the research team. However, the individual HCPs who volunteered to participate and the patient participants were not financially compensated for taking part in the study.

Participants

Patients who were rostered with the FHT, were aged ≥55 years, and had no history of MCI or dementia diagnoses to better capture the population at risk of early signs of cognitive impairment were eligible to participate [ 51 , 52 ]. It was necessary for the participants to be rostered with the FHTs to ensure that the HCPs could access their electronic medical record to confirm eligibility and record the testing sessions and results and to ensure that there was a responsible physician for referral if indicated. As the SCREEN is administered using a tablet, participants had to be able to read and think in English and discern color, have adequate hearing and vision to interact with the administering HCP, read 12-point font on the tablet, and have adequate hand and arm function to manipulate and hold the tablet. The exclusion criteria used in the study included colorblindness and any disability that might impair the individual’s ability to hold and interact with the tablet. Prospective participants were also excluded based on a diagnosis of conditions that may result in MCI or dementia-like symptoms, including major depression that required hospitalization, psychiatric disorders (eg, schizophrenia and bipolar disorder), psychopathology, epilepsy, substance use disorders, or sleep apnea (without the use of a continuous positive airway pressure machine) [ 52 ]. Patients were required to complete a minimum of 2 screening sessions spaced 3 months apart to participate in the study and, depending on when they enrolled to participate, could complete a maximum of 4 screening sessions over a year.

Data Collection Instruments

Gas-10 instrument.

A standardized protocol was used to collect demographic data, randomly administer the SCREEN and the Qmci (a validated screening tool for MCI), and administer the GAS-10 immediately before and after the completion of the first MCI screen at each visit [ 44 ]. This was to assess participants’ general anxiety as it related to screening for cognitive impairment at the time of the assessment, any change in subjective ratings after completion of the first MCI screen, and change in anxiety between appointments. The GAS-10 is a 10-item, self-report screen for anxiety in older adults [ 53 ] developed for rapid screening of anxiety in clinical settings (the GAS-10 is the short form of the full 30-item Geriatric Anxiety Scale [GAS]) [ 54 ]. While 3 subscales are identified, the GAS is reported to be a unidimensional scale that assesses general anxiety [ 55 , 56 ]. Validation of the GAS-10 suggests that it is optimal for assessing average to moderate levels of anxiety in older adults, with subscale scores that are highly and positively correlated with the GAS and high internal consistency [ 53 ]. Participants were asked to use a 4-point Likert scale (0= not at all , 1= sometimes , 2= most of the time , and 3= all of the time ) to rate how often they had experienced each symptom over the previous week, including on the day the test was administered [ 54 ]. The GAS-10 has a maximum score of 30, with higher scores indicating higher levels of anxiety [ 53 , 54 , 57 ].

HCPs completed the required training to become certified BrainFx SCREEN administrators before the start of the study. To this end, HCPs completed a web-based training program (developed and administered through the BrainFx website) that included 3 self-directed training modules. For the purpose of the study, they also participated in 1 half-day in-person training session conducted by a certified BrainFx administrator (T Milner, BrainFx chief executive officer) at one of the participating FHT locations. The SCREEN (version 0.5; beta) was administered on a tablet (ASUS ZenPad 10.1” IPS WXGA display, 1920 × 1200, powered by a quad-core 1.5 GHz, 64-bit MediaTek MTK 8163A processor with 2 GB RAM and 16-GB storage). The tablet came with a tablet stand for optional use and a dedicated stylus that is recommended for completion of a subset of questions. At the start of the study, HCPs were provided with identical tablets preloaded with the SCREEN software for use in the study. The 7 tasks on the SCREEN are summarized in Table 1 and were taken directly from the 360 based on a clustering and regression analysis of LBB records in 2016 (N=188) [ 58 ]. A detailed description of the study and SCREEN administration procedures was published by McMurray et al [ 44 ].

An activity score is generated for each of the 7 tasks on the SCREEN. It is computed based on a combination of the accuracy of the participant’s response and the processing speed (time in seconds) that it takes to complete the task. The relative contribution of accuracy and processing speed to the final activity score for each task is proprietary to BrainFx and unknown to the research team. The participant’s activity score is compared to the mean activity score for the same task at the time of testing in the LBB. The mean activity score from the LBB may be based on the global reference population (ie, all available SCREEN results in the LBB), or the administering HCP may select a specific reference population by filtering according to factors including but not limited to age, sex, or diagnosis. If the participant’s activity score is >1 SD below the LBB activity score mean for that task, it is labeled as an area of challenge . Each of the 7 tasks on the SCREEN are evaluated independently of each other, producing a report with 7 activity scores showing the participant’s score, the LBB mean score, and the SD. The report also provides an overall performance and processing speed score. The overall performance score is an average of all 7 activity scores; however, the way in which the overall processing speed score is generated remains proprietary to BrainFx and unknown to the research team. Both the overall performance and processing speed scores are similarly evaluated against the LBB and identified as an area of challenge using the criteria described previously. For the purpose of this study, participants’ mean activity scores on the SCREEN were compared to the results of people aged ≥55 years in the LBB.

The Qmci evaluated 6 cognitive domains: orientation (10 points), registration (5 points), clock drawing (15 points), delayed recall (20 points), verbal fluency (20 points), and logical memory (30 points) [ 59 ]. Administering HCPs scored the text manually, with each subtest’s points contributing to the overall score out of 100 points, and the cutoff score to distinguish normal cognition from MCI was ≤67/100 [ 60 ]. Cutoffs to account for age and education have been validated and are recommended as the Qmci is sensitive to these factors [ 60 ]. A 2019 meta-analysis of the diagnostic accuracy of MCI screening tools reported that the sensitivity and specificity of the Qmci for distinguishing MCI from normal cognition is similar to usual standard-of-care tools (eg, the MoCA, Addenbrooke Cognitive Examination–Revised, Consortium to Establish a Registry for Alzheimer’s Disease battery total score, and Sunderland Clock Drawing Test) [ 61 ]. The Qmci has also been translated into >15 different languages and has undergone psychometric evaluation across a subset of these languages. While not as broadly adopted as the MoCA 8.1 in Canada, its psychometric properties, administration time, and availability for use suggested that the Qmci was an optimal assessment tool for MCI screening in FHT settings during the study.

Psychometric Evaluation

To date, the only published psychometric evaluation of any BrainFx tool is by Searles et al [ 42 ] in Athletic Training & Sports Health Care ; it assessed the test-retest reliability of the 360 in 15 healthy adults between the ages of 20 and 25 years. This study evaluated the psychometric properties of the SCREEN and included a statistical analysis of the tool’s internal consistency, construct validity, test-retest reliability, and sensitivity and specificity. McMurray et al [ 44 ] provide a detailed description of the data collection procedures for administration of the SCREEN and Qmci completed by participants at each visit.

Validity Testing

Face validity was outside the scope of this study but was implied, and assumptions are reported in the Results section. Construct validity, whether the 7 activities that make up the SCREEN were representative of MCI, was assessed through comparison with a substantive body of literature in the domain and through principal component analysis using varimax rotation. Criterion validity measures how closely the SCREEN results corresponded to the results of the Qmci (used here as an “imperfect gold standard” for identifying MCI in older adults) [ 62 ]. A BrainFx representative hypothesized that the ecological validity of the SCREEN questions (ie, using tasks that reflect real-world activities to detect early signs of cognitive impairment) [ 63 ] makes it a more sensitive tool than other screens (T Milner, personal communication, May 2018) and allows HCPs to equate activity scores on the SCREEN with real-world functional abilities. Criterion validity was explored first using cross-tabulations to calculate the sensitivity and specificity of the SCREEN compared to those of the Qmci. Conventional screens such as the Qmci are scored by taking the sum of correct responses on the screen and a cutoff score derived from normative data to distinguish normal cognition from MCI. The SCREEN used a different method of scoring whereby each of the 7 tasks was scored and evaluated independently of each other and there were no recommended guidelines for distinguishing normal cognition from MCI based on the aggregate areas of challenge identified by the SCREEN. Therefore, to compare the sensitivity and specificity of the SCREEN against those of the Qmci, the results of both screens were coded into a binary format as 1=healthy and 2=unhealthy, where healthy denoted no areas of challenge identified through the SCREEN and a Qmci score of ≥67. Conversely, unhealthy denoted one or more areas of challenge identified through the SCREEN and a Qmci score of <67.

Criterion validity was further explored using discrepant analysis via a resolver test [ 44 ]. Following the administration of the SCREEN and Qmci, screen results were evaluated by the administering HCP. HCPs were instructed to refer the participant for follow-up with their primary care physician if the Qmci result was <67 regardless of whether any areas of challenge were identified on the SCREEN. However, HCPs could use their clinical judgment to refer a participant for physician follow-up based on the results of the SCREEN or the Qmci, and all the referral decisions were charted on the participant’s electronic medical record following each visit and screening. In discrepant analysis, the results of the imperfect gold standard [ 64 ], as was the role of the Qmci in this study, were compared with the SCREEN results. A resolver test (classified as whether the HCP referred the patient to a physician for follow-up based on their performance on the SCREEN and the Qmci) was used on discordant results [ 64 , 65 ] to determine sensitivity and specificity. To this end, a new variable, Referral to a Physician for Cognitive Impairment , was coded as the true status (1=no referral; 2=referral was made) and compared to the Qmci as the imperfect gold standard (1=healthy; 2=unhealthy).

Reliability Testing

The reliability of a screening instrument is its ability to consistently measure an attribute and how well its component measures fit together conceptually. Internal consistency identifies whether the items in a multi-item scale are measuring the same underlying construct; the internal consistency of the SCREEN was assessed using the Cronbach α. Test-retest reliability refers to the ability of a measurement instrument to reproduce results over ≥2 occasions (assuming the underlying conditions have not changed) and was assessed using paired t tests (2-tailed), ICC, and the κ coefficient. In this study, participants completed both the SCREEN and the Qmci in the same sitting in a random sequence on at least 2 different occasions spaced 3 months apart (administration procedures are described elsewhere) [ 44 ]. In some instances, the screens were administered to the same participant on 4 separate occasions spaced 3 months apart each, and this provided up to 3 separate opportunities to conduct test-retest reliability analyses and investigate the effects of repeated practice. There are no clear guidelines on the optimal time between tests [ 66 , 67 ]; however, Streiner and Kottner [ 68 ] and Streiner [ 69 ] recommend longer periods between tests (eg, at least 10-14 days) to avoid recall bias, and greater practice effects have been experienced with shorter test-retest intervals [ 32 ].

Analysis of the quantitative data was completed using Stata (version 17.0; StataCorp). Assumptions of normality were not violated, so parametric tests were used. Collected data were reported using frequencies and percentages and compared using the chi-square or Fisher exact test as necessary. Continuous data were analyzed for central tendency and variability; categoric data were presented as proportions. Normality was tested using the Shapiro-Wilk test, and nonparametric data were tested using the Mann-Whitney U test. A P value of .05 was considered statistically significant, with 95% CIs provided where appropriate. We powered the exploratory analysis to validate the SCREEN using an estimated effect size of 12%—understanding that Canadian prevalence rates of MCI were not available [ 1 ]—and determined that the study required at least 162 participants. For test-retest reliability, using 90% power and a 5% type-I error rate, a minimum of 67 test results was required.

The time taken for participants to complete the SCREEN was recorded by the HCPs at the time of testing; there were 6 missing HCP records of time to complete the SCREEN. For these 6 cases of missing data, we imputed the mean time to complete the SCREEN by all participants who were tested by that HCP and used this to populate the missing cells [ 70 ]. There were 3 cases of missing data related to the SCREEN reports. More specifically, the SCREEN report generated by BrainFx did not include 1 or 2 data points each for the route finding, divided attention, and prioritizing tasks. The clinical notes provided by the HCP at the time of SCREEN administration did not indicate that the participant had not completed those questions, and it was not possible to determine the root cause of the missing data in report generation according to BrainFx (M Milner, personal communication, July 7, 2020). For continuous variables in analyses such as exploratory factor analysis, Cronbach α, and t test, missing values were imputed using the mean. However, for the coded healthy and unhealthy categorical variables, values were not imputed.

Data collection began in January 2019 and was to conclude on May 31, 2020. However, the emergence of the global COVID-19 pandemic resulted in the FHTs and Wilfrid Laurier University prohibiting all in-person research starting on March 16, 2020.

Participant Demographics

A total of 154 participants were recruited for the study, and 20 (13%) withdrew following their first visit to the FHT. The data of 65% (13/20) of the participants who withdrew were included in the final analysis, and the data of the remaining 35% (7/20) were removed, either due to their explicit request (3/7, 43%) or because technical issues at the time of testing rendered their data unusable (4/7, 57%). These technical issues were related to software issues (eg, any instance in which the patient or HCP interacted with the SCREEN software and followed the instructions provided, the software did not work as expected [ie, objects did not move where they were dragged or tapping on objects failed to highlight the object], and the question could not be completed). After attrition, a total of 147 individuals aged ≥55 years with no previous diagnosis of MCI or dementia participated in the study ( Table 2 ). Of the 147 participants, 71 (48.3%) took part in only 1 round of screening on visit 1 (due to COVID-19 restrictions imposed on in-person research that prevented a second visit). The remaining 51.7% (76/147) of the participants took part in ≥2 rounds of screening across multiple visits (76/147, 51.7% participated in 2 rounds; 22/147, 15% participated in 3 rounds; and 13/147, 8.8% participated in 4 rounds of screening).

The sample population was 65.3% (96/147) female (mean 70.2, SD 7.9 years) and 34.7% (51/147) male (mean 72.5, SD 8.1 years), with age ranging from 55 to 88 years; 65.3% (96/147) achieved the equivalent of or higher than a college diploma or certificate ( Table 2 ); and 32.7% (48/147) self-reported living with one or more chronic medical conditions ( Table 3 ). At the time of screening, 73.5% (108/147) of participants were also taking medications with side effects that may include impairments to memory and thinking abilities [ 71 - 75 ]; therefore, medication use was accounted for in a subset of the analyses. Finally, 84.4% (124/147) of participants self-reported regularly using technology (eg, smartphone, laptop, or tablet) with high proficiency. A random sequence generator was used to determine the order for administering the MCI screens; the SCREEN was administered first 51.9% (134/258) of the time.

Construct Validity

Construct validity was assessed through a review of relevant peer-reviewed literature that compared constructs included in the SCREEN with those identified in the literature as 2 of the most sensitive tools for MCI screening: the MoCA 8.1 [ 76 ] and the Qmci [ 25 ]. Memory, language, and verbal skills are assessed in the MoCA and Qmci but are absent from the SCREEN. Tests of verbal fluency and logical memory have been shown to be particularly sensitive to early cognitive changes [ 77 , 78 ] but are similarly absent from the SCREEN.

Exploratory factor analysis was performed to examine the SCREEN’s ability to reliably measure risk of MCI. The Kaiser-Meyer-Olkin measure yielded a value of 0.79, exceeding the commonly accepted threshold of 0.70, indicating that the sample was adequate for factor analysis. The Bartlett test of sphericity returned a chi-square value of χ 2 21 =167.1 ( P <.001), confirming the presence of correlations among variables suitable for factor analysis. A principal component analysis revealed 2 components with eigenvalues of >1, cumulatively accounting for 52.12% of the variance, with the first factor alone explaining 37.8%. After the varimax rotation, the 2 factors exhibited distinct patterns of loadings, with the visual-spatial ability factor loading predominantly on the second factor. The SCREEN tasks, except for visual-spatial ability, loaded substantially on the factors (>0.5), suggesting that the SCREEN possesses good convergent validity for assessing the risk of MCI.

Criterion Validity

The coding of SCREEN scores into a binary healthy and unhealthy outcome standardized the dependent variable to allow for criterion testing. Criterion validity was assessed using cross-tabulations and the analysis of confusion matrices and provided insights into the sensitivity and specificity of the SCREEN when compared to the Qmci. Of the 144 cases considered, 20 (13.9%) were true negatives, and 74 (51.4%) were true positives. The SCREEN’s sensitivity, which reflects its capacity to accurately identify healthy individuals (true positives), was 63.25% (74 correct identifications/117 actual positives). The specificity of the test, indicating its ability to accurately identify unhealthy individuals (true negatives), was 74.07% (20 correct identifications/27 actual negatives). Then, sensitivity and specificity were derived using discrepant analysis and a resolver test previously described (whether the HCP referred the participant to a physician following the screens). The results were identical, the estimate of the SCREEN sensitivity was 63.3% (74/117), and the estimate of the specificity was 74% (20/27).

Internal Reliability

A Cronbach α=0.70 is acceptable, and at least 0.90 is required for clinical instruments [ 79 ]. The estimate of internal consistency for the SCREEN (N=147) was Cronbach α=0.63.

Test-Retest Reliability

Test-retest reliability analyses were conducted using ICC for the SCREEN activity scores and the κ coefficient for the healthy and unhealthy classifications. Guidelines for interpretation of the ICC suggest that anything <0.5 indicates poor reliability and anything between 0.5 and 0.75 suggests moderate reliability [ 80 ]; the ICC for the SCREEN activity scores was 0.54. With respect to the κ coefficient, a κ value of <0.2 is considered to have no level of agreement, a κ value of 0.21 to 0.39 is considered minimal, a κ value of 0.4 to 0.59 is considered weak agreement, and anything >0.8 suggests strong to almost perfect agreement [ 81 ]. The κ coefficient for healthy and unhealthy classifications was 0.15.

Analysis of the Factors Impacting Healthy and Unhealthy Results

The Spearman rank correlation was used to assess the relationships between participants’ overall activity score on the SCREEN and their total time to complete the SCREEN; age, sex, and self-reported levels of education; technology use; medication use; amount of sleep; and level of anxiety (as measured using the GAS-10) at the time of SCREEN administration. Lower overall activity scores were moderately correlated with being older ( r s142 =–0.57; P <.001) and increased total time to complete the SCREEN ( r s142 =0.49; P <.001). There was also a moderate inverse relationship between overall activity score and total time to compete the SCREEN ( r s142 =–0.67; P <.001) whereby better performance was associated with quicker task completion. There were weak positive associations between overall activity score and increased technology use ( r s142 =0.34; P <.001) and higher level of education ( r s142 =0.21; P =.01).

A logistic regression model was used to predict the SCREEN result using data from 144 observations. The model’s predictors explain approximately 21.33% of the variance in the outcome variable. The likelihood ratio test indicates that the model provides a significantly better fit to the data than a model without predictors ( P <.001).

The SCREEN outcome variable ( healthy vs unhealthy ) was associated with the predictor variables sex and total time to complete the SCREEN. More specifically, female participants were more likely to obtain healthy SCREEN outcomes ( P =.007; 95% CI 0.32-2.05). For all participants, the longer it took to complete the SCREEN, the less likely they were to achieve a healthy SCREEN outcome ( P =.002; 95% CI –0.33 to –0.07). Age ( P =.25; 95% CI –0.09 to 0.02), medication use ( P =.96; 95% CI –0.9 to 0.94), technology use ( P =.44; 95% CI –0.28 to 0.65), level of education ( P =.14; 95% CI –0.09 to 0.64), level of anxiety ( P =.26; 95% CI –1.13 to 0.3), and hours of sleep ( P =.08; 95% CI –0.06 to 0.93) were not significant.

Impact of Practice Effects

The SCREEN was administered approximately 3 months apart, and separate, paired-sample t tests were performed to compare SCREEN outcomes between visits 1 and 2 (76/147, 51.7%; Table 4 ), visits 2 and 3 (22/147, 15%), and visits 3 and 4 (13/147, 8.8%). Declining visits were partially attributable to the early shutdown of data collection due to the COVID-19 pandemic, and therefore, comparisons between visits 2 and 3 or visits 3 and 4 were not reported. Compared to participants’ SCREEN performance on visit 1, their overall mean activity score and overall processing time improved on their second administration of the SCREEN (score: t 75 =–2.86 and P =.005; processing time: t 75 =–2.98 and P =.004). Even though the 7 task-specific activity scores on the SCREEN also increased between visits 1 and 2, these improvements were not significant, indicating that the difference in overall activity scores was cumulative and not attributable to a specific task ( Table 4 ).

Principal Findings

Our study aimed to evaluate the effectiveness and reliability of the BrainFx SCREEN in detecting MCI in primary care settings. The research took place during the COVID-19 pandemic, which influenced the study’s execution and timeline. Despite these challenges, the findings offer valuable insights into cognitive impairment screening.

Brief MCI screening tools help time-strapped primary care physicians determine whether referral for a definitive battery of more time-consuming and expensive tests is warranted. These tools must optimize and balance the need for time efficiency while also being psychometrically valid and easily administered [ 82 ]. The importance of brevity is determined by a number of factors, including the clinical setting. Screens that can be completed in approximately ≤5 minutes [ 13 ] are recommended for faster-paced clinical settings (eg, emergency rooms and preoperative screens), whereas those that can be completed in 5 to 10 minutes or less are better suited to primary care settings [ 82 - 84 ]. Identifying affordable, psychometrically tested screening tests for MCI that integrate into clinical workflows and are easy to consistently administer and complete may help with the following:

Initiating appropriate diagnostic tests for signs and symptoms at an earlier stage
Normalizing and destigmatizing cognitive testing for older adults
Expediting referrals
Allowing for timely access to programs and services that can support aging in place or delay institutionalization
Reducing risk
Improving the psychosocial well-being of patients and their care partners by increasing access to information and resources that aid with future planning and decision-making [ 85 , 86 ]

Various cognitive tests are commonly used for detecting MCI. These include the Addenbrook Cognitive Examination–Revised, Consortium to Establish a Registry for Alzheimer’s Disease, Sunderland Clock Drawing Test, Informant Questionnaire on Cognitive Decline in the Elderly, Memory Alternation Test, MMSE, MoCA 8.1, and Qmci [ 61 , 87 ]. The Addenbrook Cognitive Examination–Revised, Consortium to Establish a Registry for Alzheimer’s Disease, MoCA 8.1, Qmci, and Memory Alternation Test are reported to have similar diagnostic accuracy [ 61 , 88 ]. The HCPs participating in this study reported using the MoCA 8.1 as their primary screening tool for MCI along with other assessments such as the MMSE and Trail Making Test parts A and B.

Recent research highlights the growing use of digital tools [ 51 , 89 , 90 ], mobile technology [ 91 , 92 ], virtual reality [ 93 , 94 ], and artificial intelligence [ 95 ] to improve early identification of MCI. Demeyere et al [ 51 ] developed the tablet-based, 10-item Oxford Cognitive Screen–Plus to detect slight changes in cognitive impairment across 5 domains of cognition (memory, attention, number, praxis, and language), which has been validated among neurologically healthy older adults. Statsenko et al [ 96 ] have explored improvement of the predictive capabilities of tests using artificial intelligence. Similarly, there is an emerging focus on the use of machine learning techniques to detect dementia leveraging routinely collected clinical data [ 97 , 98 ]. This progression signifies a shift toward more technologically advanced, efficient, and potentially more accurate diagnostic approaches in the detection of MCI.

Whatever the modality, screening tools should be quick to administer, demonstrate consistent results over time and between different evaluators, cover all major cognitive areas, and be straightforward to both administer and interpret [ 99 ]. However, highly sensitive tests such as those suggested for screening carry a significant risk of false-positive diagnoses [ 15 ]. Given the high potential for harm of false positives, it is important to validate the psychometric properties of screening tests across different populations and understand how factors such as age and education can influence the results [ 99 ].

Our study did not assess the face validity of the SCREEN, but participating occupational therapists were comfortable with the test regimen. Nonetheless, the research team noted the absence of verbal fluency and memory tests in the SCREEN, both of which McDonnell et al [ 100 ] identified as being more sensitive to the more commonly seen amnesic MCI. Two of the most sensitive tools for MCI screening, the MoCA 8.1 [ 76 ] and Qmci [ 25 ], assess memory, language, and verbal skills, and tests of verbal fluency and logical memory have been shown to be particularly sensitive to early cognitive changes [ 77 , 78 ].

The constructs included in the SCREEN ( Table 1 ) were selected based on a single non–peer-reviewed study [ 58 ] using the 360 and traumatic brain injury data (N=188) that identified the constructs as predictive of brain injury. The absence of tasks that measure verbal fluency or logical memory in the SCREEN appears to weaken claims of construct validity. The principal component analysis of the SCREEN assessment identified 2 components accounting for 52.12% of the total variance. The first component was strongly associated with abstract reasoning, constructive ability, and divided attention, whereas the second was primarily influenced by visual-spatial abilities. This indicates that constructs related to perception, attention, and memory are central to the SCREEN scores.

The SCREEN’s binary outcome (healthy or unhealthy) created by the research team was based on comparisons with the Qmci. However, the method of identifying areas of challenge in the SCREEN by comparing the individual’s mean score on each of the 7 tasks with the mean scores of a global or filtered cohort in the LBB introduces potential biases or errors. These could arise from a surge in additions to the LBB from patients with specific characteristics, self-selection of participants, poorly trained SCREEN administrators, inclusion of nonstandard test results, underuse of appropriate filters, and underreporting of clinical conditions or factors such as socioeconomic status that impact performance in standardized cognitive tests.

The proprietary method of analyzing and reporting SCREEN results complicates traditional sensitivity and specificity measurement. Our testing indicated a sensitivity of 63.25% and specificity of 74.07% for identifying healthy (those without MCI) and unhealthy (those with MCI) individuals. The SCREEN’s Cronbach α=.63, slightly below the threshold for clinical instruments, and reliability scores that were lower than the ideal standards suggest a higher-than-acceptable level of random measurement error in its constructs. The lower reliability may also stem from an inadequate sample size or a limited number of scale items.

The SCREEN’s results are less favorable compared to those of other digital MCI screening tools that similarly enable evaluation of specific cognitive domains but also provide validated, norm-referenced cutoff scores and methods for cumulative scoring in clinical settings (Oxford Cognitive Screen–Plus) [ 51 ] or of validated MCI screening tools used in primary care (eg, MoCA 8.1, Qmci, and MMSE) [ 51 , 87 ]. The SCREEN’s unique scoring algorithm and the dynamic denominator in data analysis necessitate caution in comparing these results to those of other tools with fixed scoring algorithms and known sensitivities [ 101 , 102 ]. We found the SCREEN to have lower-than-expected internal reliability, suggesting significant random measurement error. Test-retest reliability was weak for the healthy or unhealthy outcome but stronger for overall activity scores between tests. The variability in identifying areas of challenge could relate to technological difficulties or variability from comparisons with a growing database of test results.

Potential reasons for older adults’ poorer scores on timed tests include the impact of sensorimotor decline on touch screen sensation and reaction time [ 38 , 103 ], anxiety related to taking a computer-enabled test [ 104 - 106 ], or the anticipated consequences of a negative outcome [ 107 ]. However, these effects were unlikely to have influenced the results of this study. Practice effects were observed [ 29 , 108 ], but the SCREEN’s novelty suggests that familiarity is not gained through prepreparation or word of mouth as this sample was self-selected and not randomized. Future research might also explore the impact of digital literacy and cultural differences in the interpretation of software constructs or icons on MCI screening in a randomized, older adult sample.

Limitations

This study had methodological limitations that warrant attention. The small sample size and the demographic distribution of the 147 participants aged ≥55 years, with most (96/147, 65.3%) being female and well educated, limits the generalizability of the findings to different populations. The study’s design, aiming to explore the sensitivity of the SCREEN for early detection of MCI, necessitated the exclusion of individuals with a previous diagnosis of MCI or dementia. This exclusion criterion might have impacted the study’s ability to thoroughly assess the SCREEN’s effectiveness in a more varied clinical context. The requirement for participants to read and comprehend English introduced another limitation to our study. This criterion potentially limited the SCREEN tool’s applicability across diverse linguistic backgrounds as individuals with language-based impairments or those not proficient in English may face challenges in completing the assessment [ 51 ]. Such limitations could impact the generalizability of our findings to non–English-speaking populations or to those with language impairments, underscoring the need for further research to evaluate the SCREEN tool’s effectiveness in broader clinical and linguistic contexts.

Financial constraints played a role in limiting the study’s scope. Due to funding limitations, it was not possible to include specialist assessments and a battery of neuropsychiatric tests generally considered the gold standard to confirm or rule out an MCI diagnosis. Therefore, the study relied on differential verification through 2 imperfect reference standards: a comparison with the Qmci (the tool with the highest published sensitivity to MCI in 2019, when the study was designed) and the clinical judgment of the administering HCP, particularly in decisions regarding referrals for further clinical assessment. Furthermore, while an economic feasibility assessment was considered, the research team determined that it should follow, not precede, an evaluation of the SCREEN’s validity and reliability.

The proprietary nature of the algorithm used for scoring the SCREEN posed another challenge. Without access to this algorithm, the research team had to use a novel comparative statistical approach, coding patient results into a binary variable: healthy (SCREEN=no areas of challenge OR Qmci≥67 out of 100) or unhealthy (SCREEN=one or more areas of challenge OR Qmci<67 out of 100). This may have introduced a higher level of error into our statistical analysis. Furthermore, the process for determining areas of challenge on the SCREEN involves comparing a participant’s result to the existing SCREEN results in the LBB at the time of testing. By the end of this study, the LBB contained 632 SCREEN results for adults aged ≥55 years, with this study contributing 258 of those results. The remaining 366 original SCREEN results, 64% of which were completed by individuals who self-identified as having a preexisting diagnosis or conditions associated with cognitive impairment (eg, traumatic brain injury, concussion, or stroke), could have led to an overestimation of the means and SDs of the study participants’ results at the outset of the study.

Unlike other cognitive screening tools, the SCREEN allows for filtering of results to compare different patient cohorts in the LBB using criteria such as age and education. However, at this stage of the LBB’s development, using such filters can significantly reduce the reliability of the results due to a smaller comparator population (ie, the denominator used to calculate the mean and SD). This, in turn, affects the significance of the results. Moreover, the constantly changing LBB data set makes it challenging to meaningfully compare an individual’s results over time as the evolving denominator affects the accuracy and relevance of these comparisons. Finally, the significant improvement in SCREEN scores between the first and second visits suggests the presence of practice effects, which could have influenced the reliability and validity of the findings.

Conclusions

In a primary care setting, where MCI screening tools are essential and recommended for those with concerns [ 85 ], certain criteria are paramount: time efficiency, ease of administration, and robust psychometric properties [ 82 ]. Our analysis of the BrainFx SCREEN suggests that, despite its innovative approach and digital delivery, it currently falls short in meeting these criteria. The SCREEN’s comparatively longer administration time and lower-than-expected reliability scores suggest that it may not be the most effective tool for MCI screening of older adults in a primary care setting at this time.

It is important to note that, in the wake of the COVID-19 pandemic, and with an aging population living and aging by design or necessity in a community setting, there is growing interest in digital solutions, including web-based applications and platforms to both collect digital biomarkers and deliver cognitive training and other interventions [ 109 , 110 ]. However, new normative standards are required when adapting cognitive tests to digital formats [ 92 ] as the change in medium can significantly impact test performance and results interpretation. Therefore, we recommend caution when interpreting our study results and encourage continued research and refinement of tools such as the SCREEN. This ongoing process will ensure that current and future MCI screening tools are effective, reliable, and relevant in meeting the needs of our aging population, particularly in primary care settings where early detection and intervention are key.

Acknowledgments

The researchers gratefully acknowledge the Ontario Centres of Excellence Health Technologies Fund for their financial support of this study; the executive directors and clinical leads in each of the Family Health Team study locations; the participants and their friends and families who took part in the study; and research assistants Sharmin Sharker, Kelly Zhu, and Muhammad Umair for their contributions to data management and statistical analysis.

Data Availability

The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions

JM contributed to the conceptualization, methodology, validation, formal analysis, data curation, writing—original draft, writing—review and editing, visualization, supervision, and funding acquisition. AML contributed to the conceptualization, methodology, validation, investigation, formal analysis, data curation, writing—original draft, writing—review and editing, visualization, and project administration. WP contributed to the validation, formal analysis, data curation, writing—original draft, writing—review and editing, and visualization. Finally, PH contributed to conceptualization, methodology, writing—review and editing, supervision, and funding acquisition.

Conflicts of Interest

None declared.

Casagrande M, Marselli G, Agostini F, Forte G, Favieri F, Guarino A. The complex burden of determining prevalence rates of mild cognitive impairment: a systematic review. Front Psychiatry. 2022;13:960648. [ FREE Full text ] [ CrossRef ] [ Medline ]
Petersen RC, Caracciolo B, Brayne C, Gauthier S, Jelic V, Fratiglioni L. Mild cognitive impairment: a concept in evolution. J Intern Med. Mar 2014;275(3):214-228. [ FREE Full text ] [ CrossRef ] [ Medline ]
Knopman DS, Petersen RC. Mild cognitive impairment and mild dementia: a clinical perspective. Mayo Clin Proc. Oct 2014;89(10):1452-1459. [ FREE Full text ] [ CrossRef ] [ Medline ]
Anderson ND. State of the science on mild cognitive impairment (MCI). CNS Spectr. Feb 2019;24(1):78-87. [ CrossRef ] [ Medline ]
Tangalos EG, Petersen RC. Mild cognitive impairment in geriatrics. Clin Geriatr Med. Nov 2018;34(4):563-589. [ CrossRef ] [ Medline ]
Ng R, Maxwell C, Yates E, Nylen K, Antflick J, Jette N, et al. Brain disorders in Ontario: prevalence, incidence and costs from health administrative data. Institute for Clinical Evaluative Sciences. 2015. URL: https://www.ices.on.ca/publications/research-reports/brain-disorders-in-ontario-prevalence-incidence-and-costs-from-health-administrative-data/ [accessed 2024-04-01]
Centers for Disease ControlPrevention (CDC). Self-reported increased confusion or memory loss and associated functional difficulties among adults aged ≥ 60 years - 21 states, 2011. MMWR Morb Mortal Wkly Rep. May 10, 2013;62(18):347-350. [ FREE Full text ] [ Medline ]
Petersen RC, Lopez O, Armstrong MJ, Getchius TS, Ganguli M, Gloss D, et al. Practice guideline update summary: mild cognitive impairment: report of the guideline development, dissemination, and implementation subcommittee of the American Academy of Neurology. Neurology. Jan 16, 2018;90(3):126-135. [ FREE Full text ] [ CrossRef ] [ Medline ]
Song WX, Wu WW, Zhao YY, Xu HL, Chen GC, Jin SY, et al. Evidence from a meta-analysis and systematic review reveals the global prevalence of mild cognitive impairment. Front Aging Neurosci. 2023;15:1227112. [ FREE Full text ] [ CrossRef ] [ Medline ]
Chen Y, Denny KG, Harvey D, Farias ST, Mungas D, DeCarli C, et al. Progression from normal cognition to mild cognitive impairment in a diverse clinic-based and community-based elderly cohort. Alzheimers Dement. Apr 2017;13(4):399-405. [ FREE Full text ] [ CrossRef ] [ Medline ]
Langa KM, Levine DA. The diagnosis and management of mild cognitive impairment: a clinical review. JAMA. Dec 17, 2014;312(23):2551-2561. [ FREE Full text ] [ CrossRef ] [ Medline ]
Zhang Y, Natale G, Clouston S. Incidence of mild cognitive impairment, conversion to probable dementia, and mortality. Am J Alzheimers Dis Other Demen. 2021;36:15333175211012235. [ FREE Full text ] [ CrossRef ] [ Medline ]
Prince M, Bryce R, Ferri CP. World Alzheimer report 2011: the benefits of early diagnosis and intervention. Alzheimer’s Disease International. 2011. URL: https://www.alzint.org/u/WorldAlzheimerReport2011.pdf [accessed 2024-04-01]
Patnode CD, Perdue LA, Rossom RC, Rushkin MC, Redmond N, Thomas RG, et al. Screening for cognitive impairment in older adults: updated evidence report and systematic review for the US preventive services task force. JAMA. Feb 25, 2020;323(8):764-785. [ FREE Full text ] [ CrossRef ] [ Medline ]
Canadian Task Force on Preventive Health Care, Pottie K, Rahal R, Jaramillo A, Birtwhistle R, Thombs BD, et al. Recommendations on screening for cognitive impairment in older adults. CMAJ. Jan 05, 2016;188(1):37-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
Tahami Monfared AA, Phan NT, Pearson I, Mauskopf J, Cho M, Zhang Q, et al. A systematic review of clinical practice guidelines for Alzheimer's disease and strategies for future advancements. Neurol Ther. Aug 2023;12(4):1257-1284. [ FREE Full text ] [ CrossRef ] [ Medline ]
Mattke S, Jun H, Chen E, Liu Y, Becker A, Wallick C. Expected and diagnosed rates of mild cognitive impairment and dementia in the U.S. medicare population: observational analysis. Alzheimers Res Ther. Jul 22, 2023;15(1):128. [ FREE Full text ] [ CrossRef ] [ Medline ]
Manly JJ, Tang MX, Schupf N, Stern Y, Vonsattel JP, Mayeux R. Frequency and course of mild cognitive impairment in a multiethnic community. Ann Neurol. Apr 2008;63(4):494-506. [ FREE Full text ] [ CrossRef ] [ Medline ]
Black CM, Ambegaonkar BM, Pike J, Jones E, Husbands J, Khandker RK. The diagnostic pathway from cognitive impairment to dementia in Japan: quantification using real-world data. Alzheimer Dis Assoc Disord. 2019;33(4):346-353. [ CrossRef ] [ Medline ]
Ritchie CW, Black CM, Khandker RK, Wood R, Jones E, Hu X, et al. Quantifying the diagnostic pathway for patients with cognitive impairment: real-world data from seven European and north American countries. J Alzheimers Dis. 2018;62(1):457-466. [ CrossRef ] [ Medline ]
Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. Nov 1975;12(3):189-198. [ CrossRef ] [ Medline ]
Tsoi KK, Chan JY, Hirai HW, Wong SY, Kwok TC. Cognitive tests to detect dementia: a systematic review and meta-analysis. JAMA Intern Med. Sep 2015;175(9):1450-1458. [ CrossRef ] [ Medline ]
Lopez MN, Charter RA, Mostafavi B, Nibut LP, Smith WE. Psychometric properties of the Folstein mini-mental state examination. Assessment. Jun 2005;12(2):137-144. [ CrossRef ] [ Medline ]
Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. Apr 2005;53(4):695-699. [ CrossRef ] [ Medline ]
O'Caoimh R, Timmons S, Molloy DW. Screening for mild cognitive impairment: comparison of "MCI specific" screening instruments. J Alzheimers Dis. 2016;51(2):619-629. [ FREE Full text ] [ CrossRef ] [ Medline ]
Trzepacz PT, Hochstetler H, Wang S, Walker B, Saykin AJ, Alzheimer’s Disease Neuroimaging Initiative. Relationship between the Montreal cognitive assessment and mini-mental state examination for assessment of mild cognitive impairment in older adults. BMC Geriatr. Sep 07, 2015;15:107. [ FREE Full text ] [ CrossRef ] [ Medline ]
Nasreddine ZS, Phillips N, Chertkow H. Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology. Mar 06, 2012;78(10):765-766. [ CrossRef ] [ Medline ]
Monroe T, Carter M. Using the Folstein Mini Mental State Exam (MMSE) to explore methodological issues in cognitive aging research. Eur J Ageing. Sep 2012;9(3):265-274. [ FREE Full text ] [ CrossRef ] [ Medline ]
Damian AM, Jacobson SA, Hentz JG, Belden CM, Shill HA, Sabbagh MN, et al. The Montreal cognitive assessment and the mini-mental state examination as screening instruments for cognitive impairment: item analyses and threshold scores. Dement Geriatr Cogn Disord. 2011;31(2):126-131. [ CrossRef ] [ Medline ]
Kaufer DI, Williams CS, Braaten AJ, Gill K, Zimmerman S, Sloane PD. Cognitive screening for dementia and mild cognitive impairment in assisted living: comparison of 3 tests. J Am Med Dir Assoc. Oct 2008;9(8):586-593. [ CrossRef ] [ Medline ]
Gagnon C, Saillant K, Olmand M, Gayda M, Nigam A, Bouabdallaoui N, et al. Performances on the Montreal cognitive assessment along the cardiovascular disease continuum. Arch Clin Neuropsychol. Jan 17, 2022;37(1):117-124. [ FREE Full text ] [ CrossRef ] [ Medline ]
Cooley SA, Heaps JM, Bolzenius JD, Salminen LE, Baker LM, Scott SE, et al. Longitudinal change in performance on the Montreal cognitive assessment in older adults. Clin Neuropsychol. 2015;29(6):824-835. [ FREE Full text ] [ CrossRef ] [ Medline ]
O'Caoimh R, Gao Y, McGlade C, Healy L, Gallagher P, Timmons S, et al. Comparison of the quick mild cognitive impairment (Qmci) screen and the SMMSE in screening for mild cognitive impairment. Age Ageing. Sep 2012;41(5):624-629. [ FREE Full text ] [ CrossRef ] [ Medline ]
O'Caoimh R, Molloy DW. Comparing the diagnostic accuracy of two cognitive screening instruments in different dementia subtypes and clinical depression. Diagnostics (Basel). Aug 08, 2019;9(3):93. [ FREE Full text ] [ CrossRef ] [ Medline ]
Clarnette R, O'Caoimh R, Antony DN, Svendrovski A, Molloy DW. Comparison of the Quick Mild Cognitive Impairment (Qmci) screen to the Montreal Cognitive Assessment (MoCA) in an Australian geriatrics clinic. Int J Geriatr Psychiatry. Jun 2017;32(6):643-649. [ CrossRef ] [ Medline ]
Glynn K, Coen R, Lawlor BA. Is the Quick Mild Cognitive Impairment screen (QMCI) more accurate at detecting mild cognitive impairment than existing short cognitive screening tests? A systematic review of the current literature. Int J Geriatr Psychiatry. Dec 2019;34(12):1739-1746. [ CrossRef ] [ Medline ]
Lee MT, Chang WY, Jang Y. Psychometric and diagnostic properties of the Taiwan version of the quick mild cognitive impairment screen. PLoS One. 2018;13(12):e0207851. [ FREE Full text ] [ CrossRef ] [ Medline ]
Wallace SE, Donoso Brown EV, Simpson RC, D'Acunto K, Kranjec A, Rodgers M, et al. A comparison of electronic and paper versions of the Montreal cognitive assessment. Alzheimer Dis Assoc Disord. 2019;33(3):272-278. [ CrossRef ] [ Medline ]
Gagnon C, Olmand M, Dupuy EG, Besnier F, Vincent T, Grégoire CA, et al. Videoconference version of the Montreal cognitive assessment: normative data for Quebec-French people aged 50 years and older. Aging Clin Exp Res. Jul 2022;34(7):1627-1633. [ FREE Full text ] [ CrossRef ] [ Medline ]
Friemel TN. The digital divide has grown old: determinants of a digital divide among seniors. New Media & Society. Jun 12, 2014;18(2):313-331. [ CrossRef ]
Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P T. May 2014;39(5):356-364. [ FREE Full text ] [ Medline ]
Searles C, Farnsworth JL, Jubenville C, Kang M, Ragan B. Test–retest reliability of the BrainFx 360® performance assessment. Athl Train Sports Health Care. Jul 2019;11(4):183-191. [ CrossRef ]
Jones C, Miguel-Cruz A, Brémault-Phillips S. Technology acceptance and usability of the BrainFx SCREEN in Canadian military members and veterans with posttraumatic stress disorder and mild traumatic brain injury: mixed methods UTAUT study. JMIR Rehabil Assist Technol. May 13, 2021;8(2):e26078. [ FREE Full text ] [ CrossRef ] [ Medline ]
McMurray J, Levy A, Holyoke P. Psychometric evaluation and workflow integration study of a tablet-based tool to detect mild cognitive impairment in older adults: protocol for a mixed methods study. JMIR Res Protoc. May 21, 2021;10(5):e25520. [ FREE Full text ] [ CrossRef ] [ Medline ]
Wilansky P, Eklund JM, Milner T, Kreindler D, Cheung A, Kovacs T, et al. Cognitive behavior therapy for anxious and depressed youth: improving homework adherence through mobile technology. JMIR Res Protoc. Nov 10, 2016;5(4):e209. [ FREE Full text ] [ CrossRef ] [ Medline ]
Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak. Jan 09, 2006;6:3. [ FREE Full text ] [ CrossRef ] [ Medline ]
Goodhue DL, Thompson RL. Task-technology fit and individual performance. MIS Q. Jun 1995;19(2):213-236. [ CrossRef ]
Beuscher L, Grando VT. Challenges in conducting qualitative research with individuals with dementia. Res Gerontol Nurs. Jan 2009;2(1):6-11. [ FREE Full text ] [ CrossRef ] [ Medline ]
Howe E. Informed consent, participation in research, and the Alzheimer's patient. Innov Clin Neurosci. May 2012;9(5-6):47-51. [ FREE Full text ] [ Medline ]
Thorogood A, Mäki-Petäjä-Leinonen A, Brodaty H, Dalpé G, Gastmans C, Gauthier S, et al. Global Alliance for GenomicsHealth‚ AgeingDementia Task Team. Consent recommendations for research and international data sharing involving persons with dementia. Alzheimers Dement. Oct 2018;14(10):1334-1343. [ FREE Full text ] [ CrossRef ] [ Medline ]
Demeyere N, Haupt M, Webb SS, Strobel L, Milosevich ET, Moore MJ, et al. Introducing the tablet-based Oxford Cognitive Screen-Plus (OCS-Plus) as an assessment tool for subtle cognitive impairments. Sci Rep. Apr 12, 2021;11(1):8000. [ FREE Full text ] [ CrossRef ] [ Medline ]
Nasreddine ZS, Patel BB. Validation of Montreal cognitive assessment, MoCA, alternate French versions. Can J Neurol Sci. Sep 2016;43(5):665-671. [ CrossRef ] [ Medline ]
Mueller AE, Segal DL, Gavett B, Marty MA, Yochim B, June A, et al. Geriatric anxiety scale: item response theory analysis, differential item functioning, and creation of a ten-item short form (GAS-10). Int Psychogeriatr. Jul 2015;27(7):1099-1111. [ CrossRef ] [ Medline ]
Segal DL, June A, Payne M, Coolidge FL, Yochim B. Development and initial validation of a self-report assessment tool for anxiety among older adults: the Geriatric Anxiety Scale. J Anxiety Disord. Oct 2010;24(7):709-714. [ CrossRef ] [ Medline ]
Balsamo M, Cataldi F, Carlucci L, Fairfield B. Assessment of anxiety in older adults: a review of self-report measures. Clin Interv Aging. 2018;13:573-593. [ FREE Full text ] [ CrossRef ] [ Medline ]
Gatti A, Gottschling J, Brugnera A, Adorni R, Zarbo C, Compare A, et al. An investigation of the psychometric properties of the Geriatric Anxiety Scale (GAS) in an Italian sample of community-dwelling older adults. Aging Ment Health. Sep 2018;22(9):1170-1178. [ CrossRef ] [ Medline ]
Yochim BP, Mueller AE, June A, Segal DL. Psychometric properties of the Geriatric Anxiety Scale: comparison to the beck anxiety inventory and geriatric anxiety inventory. Clin Gerontol. Dec 06, 2010;34(1):21-33. [ CrossRef ]
Recent concussion (< 6 months ago) analysis result. Daisy Intelligence. 2016. URL: https://www.daisyintelligence.com/retail-solutions/ [accessed 2024-04-01]
Malloy DW, O'Caoimh R. The Quick Guide: Scoring and Administration Instructions for The Quick Mild Cognitive Impairment (Qmci) Screen. Waterford, Ireland. Newgrange Press; 2017.
O'Caoimh R, Gao Y, Svendovski A, Gallagher P, Eustace J, Molloy DW. Comparing approaches to optimize cut-off scores for short cognitive screening instruments in mild cognitive impairment and dementia. J Alzheimers Dis. 2017;57(1):123-133. [ FREE Full text ] [ CrossRef ] [ Medline ]
Breton A, Casey D, Arnaoutoglou NA. Cognitive tests for the detection of mild cognitive impairment (MCI), the prodromal stage of dementia: meta-analysis of diagnostic accuracy studies. Int J Geriatr Psychiatry. Feb 2019;34(2):233-242. [ CrossRef ] [ Medline ]
Umemneku Chikere CM, Wilson K, Graziadio S, Vale L, Allen AJ. Diagnostic test evaluation methodology: a systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update. PLoS One. 2019;14(10):e0223832. [ FREE Full text ] [ CrossRef ] [ Medline ]
Espinosa A, Alegret M, Boada M, Vinyes G, Valero S, Martínez-Lage P, et al. Ecological assessment of executive functions in mild cognitive impairment and mild Alzheimer's disease. J Int Neuropsychol Soc. Sep 2009;15(5):751-757. [ FREE Full text ] [ CrossRef ] [ Medline ]
Hawkins DM, Garrett JA, Stephenson B. Some issues in resolution of diagnostic tests using an imperfect gold standard. Stat Med. Jul 15, 2001;20(13):1987-2001. [ CrossRef ] [ Medline ]
Hadgu A, Dendukuri N, Hilden J. Evaluation of nucleic acid amplification tests in the absence of a perfect gold-standard test: a review of the statistical and epidemiologic issues. Epidemiology. Sep 2005;16(5):604-612. [ CrossRef ] [ Medline ]
Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol. Aug 2003;56(8):730-735. [ CrossRef ] [ Medline ]
Paiva CE, Barroso EM, Carneseca EC, de Pádua Souza C, Dos Santos FT, Mendoza López RV, et al. A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review. BMC Med Res Methodol. Jan 21, 2014;14:8. [ FREE Full text ] [ CrossRef ] [ Medline ]
Streiner DL, Kottner J. Recommendations for reporting the results of studies of instrument and scale development and testing. J Adv Nurs. Sep 2014;70(9):1970-1979. [ CrossRef ] [ Medline ]
Streiner DL. A checklist for evaluating the usefulness of rating scales. Can J Psychiatry. Mar 1993;38(2):140-148. [ CrossRef ] [ Medline ]
Peyre H, Leplège A, Coste J. Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the SF-36 in the French 2003 decennial health survey. Qual Life Res. Mar 2011;20(2):287-300. [ CrossRef ] [ Medline ]
Nevado-Holgado AJ, Kim CH, Winchester L, Gallacher J, Lovestone S. Commonly prescribed drugs associate with cognitive function: a cross-sectional study in UK Biobank. BMJ Open. Nov 30, 2016;6(11):e012177. [ FREE Full text ] [ CrossRef ] [ Medline ]
Moore AR, O'Keeffe ST. Drug-induced cognitive impairment in the elderly. Drugs Aging. Jul 1999;15(1):15-28. [ CrossRef ] [ Medline ]
Rogers J, Wiese BS, Rabheru K. The older brain on drugs: substances that may cause cognitive impairment. Geriatr Aging. 2008;11(5):284-289. [ FREE Full text ]
Marvanova M. Drug-induced cognitive impairment: effect of cardiovascular agents. Ment Health Clin. Jul 2016;6(4):201-206. [ FREE Full text ] [ CrossRef ] [ Medline ]
Espeland MA, Rapp SR, Manson JE, Goveas JS, Shumaker SA, Hayden KM, et al. WHIMSYWHIMS-ECHO Study Groups. Long-term effects on cognitive trajectories of postmenopausal hormone therapy in two age groups. J Gerontol A Biol Sci Med Sci. Jun 01, 2017;72(6):838-845. [ FREE Full text ] [ CrossRef ] [ Medline ]
Luis CA, Keegan AP, Mullan M. Cross validation of the Montreal cognitive assessment in community dwelling older adults residing in the Southeastern US. Int J Geriatr Psychiatry. Feb 2009;24(2):197-201. [ CrossRef ] [ Medline ]
Cunje A, Molloy DW, Standish TI, Lewis DL. Alternate forms of logical memory and verbal fluency tasks for repeated testing in early cognitive changes. Int Psychogeriatr. Feb 2007;19(1):65-75. [ CrossRef ] [ Medline ]
Molloy DW, Standish TI, Lewis DL. Screening for mild cognitive impairment: comparing the SMMSE and the ABCS. Can J Psychiatry. Jan 2005;50(1):52-58. [ CrossRef ] [ Medline ]
Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 4th edition. Oxford, UK. Oxford University Press; 2008.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. Jun 2016;15(2):155-163. [ FREE Full text ] [ CrossRef ] [ Medline ]
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282. [ FREE Full text ] [ Medline ]
Zhuang L, Yang Y, Gao J. Cognitive assessment tools for mild cognitive impairment screening. J Neurol. May 2021;268(5):1615-1622. [ CrossRef ] [ Medline ]
Zhang J, Wang L, Deng X, Fei G, Jin L, Pan X, et al. Five-minute cognitive test as a new quick screening of cognitive impairment in the elderly. Aging Dis. Dec 2019;10(6):1258-1269. [ FREE Full text ] [ CrossRef ] [ Medline ]
Feldman HH, Jacova C, Robillard A, Garcia A, Chow T, Borrie M, et al. Diagnosis and treatment of dementia: 2. Diagnosis. CMAJ. Mar 25, 2008;178(7):825-836. [ FREE Full text ] [ CrossRef ] [ Medline ]
Sabbagh MN, Boada M, Borson S, Chilukuri M, Dubois B, Ingram J, et al. Early detection of mild cognitive impairment (MCI) in primary care. J Prev Alzheimers Dis. 2020;7(3):165-170. [ CrossRef ] [ Medline ]
Milne A. Dementia screening and early diagnosis: the case for and against. Health Risk Soc. Mar 05, 2010;12(1):65-76. [ CrossRef ]
Screening tools to identify adults with cognitive impairment associated with dementia: diagnostic accuracy. Canadian Agency for Drugs and Technologies in Health. 2014. URL: https://www.cadth.ca/sites/default/files/pdf/htis/nov-2014/RB0752%20Cognitive%20Assessments%20for%20Dementia%20Final.pdf [accessed 2024-04-01]
Chehrehnegar N, Nejati V, Shati M, Rashedi V, Lotfi M, Adelirad F, et al. Early detection of cognitive disturbances in mild cognitive impairment: a systematic review of observational studies. Psychogeriatrics. Mar 2020;20(2):212-228. [ CrossRef ] [ Medline ]
Chan JY, Yau ST, Kwok TC, Tsoi KK. Diagnostic performance of digital cognitive tests for the identification of MCI and dementia: a systematic review. Ageing Res Rev. Dec 2021;72:101506. [ CrossRef ] [ Medline ]
Cubillos C, Rienzo A. Digital cognitive assessment tests for older adults: systematic literature review. JMIR Ment Health. Dec 08, 2023;10:e47487. [ FREE Full text ] [ CrossRef ] [ Medline ]
Chen R, Foschini L, Kourtis L, Signorini A, Jankovic F, Pugh M, et al. Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019. Presented at: KDD '19; August 4-8, 2019;2145; Anchorage, AK. URL: https://dl.acm.org/doi/10.1145/3292500.3330690 [ CrossRef ]
Koo BM, Vizer LM. Mobile technology for cognitive assessment of older adults: a scoping review. Innov Aging. Jan 2019;3(1):igy038. [ FREE Full text ] [ CrossRef ] [ Medline ]
Zygouris S, Ntovas K, Giakoumis D, Votis K, Doumpoulakis S, Segkouli S, et al. A preliminary study on the feasibility of using a virtual reality cognitive training application for remote detection of mild cognitive impairment. J Alzheimers Dis. 2017;56(2):619-627. [ CrossRef ] [ Medline ]
Liu Q, Song H, Yan M, Ding Y, Wang Y, Chen L, et al. Virtual reality technology in the detection of mild cognitive impairment: a systematic review and meta-analysis. Ageing Res Rev. Jun 2023;87:101889. [ CrossRef ] [ Medline ]
Fayemiwo MA, Olowookere TA, Olaniyan OO, Ojewumi TO, Oyetade IS, Freeman S, et al. Immediate word recall in cognitive assessment can predict dementia using machine learning techniques. Alzheimers Res Ther. Jun 15, 2023;15(1):111. [ FREE Full text ] [ CrossRef ] [ Medline ]
Statsenko Y, Meribout S, Habuza T, Almansoori TM, van Gorkom KN, Gelovani JG, et al. Patterns of structure-function association in normal aging and in Alzheimer's disease: screening for mild cognitive impairment and dementia with ML regression and classification models. Front Aging Neurosci. 2022;14:943566. [ FREE Full text ] [ CrossRef ] [ Medline ]
Roebuck-Spencer TM, Glen T, Puente AE, Denney RL, Ruff RM, Hostetter G, et al. Cognitive screening tests versus comprehensive neuropsychological test batteries: a national academy of neuropsychology education paper†. Arch Clin Neuropsychol. Jun 01, 2017;32(4):491-498. [ CrossRef ] [ Medline ]
Jammeh EA, Carroll CB, Pearson SW, Escudero J, Anastasiou A, Zhao P, et al. Machine-learning based identification of undiagnosed dementia in primary care: a feasibility study. BJGP Open. Jul 2018;2(2):bjgpopen18X101589. [ FREE Full text ] [ CrossRef ] [ Medline ]
Riello M, Rusconi E, Treccani B. The role of brief global cognitive tests and neuropsychological expertise in the detection and differential diagnosis of dementia. Front Aging Neurosci. 2021;13:648310. [ FREE Full text ] [ CrossRef ] [ Medline ]
McDonnell M, Dill L, Panos S, Amano S, Brown W, Giurgius S, et al. Verbal fluency as a screening tool for mild cognitive impairment. Int Psychogeriatr. Sep 2020;32(9):1055-1062. [ FREE Full text ] [ CrossRef ] [ Medline ]
Wojtowicz A, Larner AJ. Diagnostic test accuracy of cognitive screeners in older people. Prog Neurol Psychiatry. Mar 20, 2017;21(1):17-21. [ CrossRef ]
Larner AJ. Cognitive screening instruments for the diagnosis of mild cognitive impairment. Prog Neurol Psychiatry. Apr 07, 2016;20(2):21-26. [ CrossRef ]
Heintz BD, Keenan KG. Spiral tracing on a touchscreen is influenced by age, hand, implement, and friction. PLoS One. 2018;13(2):e0191309. [ FREE Full text ] [ CrossRef ] [ Medline ]
Laguna K, Babcock RL. Computer anxiety in young and older adults: implications for human-computer interactions in older populations. Comput Human Behav. Aug 1997;13(3):317-326. [ CrossRef ]
Wild KV, Mattek NC, Maxwell SA, Dodge HH, Jimison HB, Kaye JA. Computer-related self-efficacy and anxiety in older adults with and without mild cognitive impairment. Alzheimers Dement. Nov 2012;8(6):544-552. [ FREE Full text ] [ CrossRef ] [ Medline ]
Wiechmann D, Ryan AM. Reactions to computerized testing in selection contexts. Int J Sel Assess. Jul 30, 2003;11(2-3):215-229. [ CrossRef ]
Gass CS, Curiel RE. Test anxiety in relation to measures of cognitive and intellectual functioning. Arch Clin Neuropsychol. Aug 2011;26(5):396-404. [ CrossRef ] [ Medline ]
Barbic D, Kim B, Salehmohamed Q, Kemplin K, Carpenter CR, Barbic SP. Diagnostic accuracy of the Ottawa 3DY and short blessed test to detect cognitive dysfunction in geriatric patients presenting to the emergency department. BMJ Open. Mar 16, 2018;8(3):e019652. [ FREE Full text ] [ CrossRef ] [ Medline ]
Owens AP, Ballard C, Beigi M, Kalafatis C, Brooker H, Lavelle G, et al. Implementing remote memory clinics to enhance clinical care during and after COVID-19. Front Psychiatry. 2020;11:579934. [ FREE Full text ] [ CrossRef ] [ Medline ]
Geddes MR, O'Connell ME, Fisk JD, Gauthier S, Camicioli R, Ismail Z, et al. Alzheimer Society of Canada Task Force on Dementia Care Best Practices for COVID‐19. Remote cognitive and behavioral assessment: report of the Alzheimer Society of Canada task force on dementia care best practices for COVID-19. Alzheimers Dement (Amst). 2020;12(1):e12111. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by G Eysenbach, T de Azevedo Cardoso; submitted 29.01.24; peer-reviewed by J Gao, MJ Moore; comments to author 20.02.24; revised version received 05.03.24; accepted 19.03.24; published 19.04.24.

©Josephine McMurray, AnneMarie Levy, Wei Pang, Paul Holyoke. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

This paper is in the following e-collection/theme issue:

Published on 19.4.2024 in Vol 8 (2024)

This is a member publication of University of Sheffield (Jisc)

Landscape of Digital Technologies Used in the National Health Service in England: Content Analysis

Authors of this article:

Original Paper

Jake Alan Allcock 1 , MA ;
Mengdie Zhuang 2 , PhD ;
Shuyang Li 3 , PhD ;
Xin Zhao 2 , PhD

1 Department of Sociological Studies, University of Sheffield, Sheffield, United Kingdom

2 Information School, University of Sheffield, Sheffield, United Kingdom

3 Business School, University of Birmingham, Birmingham, United Kingdom

Corresponding Author:

Mengdie Zhuang, PhD

Information School

University of Sheffield

2 Whitham Road

Sheffield, S10 2AH

United Kingdom

Phone: 44 114 222 6339

Email: [email protected]

Background: In England, digital technologies are exploited to transform the way health and social care is provided and encompass a wide range of hardware devices and software that are used in all aspects of health care. However, little is known about the extent to which health care providers differ in digital health technology capabilities and how this relates to geographical and regional differences in health care capacities and resources.

Objective: This paper aims to identify the set of digital technologies that have been deployed by the National Health Services clinical commissioning groups (NHS CCGs) in England. In doing this, we respond to calls to shed light on the internal dynamics and variation in the form of digital capability in England in terms of health service regional differences and health diversity, equity, and inclusion.

Methods: We collected 135 annual reports that belong to 106 NHS CCGs in England, comprising more than 18,000 pages in total, released from 2020 to 2021. Using this data set, we identified 2163 pages related to digital technologies and labeled them using content analysis. We follow the construct taxonomy used by digital options theory, a theory from the management information systems field analyzing organizational resource investment choices, in classifying observed technologies according to digital themes —inherent design patterns that we identified and explained. We then used a hierarchical clustering method to extract groups of NHS CCGs that implement similar technology themes.

Results: We found 31 technologies from the reports and grouped them into 9 digital themes. The 9 themes were further assigned to 1 of the 3 constructs of digital options theory, the identification of patients’ requirements (we identified information portals [76/106], digital health engagement [67/106], and digital inclusion support [45/106]), the development of new work patterns (we identified telehealth [87/106], telemedicine [35/106], and care home technologies [40/106]), the realization of improvements in efficiency and public accessibility (we identified online booking [26/106], online triage [104/106], and digital mental health services [74/106]) . The 3 clusters of CCGs are identified based on the 8 themes (Hopkins=0.9914, silhouette=0.186), namely (1) digitally disengaged, (2) digitally engaged, and (3) digital torchbearer.

Conclusions: Our findings show prominent digital themes within each construct group, namely information portals, telehealth, and online triage, covering people’s fundamental health information needs. Almost half of CCGs fell into the digitally disengaged group, and all London CCGs (5/106) belonged to this group. We propose that practitioners should offer specialized assistance to regions with limited digital engagement, emphasizing digital health literacy, inclusion support, and ongoing evaluation, rather than concentrating solely on technical advancements.

Introduction

The health care services in England are in a transformational phase due to the increasing pressure to propose and review digital strategies, use continually emerging new digital technologies, and place varied emphasis on digitization according to regional needs [ 1 , 2 ]. Digital technologies are exploited to transform the way health and social care is provided and encompass a wide range of hardware devices and software that are used in all aspects of health care. These technologies are often grouped and examined by their use cases [ 3 , 4 ], such as medical consultations and treatment, patient management, and information campaigns for access to care. Some researchers emphasize the change these technologies may bring (also referred to as “innovation”) and therefore propose a different type of taxonomy. Zweifel [ 5 ] categorized technologies serving the purpose of health care innovation into 3 types, including technologies for product innovation, process innovation, and organizational innovation.

Despite these developments, however, health care practitioners and managers often find it difficult to choose and leverage different forms of digital technologies and innovation in their institutions to overcome challenges or inefficiencies [ 6 ]. Furthermore, little is known about the extent to which health care providers differ in technology capabilities regarding improving public access to health resources, and how this relates to geographical and regional differences in health care capacities and resources. In the United Kingdom, the availability and usage of medical treatments (measured by, eg, travel distance to attend treatment, waiting time to receive a treatment, and funding for a primary-care practice) vary across regional health services in England [ 7 - 10 ]. We suspect this variation may also apply to the availability of digital technologies as a result of a local strategy priority, prior digitalization level in the area, and the demographics of the local populations.

Drawing from this emerging stream of research, this study identified the digital technologies adopted in 106 National Health Service clinical commissioning groups (NHS CCGs) in England, using 135 annual reports for the years 2020-2021. The CCGs were clinically led statutory NHS bodies responsible for the planning and commissioning of health care services for their local area, which were dissolved in 2022 to be replaced by the new integrated care boards (ICBs). The CCG annual reports were selected for 2 main reasons. First, the CCGs’ structure (106 CCGs in England as of April 2021) provided a finer spatial granularity than the ICBs’ structure (42 ICBs in England as of July 2022). Therefore, they offered crucial insights and statistics for all involved NHS services across different regions, which could offer valuable information regarding differences in digital capabilities and diversity, equity, and inclusion issues across various fine-grained geographical sites. Second, the years 2020-2021 marked an important step in health care digital transformation as the CCGs afterward focused on structural change and were replaced by ICBs nationally. In addition, in response to the COVID-19 pandemic, CCGs were incentivized to make services accessible through digital platforms, aiming to ensure the continual provision of care while mitigating transmission risks. Therefore, the reports in 2020-2021 highlighted important aspects of digital technologies and themes adopted by the CCGs before the structural shift and in response to the COVID-19 pandemic, which were fed into the newly formed ICBs.

We aim to answer the following research questions: What are the digital technologies adopted by CCGs to improve their services as well as widen public access and engagement? How do they vary across regions?

We use the construct taxonomy used by digital options theory [ 11 ] to understand the different types of technologies that are used in health care institutions, and how these options support health care providers’ ability to translate their resources into performance. In the management information systems literature, digital options represent an organization’s investment in and adoption of information technologies [ 12 ]. Such adoption, as Sambamurthy and colleagues argued [ 11 ], together with changes to the organization’s technological environment, impacts an organization’s information technology capabilities. Therefore, digital options are primarily used to examine the evaluation of information technologies [ 12 , 13 ] and to assess how the technologies and digital capabilities in an organization can be transformed into performance improvement [ 11 , 14 ].

Following this set of ideas, in the context of health care, digital options represent opportunities to use new technical tools and features that will increase the service quality, efficiency, and public accessibility. Previous work involving digital options theory and health care focuses on performance improvement, for example, how cost-effective information technology solutions can enhance the financial performance of resource-constrained hospitals [ 15 ]. Little effort has been invested to map out the existing set of digital technologies in health care using the taxonomy of digital options.

Drawing on the concepts of digital options theory [ 16 , 17 ], the technologies serving the purpose of health care efficiency can be grouped using three constructs: (1) identifying patients’ requirements that involve recognizing new technical features for the digital health service (such as online community engagement and digital champion events); (2) developing new work patterns that improve internal coordination and working process (such as artificial intelligence [AI] diagnostics and digital prescribing); and (3) realizing improvements in efficiency and public accessibility (such as virtual consultation and direct online booking). We use these constructs as the theoretical lens to categorize the digital tools used in various CCGs in NHS England.

This paper aims to identify the set of digital technologies that have been deployed by the NHS CCGs in England. This study has several contributions. First, it contributes to the health care literature by providing insights into how regional differences in technology, under an NHS system in England, can vary in their capabilities and efforts in implementing digital technologies. We empirically draw the landscape of digital transformation in health care by unpacking the role of NHS CCGs in adopting technologies and by presenting the regional differences in England. In doing this, we respond to calls to shed light on the internal dynamics and variation in the form of digital capability and patient engagement in England in terms of health service regional differences and health diversity, equity, and inclusion [ 18 , 19 ]. Second, we extend digital options theory to understand its application of constructs within a health care setting. We applied and examined the 3 key approaches, proposed by Rolland et al [ 12 ], for technological options to engage patients as technology users and to improve health care service quality. The study demonstrates the complex choices faced by health care providers: while they need to address large numbers of patient queries and appointments through actionable digital options, they are constrained by regional resources and digital capabilities. Similarly, while different types of digital options could offer health care providers with new opportunities to understand patients’ needs and coordinate workflows, the actual adoption of options varies significantly across different regions.

In this study, we followed a text-mining approach and content analysis method to analyze the research data. Specifically, we used a text mining approach to extract key paragraphs and texts from the 135 NHS CCG annual reports released for 2020-2021 identifying digital technologies. We further followed content analysis methods to manually code and categorize the extracted paragraphs and texts through the lens of the digital options theory [ 16 , 17 ]. We then clustered the CCGs based on the corresponding technologies to reveal the similarities between CCGs and regional differences in digital technologies’ availability.

Data Selection

Secondary data sources, namely annual reports of each NHS CCG, were used in this paper. We decided to use annual reports as they cover each CCG’s performance and accountability in the period, ranging from performance analysis, progress on key initiatives, and public and patient involvement to actual spending, and are the most recent reports before the start of the ICB-forming stage. The reports were downloaded from individual CCG websites directly. In total, there were 106 CCGs across England as of April 1, 2021, which was reduced from 135 CCGs in 2020 with the merger of 38 CCGs into 9. This paper investigates the 135 annual reports, 18,667 pages in total, released from 2020 to 2021. We further group the results according to the 106-CCG structure as it reflected the most recent structure of the CCG systems for 2021-2022.

Data Preprocessing

Given that each report contains 44-222 pages and is labor-consuming to go through manually, highlighting the relevant texts visually can support the researchers in locating useful information for content analysis more quickly. The reports were preprocessed using the semantic matching method [ 20 ], a text mining technique, to identify and highlight texts that are relevant to author-selected keywords. The processing was implemented in Python (Python Software Foundation).

First, reports from 10 CCGs were manually screened by the first 3 researchers independently, to gather the initial set of keywords. The selected 10 CCGs covered the main geographic regions in England. The whole research team had regular meetings during and after screening to discuss the expansion of or trimming the keyword set. Frequent and relevant keywords were excluded from the set if they might cover a much larger field, such as “digital tool.” To validate the set, the second author applied semantic matching methods using the proposed keyword set on the 10 reports and manually screened the reports again to make sure the relevant contents were highlighted appropriately. The final set contained the following words: “digital,” “technology,” “AI,” “Machine Learning,” “e-,” and “online,” together with their derivations. Using this method, 2163 pages containing selected keywords were highlighted in the 135 reports.

Data Analysis

Content analysis.

After the preprocessing, we analyzed the extracted 2163 pages through content analysis. Content analysis [ 21 ] was conducted by the first author. According to digital options theory, there are three types of options when engaging with technologies, including: (1) identifying patients’ requirements that involve recognizing new technical features for the digital health service (eg, digital survey to gather patient feedback on services), (2) developing new work patterns that improve internal coordination and working process (eg, AI diagnostic tools), and (3) realizing improvements in efficiency and public accessibility (eg, appointment booking system). We first conducted the initial coding by going through the research data and grouping them into categories according to the purpose of the adopted technologies. Second, we went through the categories in detail and mapped them to the 3 options as themes. Regarding reliability, the research team had weekly meetings among all the authors throughout the analysis stage to constantly review the emerging codes and categories and to ensure agreement among researchers was reached. The codes assigned for each CCG can be found in Multimedia Appendix 1 .

In total, 31 types of technologies were extracted, covering a wide range of tools and purposes, including online booking systems, virtual consultation, health care promotion campaigns, patient access to health care records, remote monitoring of patient status, AI diagnostics, mental health online service, and digital champions podcasts (see full list in Table 1 ). Further, 9 themes were then identified by grouping the technologies based on their use cases, namely information portal, digital health engagement, digital inclusion support, telehealth, telemedicine, care home technologies, booking system, online triage, and digital mental health services.

a NHS: National Health Service.

b CCG: clinical commissioning group.

c GP: general practitioner.

d AI: artificial intelligence.

e VR: virtual reality.

f IAPT: Improving Access to Psychological Therapies.

g BLMK: Bedfordshire, Luton, and Milton Keynes.

Cluster Analysis

Hierarchical clustering [ 26 ] with Euclidean distance was used to extract common patterns from the code; by doing so the CCGs implementing similar types of technologies were grouped together. We used the identified 9 digital technology themes from the previous step to cluster the CCGs, in which the number of technology types mentioned for each theme was assigned to each CCG. For example, the Sheffield CCG reported they only had 1 technology (remote monitoring) under the telehealth theme, therefore the value of telehealth for Sheffield CCG was 1. In total, for each CCG, there were 9 values describing how engaged the CCG was with each technology theme. These values ere further standardized using the z score to achieve balanced similarity weights across all 9 themes. The silhouette and elbow method were used to select the optimal numbers for clusters [ 27 ], resulting in a structure with 3 clusters.

Ethical Considerations

This study is based on the secondary analysis of NHS CCG annual reports, which are public information. The ethics approval for the secondary analysis of all data presented in this study was obtained from the University of Sheffield Research Ethics Committee (045790).

Digital Technologies in NHS CCGs

In total, 18,667 pages from 135 annual reports were preprocessed and 2163 pages containing selected keywords were highlighted in the 135 reports, representing 106 CCGs. Further, 31 types of technologies were then extracted, covering a wide range of tools and purposes. These technologies were then grouped into 9 themes. The full codebook can be found in Multimedia Appendix 1 .

The purposes of the digital technologies are typically associated with the patients’ or health care practitioners’ needs across various use cases and regions. Generalizing across all the observed items, these technologies were first grouped under particle themes, specific to either a location (eg, care home), disease (eg, mental health), or a type of service (eg, booking appointment). We then further assigned the identified 9 themes to 1 of the 3 digital options theory constructs. Table 1 describes the digital technologies observed from the NHS CCG annual reports and their associated themes.

Information Portals

The information portal theme consisted of 6 technologies that all involved access to or communication specific health care information via an online format. In total, 72% (76/106) of CCGs contained a mention of this theme via either 1 or more of the 6 technologies. Health care promotion for raising the public’s awareness of available services (35/106, 33%) was the most popular method, followed by online service catalogues (30/106, 28.3%) and webinars designed to briefly or formally announce digital tools (30/106, 28.3%). Providing patients with their own health care records accounted for 6% (7/106). These 4 above all involve having health care information readily available online for patients to locate and use. Other tools were typically more oriented toward the sharing of health care information between health care professionals or to the patients. For example, advice and guidance (16/106, 14.8%) was a tool that allowed general practitioners (GPs) to contact specialists quickly to obtain information for a patient so they did not have to refer the patient to the specialist. In doing so they saved time and could provide appropriate treatment for the patient on the same day. The health practitioners could also share electronic patient records quickly with relevant members through an online system (47/106, 43.5%) to facilitate discussion.

Digital Health Engagement

Digital health engagement was about engaging with the public through digital means (67/106, 63.2%). Differing from the information portal theme aiming to support patients or health practitioners dealing with immediate health needs, this theme focuses on obtaining feedback on service delivery and raising awareness of a disease or healthy lifestyle in the community in a less formal manner. The most popular technologies within this category involved patients in the operations of the health care service within their area through virtual meetings or surveys (63/106, 59.4%). These activities kept patients informed and created and allowed them to give their input on the operations of the health care service. Around 12% (13/106) of the reports discussed the development of a podcast that patients could access anytime that kept them up to date with any health care developments or gave them advice for better self-care management. Around 17.9% (19/106) of reports mentioned the use of digital champions, who are individuals who help staff and patients struggling with the integration of digital technologies and help them develop their digital skills and confidence.

Digital Inclusion Support

The implementation of many digital technologies was vital to keeping the NHS operating during the COVID-19 pandemic, and there was a conscious effort to keep providing adequate health care to those without a means of accessing these services. In total, 42.5% (45/106) of CCGs included recognition of the problem of digital exclusion and how they would tackle it via an alternative digital approach. The Newcastle Gateshead CCG report reflected on the downside of the increase in digital technology, suggesting that it could impact the NHS’ free at point of care policy (eg, using a phone or the internet is not free). Methods such as providing targeted alternative communication methods (eg, text message or physical newsletters; 14/106, 13.2%) or providing essential technology (typically providing hardware to care homes; 38/106, 35.5%) were mentioned.

Telehealth is a theme that aims to incorporate digital technologies to monitor patients’ health information through real time data and provide long-distance health care. At least one form of telehealth was mentioned by 82% (87/106) of the CCGs and was seen by many of them as vital to providing safe health care during the COVID-19 pandemic. Remote monitoring (69/106, 65.1%) typically involved the use of software or hardware technologies such as pulse oximetry and digital blood pressure monitors. The information from these devices would automatically be sent to the patients’ health care record, which allowed the GP to continuously check the patient’s vitals. This allowed the GP to take quick action as needed as they would be notified if any major problems arose. Further, 26.4% (28/106) of the reports mentioned that telehealth tools also helped patients with self-management, particularly for patients with chronic conditions. In addition, this theme involved some innovation and creative use of technology by some CCGs. For example, 6.5% (7/106) of the reports discussed providing Alexa (Amazon Inc) devices to patients who were not able to use traditional computer devices (due to conditions such as vision impairments). The speech recognition software would allow the patient to keep in contact with their GPs, where they could also be monitored remotely. We note that some reports mentioned the use of telehealth tools in general terms rather than naming the specific technology implemented (such as the name or provider of the tool).

Telemedicine

Telemedicine was one of the least mentioned themes among the CCG groups, with it only being mentioned by 33% (35/106) of CCGs. Telemedicine differs from telehealth in that it seeks to provide treatment or make a diagnosis for patients using digital technologies. The more innovative approach to digital technologies would lie within this category. This includes health care practices using virtual reality headsets (2/106, 1.9%) to treat patients for their mental health by having them experience scenarios in a digital space. AI diagnostics, mentioned by 9% (10/106) of reports, makes predictions about a patient’s health (eg, heart-related issues or developing cancer), allowing the health care provider to advise a patient so they may mitigate any future health care concerns. We also found that 10.3% (11/106) of the reports mentioned technologies for online prescribing, often for a repeat prescription. Teledermatology, mentioned by 7.5% of the reports (8/106), refers to the use of static digital images to triage, diagnose, monitor, or assess skin conditions without the patient physically meeting the dermatologist. This technology required the users to submit a clear photo of their skin condition before the appointment (either in the form of online meetings or chat). This dependency on trust in technology (so that the users submit personal information through the tool and believe in the results) as well as effort in the form of hardware (such as a virtual reality headset, laptop, or mobile camera for taking pictures and making video calls) from the users, is consistent for all tools in this theme.

Care Home Technologies

We refer to care home technologies as digital technologies designed specifically for the care home or training tools for care home residents and staff to use digital technologies. The technologies under this theme contained accessibility considerations particularly designed to support people in the later stage of life and living in care homes; therefore we separated it as a single theme. Care home technologies were only mentioned by about 37.7% (40/106) of the reports. By 2021, it was predicted that more than 400,000 people would live in care homes in the United Kingdom and would likely need close monitoring and more delicate health care due to age or health conditions than older people staying at home. In total, 35.8% (38/106) of the CCGs mentioned the involvement of training staff within these settings so they could adequately use the new digital technologies to care for their residents.

Booking System

In total, 24.5% (26/106) of CCGs had started to adopt various forms of online booking systems, especially within GP practices. This is the alternative to the traditional method of a patient telephoning a GP asking for an appointment through these means. The reasons cited for the uptake of this theme by some CCGs can be attributed to three factors: (1) the integration of NHS 111 (a free-to-call single nonemergency number medical helpline) services with GP and emergency departments (EDs), (2) the desire to ease the burden on telephone lines, and (3) the development of the NHS app. Further, 2 types of technologies, namely indirect booking and direct booking, were found within this theme. Indirect online booking was mentioned by 13.2% (14/106) of the CCGs, and this was the process of NHS 111 having the ability to book patients directly into GP or ED appointments. Multiple reports mentioned that the sharing of data between services (such as GPs, NHS 111, and ED) allowed NHS 111 to filter patients through their lines and book patients into appointments that they urgently needed. For example, the Manchester CCG used the Adastra Digital solution to send information between EDs and NHS 111. This improved patient flow and eased the burden on GPs and EDs. Direct online booking is when the patient themselves can book, manage, and also cancel their own appointment online. This was mentioned by 14% (15/106) of the CCGs and many of them cited the integration of the NHS app as a convenient means of allowing patients access to book appointments themselves.

Online Triage

Virtually every CCG (104/106, 98.1%) mentioned some form of online triage that was integrated into their effort to adopt digital technologies. This mostly came from the reports covering the use of virtual consultations by their health care professionals. Many reports discussed how the COVID-19 pandemic forced them to accelerate their plans to integrate digital solutions into their health care plans, and virtual consultations became a must for many health care settings so they could continue operating safely (both for the practitioner and for the patients). Typically, this involved practitioners contacting patients via telephone or some form of video consultation services. The video consultation would usually be operated by private health care platforms such as Doctorlink (HealthHero), askmyGP (Evergreen Health Solutions Ltd), or Attend Anywhere (Induction Healthcare Group PLC), in which case the private companies created a process to help practitioners carry out this health care service (either through a telephone call, a smartphone app, or a web app). eConsult is a form of online triage mentioned by 19.6% (21/106) of CCGs and differs from the virtual consultation format as it involves patients filling out an online form and sending it to their health care practice, where it is reviewed and next steps for treatment are provided by a health care professional without directly speaking to the patient. This service was cited as being useful for prescribing repeat prescriptions and removing pressure on phone lines.

Digital Mental Health Services

Digital mental health services were popular among CCGs, being mentioned by 70.3% (74/106) of them. We decided to put these technologies into a separate theme due to the increased mental health needs during the pandemic and the innovations used to carry out traditional treatments (such as psychotherapy or counseling). Some mental health treatments involved using digital tools, such as video calls, to offer counseling services before the pandemic. These mental health services often incorporated other digital technologies such as information portals (eg, Instagram accounts) to share advice and tips for self-management. Around 38.7% (41/106) of CCGs developed their own region-specific digital mental health care services that they were able to refer patients to. For example, Mind-BLMK is available for people in Bedfordshire and Luton as well as Milton Keynes, and With Me in Mind is available for people in Doncaster, Rotherham, and North Lincolnshire. However, around 33.9% (36/106) of CCGs discussed how they had integrated private mental health services that they had partnered with to refer their patients to. Kooth was one of the more popular services as it focused on treatment for children and younger people, although other services such as Qwell [ 28 ] specialized in treatment for adults. Both sites are similar in layout, and they tailor their services to the targeted user group. For example, Kooth offers the opportunity for young people to engage in mini activities to manage their mental health, while Qwell offers more traditional long-form articles that explore different mental health issues and solutions. Both sites also offer online forums to talk to other patients of a similar age using the services, potentially sharing experiences and offering peer support to build a community between each other. Both sites offer access to therapists who specialize in the targeted age groups, as well.

Clustering was used to identify groups of CCGs implementing similar types of technologies. We first calculated Hopkins statistics for the data. The Hopkins statistic was 0.9914 (>0.5), indicating the data were highly clustered. A structure with 3 clusters was the optimal cluster structure (average silhouette width of 0.186, SD 0.163), determined using the silhouette and elbow method [ 27 ].

Resulting from hierarchical clustering, the first cluster group contained 51 CCGs, the second cluster contained 35, and the third contained 20. The CCGs differed between clusters in their digital themes. Figure 1 helps establish the differences between each group by showing the weighted proportions of technologies in each theme aggregated across CCGs in each group, which we refer to as “scores” in the following text and Figure 1 . There were 9 scores assigned to each CCG, each corresponding to a digital theme. Group 1 (orange) will be named “disengaged” (of digital technologies) due to low scores in all of the digital themes within the CCG reports within this cluster (8 out of 9 themes had scores less than 25%). Cluster 2 (blue) will be named “engaged” (with digital technologies) due to the CCG reports within this cluster having a general interest in many of the digital themes (6 out of 9 themes had scores between 25% and 50%). This is especially true for telemedicine ,where it excels compared to the other two clusters; however, there is a low score for the digital engagement and telemedicine themes. Group 3 (green) did not have this problem as the CCG reports within this cluster had a high score for technologies in the digital engagement category, as well as a very high score for those in the information portal category. Therefore, this group will be named “torchbearer” (of digital technologies).

Table 2 presents the number of CCGs in each cluster group represented by the region they belong to according to the UK Office for National Statistics. For example, in the southeast region, there were 5 CCGs that belonged to the digitally disengaged cluster and 6 CCGs that belonged to the digitally engaged cluster. Figure 2 displays the geographical distribution of cluster groups in England. The graph does not present a particular spatial pattern, but there are some notable observations. For one, all of the London CCGs fell into the digitally disengaged group. The northwest and Midlands regions have a proportionally very low presence in the digitally engaged cluster group. This may be due to them having a very high presence in the digital torchbearer group.

Principal Results

The pandemic has disrupted health care support services in the United Kingdom, resulting in the rapid adoption of digital technologies [ 29 ]. Nevertheless, the digital technology themes guiding the mass adoption of technology and the potential regional disparities remain to be explored. Based on 135 NHS CCG annual reports, this paper identified digital technology themes and associated technologies adopted by the NHS nationally in 2020 and 2021 and examined their regional differences from a digital divide perspective [ 30 ].

Informed by digital options theory [ 16 , 17 ], we identified 9 digital technology themes, which were categorized into 3 main groups, namely, the identification of patients’ requirements, the development of new work patterns that improve internal coordination and working process, and realizing improvements in efficiency and public accessibility. First, the identification of patients’ requirements includes the use of information portals (eg, retrieving and sharing health-related information), digital means (eg, using podcasts for promoting health-related information), and digital support (eg, providing hardware or alternative access for the digitally excluded) to achieve effective communication channels between health care providers and patients. Second, the development of new work patterns that improve internal coordination and working process includes digitizing existing health care services, such as remote monitoring of patient health (ie, Alexa), e-prescriptions (eg, AI diagnostics and online prescriptions), and digital training (eg, staff training on using digital tools in care homes). Finally, the realization of improvements in efficiency and public accessibility results in the use of digital methods to widen public access to health care, including creating online booking systems shared by health care providers (eg, NHS 111 and NHS apps), online triage (eg, online consultation), and online mental health services (eg, Kooth). Our data suggest that there are prominent digital themes within each group. Information portals are mostly adopted by CCGs to achieve effective health care communication among patients and health care providers. Telehealth is primarily adopted by CCGs for digitizing health care services, particularly to monitor patient health remotely. Online triage has been widely implemented by CCGs to provide patients with access to health care during the pandemic. Therefore, these top 3 themes, which cover people’s fundamental health needs, could serve as a starting point for future CCGs and other health care providers when adopting and implementing digital solutions.

In addition, our findings contribute to the new research theme of “digital health citizenship” by highlighting that digital tools and technologies extend beyond operation efficiency to wider patient engagement, and therefore reshape social relations and interactions among patients as health service users [ 2 ]. Regional differences in such social relations and interactions could be linked to the equality and inclusion issues in health service provision. Based on the 9 identified digital themes, 3 main clusters were identified: the digitally engaged, the digitally disengaged, and the digital torchbearer. It is concerning that almost half of the CCGs fell into the digitally disengaged group, showing a low uptake of the aforementioned digital themes. Interestingly, most digitally disengaged CCGs belonged to London areas. This seems to be aligned with the data released by the Office for National Statistics [ 31 ] in 2019, suggesting that London has the lowest percentage of internet nonusers in the United Kingdom by population. However, Watson et al [ 32 ] also pointed out that a lack of digital devices and private spaces for accessing online health care could also act as barriers to digital health care themes in London. The use of such digital devices and online health care services opens up a new set of digital rights, opportunities, and responsibilities for patients [ 2 ], and this needs to be balanced across regions to ensure the equality and inclusion of health care. Our data suggest that CCGs are still not sufficiently efficient when it comes to adopting digital themes.

Furthermore, our findings highlight the impact of COVID-19 on the development of a CCG’s digital technology themes. Many of the reports cited COVID-19 as accelerating the need to digitize health care. For example, the use of telehealth to increase remote monitoring of patients or encourage them to manage their own health increased during the pandemic due to patients not being able to reach their own health care practices as freely. Most of the reports were generally very positive about increasing the inclusion of digital technology. Reasons for the positives statements included making it easier to access health care, providing early treatment, providing more data to improve services, and making the health care practice more agile and efficient. However, some reports cited reasons for concern about increasing digital health care when discussing the possibility of digital exclusion for some patients. Further, one report from the Newcastle Gateshead CCG raised concerns about how the integration of digital technology may impact the NHS policy of being free at point of use. This issue is consistent with concerns expressed by other researchers. For example, Clare [ 33 ] and Eruchalu et al [ 34 ] have noted the potential limitations of telehealth and telemedicine due to broadband connectivity issues resulting from socioeconomic disparities among regions, particularly among the underprivileged, the medically underserved, and in communities of color. With services becoming increasingly online (eg, many mental health services were all online) and not everyone having access to technology, there is a justifiable concern. To realize equitable benefits from health-related technologies across all populations, it is imperative to thoroughly examine and address complex issues, such as social, cultural, and economic factors that hinder accessibility and adoption among different communities. Otherwise, as pointed out by Ramsetty and Adams [ 35 ], despite technological advancements, disparities in health care access and outcomes will inevitably persist, particularly among the most vulnerable during times of crisis. Future research could further investigate this area and how these issues could be addressed.

It is essential to highlight that the absence of digital technology mentioned in a CCG report does not necessarily indicate a failure in its adoption. For example, eConsult is an online triage form that patients can fill out and send to their health care practitioner where it is reviewed and the next steps for treatment are accessed by a health care professional for the patients to take. The Kent and Medway CCG did not make any reference to this digital technology within their annual report [ 36 ], however, when investigating individual GP practices that operate under them, the technology is being used. This suggests that the lack of mention of a particular technology by a CCG group does not mean it is not being used at all but indicates the action of using such technology is not applied at the CCG level.

This research mapped out the current digital technology themes adopted by CCGs in the United Kingdom when providing health care services during the pandemic. These identified themes can be used by future health care providers to adopt digital solutions to address different health care issues, such as improving communication, digitizing their existing services, and increasing public access to health care. Furthermore, the research highlights the existence of a digital divide within CCGs in terms of adopting digital technology themes, particularly when it comes to regional disparities. The possible solutions could include providing support to the “digitally disengaged” CCGs in using various identified digital technology themes and becoming more digitally “active” or “engaged.” Caution needs to be taken when offering support as well. For example, we need to fully explore and understand the reasons behind their slow uptake of digital technologies in order to offer tailored solutions. To build upon our findings and promote equal access to digital health benefits for all communities, future research could examine the factors that contribute to regional disparities in digital technology adoption and access within and across CCGs. A comprehensive understanding of the underlying causes of these disparities could help policy makers and health care professionals focus their efforts more effectively toward bridging the digital divide and improving access to health resources and support for the public.

Although our study offers great insights into the digital technology themes adopted by CCGs, there are a few limitations that need to be addressed. Our findings are mainly based on CCG reports. The length, focus, and description details may differ slightly between these reports. In addition, there may be underreporting when it comes to digital technology themes by individual CCGs, resulting in misrepresentation when analyzing reports. Finally, the research focus was on the digital technology themes and the adoption of associated technologies rather than the impact these technologies had on patient satisfaction or patient outcomes. Future research could look at the impact of these identified digital technology themes or individual technologies and their effectiveness through in-depth interviews or questionnaire surveys with patients and health care professionals.

Acknowledgments

This study is supported by the Faculty of Social Science Social Research Internship fund, University of Sheffield.

Data Availability

All data analyzed during this study are included in this published article and its supplementary information files.

Authors' Contributions

The project was conceived and designed by MZ and SL. The analysis was carried out by JAA, MZ, and SL. All authors read and approved the final paper.

Conflicts of Interest

None declared.

Al Dahdah M, Mishra RK. Digital health for all: The turn to digitized healthcare in India. Soc Sci Med. 2023;319:114968. [ CrossRef ] [ Medline ]
Petrakaki D, Hilberg E, Waring J. The cultivation of digital health citizenship. Soc Sci Med. 2021;270:113675. [ FREE Full text ] [ CrossRef ] [ Medline ]
Kelly JT, Campbell KL, Gong E, Scuffham P. The Internet of things: impact and implications for health care delivery. J Med Internet Res. 2020;22(11):e20135. [ FREE Full text ] [ CrossRef ] [ Medline ]
Ye J. The role of health technology and informatics in a global public health emergency: practices and implications from the COVID-19 pandemic. JMIR Med Inform. 2020;8(7):e19866. [ FREE Full text ] [ CrossRef ] [ Medline ]
Zweifel P. Innovation in health care through information technology (IT): The role of incentives. Soc Sci Med. 2021;289:114441. [ CrossRef ] [ Medline ]
Denicolai S, Previtali P. Innovation strategy and digital transformation execution in healthcare: The role of the general manager. Technovation. 2023;121:102555. [ CrossRef ]
Asaria M, Doran T, Cookson R. The costs of inequality: whole-population modelling study of lifetime inpatient hospital costs in the English National Health Service by level of neighbourhood deprivation. J Epidemiol Community Health. 2016;70(10):990-996. [ FREE Full text ] [ CrossRef ] [ Medline ]
Dixon A, Le Grand J, Henderson J, Murray R, Poteliakhoff E. Is the British National Health Service equitable? The evidence on socioeconomic differences in utilization. J Health Serv Res Policy. 2007;12(2):104-109. [ CrossRef ] [ Medline ]
Propper C, Damiani M, Leckie G, Dixon J. Impact of patients' socioeconomic status on the distance travelled for hospital admission in the English National Health Service. J Health Serv Res Policy. 2007;12(3):153-159. [ CrossRef ] [ Medline ]
Kontopantelis E, Mamas MA, van Marwijk H, Ryan AM, Bower P, Guthrie B, et al. Chronic morbidity, deprivation and primary medical care spending in England in 2015-16: a cross-sectional spatial analysis. BMC Med. 2018;16(1):19. [ FREE Full text ] [ CrossRef ] [ Medline ]
Sambamurthy V, Bharadwaj A, Grover V. Shaping agility through digital options: reconceptualizing the role of information technology in contemporary firms. MIS Q. 2003;27(2):237-263. [ CrossRef ]
Rolland KH, Mathiassen L, Rai A. Managing digital platforms in user organizations: The interactions between digital options and digital debt. Inf Syst Res. 2018;29(2):419-443. [ CrossRef ]
Karimi J, Somers TM, Bhattacherjee A. The role of ERP implementation in enabling digital options: A theoretical and empirical analysis. Int J Electron Commer. 2009;13(3):7-42. [ CrossRef ]
Fichman RG. Real options and IT platform adoption: implications for theory and practice. Inf Syst Res. 2004;15(2):132-154. [ CrossRef ]
Singh R, Mindel V, Mathiassen L. IT-enabled revenue cycle transformation in resource-constrained hospitals: A collaborative digital options inquiry. J Manag Inf Syst. 2017;34(3):695-726. [ CrossRef ]
Sandberg J, Mathiassen L, Napier N. Digital options theory for IT capability investment. JAIS. 2014;15(7):422-453. [ CrossRef ]
Svahn F, Lindgren R, Mathiassen L. Applying options thinking to shape generativity in digital innovation: an action research into connected cars. 2015. Presented at: 2015 48th Hawaii International Conference on System Sciences; January 5-8, 2015;4141-4150; Kauai, HI, USA. [ CrossRef ]
Holmås TH, Monstad K, Reme SE. Regular employment for people with mental illness - An evaluation of the individual placement and support programme. Soc Sci Med. 2021;270:113691. [ FREE Full text ] [ CrossRef ] [ Medline ]
Santana P. Ageing in Portugal: regional iniquities in health and health care. Soc Sci Med. 2000;50(7-8):1025-1036. [ FREE Full text ] [ CrossRef ] [ Medline ]
Giunchiglia F, Shvaiko P. Semantic matching. Knowl Eng Rev. 2004;18(3):265-280. [ CrossRef ]
Stemler S. An overview of content analysis. Pract Assess Res Eval. 2001;7(1):17. [ CrossRef ]
Jansons P, Fyfe J, Via JD, Daly RM, Gvozdenko E, Scott D. Barriers and enablers for older adults participating in a home-based pragmatic exercise program delivered and monitored by Amazon Alexa: a qualitative study. BMC Geriatr. 2022;22(1):248. [ FREE Full text ] [ CrossRef ] [ Medline ]
Patient-Specific Medicines Optimisation | OptimiseRx. FDB (First Databank). First Databank. URL: https://www.fdbhealth.co.uk/solutions/optimiserx-medicines-optimisation [accessed 2023-08-14]
Mind BLMK - For better mental health. URL: https://www.mind-blmk.org.uk/ [accessed 2023-08-14]
With Me In Mind – Mental Health Support Teams. URL: https://www.withmeinmind.co.uk/ [accessed 2023-08-14]
Murtagh F, Contreras P. Algorithms for hierarchical clustering: an overview. WIREs Data Min Knowl Discov. 2012;2(1):86-97. [ CrossRef ]
Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math. 1987;20:53-65. [ CrossRef ]
Qwell. URL: https://www.qwell.io/ [accessed 2023-08-14]
Golinelli D, Boetto E, Carullo G, Nuzzolese AG, Landini MP, Fantini MP. Adoption of digital technologies in health care during the COVID-19 pandemic: systematic review of early scientific literature. J Med Internet Res. 2020;22(11):e22280. [ FREE Full text ] [ CrossRef ] [ Medline ]
Ragnedda M, Ruiu ML. Social capital and the three levels of digital divide. In: Ragnedda M, Muschert GW, editors. Theorizing Digital Divides. London, UK. Routledge, Taylor & Francis Group; 2017.
Exploring the UK’s digital divide. Office for National Statistics. URL: https://www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/articles/exploringtheuksdigitaldivide/2019-03-04 [accessed 2023-02-02]
Watson A, Mellotte H, Hardy A, Peters E, Keen N, Kane F. The digital divide: factors impacting on uptake of remote therapy in a South London psychological therapy service for people with psychosis. J Ment Health. 2022;31(6):825-832. [ FREE Full text ] [ CrossRef ] [ Medline ]
Clare CA. Telehealth and the digital divide as a social determinant of health during the COVID-19 pandemic. Netw Model Anal Health Inform Bioinform. 2021;10(1):26. [ FREE Full text ] [ CrossRef ] [ Medline ]
Eruchalu CN, Pichardo MS, Bharadwaj M, Rodriguez CB, Rodriguez JA, Bergmark RW, et al. The expanding digital divide: digital health access inequities during the COVID-19 pandemic in New York city. J Urban Health. 2021;98(2):183-186. [ FREE Full text ] [ CrossRef ] [ Medline ]
Ramsetty A, Adams C. Impact of the digital divide in the age of COVID-19. J Am Med Inform Assoc. 2020;27(7):1147-1148. [ FREE Full text ] [ CrossRef ] [ Medline ]
NHS Kent and Medway CCG Annual Report 2021/22. NHS Kent and Medway CCG. URL: https://www.kentandmedway.icb.nhs.uk/about-us/ccg-archive/annual-reports-and-accounts [accessed 2024-03-25]

Abbreviations

Edited by A Mavragani; submitted 15.08.23; peer-reviewed by A Davies, I Wilson; comments to author 06.02.24; revised version received 06.03.24; accepted 08.03.24; published 19.04.24.

©Jake Alan Allcock, Mengdie Zhuang, Shuyang Li, Xin Zhao. Originally published in JMIR Formative Research (https://formative.jmir.org), 19.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

Press Enter to activate screen reader mode.

Institute of Molecular Plant Biology

Our paper made it into the eth news.

Twisted pollen tubes induce infertility Plants with multiple sets of chromosomes have advantages over their relatives with a double set. But why they often start out infertile was only partially understood. Biologists at ETH Zurich have now discovered a new reason for the initial difficulties.

mode_comment Number of comments

Please follow link to the ETH News site for the whole article: LINK

IMAGES

Table of Contents
Table of Contents for Dissertation/ Research Paper & Example
Thesis Table Of Contents Format
APA Table of Contents Writing Guide (+ Example)
Table of Contents for Dissertation/ Research Paper & Example
20 Table of Contents Templates and Examples ᐅ TemplateLab

VIDEO

How to Add a Table of Contents to Wordpress in 3 Clicks!
Automatic table contents updation creating table in MS word
CSEC CXC GEOGRAPHY SBA TABLE OF CONTENTS
Table of Contents || How to Use Table Of Contents in MS-Word ?
Table of Contents
11/18 CREATE REFERENCE AND TABLE OF CONTENTS in Microsoft Word 2019/365

COMMENTS

Table of Contents
The table of contents is usually located at the beginning of the document or book, after the title page and any front matter, such as a preface or introduction. Table of Contents in Research. In Research, A Table of Contents (TOC) is a structured list of the main sections or chapters of a research paper, Thesis and Dissertation. It provides ...
How to Create an APA Table of Contents
Now you can generate your table of contents. First write the title "Contents" (in the style of a level 1 heading). Then place your cursor two lines below this and go to the References tab. Click on Table of Contents and select Custom Table of Contents…. In the popup window, select how many levels of heading you wish to include (at least ...
How to Structure the Table of Contents for a Research Paper
To summarize, the following steps will help you create a clear and concise table of contents to guide readers through your research paper: 1. Insert the table of contents after the title page. 2. List all the sections and subsections in chronological order. 3. Paginate each section. 4. Format the table of contents according to your style guide. 5.
PDF Student Paper Setup Guide, APA Style 7th Edition
Indent the first line of every paragraph of text 0.5 in. using the tab key or the paragraph-formatting function of your word-processing program. Page numbers: Put a page number in the top right corner of every page, including the title page or cover page, which is page 1. Student papers do not require a running head on any page.
Dissertation Table of Contents in Word
The table of contents is where you list the chapters and major sections of your thesis, dissertation, or research paper, alongside their page numbers. A clear and well-formatted table of contents is essential, as it demonstrates to your reader that a quality paper will follow. ... or research paper. In addition to a table of contents, you might ...
APA Format Table of Contents
Research papers, in particular, may benefit from the addition of a table of contents. APA style is the official publication style of the American Psychological Association. APA style is used in psychology courses as well as other social science classes including those in social science, behavioral sciences, and education.
How to Write a Table of Contents for Research Paper
A table of contents (TOC) is a list of a research paper's main sections and subsections, along with their page numbers. It serves as a roadmap for the reader, allowing them to find the information they need within the paper quickly. The table of contents for a research paper is usually placed at the beginning of the paper, after the title ...
Table of Contents
However, if your written report, research paper, or other longer work is more than five pages, it is helpful to include a table of contents showing the page number where each specific section is ...
Table of Contents
A table of contents is a list, usually on a page at the beginning of a piece of academic writing, which outlines the chapters or sections names with their corresponding page numbers. In addition to chapter names, it includes bullet points of the sub-chapter headings or subsection headings. It usually comes right after the title page of a ...
Table of contents for research papers
The table of contents forms an essential part of any academic paper. Through the use of headings, sub-headings, and page numbers, we can construct an accurate road map to assist reviewers, evaluators, tutors, and general readers. The table of contents shows how effective the writer is at dividing the thesis into relevant and manageable sections.
How to Effectively Create a Table of Contents for Your Research Paper
Related Questions and Answers from "Table of Contents Research Paper" 1. What is the purpose of a table of contents in a research paper? A table of contents in a research paper serves as a structured guide, allowing readers to easily navigate and locate specific sections or chapters. It provides an overview of the paper's organization and ...
Research Paper Format
Formatting a Chicago paper. The main guidelines for writing a paper in Chicago style (also known as Turabian style) are: Use a standard font like 12 pt Times New Roman. Use 1 inch margins or larger. Apply double line spacing. Indent every new paragraph ½ inch. Place page numbers in the top right or bottom center.
How To Write a Table of Contents for Academic Papers
A working table of contents should begin with a title. This title may change as you draft your text, but a working title will help you focus your thoughts as you devise the headings and plan the content for the main parts, chapters, sections and subsections that should be added beneath it. All headings, whether numbered or not, should be ...
Table of Contents
An automatic Table of Contents relies on Styles to keep track of page numbers and section titles for you automatically. Microsoft Word can scan your document and find everything in the Heading 1 style and put that on the first level of your table of contents, put any Heading 2's on the second level of your table of contents, and so on.. If you want an automatic table of contents you need to ...
Dissertation Table of Contents in Word
The table of contents is where you list the chapters and major sections of your thesis, dissertation, or research paper, alongside their page numbers. A clear and well-formatted table of contents is essential, as it demonstrates to your reader that a quality paper will follow. ... or research paper. In addition to a table of contents, you might ...
What is Academic Writing: Tips for Students
Table of Contents. Critical characteristics of academic writing ; Types of academic writing . Essays; Research paper; Literature review; ... Unlike essays, research papers are more detailed, scholarly, and based on independent research. They should demonstrate their knowledge of the topic, employ analytical rigour, and demonstrate strength in ...
Table of Contents for Dissertation/ Research Paper & Example
Open Home tab and choose the style for your table of contents (ToC next). Apply heading 1 to your chapters, heading 2 to the subheading, and if needed heading 3 to the level 3 heading. Next, you are going to create a research paper or PhD dissertation table of contents. Open References and choose ToC.
Research Paper Appendix
Research Paper Appendix | Example & Templates. Published on August 4, 2022 by Tegan George and Kirsten Dingemanse. Revised on July 18, 2023. An appendix is a supplementary document that facilitates your reader's understanding of your research but is not essential to your core argument. Appendices are a useful tool for providing additional information or clarification in a research paper ...
Table of Contents
Table of Contents March 1, 2024; 38 (5-6) PERSPECTIVE. Select this article. Protein domains of low sequence complexity—dark matter of the proteome . ... RESEARCH PAPERS. Select this article. WRN exonuclease imparts high fidelity on translesion synthesis by Y family DNA polymerases . Jung-Hoon Yoon,
Journal of Medical Internet Research
Background: With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess ...
JMIR Formative Research
Background: In England, digital technologies are exploited to transform the way health and social care is provided and encompass a wide range of hardware devices and software that are used in all aspects of health care. However, little is known about the extent to which health care providers differ in digital health technology capabilities and how this relates to geographical and regional ...
APA Format for Tables and Figures
Tables and figures are each numbered separately, in the order they are referred to in your text. For example, the first table you refer to is Table 1; the fourth figure you refer to is Figure 4. The title should clearly and straightforwardly describe the content of the table or figure. Omit articles to keep it concise.
Our paper made it into the ETH News
Our paper made it into the ETH News. Twisted pollen tubes induce infertility Plants with multiple sets of chromosomes have advantages over their relatives with a double set. But why they often start out infertile was only partially understood. Biologists at ETH Zurich have now discovered a new ...

How to Structure the Table of Contents for a Research Paper

Steps to Create a Table of Contents

What is a table of contents?

Find this useful?

What should I include in my table of contents?

Is there more than one way to format a table of contents?

Share this article:

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

How to Cite the CDC in APA

Six Product Description Generator Tools for Your Product Copy

What Is a Content Editor?

The Benefits of Using an Online Proofreading Service

6 Online AI Presentation Maker Tools

What Is Market Research?

Make sure your writing is the best it can be with our expert English proofreading and editing.

A Table of Contents in APA Format

General Guidelines

Table of Contents Format

charlesstreetresearch.com

How to Write a Table of Contents for Research Paper

What is a table of contents?

Table of Contents

Practical tips on how to write a table of contents for a research paper

Leave a Reply Cancel reply

Your Step to Success

Table of Contents

A guide to the table of contents page

Definition: Table of Contents

What is a table of contents?

How do you write a table of contents

Where do you put a table of contents?

What to include in a table of contents?

How important is a table of contents?

Everything for Your Thesis

Sample table of contents for a short dissertation:

Sample table of contents for a PhD dissertation:

Create in Microsoft Word

In a Nutshell

How to Effectively Create a Table of Contents for Your Research Paper?

Understanding the Basics of a Table of Contents

Step-by-Step Guide to Crafting Your TOC

Actionable Steps

Related Questions and Answers from “Table of Contents Research Paper”

Book Editing

Medical Editing

PhD Thesis Editing

How To Write a Table of Contents for Academic Papers

You might be interested in Services offered by Proof-Reading-Service.com

Scientific Editing

Expert Editing

Research Editing

Related Posts

Choosing the Right Journal

Example of a Quantitative Research Paper

What Is a Good H-Index Required for an Academic Position?

Acknowledgements Example for an Academic Research Paper

Our Recent Posts

Our review ratings

Explore our Categories

Microsoft Word for Dissertations

Automatic Table of Contents

Modifying the format of your Table of Contents

Bonus tip for updating fields like the Table of Contents

Have a language expert improve your writing

Dissertation Table of Contents in Word | Instructions & Examples

Instantly correct all language mistakes in your text

Table of contents

Including appendices and tables

The only proofreading tool specialized in correcting academic writing

Prevent plagiarism, run a free check.

Cite this Scribbr article

Is this article helpful?

Tegan George

What is Academic Writing: Tips for Students

Table of Contents

Research paper

Critical characteristics of academic writing

Types of academic writing

Dos and don’ts for students who are starting with academic writing

Don’ts