techniques of data representation

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• *New* Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

17 Data Visualization Techniques All Professionals Should Know

• 17 Sep 2019

There’s a growing demand for business analytics and data expertise in the workforce. But you don’t need to be a professional analyst to benefit from data-related skills.

Becoming skilled at common data visualization techniques can help you reap the rewards of data-driven decision-making , including increased confidence and potential cost savings. Learning how to effectively visualize data could be the first step toward using data analytics and data science to your advantage to add value to your organization.

Several data visualization techniques can help you become more effective in your role. Here are 17 essential data visualization techniques all professionals should know, as well as tips to help you effectively present your data.

Access your free e-book today.

What Is Data Visualization?

Data visualization is the process of creating graphical representations of information. This process helps the presenter communicate data in a way that’s easy for the viewer to interpret and draw conclusions.

There are many different techniques and tools you can leverage to visualize data, so you want to know which ones to use and when. Here are some of the most important data visualization techniques all professionals should know.

Data Visualization Techniques

The type of data visualization technique you leverage will vary based on the type of data you’re working with, in addition to the story you’re telling with your data .

Here are some important data visualization techniques to know:

• Gantt Chart
• Box and Whisker Plot
• Waterfall Chart
• Scatter Plot
• Pictogram Chart
• Highlight Table
• Bullet Graph
• Choropleth Map
• Network Diagram
• Correlation Matrices

1. Pie Chart

Pie charts are one of the most common and basic data visualization techniques, used across a wide range of applications. Pie charts are ideal for illustrating proportions, or part-to-whole comparisons.

Because pie charts are relatively simple and easy to read, they’re best suited for audiences who might be unfamiliar with the information or are only interested in the key takeaways. For viewers who require a more thorough explanation of the data, pie charts fall short in their ability to display complex information.

2. Bar Chart

The classic bar chart , or bar graph, is another common and easy-to-use method of data visualization. In this type of visualization, one axis of the chart shows the categories being compared, and the other, a measured value. The length of the bar indicates how each group measures according to the value.

One drawback is that labeling and clarity can become problematic when there are too many categories included. Like pie charts, they can also be too simple for more complex data sets.

3. Histogram

Unlike bar charts, histograms illustrate the distribution of data over a continuous interval or defined period. These visualizations are helpful in identifying where values are concentrated, as well as where there are gaps or unusual values.

Histograms are especially useful for showing the frequency of a particular occurrence. For instance, if you’d like to show how many clicks your website received each day over the last week, you can use a histogram. From this visualization, you can quickly determine which days your website saw the greatest and fewest number of clicks.

4. Gantt Chart

Gantt charts are particularly common in project management, as they’re useful in illustrating a project timeline or progression of tasks. In this type of chart, tasks to be performed are listed on the vertical axis and time intervals on the horizontal axis. Horizontal bars in the body of the chart represent the duration of each activity.

Utilizing Gantt charts to display timelines can be incredibly helpful, and enable team members to keep track of every aspect of a project. Even if you’re not a project management professional, familiarizing yourself with Gantt charts can help you stay organized.

5. Heat Map

A heat map is a type of visualization used to show differences in data through variations in color. These charts use color to communicate values in a way that makes it easy for the viewer to quickly identify trends. Having a clear legend is necessary in order for a user to successfully read and interpret a heatmap.

There are many possible applications of heat maps. For example, if you want to analyze which time of day a retail store makes the most sales, you can use a heat map that shows the day of the week on the vertical axis and time of day on the horizontal axis. Then, by shading in the matrix with colors that correspond to the number of sales at each time of day, you can identify trends in the data that allow you to determine the exact times your store experiences the most sales.

6. A Box and Whisker Plot

A box and whisker plot , or box plot, provides a visual summary of data through its quartiles. First, a box is drawn from the first quartile to the third of the data set. A line within the box represents the median. “Whiskers,” or lines, are then drawn extending from the box to the minimum (lower extreme) and maximum (upper extreme). Outliers are represented by individual points that are in-line with the whiskers.

This type of chart is helpful in quickly identifying whether or not the data is symmetrical or skewed, as well as providing a visual summary of the data set that can be easily interpreted.

7. Waterfall Chart

A waterfall chart is a visual representation that illustrates how a value changes as it’s influenced by different factors, such as time. The main goal of this chart is to show the viewer how a value has grown or declined over a defined period. For example, waterfall charts are popular for showing spending or earnings over time.

8. Area Chart

An area chart , or area graph, is a variation on a basic line graph in which the area underneath the line is shaded to represent the total value of each data point. When several data series must be compared on the same graph, stacked area charts are used.

This method of data visualization is useful for showing changes in one or more quantities over time, as well as showing how each quantity combines to make up the whole. Stacked area charts are effective in showing part-to-whole comparisons.

9. Scatter Plot

Another technique commonly used to display data is a scatter plot . A scatter plot displays data for two variables as represented by points plotted against the horizontal and vertical axis. This type of data visualization is useful in illustrating the relationships that exist between variables and can be used to identify trends or correlations in data.

Scatter plots are most effective for fairly large data sets, since it’s often easier to identify trends when there are more data points present. Additionally, the closer the data points are grouped together, the stronger the correlation or trend tends to be.

10. Pictogram Chart

Pictogram charts , or pictograph charts, are particularly useful for presenting simple data in a more visual and engaging way. These charts use icons to visualize data, with each icon representing a different value or category. For example, data about time might be represented by icons of clocks or watches. Each icon can correspond to either a single unit or a set number of units (for example, each icon represents 100 units).

In addition to making the data more engaging, pictogram charts are helpful in situations where language or cultural differences might be a barrier to the audience’s understanding of the data.

11. Timeline

Timelines are the most effective way to visualize a sequence of events in chronological order. They’re typically linear, with key events outlined along the axis. Timelines are used to communicate time-related information and display historical data.

Timelines allow you to highlight the most important events that occurred, or need to occur in the future, and make it easy for the viewer to identify any patterns appearing within the selected time period. While timelines are often relatively simple linear visualizations, they can be made more visually appealing by adding images, colors, fonts, and decorative shapes.

12. Highlight Table

A highlight table is a more engaging alternative to traditional tables. By highlighting cells in the table with color, you can make it easier for viewers to quickly spot trends and patterns in the data. These visualizations are useful for comparing categorical data.

Depending on the data visualization tool you’re using, you may be able to add conditional formatting rules to the table that automatically color cells that meet specified conditions. For instance, when using a highlight table to visualize a company’s sales data, you may color cells red if the sales data is below the goal, or green if sales were above the goal. Unlike a heat map, the colors in a highlight table are discrete and represent a single meaning or value.

13. Bullet Graph

A bullet graph is a variation of a bar graph that can act as an alternative to dashboard gauges to represent performance data. The main use for a bullet graph is to inform the viewer of how a business is performing in comparison to benchmarks that are in place for key business metrics.

In a bullet graph, the darker horizontal bar in the middle of the chart represents the actual value, while the vertical line represents a comparative value, or target. If the horizontal bar passes the vertical line, the target for that metric has been surpassed. Additionally, the segmented colored sections behind the horizontal bar represent range scores, such as “poor,” “fair,” or “good.”

14. Choropleth Maps

A choropleth map uses color, shading, and other patterns to visualize numerical values across geographic regions. These visualizations use a progression of color (or shading) on a spectrum to distinguish high values from low.

Choropleth maps allow viewers to see how a variable changes from one region to the next. A potential downside to this type of visualization is that the exact numerical values aren’t easily accessible because the colors represent a range of values. Some data visualization tools, however, allow you to add interactivity to your map so the exact values are accessible.

15. Word Cloud

A word cloud , or tag cloud, is a visual representation of text data in which the size of the word is proportional to its frequency. The more often a specific word appears in a dataset, the larger it appears in the visualization. In addition to size, words often appear bolder or follow a specific color scheme depending on their frequency.

Word clouds are often used on websites and blogs to identify significant keywords and compare differences in textual data between two sources. They are also useful when analyzing qualitative datasets, such as the specific words consumers used to describe a product.

16. Network Diagram

Network diagrams are a type of data visualization that represent relationships between qualitative data points. These visualizations are composed of nodes and links, also called edges. Nodes are singular data points that are connected to other nodes through edges, which show the relationship between multiple nodes.

There are many use cases for network diagrams, including depicting social networks, highlighting the relationships between employees at an organization, or visualizing product sales across geographic regions.

17. Correlation Matrix

A correlation matrix is a table that shows correlation coefficients between variables. Each cell represents the relationship between two variables, and a color scale is used to communicate whether the variables are correlated and to what extent.

Correlation matrices are useful to summarize and find patterns in large data sets. In business, a correlation matrix might be used to analyze how different data points about a specific product might be related, such as price, advertising spend, launch date, etc.

Other Data Visualization Options

While the examples listed above are some of the most commonly used techniques, there are many other ways you can visualize data to become a more effective communicator. Some other data visualization options include:

• Bubble clouds
• Circle views
• Dendrograms
• Dot distribution maps
• Open-high-low-close charts
• Polar areas
• Radial trees
• Ring Charts
• Sankey diagram
• Span charts
• Streamgraphs
• Wedge stack graphs
• Violin plots

Tips For Creating Effective Visualizations

Creating effective data visualizations requires more than just knowing how to choose the best technique for your needs. There are several considerations you should take into account to maximize your effectiveness when it comes to presenting data.

Related : What to Keep in Mind When Creating Data Visualizations in Excel

One of the most important steps is to evaluate your audience. For example, if you’re presenting financial data to a team that works in an unrelated department, you’ll want to choose a fairly simple illustration. On the other hand, if you’re presenting financial data to a team of finance experts, it’s likely you can safely include more complex information.

Another helpful tip is to avoid unnecessary distractions. Although visual elements like animation can be a great way to add interest, they can also distract from the key points the illustration is trying to convey and hinder the viewer’s ability to quickly understand the information.

Finally, be mindful of the colors you utilize, as well as your overall design. While it’s important that your graphs or charts are visually appealing, there are more practical reasons you might choose one color palette over another. For instance, using low contrast colors can make it difficult for your audience to discern differences between data points. Using colors that are too bold, however, can make the illustration overwhelming or distracting for the viewer.

Related : Bad Data Visualization: 5 Examples of Misleading Data

Visuals to Interpret and Share Information

No matter your role or title within an organization, data visualization is a skill that’s important for all professionals. Being able to effectively present complex data through easy-to-understand visual representations is invaluable when it comes to communicating information with members both inside and outside your business.

There’s no shortage in how data visualization can be applied in the real world. Data is playing an increasingly important role in the marketplace today, and data literacy is the first step in understanding how analytics can be used in business.

Are you interested in improving your analytical skills? Learn more about Business Analytics , our eight-week online course that can help you use data to generate insights and tackle business decisions.

This post was updated on January 20, 2022. It was originally published on September 17, 2019.

About the Author

17 Essential Data Visualization Techniques, Concepts & Methods To Improve Your Business – Fast

“By visualizing information, we turn it into a landscape that you can explore with your eyes. A sort of information map. And when you’re lost in information, an information map is kind of useful.” – David McCandless

Did you know? 90% of the information transmitted to the brain is visual.

Concerning professional growth, development, and evolution, using data-driven insights to formulate actionable strategies and implement valuable initiatives is essential. Digital data not only provides astute insights into critical elements of your business, but if presented in an inspiring, digestible, and logical format, it can tell a tale that everyone within the organization can get behind.

Data visualization methods refer to the creation of graphical representations of information. Visualization plays a crucial part in data analytics and helps interpret big data in a real-time structure by utilizing complex sets of numerical or factual figures.

With the seemingly infinite streams of data readily available to today's businesses across industries, the challenge lies in data interpretation , which is the most valuable insight into the individual organization as well as its aims, goals, and long-term objectives.

That's where data visualization comes in.

Due to how the human brain processes information, presenting insights in charts or graphs to visualize significant amounts of complex data is more accessible than relying on spreadsheets or reports.

Visualizations offer a swift, intuitive, and simpler way of conveying critical concepts universally – and it's possible to experiment with different scenarios by making tiny adjustments.

Recent studies discovered that the use of visualizations in data analytics could shorten business meetings by 24% . Moreover, a business intelligence strategy with visualization capabilities boasts a ROI of $13.01 back on every dollar spent.

Therefore, the visualization of data is critical to the sustained success of your business and to help you yield the possible value from this tried and tested means of analyzing and presenting vital information. To keep putting its value into perspective, let’s start by listing a few of the benefits businesses can reap from efficient visuals. 

Benefits Of Data Visualization Skills & Techniques

As we just mentioned in the introduction, using visuals to boost your analytical strategy can significantly improve your company’s return on investment as well as set it apart from competitors by involving every single employee and team member in the analysis process. This is possible thanks to the user-friendly approach of modern online data analysis tools that allow an average user, without the need for any technical knowledge, to use data in the shape of interactive graphs in their decisions making process. Let’s look at some of the benefits data visualization skills can provide to an organization. 

• Boosts engagement: Generating reports has been a tedious and time-consuming task since businesses and analytics came together. Not only are static reports full of numbers and text quickly outdated, but they are also harder to understand for non-technical users. How can you get your employees to be motivated and work towards company goals when they might not even understand them? Data visualizations put together in intuitive dashboards can make the analysis process more dynamic and understandable while keeping the audience engaged.  
• Makes data accessible: Following up on the accessibility point, imagine you are an employee that has never worked with data before. Trying to extract relevant conclusions from a bunch of numbers on a spreadsheet can become an unbearable task. Data visualizations relieve them from that burden by providing easy access to relevant performance insights. By looking at well-made graphs, employees can find improvement opportunities in real-time and apply them to their strategies. For instance, your marketing team can monitor the development of their campaigns and easily understand at a glance if something is not going as expected or if they exceeded their initial expectations. 
• They save time: No matter the business size, it is very likely that you are working with raw data coming from various sources. Working with this raw data as it is can present many challenges, one of them being the amount of time that it takes to analyze and extract conclusions from it. A time that could be spent on other important organizational or operational tasks. With the right data visualization tools and techniques, this is not an issue, as you can quickly visualize critical performance indicators in stunning graphs within seconds.  Like this, you can build a complete story, find relationships, make comparisons, and navigate through the data to find hidden insights that might otherwise remain untapped. 

17 Essential Data Visualization Techniques

 Now that you have a better understanding of how visuals can boost your relationship with data, it is time to go through the top techniques, methods, and skills needed to extract the maximum value out of this analytical practice. Here are 17 different types of data visualization techniques you should know.

1. Know Your Audience

This is one of the most overlooked yet vital concepts around.

In the grand scheme of things, the World Wide Web and Information Technology as a concept are in their infancy - and data visualization is an even younger branch of digital evolution.

That said, some of the most accomplished entrepreneurs and executives find it difficult to digest more than a pie chart, bar chart, or a neatly presented visual, nor do they have the time to delve deep into data. Therefore, ensuring that your content is both inspiring and tailored to your audience is one of the most essential data visualization techniques imaginable.

Some stakeholders within your organization or clients and partners will be happy with a simple pie chart, but others will be looking to you to delve deeper into the insights you’ve gathered. For maximum impact and success, you should always conduct research about those you’re presenting to prior to a meeting and collate your report to ensure your visuals and level of detail meet their needs exactly.

2. Set Your Goals

Like any business-based pursuit, from brand storytelling right through to digital selling and beyond - with the visualization of your data, your efforts are only as effective as the strategy behind them.

To structure your visualization efforts, create a logical narrative and drill down into the insights that matter the most. It’s crucial to set a clear-cut set of aims, objectives, and goals prior to building your management reports , graphs, charts, and additional visuals.

By establishing your aims for a specific campaign or pursuit, you should sit down in a collaborative environment with others invested in the project and establish your ultimate aims in addition to the kind of data that will help you achieve them.

One of the most effective ways to guide your efforts is by using a predetermined set of relevant KPIs for your project, campaigns, or ongoing commercial efforts and using these insights to craft your visualizations.

3. Choose The Right Chart Type

One of the most effective methods of data visualization on our list; is to succeed in presenting your data effectively, you must select the right graphics for your specific project, audience, and purpose.

For instance, if you are demonstrating a change over set periods with more than a small handful of insights, a line graph is an effective means of visualization. Moreover, lines make it simple to plot multiple series together.

**click to enlarge**

An example of a line chart used to present monthly sales trends for a one-year period in a clear and glanceable format.

Here are six other effective chart types for different data visualization concepts:

a) Number charts

Real-time number charts are particularly effective when you’re looking to showcase an immediate and interactive overview of a particular key performance indicator, whether it’s a sales KPI , site visitations, engagement levels, or a percentage of evolution.

First of all, maps look great, which means they will inspire engagement in a board meeting or presentation. Secondly, a map is a quick, easy, and digestible way to present large or complex sets of geographical information for a number of purposes.

c) Pie charts

While pie charts have received a bad rep in recent years, we feel that they form a useful visualization tool that serves up important metrics in an easy-to-follow format. Pie charts prove particularly useful when demonstrating the proportional composition of a certain variable over a static timeframe. And as such, pie charts will be a valuable item in your visualization arsenal.

d) Gauge charts

This example shows the operating expense ratio, strongly related to the profit and loss area of your finance department’s key activities, and this color-coded health gauge helps you gain access to the information you need, even at a quick glance.

Gauge charts can be effectively used with a single value or data point. Whether they're used in financial or executive dashboard reports to display progress against key performance indicators, gauge charts are an excellent example of showcasing an immediate trend indication.

e) Bar or column chart

One of the most common types of visuals, the bar chart, is often used to compare two or more values in the same category, such as which product is sold the most in the women's department. Retail analytics tools allow you to visualize relevant metrics in interactive bar charts such as the one displayed below. There you can see a detailed breakdown of sales by country. This way, you can easily understand at a glance where to focus your promotional efforts, for example. 

d) Area chart  

Area charts are perfect when you want to show how different values developed over time. It combines a line and a bar chart to show how numeric values change based on a second variable. For example, we can see an area chart in action below tracking the P/E ratio. This financial analytics metric measures the value of a company’s shares compared to an industry benchmark (second variable). It gives investors an idea of how much they would pay for stock shares for each dollar of earnings. 

e) Spider chart  

Spider charts are complex visuals used to compare multivariate data with three or more quantitative variables. They are not so commonly used as bar or column graphs, but they prove extremely useful when analyzing rankings, reviews, or performance. For instance, our example below shows an employee skill analysis where three employees are being evaluated based on 6 attributes and a score. Through this, users can understand which employee is over or underperforming in each area and provide help where needed. 

f) Treemap chart

This chart type is used to display hierarchical data through rectangles that increase or decrease their size proportional to the changes in the value it represents. It is often used to display large volumes of data in a visually appealing way to help the audience extract conclusions from it. It can be divided into multiple categories, but each category needs to have a different color, as seen in our example below, where the patient drug cost per stay is divided by department.   

To find out more and expand your data visualization techniques knowledge base, you can explore our selected types of graphs and charts simple guide on how and when to use them.

4. Be Careful Not To Mislead  

As mentioned a couple of times already, well-made visuals open the analytical world to a wider audience by offering easy-to-understand access to critical information. In fact, during the COVID-19 pandemic, millions of people across the globe used graphs and charts to stay informed about the number of cases and deaths. That said, purposely or not, visuals are not always used with the best intentions. The data in them can be manipulated to show a different or more exaggerated version of the truth. This is a common tactic used in the media, politics, and advertising, and you should be aware of it not only to identify it but also to prevent it from happening to you when generating a graphic. Some of the bad practices to avoid include: 

• Truncating axes: It happens when the y-axis starts at a defined value instead of 0. This makes small differences between data points seem hyperbolic. It is widely used in politics to exaggerate particular scenarios. 
• Omitting data : As its name suggests, this involves omitting specific data sets from the visual. This could either be intentional to hide a specific trend or unintentional to ensure the chart is not crowded. To prevent it, double-check that something is not critical to the context and overall understanding of the chart before omitting it. 
• Correlating causation : It is the assumption that because two variables changed simultaneously, one caused the other. This should not be taken as an assumption, and causation should always be confirmed. 

Learn more about this data visualization methodology by exploring our guide on misleading data visualizations . 

5. Take Advantage Of Color Theory

The most straightforward of our selected data visualization techniques - selecting the right color scheme for your presentational assets will help enhance your efforts significantly. 

Colors not only help in highlighting or emphasizing areas of focus, but they are also proven to be a key factor in the user’s decision-making process, as specific colors are known to cause certain emotions in people. Therefore, putting some thought into the process is very important. For instance, you should consider preconceived color associations that users might have, such as associating lighter colors with lower or median values or red and green showing negative and positive results. Taking advantage of these natural associations can help you build visuals that will be automatically engaging and understandable for the audience. 

On that same note, using a color palette that matches the business’s branding will also make the visuals more engaging and familiar. If you choose to go this route, ensure you respect the text and the use of white space. The principles of color theory will have a notable impact on the overall success of your visualization model. That said, you should always try to keep your color scheme consistent throughout your data visualizations, using clear contrasts to distinguish between elements (e.g., positive trends in green and negative trends in red). As a guide, people, on the whole, use red, green, blue, and yellow as they can be recognized and deciphered with ease.

6. Prioritize Simplicity  

Another technique that should not be ignored is always to keep your design simple and understandable, as that is the key to a successful visual. To do so, you should avoid cluttering the graph with unnecessary elements such as too many labels, distracting patterns or images, and colors that are too bright, among other things. Another important thing to consider to ensure simplicity is to use fonts that are classic and easy to understand. Avoid italics or other “artistic” fonts to prevent your text from taking the attention away from the main message of your graph. 

Most importantly, when designing your visuals, stay away from 3D effects and any other element that can make the graphic overwhelming to the eye, such as borders, color gradients, and others. As mentioned in the previous point, stick to a light color palette that is not tiring to the eye. In a business context, it is also a good idea to use the colors, font, and overall brand identity of the business to boost the audience’s engagement towards your visuals. 

In the context of generating a dashboard or report where you need to include multiple visuals, it is recommended to avoid cluttering them with too many graphs. Stick only to the ones that will help you tell a compelling story. More on this point later in the post. 

7. Handle Your Big Data

With an overwhelming level of data and insights available in today’s digital world - with roughly 1.7 megabytes of data to be generated per second for every human being on the planet by the year 2020 - handling, interpreting, and presenting this rich wealth of insight does prove to be a real challenge.

To help you handle your big data and break it down for the most focused, logical, and digestible visualizations possible, here are some essential tips:

• Discover which data is available to you and your organization, decide which is the most valuable, and label each branch of information clearly to make it easy to separate, analyze, and decipher.
• Ensure that all of your colleagues, staff, and team members understand where your data comes from and how to access it to ensure the smooth handling of insights across departments.
• Keep your data protected and your data handling systems simple, digestible, and updated to make the visualization process as straightforward and intuitive as humanly possible.
• Ensure that you use business dashboards that present your most valuable insights in one easy-to-access, interactive space - accelerating the visualization process while also squeezing the maximum value from your information.

8. Use Ordering, Layout, And Hierarchy To Prioritize

Following on our previous point, once you’ve categorized your data and broken it down into the branches of information that you deem to be most valuable to your organization, you should dig deeper, creating a clearly labeled hierarchy of your data, prioritizing it by using a system that suits you (color-coded, numeric, etc.) while assigning each data set a visualization model or chart type that will showcase it to the best of its ability.

Of course, your hierarchy, ordering, and layout will be in a state of constant evolution, but by putting a system in place, you will turn your visualization efforts speedier, simpler, and more successful.

9. Utilize Word Clouds And Network Diagrams

To handle semi-structured or decidedly unstructured sets of data efficiently, you should consult the services of network diagrams or cloud words.

A network diagram is often utilized to draw a graphical chart of a network. This style of layout is useful for network engineers, designers, and data analysts while compiling comprehensive network documentation.

Akin to network diagrams, word clouds offer a digestible means of presenting complex sets of unstructured information. But, as opposed to graphical assets, a word cloud is an image developed with words used for particular text or subject, in which the size of each word indicates its frequency or importance within the context of the information.

10. Use Text Carefully  

So far, we’ve made it abundantly clear that the human brain processes visuals better than text. However, that doesn’t mean you should exclude text altogether. When building efficient graphics with your data, the use of text plays a fundamental role in making the graphs understandable for the audience. That said, it should be used carefully and with a clear purpose. 

The most common text elements you can find in data visualizations are often captions, labels, legends, or tooltips, to name a few. Let’s look at each of them in a bit more detail. 

• Captions : The caption occupies the top place in a graph or chart, telling the user what he or she should look for in that visual. When it comes to captions, you should always avoid verbosity. Keep them short and concise, and always add the units of measurement. 
• Labels: Labels describe a value associated with a specific data point in the chart. Here it is important to keep them short, as too long labels can crowd the visual and make it hard to understand. 
• Legends: A legend is a side section of a chart that gives a brief description to help users understand the data being displayed. For example, what each color means. A good practice when it comes to legends is to arrange them per order of appearance. 
• Tooltip: A tooltip is a visualization technique that allows you to add extra information to your graphs to make them more clear. Now, adding them under each data point would totally overcrowed them. Instead, you should rely on interactive tooltips that show the extra text once the user hovers over the data point. 

By following these best practices, you will ensure your text brings added value to your visuals instead of making them crowded and harder to read. 

11. Include Comparisons

This may be the briefest of our data visualization methods, but it’s important nonetheless: when you’re presenting your information and insights, you should include as many tangible comparisons as possible. By presenting two graphs, charts, and diagrams together, each showing contrasting versions of the same information over a particular timeframe, such as monthly sales records for 2016 and 2017 presented next to one another, you will provide a clear-cut guide on the impact of your data, highlighting strengths, weaknesses, trends, peaks, and troughs that everyone can ponder and act upon.

12. Tell Your Tale

Similar to content marketing, when you're presenting your data in a visual format with the aim of communicating an important message or goal, telling your story will engage your audience and make it easy for people to understand with minimal effort.

Scientific studies confirm that humans, at large, respond better to a well-told story, and by taking this approach to your visualization pursuits, you will not only dazzle your colleagues, partners, and clients with your reports and presentations, but you will increase your chances of conveying your most critical messages, getting the buy-in and response you need to make the kind of changes that will result in long-term growth, evolution, and success.

To do so, you should collate your information, thinking in terms of a writer, establishing a clear-cut beginning, middle, and end, as well as a conflict and resolution, building tension during your narrative to add maximum impact to your various visualizations.

13. Merge It All Together

Expanding on the point above, in order to achieve an efficient data storytelling process with the help of visuals, it is also necessary to merge it all together into one single location. In the past, this was done with the help of endless PowerPoint presentations or Excel sheets. However, this is no longer the case, thanks to modern dashboard technology. 

Dashboards are analytical tools that allow users to visualize their most important performance indicators all on one screen. This way, you avoid losing time by looking at static graphs that make the process tedious. Instead, you get the possibility to interact and navigate them to extract relevant conclusions in real time. Now, dashboard design has its own set of best practices that you can explore. However, they are still similar to the ones mentioned throughout this post. Let’s look at an example of a sales dashboard to put all of this into perspective. 

As seen in the image above, this sales dashboard provides a complete picture of the performance of the sales department. With a mix of metrics that show current and historical data, users can take a look into the past to understand certain trends and patterns and build an efficient story to support their strategic decisions. 

14. Make It Interactive 

Even though graphs, charts, infographics, and other types of visuals have been a part of our world for decades now, the use of data has always been exclusively reserved for people with knowledge of the subject, leaving non-technical users behind. Luckily, this thought has changed over the years, as experts have realized the great affinity that humans have with visuals. This has created a shift in the DataViz industry, where designers have begun to prioritize aesthetics and design as a way to convey information in an understandable way. Part of this change has been to introduce interactivity as a key element in their graphics. Making it one of the biggest advantages of data visualization today. 

In short, interactive elements help businesses and users bring their visuals to life by giving them the power to explore and navigate the data and extract powerful insights from it. Tools such as datapine provide multiple interactivity features that are easy to implement and use. For instance, a drill-down filter enables users to dig into lower levels of hierarchical data without having to jump to another chart. This is valuable in a number of opportunities; for example, when looking at sales by country, you can use a drill down filter to click on a specific country, and the whole chart will change to show sales by city of that country. 

Another valuable interactivity option is the time interval widget. This feature allows you to add specific buttons to your charts that enable you to change the period of the data being displayed. For example, if you are looking at a bar chart showing sales by month and realize that a particular month is lower than expected, the time interval widget will allow you to dig deeper into that particular month by looking at the weekly or even daily performance. 

To learn more about the topic of interactivity, check out our guide on the top interactive dashboard features. 

15. Consider The End Device

As we almost reached the end of our list of insightful data visualization methods, we couldn’t leave a fundamental point behind. We live in a fast-paced world where decisions need to be made on the go. In fact, according to Statista, 56,89% of the global online traffic corresponds to mobile internet traffic. With that in mind, it is fundamental to consider device versatility when it comes to building your visuals and ensuring an excellent user experience.   

We already mentioned the importance of merging all your visuals together into one intuitive business dashboard to tell a complete story. When it comes to generating visuals for mobile, the same principles apply. Considering that these screens are smaller than desktops, you should make sure only to include the graphs and charts that will help you convey the message you want to portray. You should also consider the size of labels and buttons, as they can be harder to see on a smaller device. Once you have managed all these points, you need to test on different devices to ensure that everything runs smoothly.  

16. Apply Visualization Tools For The Digital Age

We live in a fast-paced, hyper-connected digital age that is far removed from the pen and paper or even copy and paste mentality of the yesteryears - and as such, to make a roaring visualization success, you should use the digital tools that will help you make the best possible decisions while gathering your data in the most efficient, effective way.

A task-specific, interactive online dashboard or tool offers a digestible, intuitive, comprehensive, and interactive means of collecting, collating, arranging, and presenting data with ease - ensuring that your techniques have the most possible impact while taking up a minimal amount of your time.

17. Never Stop Learning

As you’ve learned throughout this list of 17 techniques of data visualization, building graphics is a process that requires a lot of skills and thoughtful consideration. While following these best practices should help you build successful visuals for multiple purposes, this is a process that requires practice and consistency. For that reason, our last piece of advice is never to stop learning. After your visuals are generated, gather feedback from your audience and rethink your process to make it a bit better on every ocasion. As the old saying goes, practice makes perfect. So don’t be afraid to look at your work with a critical eye. 

Summary & Next Steps 

As seen throughout this guide, data visualizations allow users and businesses to make large volumes of relevant data more accessible and understandable. With markets becoming more competitive by the day, the need to leverage the power of data analytics becomes an obligation instead of a choice, and companies that understand that will have a huge competitive advantage. 

We hope these data visualization concepts served to help propel your efforts to new successful heights. To enhance your ongoing activities, explore our cutting-edge business intelligence and online data visualization tool.

To summarize our detailed article, here is an overview of the best data visualization techniques:

• Know your audience
• Set your goals
• Choose the right chart type
• Be careful not to mislead
• Take advantage of color theory
• Prioritize simplicity
• Handle your big data
• Use ordering, layout, and hierarchy to prioritize
• Utilize word clouds and network diagrams
• Use text carefully
• Include comparisons
• Tell your tale
• Merge it all together
• Make it interactive
• Consider the end device
• Apply visualization tools for the digital age
• Never stop learning

To get a more in-depth insight into what visualization techniques can do for you, try our 14-day trial completely free!

All Courses

• Interview Questions
• Free Courses
• Career Guide
• PGP in Data Science and Business Analytics
• PG Program in Data Science and Business Analytics Classroom
• PGP in Data Science and Engineering (Data Science Specialization)
• PGP in Data Science and Engineering (Bootcamp)
• PGP in Data Science & Engineering (Data Engineering Specialization)
• Master of Data Science (Global) – Deakin University
• MIT Data Science and Machine Learning Course Online
• Master’s (MS) in Data Science Online Degree Programme
• MTech in Data Science & Machine Learning by PES University
• Data Analytics Essentials by UT Austin
• Data Science & Business Analytics Program by McCombs School of Business
• MTech In Big Data Analytics by SRM
• M.Tech in Data Engineering Specialization by SRM University
• M.Tech in Big Data Analytics by SRM University
• PG in AI & Machine Learning Course
• Weekend Classroom PG Program For AI & ML
• AI for Leaders & Managers (PG Certificate Course)
• Artificial Intelligence Course for School Students
• IIIT Delhi: PG Diploma in Artificial Intelligence
• Machine Learning PG Program
• MIT No-Code AI and Machine Learning Course
• Study Abroad: Masters Programs
• MS in Information Science: Machine Learning From University of Arizon
• SRM M Tech in AI and ML for Working Professionals Program
• UT Austin Artificial Intelligence (AI) for Leaders & Managers
• UT Austin Artificial Intelligence and Machine Learning Program Online
• MS in Machine Learning
• IIT Roorkee Full Stack Developer Course
• IIT Madras Blockchain Course (Online Software Engineering)
• IIIT Hyderabad Software Engg for Data Science Course (Comprehensive)
• IIIT Hyderabad Software Engg for Data Science Course (Accelerated)
• IIT Bombay UX Design Course – Online PG Certificate Program
• Online MCA Degree Course by JAIN (Deemed-to-be University)
• Cybersecurity PG Course
• Online Post Graduate Executive Management Program
• Product Management Course Online in India
• NUS Future Leadership Program for Business Managers and Leaders
• PES Executive MBA Degree Program for Working Professionals
• Online BBA Degree Course by JAIN (Deemed-to-be University)
• MBA in Digital Marketing or Data Science by JAIN (Deemed-to-be University)
• Master of Business Administration- Shiva Nadar University
• Post Graduate Diploma in Management (Online) by Great Lakes
• Online MBA Program by Shiv Nadar University
• Cloud Computing PG Program by Great Lakes
• University Programs
• Stanford Design Thinking Course Online
• Design Thinking : From Insights to Viability
• PGP In Strategic Digital Marketing
• Post Graduate Diploma in Management
• Master of Business Administration Degree Program
• MS in Business Analytics in USA
• MS in Machine Learning in USA
• Study MBA in Germany at FOM University
• M.Sc in Big Data & Business Analytics in Germany
• Study MBA in USA at Walsh College
• MS Data Analytics
• MS Artificial Intelligence and Machine Learning
• MS in Data Analytics
• Master of Business Administration (MBA)
• MS in Information Science: Machine Learning
• MS in Machine Learning Online
• MIT Data Science Program
• AI For Leaders Course
• Data Science and Business Analytics Course
• Cyber Security Course
• PG Program Online Artificial Intelligence Machine Learning
• PG Program Online Cloud Computing Course
• Data Analytics Essentials Online Course
• MIT Programa Ciencia De Dados Machine Learning
• MIT Programa Ciencia De Datos Aprendizaje Automatico
• Program PG Ciencia Datos Analitica Empresarial Curso Online
• Mit Programa Ciencia De Datos Aprendizaje Automatico
• Online Data Science Business Analytics Course
• Online Ai Machine Learning Course
• Online Full Stack Software Development Course
• Online Cloud Computing Course
• Cybersecurity Course Online
• Online Data Analytics Essentials Course
• Ai for Business Leaders Course
• Mit Data Science Program
• No Code Artificial Intelligence Machine Learning Program
• MS Information Science Machine Learning University Arizona
• Wharton Online Advanced Digital Marketing Program
• Data Science
• Introduction to Data Science
• Data Scientist Skills
• Get Into Data Science From Non IT Background
• Data Scientist Salary
• Data Science Job Roles
• Data Science Resume
• Data Scientist Interview Questions
• Data Science Solving Real Business Problems
• Business Analyst Vs Data Scientis
• Data Science Applications
• Must Watch Data Science Movies
• Data Science Projects
• Free Datasets for Analytics
• Data Analytics Project Ideas
• Mean Square Error Explained
• Hypothesis Testing in R
• Understanding Distributions in Statistics
• Bernoulli Distribution
• Inferential Statistics
• Analysis of Variance (ANOVA)
• Sampling Techniques
• Outlier Analysis Explained
• Outlier Detection
• Data Science with K-Means Clustering
• Support Vector Regression
• Multivariate Analysis
• What is Regression?
• An Introduction to R – Square
• Why is Time Complexity essential?
• Gaussian Mixture Model
• Genetic Algorithm
• Business Analytics
• What is Business Analytics?
• Business Analytics Career
• Major Misconceptions About a Career in Business Analytics
• Business Analytics and Business Intelligence Possible Career Paths for Analytics Professionals
• Business Analytics Companies
• Business Analytics Tools
• Business Analytics Jobs
• Business Analytics Course
• Difference Between Business Intelligence and Business Analytics
• Python Tutorial for Beginners
• Python Cheat Sheet
• Career in Python
• Python Developer Salary
• Python Interview Questions
• Python Project for Beginners
• Python Books
• Python Real World Examples
• Python 2 Vs. Python 3
• Free Online Courses for Python
• Flask Vs. Django
• Python Stack
• Python Switch Case
• Python Main
• Data Types in Python
• Mutable & Immutable in Python
• Python Dictionary
• Python Queue
• Iterator in Python
• Regular Expression in Python
• Eval in Python
• Classes & Objects in Python
• OOPs Concepts in Python
• Inheritance in Python
• Abstraction in Python
• Polymorphism in Python
• Fibonacci Series in Python
• Factorial Program in Python
• Armstrong Number in Python
• Reverse a String in Python
• Prime Numbers in Python
• Pattern Program in Python
• Palindrome in Python
• Convert List to String in Python
• Append Function in Python
• REST API in Python
• Python Web Scraping using BeautifulSoup
• Scrapy Tutorial
• Web Scraping using Python
• Jupyter Notebook
• Spyder Python IDE
• Free Data Science Course
• Free Data Science Courses
• Data Visualization Courses
• Data Science Course In Hyderabad
• Data Science Course In Chennai
• Data Science Course In Pune
• Data Science Course In Bangalore
• Data Science Course In Mumbai
• Data Science Course In Dehli NCR

Understanding Data Visualization Techniques

• Benefits of good data visualization
• Data Visualization Techniques
• List of Methods to Visualize Data
• Five Number Summary of Box Plot
• Histograms are based on area, not height of bars
• Histogram Vs Bar Chart
• Word Clouds and Network Diagrams for Unstructured Data
• FAQs Related to Data Visualization

Data visualization  is a graphical representation of information and data. By using visual elements like charts, graphs, and maps, data visualization tools provide an accessible way to see and understand trends, outliers, and patterns in data. This blog on data visualization techniques will help you understand detailed techniques and benefits.

In the world of Big Data, data visualization in Python tools and technologies are essential to analyze massive amounts of information and make data-driven decisions. 

Contributed by: Dinesh

Our eyes are  drawn to colours and patterns . We can quickly identify red from blue, and square from the circle. Our culture is visual, including everything from art and advertisements to TV and movies.

Data visualization is another form of visual art that grabs our interest and keeps our eyes on the message. When we see a chart, we  quickly see trends and outliers . If we can see something, we internalize it quickly. It’s storytelling with a purpose. If you’ve ever stared at a massive spreadsheet of data and couldn’t see a trend, you know how much more effective a visualization can be. The uses of Data Visualization as follows.

• Powerful way to explore data with presentable results.
• Primary use is the pre-processing portion of the data mining process.
• Supports the data cleaning process by finding incorrect and missing values.
• For variable derivation and selection means to determine which variable to include and discarded in the analysis.
• Also play a role in combining categories as part of the data reduction process.
• Word Cloud/Network diagram

E nrol Now – Data Visualization Using Tableau course for free offered by Great Learning Academy .

The image above is a box plot .  A boxplot is a standardized way of displaying the distribution of data based on a five-number summary (“minimum”, first quartile (Q1), median, third quartile (Q3), and “maximum”). It can tell you about your outliers and what their values are. It can also tell you if your data is symmetrical, how tightly your data is grouped, and if and how your data is skewed.

A box plot is a graph that gives you a good indication of how the values in the data are spread out. Although box plots may seem primitive in comparison to a histogram or density plot, they have the advantage of taking up less space, which is useful when comparing distributions between many groups or datasets. For some distributions/datasets, you will find that you need more information than the measures of central tendency (median, mean, and mode). You need to have information on the variability or dispersion of the data.

• Column Chart: It is also called a vertical bar chart where each category is represented by a rectangle. The height of the rectangle is proportional to the values that are plotted.
• Bar Graph: It has rectangular bars in which the lengths are proportional to the values which are represented.
• Stacked Bar Graph: It is a bar style graph that has various components stacked together so that apart from the bar, the components can also be compared to each other.
• Stacked Column Chart: It is similar to a stacked bar; however, the data is stacked horizontally.
• Area Chart: It combines the line chart and bar chart to show how the numeric values of one or more groups change over the progress of a viable area.
• Dual Axis Chart: It combines a column chart and a line chart and then compares the two variables.
• Line Graph: The data points are connected through a straight line; therefore, creating a representation of the changing trend.
• Mekko Chart: It can be called a two-dimensional stacked chart with varying column widths.
• Pie Chart: It is a chart where various components of a data set are presented in the form of a pie which represents their proportion in the entire data set.
• Waterfall Chart: With the help of this chart, the increasing effect of sequentially introduced positive or negative values can be understood.
• Bubble Chart: It is a multi-variable graph that is a hybrid of Scatter Plot and a Proportional Area Chart.
• Scatter Plot Chart: It is also called a scatter chart or scatter graph. Dots are used to denote values for two different numeric variables.
• Bullet Graph: It is a variation of a bar graph. A bullet graph is used to swap dashboard gauges and meters.
• Funnel Chart: The chart determines the flow of users with the help of a business or sales process.
• Heat Map: It is a technique of data visualization that shows the level of instances as color in two dimensions.

A histogram is a graphical display of data using bars of different heights. In a histogram, each bar groups numbers into ranges. Taller bars show that more data falls in that range. A histogram displays the shape and spread of continuous sample data. 

It is a plot that lets you discover, and show, the underlying frequency distribution (shape) of a set of continuous data. This allows the inspection of the data for its underlying distribution (e.g., normal distribution), outliers, skewness, etc. It is an accurate representation of the distribution of numerical data, it relates only one variable. Includes bin or bucket- the range of values that divide the entire range of values into a series of intervals and then count how many values fall into each interval.

Bins are consecutive, non- overlapping intervals of a variable. As the adjacent bins leave no gaps, the rectangles of histogram touch each other to indicate that the original value is continuous.

In a histogram, the height of the bar does not necessarily indicate how many occurrences of scores there were within each bin. It is the product of height multiplied by the width of the bin that indicates the frequency of occurrences within that bin. One of the reasons that the height of the bars is often incorrectly assessed as indicating the frequency and not the area of the bar is because a lot of histograms often have equally spaced bars (bins), and under these circumstances, the height of the bin does reflect the frequency.

Also Read: Machine Learning Interview Questions

The major difference is that a histogram is only used to plot the frequency of score occurrences in a continuous data set that has been divided into classes, called bins. Bar charts, on the other hand, can be used for a lot of other types of variables including ordinal and nominal data sets.

A heat map is data analysis software that uses colour the way a bar graph uses height and width: as a data visualization tool. If you’re looking at a web page and you want to know which areas get the most attention, a heat map shows you in a visual way that’s easy to assimilate and make decisions from. It is a graphical representation of data where the individual values contained in a matrix are represented as colours. Useful for two purposes: for visualizing correlation tables and for visualizing missing values in the data. In both cases, the information is conveyed in a two-dimensional table. Note that heat maps are useful when examining a large number of values, but they are not a replacement for more precise graphical displays, such as bar charts, because colour differences cannot be perceived accurately.

Also Read: Top Data Mining Tools

The simplest technique, a line plot is used to plot the relationship or dependence of one variable on another. To plot the relationship between the two variables, we can simply call the plot function.

Bar charts are used for comparing the quantities of different categories or groups. Values of a category are represented with the help of bars and they can be configured with vertical or horizontal bars, with the length or height of each bar representing the value.

It is a circular statistical graph which decides slices to illustrate numerical proportion. Here the arc length of each slide is proportional to the quantity it represents. As a rule, they are used to compare the parts of a whole and are most effective when there are limited components and when text and percentages are included to describe the content. However, they can be difficult to interpret because the human eye has a hard time estimating areas and comparing visual angles.

Scatter Charts

Another common visualization technique is a scatter plot that is a two-dimensional plot representing the joint variation of two data items. Each marker (symbols such as dots, squares and plus signs) represents an observation. The marker position indicates the value for each observation. When you assign more than two measures, a scatter plot matrix is produced that is a series scatter plot displaying every possible pairing of the measures that are assigned to the visualization. Scatter plots are used for examining the relationship, or correlations, between X and Y variables.

Bubble Charts

It is a variation of scatter chart in which the data points are replaced with bubbles, and an additional dimension of data is represented in the size of the bubbles.

Timeline Charts

Timeline charts illustrate events, in chronological order — for example the progress of a project, advertising campaign, acquisition process — in whatever unit of time the data was recorded — for example week, month, year, quarter. It shows the chronological sequence of past or future events on a timescale.

A treemap is a visualization that displays hierarchically organized data as a set of nested rectangles, parent elements being tiled with their child elements. The sizes and colours of rectangles are proportional to the values of the data points they represent. A leaf node rectangle has an area proportional to the specified dimension of the data. Depending on the choice, the leaf node is coloured, sized or both according to chosen attributes. They make efficient use of space, thus display thousands of items on the screen simultaneously.

The variety of big data brings challenges because semi-structured, and unstructured data require new visualization techniques. A word cloud visual represents the frequency of a word within a body of text with its relative size in the cloud. This technique is used on unstructured data as a way to display high- or low-frequency words.

Another visualization technique that can be used for semi-structured or unstructured data is the network diagram. Network diagrams represent relationships as nodes (individual actors within the network) and ties (relationships between the individuals). They are used in many applications, for example for analysis of social networks or mapping product sales across geographic areas.

Learn all about Data Visualization with Power BI with this free course.

• What are the techniques of Visualization?

A : The visualization techniques include Pie and Donut Charts, Histogram Plot, Scatter Plot, Kernel Density Estimation for Non-Parametric Data, Box and Whisker Plot for Large Data, Word Clouds and Network Diagrams for Unstructured Data, and Correlation Matrices.

• What are the types of visualization?

A : The various types of visualization include Column Chart, Line Graph, Bar Graph, Stacked Bar Graph, Dual-Axis Chart, Pie Chart, Mekko Chart, Bubble Chart, Scatter Chart, and Bullet Graph.

• What are the various visualization techniques used in data analysis?

A: Various visualization techniques are used in data analysis. A few of them include Box and Whisker Plot for Large Data, Histogram Plot, and Word Clouds and Network Diagrams for Unstructured Data, to name a few.

• How do I start visualizing?

A: You need to have a basic understanding of data and present it without misleading the data. Once you understand it, you can further take up an online course or tutorials.

• What are the two basic types of data visualization?

A: The two very basic types of data visualization are exploration and explanation.

• Which is the best visualization tool?

A: Some of the best visualization tools include Visme, Tableau, Infogram, Whatagraph, Sisense, DataBox, ChartBlocks, DataWrapper, etc.

These are some of the Visualization techniques used to represent data effectively for their better understanding and interpretation. We hope this article was useful. You can also upskill with our free courses on Great Learning Academy .

Top Free Courses

Top 30 Python Libraries To Know in 2024

Python dictionary append: How to Add Key/Value Pair?

¿Qué es la Ciencia de Datos? – Una Guía Completa [2024]

What is Data Science? – The Complete Guide

What is Time Complexity And Why Is It Essential?

Python NumPy Tutorial – 2024

Leave a comment cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Table of contents

tableau.com is not available in your region.

• Warning : Invalid argument supplied for foreach() in /home/customer/www/opendatascience.com/public_html/wp-includes/nav-menu.php on line 95 Warning : array_merge(): Expected parameter 2 to be an array, null given in /home/customer/www/opendatascience.com/public_html/wp-includes/nav-menu.php on line 102
• ODSC EUROPE
• AI+ Training
• Speak at ODSC

• Data Engineering
• Data Visualization
• Deep Learning
• Generative AI
• Machine Learning
• NLP and LLMs
• Business & Use Cases
• Career Advice
• Write for us
• ODSC Community Slack Channel
• Upcoming Webinars

Mastering the Art of Data Visualization: Tips & Techniques

Data Visualization Modeling posted by ODSC Community October 5, 2023 ODSC Community

In the digital era, data visualization stands as an indispensable tool in the realm of business intelligence . It represents the graphical display of data and information, transforming complex datasets into intuitive and understandable visuals.

By implementing data visualization, businesses can reap multifaceted benefits:

• Simplified Data Interpretation : Complex data are converted into easily comprehensible visuals, enabling quick interpretation and understanding
• Enhanced Decision-making Process : Visual data representation aids in identifying patterns, trends, and outliers, fostering informed decision-making
• Improved Information Retention : Visuals increase information retention and recall, promoting long-term strategic planning

Remember that effective data visualization acts not only as a summary of your data but also as a guide towards informed decisions. By harnessing the power of visual grammar rules and frameworks like the McCandless Method or Kaiser Fung’s Junk Charts Trifecta Checkup, you can elevate your business intelligence strategy to new heights.

Key Principles and Best Practices for Data Visualization

Understanding the key principles and best practices for data visualization can significantly improve your ability to present data in a meaningful, digestible way.

Visual Grammar Rules

Visual grammar rules are fundamental to creating effective visual presentations. These include clarity, simplicity, and emphasis on the information rather than the graphic design itself.

• Clarity: The purpose of your visualization should be immediately clear to your audience. Avoid unnecessary complexity in charts, graphs, and diagrams.
• Simplicity: Keep designs as simple as possible while still conveying the necessary information. Extraneous elements can distract from the data you’re presenting.
• Emphasis on Information: The main focus should always be on the data, not on the aesthetics of the presentation. Avoid using overly flashy designs or effects that could detract from your message.

Consider these points when designing your own visualizations.

Consider a bar chart showing annual sales figures for your company’s product range. Making each bar a different color might make the chart look more appealing, but it can confuse the audience if there’s no clear reason for the color variation. Instead, use one color for all bars and differentiate them by labeling each one with the product name and sales figure.

This approach applies clarity , simplicity , and an emphasis on information , demonstrating how these visual grammar rules enhance data visualization best practices.

Incorporate these design principles into your own work to ensure your visualizations communicate effectively, keeping your audience engaged and informed without overwhelming them with unnecessary detail or confusing graphics.

Moving forward, let’s delve deeper into organizing visualization through popular frameworks that help structure data effectively. Using such frameworks will enable you to deliver insights more coherently, making it easier for your audience to understand complex datasets.

Frameworks for Organizing Visualization

Organizing data visualization is an art as much as it is a science. It’s where design principles meet visual grammar rules, creating effective and engaging visuals. Two popular frameworks that encapsulate data visualization best practices are the McCandless Method and Kaiser Fung’s Junk Charts Trifecta Checkup .

The McCandless Method, coined by David McCandless, advocates for a balance between information and design. This involves considering aspects like:

• Ensuring data accuracy
• Emphasizing clarity and precision
• Incorporating a meaningful color palette
• Utilizing pre-attentive attributes effectively to guide viewer attention

Meanwhile, Kaiser Fung’s Junk Charts Trifecta Checkup takes a more analytical approach to visualization. It encourages us to ask three key questions:

• What does the chart show?
• What does the data say?
• What relevant factor is missing?

By following these frameworks, you can create striking visuals that not only look good but also communicate your message effectively.

Effective Techniques for Data Visualization

Let’s dive into the world of Power BI to explore its features and learn some useful data visualization tips and tricks for data visualization. Power BI, a business analytics tool developed by Microsoft, offers interactive visualizations with self-service business intelligence capabilities.

Using Power BI Tips and Tricks

To begin with, Power BI stands out in the crowd due to its ability to produce beautiful reports with interactive visualizations. It allows sharing these reports directly within the platform or embedding them in an app or website. The tool’s drag-and-drop feature simplifies creating complex dashboards, making it user-friendly even for beginners.

One of the most potent features of Power BI is Quick Insights . This function finds patterns, trends, and correlations in the data automatically. To use it effectively:

• Select a dataset
• Click ‘Get Insights’
• Wait for Power BI to do its magic!

Another valuable feature is Natural Language Querying . With this feature, you can type in questions about your data in natural language and get immediate answers. For example, if you have an e-commerce business and want to know your best-selling product last month, just type “What was my top-selling product last month?” into the query box.

The Q&A Visual takes this one step further by allowing users to ask questions directly on the report page and presenting answers in a visual format. To harness this feature:

• Drag the Q&A button onto your report page
• Start asking questions!

Don’t forget about Bookmarking . This tool allows you to save a customized view of a report (filters and slicers) and return to it at any point. This is particularly useful when dealing with large datasets.

Lastly, consider using Data Drill Down for hierarchical data visualization. This technique helps users navigate from general overviews down to specific details in just a few clicks.

In the realm of data visualization, Power BI is a game-changer with its advanced features and user-friendly interface. Harness these data visualization tips and techniques to create compelling visualizations and gain valuable insights from your data. Up next, we’ll delve into more intriguing aspects of data visualization: keyboard shortcuts and custom themes. Stay tuned!

Utilizing Keyboard Shortcuts and Custom Themes

Efficiency in data visualization is crucial. One way to boost your productivity is through keyboard shortcuts . They provide a fast, seamless way to navigate your workspace, execute commands, and manipulate data. For instance, in Power BI, you can use Ctrl + M to start a new measure or Alt + Shift + F10 to access the context menu.

Complementing the use of shortcuts, custom themes are essential for enhancing visual appeal. They not only add color and style but also consistency across your visuals. You can create custom themes within Power BI by going to View > Themes > Customize current theme . From there, you can tweak colors, text properties, and visual elements to match your brand or preference.

Remember: The right combination of keyboard efficiency and design aesthetics elevates the impact of your data visualization techniques.

Data Modeling and Drill-Through Techniques

Data modeling plays a crucial role in data visualization. It’s the process of creating a visual representation of data, which can help to understand complex patterns and relationships. Using data modeling effectively allows you to uncover insights that would be difficult to grasp in raw, unprocessed data.

To illustrate, it’s like taking a jumbled pile of puzzle pieces and organizing them into an understandable image. When you organize your data in this way, it becomes easier for everyone to understand.

One of the most powerful techniques in data modeling is drill-through. This technique allows users to navigate from a summary view into detailed data. For instance, if you’re viewing sales data by region, a drill-through could allow you to click on one region and see the individual sales by city or even by store.

Here are some tips for implementing drill-through techniques:

• Plan Ahead : Define what detailed information would be useful before setting up your drill-throughs.
• Limit Your Layers : Too many layers can confuse users. Stick to a few key details.
• Use Clear Labels : Make sure it’s obvious what each layer represents.

These techniques, when used correctly, can dramatically enhance your ability to communicate complex information through your visualizations.

Real-Time Dashboards and Explaining Data

Access to real-time dashboards in data visualization provides a game-changing advantage. These dynamic tools compile and display data as it enters the system, allowing for immediate analysis and action. Benefits of utilizing real-time dashboards include:

• Keeping stakeholders informed with up-to-the-minute data
• Enabling rapid response to emerging trends or issues
• Facilitating ongoing optimization of strategies based on live data

To extract maximum value from your real-time dashboards, it’s essential to adequately explain the data they present. Densely packed data or complex graphs can be overwhelming without clear, concise explanations. Here are some techniques to effectively communicate complex data:

• Simplicity : Distil complex ideas into simple, understandable terms. Avoid jargon where possible.
• Context : Provide relevant background information to help readers understand why the data matters.
• Visual Aids : Use charts, graphs, and infographics to represent data visually, making it easier to digest.
• Narrative : Weave a story around the data to make it more engaging and relatable.

By employing these techniques, you can ensure that your audience not only sees the numbers but also comprehends their implications. Remember that balancing real-time insight with effective explanations can greatly improve your decision-making process in any online business venture.

Through this article, we’ve unlocked the potential of data visualization , from understanding its benefits to adopting best practices. We’ve dived into Visual Grammar Rules and explored frameworks like the McCandless Method and Kaiser Fung’s Junk Charts Trifecta Checkup . Power BI has been our companion, guiding us through data visualization tips, tricks, and techniques such as keyboard shortcuts, custom themes, data modeling, and drill-through techniques.

We encourage you to leverage these insights in your quest for effective data visualization. Remember, the key is to keep experimenting and learning. Let’s transform complex data into understandable visuals together. After all, a picture is worth a thousand words.

About the author on data visualization tips:

ODSC Community

The Open Data Science community is passionate and diverse, and we always welcome contributions from data science professionals! All of the articles under this profile are from our community, with individual authors mentioned in the text itself.

OpenAI Has Eyes For Hollywood Thanks to Sora

AI and Data Science News posted by ODSC Team Mar 25, 2024 In an unsurprising move, OpenAI is setting the stage for an entry into the movie business...

Is Synthetic Data a Reliable Option for Training Machine Learning Models?

Machine Learning posted by Zac Amos Mar 25, 2024 Getting enough of the right data is one of the most persistent challenges in machine learning....

How to Organize and Motivate a Biotech Data Science Team

East 2024 Business + Management posted by ODSC Community Mar 25, 2024 Editor’s note: Eric Ma, PhD is a speaker for ODSC East this April 23-25. Be sure...

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

2.1: Types of Data Representation

• Last updated
• Save as PDF
• Page ID 5696

Two common types of graphic displays are bar charts and histograms. Both bar charts and histograms use vertical or horizontal bars to represent the number of data points in each category or interval. The main difference graphically is that in a  bar chart  there are spaces between the bars and in a  histogram  there are not spaces between the bars. Why does this subtle difference exist and what does it imply about graphic displays in general?

Displaying Data

It is often easier for people to interpret relative sizes of data when that data is displayed graphically. Note that a  categorical variable  is a variable that can take on one of a limited number of values and a  quantitative variable  is a variable that takes on numerical values that represent a measurable quantity. Examples of categorical variables are tv stations, the state someone lives in, and eye color while examples of quantitative variables are the height of students or the population of a city. There are a few common ways of displaying data graphically that you should be familiar with. 

A  pie chart  shows the relative proportions of data in different categories.  Pie charts  are excellent ways of displaying categorical data with easily separable groups. The following pie chart shows six categories labeled A−F.  The size of each pie slice is determined by the central angle. Since there are 360 o  in a circle, the size of the central angle θ A  of category A can be found by:

CK-12 Foundation -  https://www.flickr.com/photos/slgc/16173880801  - CCSA

A  bar chart  displays frequencies of categories of data. The bar chart below has 5 categories, and shows the TV channel preferences for 53 adults. The horizontal axis could have also been labeled News, Sports, Local News, Comedy, Action Movies. The reason why the bars are separated by spaces is to emphasize the fact that they are categories and not continuous numbers. For example, just because you split your time between channel 8 and channel 44 does not mean on average you watch channel 26. Categories can be numbers so you need to be very careful.

CK-12 Foundation -  https://www.flickr.com/photos/slgc/16173880801  - CCSA

A  histogram  displays frequencies of quantitative data that has been sorted into intervals. The following is a histogram that shows the heights of a class of 53 students. Notice the largest category is 56-60 inches with 18 people.

A  boxplot  (also known as a  box and whiskers plot ) is another way to display quantitative data. It displays the five 5 number summary (minimum, Q1,  median , Q3, maximum). The box can either be vertically or horizontally displayed depending on the labeling of the axis. The box does not need to be perfectly symmetrical because it represents data that might not be perfectly symmetrical.

Earlier, you were asked about the difference between histograms and bar charts. The reason for the space in bar charts but no space in histograms is bar charts graph categorical variables while histograms graph quantitative variables. It would be extremely improper to forget the space with bar charts because you would run the risk of implying a spectrum from one side of the chart to the other. Note that in the bar chart where TV stations where shown, the station numbers were not listed horizontally in order by size. This was to emphasize the fact that the stations were categories.

Create a boxplot of the following numbers in your calculator.

8.5, 10.9, 9.1, 7.5, 7.2, 6, 2.3, 5.5

Enter the data into L1 by going into the Stat menu.

CK-12 Foundation - CCSA

Then turn the statplot on and choose boxplot.

Use Zoomstat to automatically center the window on the boxplot.

Create a pie chart to represent the preferences of 43 hungry students.

• Other – 5
• Burritos – 7
• Burgers – 9
• Pizza – 22

Create a bar chart representing the preference for sports of a group of 23 people.

• Football – 12
• Baseball – 10
• Basketball – 8
• Hockey – 3

Create a histogram for the income distribution of 200 million people.

• Below $50,000 is 100 million people
• Between $50,000 and $100,000 is 50 million people
• Between $100,000 and $150,000 is 40 million people
• Above $150,000 is 10 million people

1. What types of graphs show categorical data?

2. What types of graphs show quantitative data?

A math class of 30 students had the following grades:

3. Create a bar chart for this data.

4. Create a pie chart for this data.

5. Which graph do you think makes a better visual representation of the data?

A set of 20 exam scores is 67, 94, 88, 76, 85, 93, 55, 87, 80, 81, 80, 61, 90, 84, 75, 93, 75, 68, 100, 98

6. Create a histogram for this data. Use your best judgment to decide what the intervals should be.

7. Find the  five number summary  for this data.

8. Use the  five number summary  to create a boxplot for this data.

9. Describe the data shown in the boxplot below.

10. Describe the data shown in the histogram below.

A math class of 30 students has the following eye colors:

11. Create a bar chart for this data.

12. Create a pie chart for this data.

13. Which graph do you think makes a better visual representation of the data?

14. Suppose you have data that shows the breakdown of registered republicans by state. What types of graphs could you use to display this data?

15. From which types of graphs could you obtain information about the spread of the data? Note that spread is a measure of how spread out all of the data is.

Review (Answers)

To see the Review answers, open this  PDF file  and look for section 15.4. 

Additional Resources

PLIX: Play, Learn, Interact, eXplore - Baby Due Date Histogram

Practice: Types of Data Representation

Real World: Prepare for Impact

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Praxis Core Math

Course: praxis core math   >   unit 1, data representations | lesson.

• Data representations | Worked example
• Center and spread | Lesson
• Center and spread | Worked example
• Random sampling | Lesson
• Random sampling | Worked example
• Scatterplots | Lesson
• Scatterplots | Worked example
• Interpreting linear models | Lesson
• Interpreting linear models | Worked example
• Correlation and Causation | Lesson
• Correlation and causation | Worked example
• Probability | Lesson
• Probability | Worked example

What are data representations?

• How much of the data falls within a specified category or range of values?
• What is a typical value of the data?
• How much spread is in the data?
• Is there a trend in the data over time?
• Is there a relationship between two variables?

What skills are tested?

• Matching a data set to its graphical representation
• Matching a graphical representation to a description
• Using data representations to solve problems

How are qualitative data displayed?

• A vertical bar chart lists the categories of the qualitative variable along a horizontal axis and uses the heights of the bars on the vertical axis to show the values of the quantitative variable. A horizontal bar chart lists the categories along the vertical axis and uses the lengths of the bars on the horizontal axis to show the values of the quantitative variable. This display draws attention to how the categories rank according to the amount of data within each. Example The heights of the bars show the number of students who want to study each language. Using the bar chart, we can conclude that the greatest number of students want to study Mandarin and the least number of students want to study Latin.
• A pictograph is like a horizontal bar chart but uses pictures instead of the lengths of bars to represent the values of the quantitative variable. Each picture represents a certain quantity, and each category can have multiple pictures. Pictographs are visually interesting, but require us to use the legend to convert the number of pictures to quantitative values. Example Each represents 40 ‍   students. The number of pictures shows the number of students who want to study each language. Using the pictograph, we can conclude that twice as many students want to study French as want to study Latin.
• A circle graph (or pie chart) is a circle that is divided into as many sections as there are categories of the qualitative variable. The area of each section represents, for each category, the value of the quantitative data as a fraction of the sum of values. The fractions sum to 1 ‍   . Sometimes the section labels include both the category and the associated value or percent value for that category. Example The area of each section represents the fraction of students who want to study that language. Using the circle graph, we can conclude that just under 1 2 ‍   the students want to study Mandarin and about 1 3 ‍   want to study Spanish.

How are quantitative data displayed?

• Dotplots use one dot for each data point. The dots are plotted above their corresponding values on a number line. The number of dots above each specific value represents the count of that value. Dotplots show the value of each data point and are practical for small data sets. Example Each dot represents the typical travel time to school for one student. Using the dotplot, we can conclude that the most common travel time is 10 ‍   minutes. We can also see that the values for travel time range from 5 ‍   to 35 ‍   minutes.
• Histograms divide the horizontal axis into equal-sized intervals and use the heights of the bars to show the count or percent of data within each interval. By convention, each interval includes the lower boundary but not the upper one. Histograms show only totals for the intervals, not specific data points. Example The height of each bar represents the number of students having a typical travel time within the corresponding interval. Using the histogram, we can conclude that the most common travel time is between 10 ‍   and 15 ‍   minutes and that all typical travel times are between 5 ‍   and 40 ‍   minutes.

How are trends over time displayed?

How are relationships between variables displayed.

• (Choice A)   A
• (Choice B)   B
• (Choice C)   C
• (Choice D)   D
• (Choice E)   E
• Your answer should be
• an integer, like 6 ‍  
• a simplified proper fraction, like 3 / 5 ‍  
• a simplified improper fraction, like 7 / 4 ‍  
• a mixed number, like 1   3 / 4 ‍  
• an exact decimal, like 0.75 ‍  
• a multiple of pi, like 12   pi ‍   or 2 / 3   pi ‍  
• a proper fraction, like 1 / 2 ‍   or 6 / 10 ‍  
• an improper fraction, like 10 / 7 ‍   or 14 / 8 ‍  

Things to remember

• When matching data to a representation, check that the values are graphed accurately for all categories.
• When reporting data counts or fractions, be clear whether a question asks about data within a single category or a comparison between categories.
• When finding the number or fraction of the data meeting a criteria, watch for key words such as or , and , less than , and more than .

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

Home Blog Design Understanding Data Presentations (Guide + Examples)

Understanding Data Presentations (Guide + Examples)

In this age of overwhelming information, the skill to effectively convey data has become extremely valuable. Initiating a discussion on data presentation types involves thoughtful consideration of the nature of your data and the message you aim to convey. Different types of visualizations serve distinct purposes. Whether you’re dealing with how to develop a report or simply trying to communicate complex information, how you present data influences how well your audience understands and engages with it. This extensive guide leads you through the different ways of data presentation.

Table of Contents

What is a Data Presentation?

What should a data presentation include, line graphs, treemap chart, scatter plot, how to choose a data presentation type, recommended data presentation templates, common mistakes done in data presentation.

We can label a presentation under the title of data presentation when the aim is to disclose quantitative information to an audience through the usage of visual formats and narrative techniques. The overall purpose of this kind of presentation is to simplify complex concepts, allowing the presenter to highlight trends, patterns, and insights with the core purpose of acting upon the shared information. This process requires a series of tools, such as charts, graphs, tables, infographics, dashboards, and so on, supported by concise textual explanations for better understanding and boosting retention rate.

Data presentations go beyond the mere usage of graphical elements. Seasoned presenters encompass visuals with the art of storytelling with data, so the speech skillfully connects the points through a narrative that resonates with the audience. Depending on the purpose – inspire, persuade, inform, support decision-making processes, etc. – is the data presentation format that is better suited to help us in this journey.

To nail your upcoming data presentation, ensure to count with the following elements:

• Clear Objectives: Understand the intent of your presentation before selecting the graphical layout and metaphors to make content easier to grasp.
• Engaging introduction: Use a powerful hook from the get-go. For instance, you can ask a big question or present a problem that your data will answer. Take a look at our guide on how to start a presentation for tips & insights.
• Structured Narrative: Your data presentation must tell a coherent story. This means a beginning where you present the context, a middle section in which you present the data, and an ending that uses a call-to-action. Check our guide on presentation structure for further information.
• Visual Elements: These are the charts, graphs, and other elements of visual communication we ought to use to present data. This article will cover one by one the different types of data representation methods we can use, and provide further guidance on choosing between them.
• Insights and Analysis: This is not just showcasing a graph and letting people get an idea about it. A proper data presentation includes the interpretation of that data, the reason why it’s included, and why it matters to your research.
• Conclusion & CTA: Ending your presentation with a call to action is necessary. Whether you intend to wow your audience into acquiring your services, inspire them to change the world, or whatever the purpose of your presentation, there must be a stage in which you convey all that you shared and show the path to staying in touch. Plan ahead whether you want to use a thank-you slide, a video presentation, or which method is apt and tailored to the kind of presentation you deliver.
• Q&A Session: After your speech is concluded, allocate 3-5 minutes for the audience to raise any questions about the information you disclosed. This is an extra chance to establish your authority on the topic. Check our guide on questions and answer sessions in presentations here.

Bar charts are a graphical representation of data using rectangular bars to show quantities or frequencies in an established category. They make it easy for readers to spot patterns or trends. Bar charts can be horizontal or vertical, although the vertical format is commonly known as a column chart. They display categorical, discrete, or continuous variables grouped in class intervals [1] . They include an axis and a set of labeled bars horizontally or vertically. These bars represent the frequencies of variable values or the values themselves. Numbers on the y-axis of a vertical bar chart or the x-axis of a horizontal bar chart are called the scale.

Real-Life Application of Bar Charts

Let’s say a sales manager is presenting sales to their audience. Using a bar chart, he follows these steps.

Step 1: Selecting Data

The first step is to identify the specific data you will present to your audience.

The sales manager has highlighted these products for the presentation.

• Product A: Men’s Shoes
• Product B: Women’s Apparel
• Product C: Electronics
• Product D: Home Decor

Step 2: Choosing Orientation

Opt for a vertical layout for simplicity. Vertical bar charts help compare different categories in case there are not too many categories [1] . They can also help show different trends. A vertical bar chart is used where each bar represents one of the four chosen products. After plotting the data, it is seen that the height of each bar directly represents the sales performance of the respective product.

It is visible that the tallest bar (Electronics – Product C) is showing the highest sales. However, the shorter bars (Women’s Apparel – Product B and Home Decor – Product D) need attention. It indicates areas that require further analysis or strategies for improvement.

Step 3: Colorful Insights

Different colors are used to differentiate each product. It is essential to show a color-coded chart where the audience can distinguish between products.

• Men’s Shoes (Product A): Yellow
• Women’s Apparel (Product B): Orange
• Electronics (Product C): Violet
• Home Decor (Product D): Blue

Bar charts are straightforward and easily understandable for presenting data. They are versatile when comparing products or any categorical data [2] . Bar charts adapt seamlessly to retail scenarios. Despite that, bar charts have a few shortcomings. They cannot illustrate data trends over time. Besides, overloading the chart with numerous products can lead to visual clutter, diminishing its effectiveness.

For more information, check our collection of bar chart templates for PowerPoint .

Line graphs help illustrate data trends, progressions, or fluctuations by connecting a series of data points called ‘markers’ with straight line segments. This provides a straightforward representation of how values change [5] . Their versatility makes them invaluable for scenarios requiring a visual understanding of continuous data. In addition, line graphs are also useful for comparing multiple datasets over the same timeline. Using multiple line graphs allows us to compare more than one data set. They simplify complex information so the audience can quickly grasp the ups and downs of values. From tracking stock prices to analyzing experimental results, you can use line graphs to show how data changes over a continuous timeline. They show trends with simplicity and clarity.

Real-life Application of Line Graphs

To understand line graphs thoroughly, we will use a real case. Imagine you’re a financial analyst presenting a tech company’s monthly sales for a licensed product over the past year. Investors want insights into sales behavior by month, how market trends may have influenced sales performance and reception to the new pricing strategy. To present data via a line graph, you will complete these steps.

First, you need to gather the data. In this case, your data will be the sales numbers. For example:

• January: $45,000
• February: $55,000
• March: $45,000
• April: $60,000
• May: $ 70,000
• June: $65,000
• July: $62,000
• August: $68,000
• September: $81,000
• October: $76,000
• November: $87,000
• December: $91,000

After choosing the data, the next step is to select the orientation. Like bar charts, you can use vertical or horizontal line graphs. However, we want to keep this simple, so we will keep the timeline (x-axis) horizontal while the sales numbers (y-axis) vertical.

Step 3: Connecting Trends

After adding the data to your preferred software, you will plot a line graph. In the graph, each month’s sales are represented by data points connected by a line.

Step 4: Adding Clarity with Color

If there are multiple lines, you can also add colors to highlight each one, making it easier to follow.

Line graphs excel at visually presenting trends over time. These presentation aids identify patterns, like upward or downward trends. However, too many data points can clutter the graph, making it harder to interpret. Line graphs work best with continuous data but are not suitable for categories.

For more information, check our collection of line chart templates for PowerPoint .

A data dashboard is a visual tool for analyzing information. Different graphs, charts, and tables are consolidated in a layout to showcase the information required to achieve one or more objectives. Dashboards help quickly see Key Performance Indicators (KPIs). You don’t make new visuals in the dashboard; instead, you use it to display visuals you’ve already made in worksheets [3] .

Keeping the number of visuals on a dashboard to three or four is recommended. Adding too many can make it hard to see the main points [4]. Dashboards can be used for business analytics to analyze sales, revenue, and marketing metrics at a time. They are also used in the manufacturing industry, as they allow users to grasp the entire production scenario at the moment while tracking the core KPIs for each line.

Real-Life Application of a Dashboard

Consider a project manager presenting a software development project’s progress to a tech company’s leadership team. He follows the following steps.

Step 1: Defining Key Metrics

To effectively communicate the project’s status, identify key metrics such as completion status, budget, and bug resolution rates. Then, choose measurable metrics aligned with project objectives.

Step 2: Choosing Visualization Widgets

After finalizing the data, presentation aids that align with each metric are selected. For this project, the project manager chooses a progress bar for the completion status and uses bar charts for budget allocation. Likewise, he implements line charts for bug resolution rates.

Step 3: Dashboard Layout

Key metrics are prominently placed in the dashboard for easy visibility, and the manager ensures that it appears clean and organized.

Dashboards provide a comprehensive view of key project metrics. Users can interact with data, customize views, and drill down for detailed analysis. However, creating an effective dashboard requires careful planning to avoid clutter. Besides, dashboards rely on the availability and accuracy of underlying data sources.

For more information, check our article on how to design a dashboard presentation , and discover our collection of dashboard PowerPoint templates .

Treemap charts represent hierarchical data structured in a series of nested rectangles [6] . As each branch of the ‘tree’ is given a rectangle, smaller tiles can be seen representing sub-branches, meaning elements on a lower hierarchical level than the parent rectangle. Each one of those rectangular nodes is built by representing an area proportional to the specified data dimension.

Treemaps are useful for visualizing large datasets in compact space. It is easy to identify patterns, such as which categories are dominant. Common applications of the treemap chart are seen in the IT industry, such as resource allocation, disk space management, website analytics, etc. Also, they can be used in multiple industries like healthcare data analysis, market share across different product categories, or even in finance to visualize portfolios.

Real-Life Application of a Treemap Chart

Let’s consider a financial scenario where a financial team wants to represent the budget allocation of a company. There is a hierarchy in the process, so it is helpful to use a treemap chart. In the chart, the top-level rectangle could represent the total budget, and it would be subdivided into smaller rectangles, each denoting a specific department. Further subdivisions within these smaller rectangles might represent individual projects or cost categories.

Step 1: Define Your Data Hierarchy

While presenting data on the budget allocation, start by outlining the hierarchical structure. The sequence will be like the overall budget at the top, followed by departments, projects within each department, and finally, individual cost categories for each project.

• Top-level rectangle: Total Budget
• Second-level rectangles: Departments (Engineering, Marketing, Sales)
• Third-level rectangles: Projects within each department
• Fourth-level rectangles: Cost categories for each project (Personnel, Marketing Expenses, Equipment)

Step 2: Choose a Suitable Tool

It’s time to select a data visualization tool supporting Treemaps. Popular choices include Tableau, Microsoft Power BI, PowerPoint, or even coding with libraries like D3.js. It is vital to ensure that the chosen tool provides customization options for colors, labels, and hierarchical structures.

Here, the team uses PowerPoint for this guide because of its user-friendly interface and robust Treemap capabilities.

Step 3: Make a Treemap Chart with PowerPoint

After opening the PowerPoint presentation, they chose “SmartArt” to form the chart. The SmartArt Graphic window has a “Hierarchy” category on the left.  Here, you will see multiple options. You can choose any layout that resembles a Treemap. The “Table Hierarchy” or “Organization Chart” options can be adapted. The team selects the Table Hierarchy as it looks close to a Treemap.

Step 5: Input Your Data

After that, a new window will open with a basic structure. They add the data one by one by clicking on the text boxes. They start with the top-level rectangle, representing the total budget.  

Step 6: Customize the Treemap

By clicking on each shape, they customize its color, size, and label. At the same time, they can adjust the font size, style, and color of labels by using the options in the “Format” tab in PowerPoint. Using different colors for each level enhances the visual difference.

Treemaps excel at illustrating hierarchical structures. These charts make it easy to understand relationships and dependencies. They efficiently use space, compactly displaying a large amount of data, reducing the need for excessive scrolling or navigation. Additionally, using colors enhances the understanding of data by representing different variables or categories.

In some cases, treemaps might become complex, especially with deep hierarchies.  It becomes challenging for some users to interpret the chart. At the same time, displaying detailed information within each rectangle might be constrained by space. It potentially limits the amount of data that can be shown clearly. Without proper labeling and color coding, there’s a risk of misinterpretation.

A heatmap is a data visualization tool that uses color coding to represent values across a two-dimensional surface. In these, colors replace numbers to indicate the magnitude of each cell. This color-shaded matrix display is valuable for summarizing and understanding data sets with a glance [7] . The intensity of the color corresponds to the value it represents, making it easy to identify patterns, trends, and variations in the data.

As a tool, heatmaps help businesses analyze website interactions, revealing user behavior patterns and preferences to enhance overall user experience. In addition, companies use heatmaps to assess content engagement, identifying popular sections and areas of improvement for more effective communication. They excel at highlighting patterns and trends in large datasets, making it easy to identify areas of interest.

We can implement heatmaps to express multiple data types, such as numerical values, percentages, or even categorical data. Heatmaps help us easily spot areas with lots of activity, making them helpful in figuring out clusters [8] . When making these maps, it is important to pick colors carefully. The colors need to show the differences between groups or levels of something. And it is good to use colors that people with colorblindness can easily see.

Check our detailed guide on how to create a heatmap here. Also discover our collection of heatmap PowerPoint templates .

Pie charts are circular statistical graphics divided into slices to illustrate numerical proportions. Each slice represents a proportionate part of the whole, making it easy to visualize the contribution of each component to the total.

The size of the pie charts is influenced by the value of data points within each pie. The total of all data points in a pie determines its size. The pie with the highest data points appears as the largest, whereas the others are proportionally smaller. However, you can present all pies of the same size if proportional representation is not required [9] . Sometimes, pie charts are difficult to read, or additional information is required. A variation of this tool can be used instead, known as the donut chart , which has the same structure but a blank center, creating a ring shape. Presenters can add extra information, and the ring shape helps to declutter the graph.

Pie charts are used in business to show percentage distribution, compare relative sizes of categories, or present straightforward data sets where visualizing ratios is essential.

Real-Life Application of Pie Charts

Consider a scenario where you want to represent the distribution of the data. Each slice of the pie chart would represent a different category, and the size of each slice would indicate the percentage of the total portion allocated to that category.

Step 1: Define Your Data Structure

Imagine you are presenting the distribution of a project budget among different expense categories.

• Column A: Expense Categories (Personnel, Equipment, Marketing, Miscellaneous)
• Column B: Budget Amounts ($40,000, $30,000, $20,000, $10,000) Column B represents the values of your categories in Column A.

Step 2: Insert a Pie Chart

Using any of the accessible tools, you can create a pie chart. The most convenient tools for forming a pie chart in a presentation are presentation tools such as PowerPoint or Google Slides.  You will notice that the pie chart assigns each expense category a percentage of the total budget by dividing it by the total budget.

For instance:

• Personnel: $40,000 / ($40,000 + $30,000 + $20,000 + $10,000) = 40%
• Equipment: $30,000 / ($40,000 + $30,000 + $20,000 + $10,000) = 30%
• Marketing: $20,000 / ($40,000 + $30,000 + $20,000 + $10,000) = 20%
• Miscellaneous: $10,000 / ($40,000 + $30,000 + $20,000 + $10,000) = 10%

You can make a chart out of this or just pull out the pie chart from the data.

3D pie charts and 3D donut charts are quite popular among the audience. They stand out as visual elements in any presentation slide, so let’s take a look at how our pie chart example would look in 3D pie chart format.

Step 03: Results Interpretation

The pie chart visually illustrates the distribution of the project budget among different expense categories. Personnel constitutes the largest portion at 40%, followed by equipment at 30%, marketing at 20%, and miscellaneous at 10%. This breakdown provides a clear overview of where the project funds are allocated, which helps in informed decision-making and resource management. It is evident that personnel are a significant investment, emphasizing their importance in the overall project budget.

Pie charts provide a straightforward way to represent proportions and percentages. They are easy to understand, even for individuals with limited data analysis experience. These charts work well for small datasets with a limited number of categories.

However, a pie chart can become cluttered and less effective in situations with many categories. Accurate interpretation may be challenging, especially when dealing with slight differences in slice sizes. In addition, these charts are static and do not effectively convey trends over time.

For more information, check our collection of pie chart templates for PowerPoint .

Histograms present the distribution of numerical variables. Unlike a bar chart that records each unique response separately, histograms organize numeric responses into bins and show the frequency of reactions within each bin [10] . The x-axis of a histogram shows the range of values for a numeric variable. At the same time, the y-axis indicates the relative frequencies (percentage of the total counts) for that range of values.

Whenever you want to understand the distribution of your data, check which values are more common, or identify outliers, histograms are your go-to. Think of them as a spotlight on the story your data is telling. A histogram can provide a quick and insightful overview if you’re curious about exam scores, sales figures, or any numerical data distribution.

Real-Life Application of a Histogram

In the histogram data analysis presentation example, imagine an instructor analyzing a class’s grades to identify the most common score range. A histogram could effectively display the distribution. It will show whether most students scored in the average range or if there are significant outliers.

Step 1: Gather Data

He begins by gathering the data. The scores of each student in class are gathered to analyze exam scores.

After arranging the scores in ascending order, bin ranges are set.

Step 2: Define Bins

Bins are like categories that group similar values. Think of them as buckets that organize your data. The presenter decides how wide each bin should be based on the range of the values. For instance, the instructor sets the bin ranges based on score intervals: 60-69, 70-79, 80-89, and 90-100.

Step 3: Count Frequency

Now, he counts how many data points fall into each bin. This step is crucial because it tells you how often specific ranges of values occur. The result is the frequency distribution, showing the occurrences of each group.

Here, the instructor counts the number of students in each category.

• 60-69: 1 student (Kate)
• 70-79: 4 students (David, Emma, Grace, Jack)
• 80-89: 7 students (Alice, Bob, Frank, Isabel, Liam, Mia, Noah)
• 90-100: 3 students (Clara, Henry, Olivia)

Step 4: Create the Histogram

It’s time to turn the data into a visual representation. Draw a bar for each bin on a graph. The width of the bar should correspond to the range of the bin, and the height should correspond to the frequency.  To make your histogram understandable, label the X and Y axes.

In this case, the X-axis should represent the bins (e.g., test score ranges), and the Y-axis represents the frequency.

The histogram of the class grades reveals insightful patterns in the distribution. Most students, with seven students, fall within the 80-89 score range. The histogram provides a clear visualization of the class’s performance. It showcases a concentration of grades in the upper-middle range with few outliers at both ends. This analysis helps in understanding the overall academic standing of the class. It also identifies the areas for potential improvement or recognition.

Thus, histograms provide a clear visual representation of data distribution. They are easy to interpret, even for those without a statistical background. They apply to various types of data, including continuous and discrete variables. One weak point is that histograms do not capture detailed patterns in students’ data, with seven compared to other visualization methods.

A scatter plot is a graphical representation of the relationship between two variables. It consists of individual data points on a two-dimensional plane. This plane plots one variable on the x-axis and the other on the y-axis. Each point represents a unique observation. It visualizes patterns, trends, or correlations between the two variables.

Scatter plots are also effective in revealing the strength and direction of relationships. They identify outliers and assess the overall distribution of data points. The points’ dispersion and clustering reflect the relationship’s nature, whether it is positive, negative, or lacks a discernible pattern. In business, scatter plots assess relationships between variables such as marketing cost and sales revenue. They help present data correlations and decision-making.

Real-Life Application of Scatter Plot

A group of scientists is conducting a study on the relationship between daily hours of screen time and sleep quality. After reviewing the data, they managed to create this table to help them build a scatter plot graph:

In the provided example, the x-axis represents Daily Hours of Screen Time, and the y-axis represents the Sleep Quality Rating.

The scientists observe a negative correlation between the amount of screen time and the quality of sleep. This is consistent with their hypothesis that blue light, especially before bedtime, has a significant impact on sleep quality and metabolic processes.

There are a few things to remember when using a scatter plot. Even when a scatter diagram indicates a relationship, it doesn’t mean one variable affects the other. A third factor can influence both variables. The more the plot resembles a straight line, the stronger the relationship is perceived [11] . If it suggests no ties, the observed pattern might be due to random fluctuations in data. When the scatter diagram depicts no correlation, whether the data might be stratified is worth considering.

Choosing the appropriate data presentation type is crucial when making a presentation . Understanding the nature of your data and the message you intend to convey will guide this selection process. For instance, when showcasing quantitative relationships, scatter plots become instrumental in revealing correlations between variables. If the focus is on emphasizing parts of a whole, pie charts offer a concise display of proportions. Histograms, on the other hand, prove valuable for illustrating distributions and frequency patterns. 

Bar charts provide a clear visual comparison of different categories. Likewise, line charts excel in showcasing trends over time, while tables are ideal for detailed data examination. Starting a presentation on data presentation types involves evaluating the specific information you want to communicate and selecting the format that aligns with your message. This ensures clarity and resonance with your audience from the beginning of your presentation.

1. Fact Sheet Dashboard for Data Presentation

Convey all the data you need to present in this one-pager format, an ideal solution tailored for users looking for presentation aids. Global maps, donut chats, column graphs, and text neatly arranged in a clean layout presented in light and dark themes.

Use This Template

2. 3D Column Chart Infographic PPT Template

Represent column charts in a highly visual 3D format with this PPT template. A creative way to present data, this template is entirely editable, and we can craft either a one-page infographic or a series of slides explaining what we intend to disclose point by point.

3. Data Circles Infographic PowerPoint Template

An alternative to the pie chart and donut chart diagrams, this template features a series of curved shapes with bubble callouts as ways of presenting data. Expand the information for each arch in the text placeholder areas.

4. Colorful Metrics Dashboard for Data Presentation

This versatile dashboard template helps us in the presentation of the data by offering several graphs and methods to convert numbers into graphics. Implement it for e-commerce projects, financial projections, project development, and more.

5. Animated Data Presentation Tools for PowerPoint & Google Slides

A slide deck filled with most of the tools mentioned in this article, from bar charts, column charts, treemap graphs, pie charts, histogram, etc. Animated effects make each slide look dynamic when sharing data with stakeholders.

6. Statistics Waffle Charts PPT Template for Data Presentations

This PPT template helps us how to present data beyond the typical pie chart representation. It is widely used for demographics, so it’s a great fit for marketing teams, data science professionals, HR personnel, and more.

7. Data Presentation Dashboard Template for Google Slides

A compendium of tools in dashboard format featuring line graphs, bar charts, column charts, and neatly arranged placeholder text areas. 

8. Weather Dashboard for Data Presentation

Share weather data for agricultural presentation topics, environmental studies, or any kind of presentation that requires a highly visual layout for weather forecasting on a single day. Two color themes are available.

9. Social Media Marketing Dashboard Data Presentation Template

Intended for marketing professionals, this dashboard template for data presentation is a tool for presenting data analytics from social media channels. Two slide layouts featuring line graphs and column charts.

10. Project Management Summary Dashboard Template

A tool crafted for project managers to deliver highly visual reports on a project’s completion, the profits it delivered for the company, and expenses/time required to execute it. 4 different color layouts are available.

11. Profit & Loss Dashboard for PowerPoint and Google Slides

A must-have for finance professionals. This typical profit & loss dashboard includes progress bars, donut charts, column charts, line graphs, and everything that’s required to deliver a comprehensive report about a company’s financial situation.

Overwhelming visuals

One of the mistakes related to using data-presenting methods is including too much data or using overly complex visualizations. They can confuse the audience and dilute the key message.

Inappropriate chart types

Choosing the wrong type of chart for the data at hand can lead to misinterpretation. For example, using a pie chart for data that doesn’t represent parts of a whole is not right.

Lack of context

Failing to provide context or sufficient labeling can make it challenging for the audience to understand the significance of the presented data.

Inconsistency in design

Using inconsistent design elements and color schemes across different visualizations can create confusion and visual disarray.

Failure to provide details

Simply presenting raw data without offering clear insights or takeaways can leave the audience without a meaningful conclusion.

Lack of focus

Not having a clear focus on the key message or main takeaway can result in a presentation that lacks a central theme.

Visual accessibility issues

Overlooking the visual accessibility of charts and graphs can exclude certain audience members who may have difficulty interpreting visual information.

In order to avoid these mistakes in data presentation, presenters can benefit from using presentation templates . These templates provide a structured framework. They ensure consistency, clarity, and an aesthetically pleasing design, enhancing data communication’s overall impact.

Understanding and choosing data presentation types are pivotal in effective communication. Each method serves a unique purpose, so selecting the appropriate one depends on the nature of the data and the message to be conveyed. The diverse array of presentation types offers versatility in visually representing information, from bar charts showing values to pie charts illustrating proportions. 

Using the proper method enhances clarity, engages the audience, and ensures that data sets are not just presented but comprehensively understood. By appreciating the strengths and limitations of different presentation types, communicators can tailor their approach to convey information accurately, developing a deeper connection between data and audience understanding.

[1] Government of Canada, S.C. (2021) 5 Data Visualization 5.2 Bar Chart , 5.2 Bar chart .  https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch9/bargraph-diagrammeabarres/5214818-eng.htm

[2] Kosslyn, S.M., 1989. Understanding charts and graphs. Applied cognitive psychology, 3(3), pp.185-225. https://apps.dtic.mil/sti/pdfs/ADA183409.pdf

[3] Creating a Dashboard . https://it.tufts.edu/book/export/html/1870

[4] https://www.goldenwestcollege.edu/research/data-and-more/data-dashboards/index.html

[5] https://www.mit.edu/course/21/21.guide/grf-line.htm

[6] Jadeja, M. and Shah, K., 2015, January. Tree-Map: A Visualization Tool for Large Data. In GSB@ SIGIR (pp. 9-13). https://ceur-ws.org/Vol-1393/gsb15proceedings.pdf#page=15

[7] Heat Maps and Quilt Plots. https://www.publichealth.columbia.edu/research/population-health-methods/heat-maps-and-quilt-plots

[8] EIU QGIS WORKSHOP. https://www.eiu.edu/qgisworkshop/heatmaps.php

[9] About Pie Charts.  https://www.mit.edu/~mbarker/formula1/f1help/11-ch-c8.htm

[10] Histograms. https://sites.utexas.edu/sos/guided/descriptive/numericaldd/descriptiven2/histogram/ [11] https://asq.org/quality-resources/scatter-diagram

Like this article? Please share

Data Analysis, Data Science, Data Visualization Filed under Design

Related Articles

Filed under Presentation Ideas • January 6th, 2024

All About Using Harvey Balls

Among the many tools in the arsenal of the modern presenter, Harvey Balls have a special place. In this article we will tell you all about using Harvey Balls.

Filed under Business • December 8th, 2023

How to Design a Dashboard Presentation: A Step-by-Step Guide

Take a step further in your professional presentation skills by learning what a dashboard presentation is and how to properly design one in PowerPoint. A detailed step-by-step guide is here!

Filed under Business • November 20th, 2023

Data Flow Diagram Demystified: What They Are and How to Use Them

Master the art of Data Flow Diagrams with our in-depth guide! Discover the steps to create, interpret, and leverage DFDs to optimize processes.

Leave a Reply

• How to Use Data Visualization Tools and Techniques for Business

Sep 29, 2023

data visualization

The ever-growing volume of data and its importance for business make data visualization an essential part of business strategy for many companies.

In this article, we review major data visualization instruments and name the key factors that influence the choice of visualization techniques and tools. You will learn about the most widely used tools for data visualization and get a few expert tips on how to combine data visualizations into effective dashboards. We will also show how the adoption of these tools can affect business outcomes and give several real-world examples from our experience.

What determines data visualization choices

Visualization is the first step to make sense of data. To translate and present complex data and relations in a simple way, data analysts use different methods of data visualization — charts, diagrams, maps, etc. Choosing the right technique and its setup is often the only way to make data understandable. Vice versa, poorly selected tactics won't let you unlock the full potential of data or even make it irrelevant.

5 factors that impact your choice of data visualization methods and techniques:

• Audience. It’s important to adjust data representation to the specific target audience. For example, fitness mobile app users who browse through their progress would prefer easy-to-read uncomplicated visualizations on their phones. On the other hand, if data insights are intended for researchers, specialists and C-level decision-makers who regularly work with data, you can and often have to go beyond simple charts.
• Content. The type of data you are dealing with will determine the tactics. For example, if it’s time-series metrics, you will use line charts to show the dynamics in many cases. To show the relationship between two elements, scatter plots are often used. In turn, bar charts work well for comparative analysis.
• Context. You can use different data visualization approaches and read data depending on the context. To emphasize a certain figure, for example, significant profit growth, you can use the shades of one color on the chart and highlight the highest value with the brightest one. On the contrary, to differentiate elements, you can use contrast colors.
• Dynamics. There are various types of data, and each type has a different rate of change. For example, financial results can be measured monthly or yearly, while time series and tracking data are changing constantly. Depending on the rate of change, you may consider dynamic representation (steaming) or static data visualization techniques in data mining.
• Purpose. The goal of data visualization affects the way it is implemented. In order to make a complex analysis, visualizations are compiled into dynamic and controllable dashboards equipped with different tools for visual data analytics (comparison, formatting, filtering, etc.). However, dashboards are not necessary to show a single or occasional data insight.

Are you looking for a skillful team to create effective and responsive data visualization and dashboards to deliver important insights for your business? Contact our team and tell us about your needs and requirements. Our analysts, developers and data scientists have profound experience in working with different types of data and will find a way to help you get the most of your data assets.

Cut time to insight to make decisions faster

Turn your data into value and make critical business decisions faster with powerful data analytics and visualization tools.

Data visualization techniques

Depending on these factors, you can choose different data visualization techniques and configure their features. Here are the common types of data visualization techniques:

The easiest way to show the development of one or several data sets is a chart. Charts vary from bar and line charts that show the relationship between elements over time to pie charts that demonstrate the components or proportions between the elements of one whole.

Plots allow to distribute two or more data sets over a 2D or 3D space to show the relationship between these sets and the parameters on the plot. Plots also vary. Scatter and bubble plots are some of the most widely used visualizations. When it comes to big data, analysts often use more complex box plots to visualize the relationships between large volumes of data.

Maps are popular techniques used for data visualization in different industries. They allow locating elements on relevant objects and areas — geographical maps, building plans, website layouts, etc. Among the most popular map visualizations are heatmaps, dot distribution maps, and cartograms.

Diagrams and matrices

Diagrams are usually used to demonstrate complex data relationships and links and include various types of data in one visual representation. They can be hierarchical, multidimensional, or tree-like.

Matrix is one of the advanced data visualization techniques that help determine the correlation between multiple constantly updating (steaming) data sets.

Image credit: duke.edu

Data visualization tools

Together with the demand for data visualization and analysis, the tools and solutions in this area develop fast and extensively. Novel 3D visualizations, immersive experiences and shared VR offices are getting common alongside traditional web and desktop interfaces. Here are three categories of data visualization technologies and tools for different types of users and purposes.

Data visualization tools for everyone

Tableau is one of the leaders in this field. Startups and global conglomerates like Verizon and Henkel rely on this platform to derive meaning from data and use insights for effective decision making.

Apart from a user-friendly interface and a rich library of interactive visualizations and data representation techniques, Tableau stands out for its powerful capabilities. The platform provides diverse integration options with various data storage, management, and infrastructure solutions, including Microsoft SQL Server, Databricks, Google BigQuery, Teradata, Hadoop, and Amazon Web Services.

This is a great tool for both occasional data visualizations and professional data analytics. The system can easily handle any type of data, including streaming performance data, and allows to combine visualizations into functional dashboards. Tableau, as part of Salesforce since 2019, invests in AI and augmented analytics and equips customers with tools for advanced analytics and forecasting.

Image credit: Tableau

Also in this category:

Among popular all-purpose data visualization tools in this category are easy-to-learn Visme and Datawrapper which allow to create engaging visualizations without design and coding skills. Both tools offer free basic plans to start from. ChartBlocks and Infogram are other no-code tools with dozens of templates and customization options across various data visualization methods.

Marketer's favorite Canva is a popular solution with varied visualization designs and user-friendly editors. If you use Google suite for business, consider also Looker Studio for business intelligence and reporting.

Data visualization tools for coders

This category of tools includes more sophisticated platforms for presenting data. They stand out for rich functionality to fully unlock the benefits of data visualization . These tools are often used to add visual data analytics techniques and features to data applications and scalable systems built with modern web app architecture approaches and cloud technologies.

FusionCharts is a versatile platform for creating interactive dashboards on web and mobile. It offers rich integration capabilities with support for various frontend and backend frameworks and languages, including Angular, React, Vue, ASP.NET, and PHP.

FusionCharts caters to diverse data visualization needs, offering rich customization options, pre-built themes, 100 ready-to-use charts and 2000 maps, and extensive documentation to make developers' lives easier. This explains the popularity of the platform. Over 800,000 developers and 28,000 organizations such as Dell, Apple, Adobe, and Google, already use this platform.

Image credit: FusionCharts

Sisense is another industry-grade data visualization tool with rich analytics capabilities. This cloud-based platform has a drag-and-drop interface, can handle multiple data sources, and supports natural language queries.

Sisense dashboards are highly customizable. You can personalize the look and feel, add images, text, videos, and links, add filters and drill-down features, and transform static visualizations into interactive storytelling experiences.

The platform has a strong focus on AI and ML to provide actionable insights for users. The platform stands out for its scalability and flexibility. It's easy to integrate Sisense analytics and visualizations using their flexible developer toolkit and SDKs to either build a new data application or embed dashboards and visualizations into an existing one.

Image credit: Sisense

Plotly is a popular platform mainly focused on developing data apps with Python . It offers rich data visualization tools and techniques and enables integrations with ChatGPT and LLMS to create visualizations using prompts. Plotly's open-source libraries for Python, R, JavaScript, F#, Julia, and other programming languages help developers create various interactive visualizations, including complex maps, animations, and 3D charts.

IBM Cognos Analytics is known for its NLP capabilities. The platform supports conversational data control and provides versatile tools for dashboard building and data reporting. The AI assistant uses natural language queries to build stunning visualizations and can even choose optimal visual data analysis techniques based on what insights you need to get.

If MongoDB is a part of your stack, consider also MongoDB Charts for your MongoDB data. It seamlessly integrates with the core platform's tools and offers various features for creating charts and dashboards.

Tools for complex data visualization and analytics

The growing adoption of connected technology places a lot of opportunities before companies and organizations. To deal with large volumes of multi-source often unstructured data, businesses search for more complex visualization and analytics solutions. This category includes Microsoft Azure Power BI, ELK stack Kibana, and Grafana.

Power BI is exceptional for its highly intuitive drag-and-drop interface, short learning curve, and large integration capabilities, including Salesforce and MailChimp. Not to mention moderate pricing ($10 per month for a Pro version).

Image credit: Microsoft

Thanks to Azure services, Power BI became one of the most robust data visualization and analytics tools that can handle nearly any amount and any type of data.

First of all, the platform allows you to create customized reports from different data sources and get insights in a couple of clicks. Secondly, Power BI is powerful and can easily work with streaming real-time data. Finally, it’s not only fully compatible with Azure and other Microsoft services but also can directly connect to existing apps and drive analytics to custom systems. Watch the introduction to learn more here .

Kibana is the part of the ELK Stack that turns data into actionable insights. It’s built on and designed to work with Elasticsearch data. This exclusivity, however, does not prevent it from being one of the best data visualization tools for log data.

Kibana allows you to explore various big data visualization techniques in data science — interactive charts, maps, histograms, etc. Moreover, Kibana goes beyond building standard dashboards for data visualization and analytics.

This tool will help you leverage various visual data analysis techniques in big data: combine visualizations from multiple sources to find correlations, explore trends, and add machine learning features to reveal hidden relationships between events. Drag-and-drop Kibana Lens helps you explore visualized data and get quick insights in just a few clicks. And a rich toolkit for developers and APIs come as a cherry on top.

Image credit: Elasticsearch

Grafana — a professional data visualization and analytic tool that supports a wide range of data sources, including AWS, Elasticsearch, and Prometheus.

Even though Grafana is more flexible in terms of integrations compared to Kibana, each of the systems works best with its own type of data. In the case of Grafana, it’s metrics. This visualization software is popular for building IoT applications and creating dashboards and monitoring tools for telemetry systems that use different IoT data collection methods .

Grafana allows you to visualize and compile different types of metrics data into dynamic dashboards. It has a wide variation of features, plugins, and roles which makes it perfect for complex monitoring and control systems.

Additionally, it enables alerts and notifications based on predefined rules. And finally, Grafana has perks for fast data analytics, such as creating custom filters and making annotations — adding metadata to certain events on a dashboard.

Qlik Sense is a data intelligence platform with unique capabilities and features. It provides highly interactive visualizations and dashboards that enable fast insight discovery. Every time a user clicks on an event or metric, the platform refines the context on the spot to show unique dependencies and correlations.

Qlik stands out for lightning-fast calculations and AI technologies in the core analytics functions.

These are just the major tools and techniques of data visualization. All the mentioned platforms and services are evolving very fast and introduce new features and capabilities to keep up with the market needs. Especially when it comes to the growing pace of big data and Internet of Things development.

Tips to create efficient data dashboards

Choosing the right data visualization techniques and tools is the key point to figure out when working with data. However, it is not the only one.

Often visualizations are combined into dashboards to provide analysts, management and other users with complete information on a subject. Dashboards have different functions (show changes in conditions, help track activity and location in real-time, provide remote monitoring and control of a system, etc.) and specifics (dynamic vs. static, historical vs. real-time, KPI/goals dashboards, etc.) that determine their design and features. However, there are several important factors to consider when you create a data dashboard of any type or purpose:

Tip 1. Consistency

Consistency is the key to fluency and fast dashboard navigation. It’s important to stick to specific color-coding, fonts, styles, and visualization elements when showing the same metrics across different dashboards.

Tip 2. The right choice of visualizations

No visualization is one-size-fits-all, even a line chart. It’s crucial to choose the right visualization technique for each type of data on a dashboard to ensure its usability and avoid confusion or even misinterpretation. Check the examples in this article that support this point.

Tip 3. Personalization

Not only does the audience impact the choice of individual visualizations but also determines how to create a data analysis dashboard. It’s essential to keep the goals of different end-users in mind when deciding what visualizations and data should be included in a dashboard. After all, important information for one user can be unessential or even meaningless for others.

Example: A health tracking app used by patients and doctors should have two personalized dashboards. The patient’s dashboard can include basic health data such as blood pressure, medication intake, activity tracking, while the doctor’s dashboard can combine this data with test results, EHR notes and other medical information to provide a more comprehensive picture of the patient’s condition.

Tip 4. Device constraints

Screen size is an important parameter when we are talking about multifunctional dashboards that are supposed to be used on different devices. A dashboard on a small mobile screen should provide no less value than a dashboard on a desktop screen. For this purpose, designers should consider responsiveness and provide tools and features to easily manipulate dashboards on limited smartphone screens — quickly navigate between views, drill data, compile custom reports, etc. This point is particularly important when creating UX/UI design for IoT apps for they are usually data-heavy.

Tip 5. Value

A dashboard should provide value the moment the user accesses it. However, it does not necessarily mean that all the data should be stuffed into screen one. On the contrary, visualizations should be carefully selected, grouped and aligned on every screen to immediately answer all important questions and suggest ways to further explore the data.

Tip 6. Testing

Whether you create an automated data visualization dashboard using tools like Grafana or design a custom dashboard for your system, it’s important to test your visualizations on different volumes of data and in different conditions before going live. In many cases, dashboards are developed based on test data. Once released, dashboards show real data which can be quite different from the test data. As a result, these dashboards do not look and behave as intended. Testing in different conditions helps bridge this gap and avoid inconsistency.

Examples of data visualization: real stories from our experience

We have worked on a range of data visualization and analytics projects for cleantech, logistics, healthcare, retail and IT companies and successfully integrated custom data solutions into their operations. Here are a few use cases that show different approaches to data visualization and their effect on business outcomes.

Efficient performance monitoring helps a printing company handle a 200% traffic increase with a 0% slowdown

Our long-standing client, one of the leading printing companies in the U.S., was dealing with a traffic upsurge challenge every holiday season. They needed an effective monitoring and control solution to avoid website slowdown or performance decrease.

Our dedicated team has been working with the lab's tech infrastructure for a decade. To address the challenge, we developed a custom data analytics and visualization tool based on the combination of Elastic Stack solutions. It collects and analyzes real-time performance data – website traffic, backend load, the quantity and status of submitted orders, printing status and queue. The system then visualizes data for real-time monitoring, registers failed operations and sends alerts to the support team if any problem is spotted.

From the moment this tool was integrated into the lab’s operations, the company could better manage growing traffic and go through intensive seasonal sales without slowdown or lost orders. Since then, we have upgraded the monitoring system and scaled the client's entire infrastructure to better address the constantly growing load ( microservices development services to enhance scalability and resilience, migration to modern .NET technologies , adoption of cloud and data services, etc.)

Monitor IT infrastructure in real-time to improve security and employee performance

A Finnish tech company developed a SaaS product for real-time IT infrastructure monitoring. The system was originally focused on Windows products, and we were asked to scale it up for the macOS platform.

We developed a multifunctional monitoring system to track performance and provide real-time data on the health and security of the customers' IT infrastructures. The system collects data on registered software and hardware performance, sends it to the cloud-based backend for analysis, and provides customers with visualized insights on dashboards and reports.

When integrated into the company’s operations, the system helps improve employee productivity and mitigate possible security and performance risks. It is an effective tool for IT asset tracking and management.

These are a few examples that demonstrate how effective data visualization systems and tools affect business operations and help companies deal with major performance challenges.

Leverage powerful data visualization tools with Digiteum

If you are looking for skilled tech experts to help you design and create a data visualization system for your business, consult on the use of professional data visualization tools, and integrate them into your decision-making process, we can help.

We work with a wide range of data visualization and analytics platforms (ELK Stack and Kibana, Grafana, Qlik, Sisense, Power BI, Tableau, etc.) and apply a range of techniques, including popular data visualization techniques in Machine Learning, IoT and big data. Our designers, software and data engineers create stunning visualizations and data tools for such companies as Oxford Languages, Printique, Origin Digital, feed.fm, and Diaceutics. We can help you:

• Select, configure, and integrate the right data visualization and analytics tools according to your business needs and scale.
• Design and build custom dashboards and add powerful features for data sorting, analysis, and reporting.
• Integrate data visualization techniques in business analytics that best fit your requirements and BI goals.
• Optimize time to insight and cut the cost of BI with robust data management and analytics tools.
• Design, build, and support your entire data infrastructure and operations using modern cloud-based solutions.

Check our big data services  to learn how we can help you turn your data into a source of income and growth... without losing track of your costs.

Read and understand your data faster

Integrate powerful data visualization tools and techniques to your decision-making process.

In this article, we answered the most common questions about platforms, tools and methods of data visualization: What are data visualization techniques and tools? How to use different visualization tools to unlock the power of data? How to build effective data dashboards? You can use this foundational knowledge to start working with your data and select tools that will help you extract real value from your assets.

If you need help to find the right data tools for your specific business needs, contact our team .

This post was originally published on August 10, 2018 and updated on September 16, 2021 and September 29, 2023.

Post by Digiteum Team

• FusionCharts
• Create consistent color coding, styles and visualization elements for easy navigation.
• Choose the right data visualization techniques for different types of data and use cases.
• Personalize dashboard layout and functionality to specific audience.
• Take into consideration device constraints and provide additional tools to manipulate data on small screens.
• Group and align data to maximize value, not volume.
• Test dashboard performance using different volumes of data to see how it can handle the real data flow and dynamics.

About Digiteum

Digiteum is a custom software development and IT consulting company founded in 2010. We design and develop customer-centric solutions for web, mobile, cloud, and IoT.

Design and Engineering

Technologies.

We are always looking for talented people. CHECK OPEN POSITIONS…

[email protected]

Graphical Representation of Data

Graphical representation of data is an attractive method of showcasing numerical data that help in analyzing and representing quantitative data visually. A graph is a kind of a chart where data are plotted as variables across the coordinate. It became easy to analyze the extent of change of one variable based on the change of other variables. Graphical representation of data is done through different mediums such as lines, plots, diagrams, etc. Let us learn more about this interesting concept of graphical representation of data, the different types, and solve a few examples.

Definition of Graphical Representation of Data

A graphical representation is a visual representation of data statistics-based results using graphs, plots, and charts. This kind of representation is more effective in understanding and comparing data than seen in a tabular form. Graphical representation helps to qualify, sort, and present data in a method that is simple to understand for a larger audience. Graphs enable in studying the cause and effect relationship between two variables through both time series and frequency distribution. The data that is obtained from different surveying is infused into a graphical representation by the use of some symbols, such as lines on a line graph, bars on a bar chart, or slices of a pie chart. This visual representation helps in clarity, comparison, and understanding of numerical data.

Representation of Data

The word data is from the Latin word Datum, which means something given. The numerical figures collected through a survey are called data and can be represented in two forms - tabular form and visual form through graphs. Once the data is collected through constant observations, it is arranged, summarized, and classified to finally represented in the form of a graph. There are two kinds of data - quantitative and qualitative. Quantitative data is more structured, continuous, and discrete with statistical data whereas qualitative is unstructured where the data cannot be analyzed.

Principles of Graphical Representation of Data

The principles of graphical representation are algebraic. In a graph, there are two lines known as Axis or Coordinate axis. These are the X-axis and Y-axis. The horizontal axis is the X-axis and the vertical axis is the Y-axis. They are perpendicular to each other and intersect at O or point of Origin. On the right side of the Origin, the Xaxis has a positive value and on the left side, it has a negative value. In the same way, the upper side of the Origin Y-axis has a positive value where the down one is with a negative value. When -axis and y-axis intersect each other at the origin it divides the plane into four parts which are called Quadrant I, Quadrant II, Quadrant III, Quadrant IV. This form of representation is seen in a frequency distribution that is represented in four methods, namely Histogram, Smoothed frequency graph, Pie diagram or Pie chart, Cumulative or ogive frequency graph, and Frequency Polygon.

Advantages and Disadvantages of Graphical Representation of Data

Listed below are some advantages and disadvantages of using a graphical representation of data:

• It improves the way of analyzing and learning as the graphical representation makes the data easy to understand.
• It can be used in almost all fields from mathematics to physics to psychology and so on.
• It is easy to understand for its visual impacts.
• It shows the whole and huge data in an instance.
• It is mainly used in statistics to determine the mean, median, and mode for different data

The main disadvantage of graphical representation of data is that it takes a lot of effort as well as resources to find the most appropriate data and then represent it graphically.

Rules of Graphical Representation of Data

While presenting data graphically, there are certain rules that need to be followed. They are listed below:

• Suitable Title: The title of the graph should be appropriate that indicate the subject of the presentation.
• Measurement Unit: The measurement unit in the graph should be mentioned.
• Proper Scale: A proper scale needs to be chosen to represent the data accurately.
• Index: For better understanding, index the appropriate colors, shades, lines, designs in the graphs.
• Data Sources: Data should be included wherever it is necessary at the bottom of the graph.
• Simple: The construction of a graph should be easily understood.
• Neat: The graph should be visually neat in terms of size and font to read the data accurately.

Uses of Graphical Representation of Data

The main use of a graphical representation of data is understanding and identifying the trends and patterns of the data. It helps in analyzing large quantities, comparing two or more data, making predictions, and building a firm decision. The visual display of data also helps in avoiding confusion and overlapping of any information. Graphs like line graphs and bar graphs, display two or more data clearly for easy comparison. This is important in communicating our findings to others and our understanding and analysis of the data.

Types of Graphical Representation of Data

Data is represented in different types of graphs such as plots, pies, diagrams, etc. They are as follows,

Related Topics

Listed below are a few interesting topics that are related to the graphical representation of data, take a look.

• x and y graph
• Frequency Polygon
• Cumulative Frequency

Examples on Graphical Representation of Data

Example 1 : A pie chart is divided into 3 parts with the angles measuring as 2x, 8x, and 10x respectively. Find the value of x in degrees.

We know, the sum of all angles in a pie chart would give 360º as result. ⇒ 2x + 8x + 10x = 360º ⇒ 20 x = 360º ⇒ x = 360º/20 ⇒ x = 18º Therefore, the value of x is 18º.

Example 2: Ben is trying to read the plot given below. His teacher has given him stem and leaf plot worksheets. Can you help him answer the questions? i) What is the mode of the plot? ii) What is the mean of the plot? iii) Find the range.

Solution: i) Mode is the number that appears often in the data. Leaf 4 occurs twice on the plot against stem 5.

Hence, mode = 54

ii) The sum of all data values is 12 + 14 + 21 + 25 + 28 + 32 + 34 + 36 + 50 + 53 + 54 + 54 + 62 + 65 + 67 + 83 + 88 + 89 + 91 = 958

To find the mean, we have to divide the sum by the total number of values.

Mean = Sum of all data values ÷ 19 = 958 ÷ 19 = 50.42

iii) Range = the highest value - the lowest value = 91 - 12 = 79

go to slide go to slide

Book a Free Trial Class

Practice Questions on Graphical Representation of Data

Faqs on graphical representation of data, what is graphical representation.

Graphical representation is a form of visually displaying data through various methods like graphs, diagrams, charts, and plots. It helps in sorting, visualizing, and presenting data in a clear manner through different types of graphs. Statistics mainly use graphical representation to show data.

What are the Different Types of Graphical Representation?

The different types of graphical representation of data are:

• Stem and leaf plot
• Scatter diagrams
• Frequency Distribution

Is the Graphical Representation of Numerical Data?

Yes, these graphical representations are numerical data that has been accumulated through various surveys and observations. The method of presenting these numerical data is called a chart. There are different kinds of charts such as a pie chart, bar graph, line graph, etc, that help in clearly showcasing the data.

What is the Use of Graphical Representation of Data?

Graphical representation of data is useful in clarifying, interpreting, and analyzing data plotting points and drawing line segments , surfaces, and other geometric forms or symbols.

What are the Ways to Represent Data?

Tables, charts, and graphs are all ways of representing data, and they can be used for two broad purposes. The first is to support the collection, organization, and analysis of data as part of the process of a scientific study.

What is the Objective of Graphical Representation of Data?

The main objective of representing data graphically is to display information visually that helps in understanding the information efficiently, clearly, and accurately. This is important to communicate the findings as well as analyze the data.

The Behavioral and Social Sciences: Achievements and Opportunities (1988)

Chapter: 5. methods of data collection, representation, and anlysis.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

l - Methods of Data Collection, Representation Analysis , and

SMethods of Data Collection. Representation, and This chapter concerns research on collecting, representing, and analyzing the data that underlie behavioral and social sciences knowledge. Such research, methodological in character, includes ethnographic and historical approaches, scaling, axiomatic measurement, and statistics, with its important relatives, econometrics and psychometrics. The field can be described as including the self-conscious study of how scientists draw inferences and reach conclusions from observations. Since statistics is the largest and most prominent of meth- odological approaches and is used by researchers in virtually every discipline, statistical work draws the lion's share of this chapter's attention. Problems of interpreting data arise whenever inherent variation or measure- ment fluctuations create challenges to understand data or to judge whether observed relationships are significant, durable, or general. Some examples: Is a sharp monthly (or yearly) increase in the rate of juvenile delinquency (or unemployment) in a particular area a matter for alarm, an ordinary periodic or random fluctuation, or the result of a change or quirk in reporting method? Do the temporal patterns seen in such repeated observations reflect a direct causal mechanism, a complex of indirect ones, or just imperfections in the Analysis 167

168 / The Behavioral and Social Sciences data? Is a decrease in auto injuries an effect of a new seat-belt law? Are the disagreements among people describing some aspect of a subculture too great to draw valid inferences about that aspect of the culture? Such issues of inference are often closely connected to substantive theory and specific data, and to some extent it is difficult and perhaps misleading to treat methods of data collection, representation, and analysis separately. This report does so, as do all sciences to some extent, because the methods developed often are far more general than the specific problems that originally gave rise to them. There is much transfer of new ideas from one substantive field to another—and to and from fields outside the behavioral and social sciences. Some of the classical methods of statistics arose in studies of astronomical observations, biological variability, and human diversity. The major growth of the classical methods occurred in the twentieth century, greatly stimulated by problems in agriculture and genetics. Some methods for uncovering geometric structures in data, such as multidimensional scaling and factor analysis, orig- inated in research on psychological problems, but have been applied in many other sciences. Some time-series methods were developed originally to deal with economic data, but they are equally applicable to many other kinds of data. Within the behavioral and social sciences, statistical methods have been developed in and have contributed to an enormous variety of research, includ- ing: · In economics: large-scale models of the U.S. economy; effects of taxa- tion, money supply, and other government fiscal and monetary policies; theories of duopoly, oligopoly, and rational expectations; economic effects of slavery. · In psychology: test calibration; the formation of subjective probabilities, their revision in the light of new information, and their use in decision making; psychiatric epidemiology and mental health program evaluation. · In sociology and other fields: victimization and crime rates; effects of incarceration and sentencing policies; deployment of police and fire-fight- ing forces; discrimination, antitrust, and regulatory court cases; social net- works; population growth and forecasting; and voting behavior. Even such an abridged listing makes clear that improvements in method- ology are valuable across the spectrum of empirical research in the behavioral and social sciences as well as in application to policy questions. Clearly, meth- odological research serves many different purposes, and there is a need to develop different approaches to serve those different purposes, including ex- ploratory data analysis, scientific inference about hypotheses and population parameters, individual decision making, forecasting what will happen in the event or absence of intervention, and assessing causality from both randomized experiments and observational data.

Methods of Data Collection, Representation, and Analysis / 169 This discussion of methodological research is divided into three areas: de- sign, representation, and analysis. The efficient design of investigations must take place before data are collected because it involves how much, what kind of, and how data are to be collected. What type of study is feasible: experi- mental, sample survey, field observation, or other? What variables should be measured, controlled, and randomized? How extensive a subject pool or ob- servational period is appropriate? How can study resources be allocated most effectively among various sites, instruments, and subsamples? The construction of useful representations of the data involves deciding what kind of formal structure best expresses the underlying qualitative and quanti- tative concepts that are being used in a given study. For example, cost of living is a simple concept to quantify if it applies to a single individual with unchang- ing tastes in stable markets (that is, markets offering the same array of goods from year to year at varying prices), but as a national aggregate for millions of households and constantly changing consumer product markets, the cost of living is not easy to specify clearly or measure reliably. Statisticians, economists, sociologists, and other experts have long struggled to make the cost of living a precise yet practicable concept that is also efficient to measure, and they must continually modify it to reflect changing circumstances. Data analysis covers the final step of characterizing and interpreting research findings: Can estimates of the relations between variables be made? Can some conclusion be drawn about correlation, cause and effect, or trends over time? How uncertain are the estimates and conclusions and can that uncertainty be reduced by analyzing the data in a different way? Can computers be used to display complex results graphically for quicker or better understanding or to suggest different ways of proceeding? Advances in analysis, data representation, and research design feed into and reinforce one another in the course of actual scientific work. The intersections between methodological improvements and empirical advances are an impor- tant aspect of the multidisciplinary thrust of progress in the behavioral and . socla. . sciences. DESIGNS FOR DATA COLLECTION Four broad kinds of research designs are used in the behavioral and social sciences: experimental, survey, comparative, and ethnographic. Experimental designs, in either the laboratory or field settings, systematically manipulate a few variables while others that may affect the outcome are held constant, randomized, or otherwise controlled. The purpose of randomized experiments is to ensure that only one or a few variables can systematically affect the results, so that causes can be attributed. Survey designs include the collection and analysis of data from censuses, sample surveys, and longitudinal studies and the examination of various relationships among the observed phe-

170 / The Behavioral and Social Sciences nomena. Randomization plays a different role here than in experimental de- signs: it is used to select members of a sample so that the sample is as repre- sentative of the whole population as possible. Comparative designs involve the retrieval of evidence that is recorded in the flow of current or past events in different times or places and the interpretation and analysis of this evidence. Ethnographic designs, also known as participant-observation designs, involve a researcher in intensive and direct contact with a group, community, or pop- ulation being studied, through participation, observation, and extended inter- vlewlng. Experimental Designs Laboratory Experiments Laboratory experiments underlie most of the work reported in Chapter 1, significant parts of Chapter 2, and some of the newest lines of research in Chapter 3. Laboratory experiments extend and adapt classical methods of de- sign first developed, for the most part, in the physical and life sciences and agricultural research. Their main feature is the systematic and independent manipulation of a few variables and the strict control or randomization of all other variables that might affect the phenomenon under study. For example, some studies of animal motivation involve the systematic manipulation of amounts of food and feeding schedules while other factors that may also affect motiva- tion, such as body weight, deprivation, and so on, are held constant. New designs are currently coming into play largely because of new analytic and computational methods (discussed below, in "Advances in Statistical Inference and Analysis". Two examples of empirically important issues that demonstrate the need for broadening classical experimental approaches are open-ended responses and lack of independence of successive experimental trials. The first concerns the design of research protocols that do not require the strict segregation of the events of an experiment into well-defined trials, but permit a subject to respond at will. These methods are needed when what is of interest is how the respond- ent chooses to allocate behavior in real time and across continuously available alternatives. Such empirical methods have long been used, but they can gen- erate very subtle and difficult problems in experimental design and subsequent analysis. As theories of allocative behavior of all sorts become more sophisti- cated and precise, the experimental requirements become more demanding, so the need to better understand and solve this range of design issues is an outstanding challenge to methodological ingenuity. The second issue arises in repeated-trial designs when the behavior on suc- cessive trials, even if it does not exhibit a secular trend (such as a learning curve), is markedly influenced by what has happened in the preceding trial or trials. The more naturalistic the experiment and the more sensitive the meas-

Methods of Data Collection, Representation, and Analysis / 171 urements taken, the more likely it is that such effects will occur. But such sequential dependencies in observations cause a number of important concep- tual and technical problems in summarizing the data and in testing analytical models, which are not yet completely understood. In the absence of clear solutions, such effects are sometimes ignored by investigators, simplifying the data analysis but leaving residues of skepticism about the reliability and sig- nificance of the experimental results. With continuing development of sensitive measures in repeated-trial designs, there is a growing need for more advanced concepts and methods for dealing with experimental results that may be influ- enced by sequential dependencies. Randomized Field Experiments The state of the art in randomized field experiments, in which different policies or procedures are tested in controlled trials under real conditions, has advanced dramatically over the past two decades. Problems that were once considered major methodological obstacles such as implementing random- ized field assignment to treatment and control groups and protecting the ran- domization procedure from corruption have been largely overcome. While state-of-the-art standards are not achieved in every field experiment, the com- mitment to reaching them is rising steadily, not only among researchers but also among customer agencies and sponsors. The health insurance experiment described in Chapter 2 is an example of a major randomized field experiment that has had and will continue to have important policy reverberations in the design of health care financing. Field experiments with the negative income tax (guaranteed minimum income) con- ducted in the 1970s were significant in policy debates, even before their com- pletion, and provided the most solid evidence available on how tax-based income support programs and marginal tax rates can affect the work incentives and family structures of the poor. Important field experiments have also been carried out on alternative strategies for the prevention of delinquency and other criminal behavior, reform of court procedures, rehabilitative programs in men- tal health, family planning, and special educational programs, among other areas. In planning field experiments, much hinges on the definition and design of the experimental cells, the particular combinations needed of treatment and control conditions for each set of demographic or other client sample charac- teristics, including specification of the minimum number of cases needed in each cell to test for the presence of effects. Considerations of statistical power, client availability, and the theoretical structure of the inquiry enter into such specifications. Current important methodological thresholds are to find better ways of predicting recruitment and attrition patterns in the sample, of designing experiments that will be statistically robust in the face of problematic sample

172 / The Behavioral and Social Sciences recruitment or excessive attrition, and of ensuring appropriate acquisition and analysis of data on the attrition component of the sample. Also of major significance are improvements in integrating detailed process and outcome measurements in field experiments. To conduct research on pro- gram effects under held conditions requires continual monitoring to determine exactly what is being done—the process how it corresponds to what was projected at the outset. Relatively unintrusive, inexpensive, and effective im- plementation measures are of great interest. There is, in parallel, a growing emphasis on designing experiments to evaluate distinct program components in contrast to summary measures of net program effects. Finally, there is an important opportunity now for further theoretical work to model organizational processes in social settings and to design and select outcome variables that, in the relatively short time of most field experiments, can predict longer-term effects: For example, in job-training programs, what are the effects on the community (role models, morale, referral networks) or on individual skills, motives, or knowledge levels that are likely to translate into sustained changes in career paths and income levels? Survey Designs Many people have opinions about how societal mores, economic conditions, and social programs shape lives and encourage or discourage various kinds of behavior. People generalize from their own cases, and from the groups to which they belong, about such matters as how much it costs to raise a child, the extent to which unemployment contributes to divorce, and so on. In fact, however, effects vary so much from one group to another that homespun generalizations are of little use. Fortunately, behavioral and social scientists have been able to bridge the gaps between personal perspectives and collective realities by means of survey research. In particular, governmental information systems include volumes of extremely valuable survey data, and the facility of modern com- puters to store, disseminate, and analyze such data has significantly improved empirical tests and led to new understandings of social processes. Within this category of research designs, two major types are distinguished: repeated cross-sectional surveys and longitudinal panel surveys. In addition, and cross-cutting these types, there is a major effort under way to improve and refine the quality of survey data by investigating features of human memory and of question formation that affect survey response. Repeated cross-sectional designs can either attempt to measure an entire population as does the oldest U.S. example, the national decennial census or they can rest on samples drawn from a population. The general principle is to take independent samples at two or more times, measuring the variables of interest, such as income levels, housing plans, or opinions about public affairs, in the same way. The General Social Survey, collected by the National Opinion Research Center with National Science Foundation support, is a repeated cross-

Methods of Data Collection, Representation, and Analysis / 173 sectional data base that was begun in 1972. One methodological question of particular salience in such data is how to adjust for nonresponses and "don't know" responses. Another is how to deal with self-selection bias. For example, to compare the earnings of women and men in the labor force, it would be mistaken to first assume that the two samples of labor-force participants are randomly selected from the larger populations of men and women; instead, one has to consider and incorporate in the analysis the factors that determine who is in the labor force. In longitudinal panels, a sample is drawn at one point in time and the relevant variables are measured at this and subsequent times for the same people. In more complex versions, some fraction of each panel may be replaced or added to periodically, such as expanding the sample to include households formed by the children of the original sample. An example of panel data developed in this way is the Panel Study of Income Dynamics (PSID), conducted by the University of Michigan since 1968 (discussed in Chapter 35. Comparing the fertility or income of different people in different circum- stances at the same time to kind correlations always leaves a large proportion of the variability unexplained, but common sense suggests that much of the unexplained variability is actually explicable. There are systematic reasons for individual outcomes in each person's past achievements, in parental models, upbringing, and earlier sequences of experiences. Unfortunately, asking people about the past is not particularly helpful: people remake their views of the past to rationalize the present and so retrospective data are often of uncertain va- lidity. In contrast, generation-long longitudinal data allow readings on the sequence of past circumstances uncolored by later outcomes. Such data are uniquely useful for studying the causes and consequences of naturally occur- ring decisions and transitions. Thus, as longitudinal studies continue, quant,i- tative analysis is becoming feasible about such questions as: How are the de- cisions of individuals affected by parental experience? Which aspects of early decisions constrain later opportunities? And how does detailed background experience leave its imprint? Studies like the two-decade-long PSID are bring- ing within grasp a complete generational cycle of detailed data on fertility, work life, household structure, and income. Advances in Longitudinal Designs Large-scale longitudinal data collection projects are uniquely valuable as vehicles for testing and improving survey research methodology. In ways that lie beyond the scope of a cross-sectional survey, longitudinal studies can some- times be designed without significant detriment to their substantive inter- ests to facilitate the evaluation and upgrading of data quality; the analysis of relative costs and effectiveness of alternative techniques of inquiry; and the standardization or coordination of solutions to problems of method, concept, and measurement across different research domains.

174 / The Behavioral and Social Sciences Some areas of methodological improvement include discoveries about the impact of interview mode on response (mail, telephone, face-to-face); the effects of nonresponse on the representativeness of a sample (due to respondents' refusal or interviewers' failure to contact); the effects on behavior of continued participation over time in a sample survey; the value of alternative methods of adjusting for nonresponse and incomplete observations (such as imputation of missing data, variable case weighting); the impact on response of specifying different recall periods, varying the intervals between interviews, or changing the length of interviews; and the comparison and calibration of results obtained by longitudinal surveys, randomized field experiments, laboratory studies, one- time surveys, and administrative records. It should be especially noted that incorporating improvements in method- ology and data quality has been and will no doubt continue to be crucial to the growing success of longitudinal studies. Panel designs are intrinsically more vulnerable than other designs to statistical biases due to cumulative item non- response, sample attrition, time-in-sample effects, and error margins in re- peated measures, all of which may produce exaggerated estimates of change. Over time, a panel that was initially representative may become much less representative of a population, not only because of attrition in the sample, but also because of changes in immigration patterns, age structure, and the like. Longitudinal studies are also subject to changes in scientific and societal con- texts that may create uncontrolled drifts over time in the meaning of nominally stable questions or concepts as well as in the underlying behavior. Also, a natural tendency to expand over time the range of topics and thus the interview lengths, which increases the burdens on respondents, may lead to deterioration of data quality or relevance. Careful methodological research to understand and overcome these problems has been done, and continued work as a com- ponent of new longitudinal studies is certain to advance the overall state of the art. Longitudinal studies are sometimes pressed for evidence they are not de- signed to produce: for example, in important public policy questions concern- ing the impact of government programs in such areas as health promotion, disease prevention, or criminal justice. By using research designs that combine field experiments (with randomized assignment to program and control con- ditions) and longitudinal surveys, one can capitalize on the strongest merits of each: the experimental component provides stronger evidence for casual state- ments that are critical for evaluating programs and for illuminating some fun- damental theories; the longitudinal component helps in the estimation of long- term program effects and their attenuation. Coupling experiments to ongoing longitudinal studies is not often feasible, given the multiple constraints of not disrupting the survey, developing all the complicated arrangements that go into a large-scale field experiment, and having the populations of interest over- lap in useful ways. Yet opportunities to join field experiments to surveys are

Methods of Data Collection, Representation, and Analysis / 175 of great importance. Coupled studies can produce vital knowledge about the empirical conditions under which the results of longitudinal surveys turn out to be similar to—or divergent from those produced by randomized field experiments. A pattern of divergence and similarity has begun to emerge in coupled studies; additional cases are needed to understand why some naturally occurring social processes and longitudinal design features seem to approxi- mate formal random allocation and others do not. The methodological impli- cations of such new knowledge go well beyond program evaluation and survey research. These findings bear directly on the confidence scientists and oth- ers can have in conclusions from observational studies of complex behavioral and social processes, particularly ones that cannot be controlled or simulated within the confines of a laboratory environment. Memory and the Framing of questions A very important opportunity to improve survey methods lies in the reduc- tion of nonsampling error due to questionnaire context, phrasing of questions, and, generally, the semantic and social-psychological aspects of surveys. Survey data are particularly affected by the fallibility of human memory and the sen- sitivity of respondents to the framework in which a question is asked. This sensitivity is especially strong for certain types of attitudinal and opinion ques- tions. Efforts are now being made to bring survey specialists into closer contact with researchers working on memory function, knowledge representation, and language in order to uncover and reduce this kind of error. Memory for events is often inaccurate, biased toward what respondents believe to be true or should be true—about the world. In many cases in which data are based on recollection, improvements can be achieved by shifting to techniques of structured interviewing and calibrated forms of memory elic- itation, such as specifying recent, brief time periods (for example, in the last seven days) within which respondents recall certain types of events with ac- ceptable accuracy. Experiments on individual decision making show that the way a question is framed predictably alters the responses. Analysts of survey data find that some small changes in the wording of certain kinds of questions can produce large differences in the answers, although other wording changes have little effect. Even simply changing the order in which some questions are presented can produce large differences, although for other questions the order of presenta- tion does not matter. For example, the following questions were among those asked in one wave of the General Social Survey: · "Taking things altogether, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy?" · "Taken altogether how would you say things are these days—would you say you are very happy, pretty happy, or not too happy?"

176 / The Behavioral and Social Sciences Presenting this sequence in both directions on different forms showed that the order affected answers to the general happiness question but did not change the marital happiness question: responses to the specific issue swayed subse- quent responses to the general one, but not vice versa. The explanations for and implications of such order effects on the many kinds of questions and sequences that can be used are not simple matters. Further experimentation on the design of survey instruments promises not only to improve the accuracy and reliability of survey research, but also to advance understanding of how people think about and evaluate their behavior from day to day. Comparative Designs Both experiments and surveys involve interventions or questions by the scientist, who then records and analyzes the responses. In contrast, many bodies of social and behavioral data of considerable value are originally derived from records or collections that have accumulated for various nonscientific reasons, quite often administrative in nature, in firms, churches, military or- ganizations, and governments at all levels. Data of this kind can sometimes be subjected to careful scrutiny, summary, and inquiry by historians and social scientists, and statistical methods have increasingly been used to develop and evaluate inferences drawn from such data. Some of the main comparative approaches are'. cross-national aggregate comparisons, selective comparison of a limited number of cases, and historical case studies. Among the more striking problems facing the scientist using such data are the vast differences in what has been recorded by different agencies whose behavior is being compared (this is especially true for parallel agencies in different nations), the highly unrepresentative or idiosyncratic sampling that can occur in the collection of such data, and the selective preservation and destruction of records. Means to overcome these problems form a substantial methodological research agenda in comparative research. An example of the method of cross-national aggregative comparisons is found in investigations by political scientists and sociologists of the factors that underlie differences in the vitality of institutions of political democracy in different societies. Some investigators have stressed the existence of a large middle class, others the level of education of a population, and still others the development of systems of mass communication. In cross-national aggregate comparisons, a large number of nations are arrayed according to some measures of political democracy and then attempts are made to ascertain the strength of correlations between these and the other variables. In this line of analysis it is possible to use a variety of statistical cluster and regression techniques to isolate and assess the possible impact of certain variables on the institutions under study. While this kind of research is cross-sectional in character, statements about historical processes are often invoked to explain the correlations.

Methods of Data Collection, Representation, and Analysis / 177 More limited selective comparisons, applied by many of the classic theorists, involve asking similar kinds of questions but over a smaller range of societies. Why did democracy develop in such different ways in America, France, and England? Why did northeastern Europe develop rational bourgeois capitalism, in contrast to the Mediterranean and Asian nations? Modern scholars have turned their attention to explaining, for example, differences among types of fascism between the two World Wars, and similarities and differences among modern state welfare systems, using these comparisons to unravel the salient causes. The questions asked in these instances are inevitably historical ones. Historical case studies involve only one nation or region, and so they may not be geographically comparative. However, insofar as they involve tracing the transformation of a society's major institutions and the role of its main shaping events, they involve a comparison of different periods of a nation's or a region's history. The goal of such comparisons is to give a systematic account of the relevant differences. Sometimes, particularly with respect to the ancient societies, the historical record is very sparse, and the methods of history and archaeology mesh in the reconstruction of complex social arrangements and patterns of change on the basis of few fragments. Like all research designs, comparative ones have distinctive vulnerabilities and advantages: One of the main advantages of using comparative designs is that they greatly expand the range of data, as well as the amount of variation in those data, for study. Consequently, they allow for more encompassing explanations and theories that can relate highly divergent outcomes to one another in the same framework. They also contribute to reducing any cultural biases or tendencies toward parochialism among scientists studying common human phenomena. One main vulnerability in such designs arises from the problem of achieving comparability. Because comparative study involves studying societies and other units that are dissimilar from one another, the phenomena under study usually occur in very different contexts—so different that in some cases what is called an event in one society cannot really be regarded as the same type of event in another. For example, a vote in a Western democracy is different from a vote in an Eastern bloc country, and a voluntary vote in the United States means something different from a compulsory vote in Australia. These circumstances make for interpretive difficulties in comparing aggregate rates of voter turnout in different countries. The problem of achieving comparability appears in historical analysis as well. For example, changes in laws and enforcement and recording procedures over time change the definition of what is and what is not a crime, and for that reason it is difficult to compare the crime rates over time. Comparative re- searchers struggle with this problem continually, working to fashion equivalent measures; some have suggested the use of different measures (voting, letters to the editor, street demonstration) in different societies for common variables

178 / The Behavioral and Social Sciences (political participation), to try to take contextual factors into account and to achieve truer comparability. A second vulnerability is controlling variation. Traditional experiments make conscious and elaborate efforts to control the variation of some factors and thereby assess the causal significance of others. In surveys as well as experi- ments, statistical methods are used to control sources of variation and assess suspected causal significance. In comparative and historical designs, this kind of control is often difficult to attain because the sources of variation are many and the number of cases few. Scientists have made efforts to approximate such control in these cases of "many variables, small N." One is the method of paired comparisons. If an investigator isolates 15 American cities in which racial violence has been recurrent in the past 30 years, for example, it is helpful to match them with IS cities of similar population size, geographical region, and size of minorities- such characteristics are controls—and then search for sys- tematic differences between the two sets of cities. Another method is to select, for comparative purposes, a sample of societies that resemble one another in certain critical ways, such as size, common language, and common level of development, thus attempting to hold these factors roughly constant, and then seeking explanations among other factors in which the sampled societies differ from one another. Ethnographic Designs Traditionally identified with anthropology, ethnographic research designs are playing increasingly significant roles in most of the behavioral and social sciences. The core of this methodology is participant-observation, in which a researcher spends an extended period of time with the group under study, ideally mastering the local language, dialect, or special vocabulary, and partic- ipating in as many activities of the group as possible. This kind of participant- observation is normally coupled with extensive open-ended interviewing, in which people are asked to explain in depth the rules, norms, practices, and beliefs through which (from their point of view) they conduct their lives. A principal aim of ethnographic study is to discover the premises on which those rules, norms, practices, and beliefs are built. The use of ethnographic designs by anthropologists has contributed signif- icantly to the building of knowledge about social and cultural variation. And while these designs continue to center on certain long-standing features— extensive face-to-face experience in- the community, linguistic competence, participation, and open-ended interviewing- there are newer trends in eth- nographic work. One major trend concerns its scale. Ethnographic methods were originally developed largely for studying small-scale groupings known variously as village, folk, primitive, preliterate, or simple societies. Over the decades, these methods have increasingly been applied to the study of small

Methods of Data Collection, Representation, and Analysis / 179 groups and networks within modern (urban, industrial, complex) society, in- cluding the contemporary United States. The typical subjects of ethnographic study in modern society are small groups or relatively small social networks, such as outpatient clinics, medical schools, religious cults and churches, ethn- ically distinctive urban neighborhoods, corporate offices and factories, and government bureaus and legislatures. As anthropologists moved into the study of modern societies, researchers in other disciplines particularly sociology, psychology, and political science- began using ethnographic methods to enrich and focus their own insights and findings. At the same time, studies of large-scale structures and processes have been aided by the use of ethnographic methods, since most large-scale changes work their way into the fabric of community, neighborhood, and family, af- fecting the daily lives of people. Ethnographers have studied, for example, the impact of new industry and new forms of labor in "backward" regions; the impact of state-level birth control policies on ethnic groups; and the impact on residents in a region of building a dam or establishing a nuclear waste dump. Ethnographic methods have also been used to study a number of social pro- cesses that lend themselves to its particular techniques of observation and interview—processes such as the formation of class and racial identities, bu- reaucratic behavior, legislative coalitions and outcomes, and the formation and shifting of consumer tastes. Advances in structured interviewing (see above) have proven especially pow- erful in the study of culture. Techniques for understanding kinship systems, concepts of disease, color terminologies, ethnobotany, and ethnozoology have been radically transformed and strengthened by coupling new interviewing methods with modem measurement and scaling techniques (see below). These techniques have made possible more precise comparisons among cultures and identification of the most competent and expert persons within a culture. The next step is to extend these methods to study the ways in which networks of propositions (such as boys like sports, girls like babies) are organized to form belief systems. Much evidence suggests that people typically represent the world around them by means of relatively complex cognitive models that in- volve interlocking propositions. The techniques of scaling have been used to develop models of how people categorize objects, and they have great potential for further development, to analyze data pertaining to cultural propositions. Ideological Systems Perhaps the most fruitful area for the application of ethnographic methods in recent years has been the systematic study of ideologies in modern society. Earlier studies of ideology were in small-scale societies that were rather ho- mogeneous. In these studies researchers could report on a single culture, a uniform system of beliefs and values for the society as a whole. Modern societies are much more diverse both in origins and number of subcultures, related to

180 / The Behavioral and Social Sciences different regions, communities, occupations, or ethnic groups. Yet these sub- cultures and ideologies share certain underlying assumptions or at least must find some accommodation with the dominant value and belief systems in the society. The challenge is to incorporate this greater complexity of structure and process into systematic descriptions and interpretations. One line of work carried out by researchers has tried to track the ways in which ideologies are created, transmitted, and shared among large populations that have tradition- ally lacked the social mobility and communications technologies of the West. This work has concentrated on large-scale civilizations such as China, India, and Central America. Gradually, the focus has generalized into a concern with the relationship between the great traditions—the central lines of cosmopolitan Confucian, Hindu, or Mayan culture, including aesthetic standards, irrigation technologies, medical systems, cosmologies and calendars, legal codes, poetic genres, and religious doctrines and rites and the little traditions, those iden- tified with rural, peasant communities. How are the ideological doctrines and cultural values of the urban elites, the great traditions, transmitted to local communities? How are the little traditions, the ideas from the more isolated, less literate, and politically weaker groups in society, transmitted to the elites? India and southern Asia have been fruitful areas for ethnographic research on these questions. The great Hindu tradition was present in virtually all local contexts through the presence of high-caste individuals in every community. It operated as a pervasive standard of value for all members of society, even in the face of strong little traditions. The situation is surprisingly akin to that of modern, industrialized societies. The central research questions are the degree and the nature of penetration of dominant ideology, even in groups that appear marginal and subordinate and have no strong interest in sharing the dominant value system. In this connection the lowest and poorest occupational caste— the untouchables- serves as an ultimate test of the power of ideology and cultural beliefs to unify complex hierarchical social systems. Historical Reconstruction Another current trend in ethnographic methods is its convergence with archival methods. One joining point is the application of descriptive and in- terpretative procedures used by ethnographers to reconstruct the cultures that created historical documents, diaries, and other records, to interview history, so to speak. For example, a revealing study showed how the Inquisition in the Italian countryside between the 1570s and 1640s gradually worked subtle changes in an ancient fertility cult in peasant communities; the peasant beliefs and rituals assimilated many elements of witchcraft after learning them from their persecutors. A good deal of social history particularly that of the fam- ily has drawn on discoveries made in the ethnographic study of primitive societies. As described in Chapter 4, this particular line of inquiry rests on a marriage of ethnographic, archival, and demographic approaches.

Methods of Data Collection, Representation, and Analysis / 181 Other lines of ethnographic work have focused on the historical dimensions of nonliterate societies. A strikingly successful example in this kind of effort is a study of head-hunting. By combining an interpretation of local oral tradition with the fragmentary observations that were made by outside observers (such as missionaries, traders, colonial officials), historical fluctuations in the rate and significance of head-hunting were shown to be partly in response to such international forces as the great depression and World War II. Researchers are also investigating the ways in which various groups in contemporary societies invent versions of traditions that may or may not reflect the actual history of the group. This process has been observed among elites seeking political and cultural legitimation and among hard-pressed minorities (for example, the Basque in Spain, the Welsh in Great Britain) seeking roots and political mo- . .1 . . . olllzatlon in a arger society. Ethnography is a powerful method to record, describe, and interpret the system of meanings held by groups and to discover how those meanings affect the lives of group members. It is a method well adapted to the study of situations in which people interact with one another and the researcher can interact with them as well, so that information about meanings can be evoked and observed. Ethnography is especially suited to exploration and elucidation of unsuspected connections; ideally, it is used in combination with other methods—experi- mental, survey, or comparative to establish with precision the relative strengths and weaknesses of such connections. By the same token, experimental, survey, and comparative methods frequently yield connections, the meaning of which is unknown; ethnographic methods are a valuable way to determine them. MODELS FOR REPRESENTING PHENOMENA The objective of any science is to uncover the structure and dynamics of the phenomena that are its subject, as they are exhibited in the data. Scientists continuously try to describe possible structures and ask whether the data can, with allowance for errors of measurement, be described adequately in terms of them. Over a long time, various families of structures have recurred throughout many fields of science; these structures have become objects of study in their own right, principally by statisticians, other methodological specialists, applied mathematicians, and philosophers of logic and science. Methods have evolved to evaluate the adequacy of particular structures to account for particular types of data. In the interest of clarity we discuss these structures in this section and the analytical methods used for estimation and evaluation of them in the next section, although in practice they are closely intertwined. A good deal of mathematical and statistical modeling attempts to describe the relations, both structural and dynamic, that hold among variables that are presumed to be representable by numbers. Such models are applicable in the behavioral and social sciences only to the extent that appropriate numerical

182 / The Behavioral and Social Sciences measurement can be devised for the relevant variables. In many studies the phenomena in question and the raw data obtained are not intrinsically nu- merical, but qualitative, such as ethnic group identifications. The identifying numbers used to code such questionnaire categories for computers are no more than labels, which could just as well be letters or colors. One key question is whether there is some natural way to move from the qualitative aspects of such data to a structural representation that involves one of the well-understood numerical or geometric models or whether such an attempt would be inherently inappropriate for the data in question. The decision as to whether or not particular empirical data can be represented in particular numerical or more complex structures is seldom simple, and strong intuitive biases or a priori assumptions about what can and cannot be done may be misleading. Recent decades have seen rapid and extensive development and application of analytical methods attuned to the nature and complexity of social science data. Examples of nonnumerical modeling are increasing. Moreover, the wide- spread availability of powerful computers is probably leading to a qualitative revolution, it is affecting not only the ability to compute numerical solutions to numerical models, but also to work out the consequences of all sorts of structures that do not involve numbers at all. The following discussion gives some indication of the richness of past progress and of future prospects al- though it is by necessity far from exhaustive. In describing some of the areas of new and continuing research, we have organized this section on the basis of whether the representations are funda- mentally probabilistic or not. A further useful distinction is between represen- tations of data that are highly discrete or categorical in nature (such as whether a person is male or female) and those that are continuous in nature (such as a person's height). Of course, there are intermediate cases involving both types of variables, such as color stimuli that are characterized by discrete hues (red, green) and a continuous luminance measure. Probabilistic models lead very naturally to questions of estimation and statistical evaluation of the correspon- dence between data and model. Those that are not probabilistic involve addi- tional problems of dealing with and representing sources of variability that are not explicitly modeled. At the present time, scientists understand some aspects of structure, such as geometries, and some aspects of randomness, as embodied in probability models, but do not yet adequately understand how to put the two together in a single unibed model. Table 5-1 outlines the way we have organized this discussion and shows where the examples in this section lie. Probability Models Some behavioral and social sciences variables appear to be more or less continuous, for example, utility of goods, loudness of sounds, or risk associated with uncertain alternatives. Many other variables, however, are inherently cat-

Methods of Data Collection, Representation, and Analysis / 183 TABLE S- 1 A Classification of Structural Models Nature of the Variables Nature of the Representation Categorical Continuous Probabilistic Log-linear and Multi-item related models measurement Event histories Nonlinear, nonadditive models Geometric and Clustering Scaling algebraic Network models Ordered factorial systems egorical, often with only two or a few values possible: for example, whether a person is in or out of school, employed or not employed, identifies with a major political party or political ideology. And some variables, such as moral attitudes, are typically measured in research with survey questions that allow only categorical responses. Much of the early probability theory was formulated only for continuous variables; its use with categorical variables was not really justified, and in some cases it may have been misleading. Recently, very sig- nificant advances have been made in how to deal explicitly with categorical variables. This section first describes several contemporary approaches to models involving categorical variables, followed by ones involving continuous repre- sentations. Log-Linear Models for Categorical Variables Many recent models for analyzing categorical data of the kind usually dis- played as counts (cell frequencies) in multidimensional contingency tables are subsumed under the general heading of log-linear models, that is, linear models in the natural logarithms of the expected counts in each cell in the table. These recently developed forms of statistical analysis allow one to partition variability due to various sources in the distribution of categorical attributes, and to isolate the effects of particular variables or combinations of them. Present log-linear models were first developed and used by statisticians and sociologists and then found extensive application in other social and behavioral sciences disciplines. When applied, for instance, to the analysis of social mo- bility, such models separate factors of occupational supply and demand from other factors that impede or propel movement up and down the social hier- archy. With such models, for example, researchers discovered the surprising fact that occupational mobility patterns are strikingly similar in many nations of the world (even among disparate nations like the United States and most of the Eastem European socialist countries), and from one time period to another, once allowance is made for differences in the distributions of occupations. The

184 / The Behavioral and Social Sciences log-linear and related kinds of models have also made it possible to identify and analyze systematic differences in mobility among nations and across time. As another example of applications, psychologists and others have used log- linear models to analyze attitudes and their determinants and to link attitudes to behavior. These methods have also diffused to and been used extensively in the medical and biological sciences. Regression Modelsfor Categorical Variables Models that permit one variable to be explained or predicted by means of others, called regression models, are the workhorses of much applied statistics; this is especially true when the dependent (explained) variable is continuous. For a two-valued dependent variable, such as alive or dead, models and ap- proximate theory and computational methods for one explanatory variable were developed in biometry about 50 years ago. Computer programs able to handle many explanatory variables, continuous or categorical, are readily avail- able today. Even now, however, the accuracy of the approximate theory on . given c .ata IS an open question. Using classical utility theory, economists have developed discrete choice models that turn out to be somewhat related to the log-linear and categorical regression models. Models for limited dependent variables, especially those that cannot take on values above or below a certain level (such as weeks unemployed, number of children, and years of schooling) have been used profitably in economics and in some other areas. For example, censored normal variables (called tobits in economics), in which observed values outside certain limits are simply counted, have been used in studying decisions to go on in school. It will require further research and development to incorporate infor- mation about limited ranges of variables fully into the main multivariate meth- odologies. In addition, with respect to the assumptions about distribution and functional form conventionally made in discrete response models, some new methods are now being developed that show promise of yielding reliable in- ferences without making unrealistic assumptions; further research in this area . ~ promises slgnl~cant progress. One problem arises from the fact that many of the categorical variables collected by the major data bases are ordered. For example, attitude surveys frequently use a 3-, 5-, or 7-point scale (from high to low) without specifying numerical intervals between levels. Social class and educational levels are often described by ordered categories. Ignoring order information, which many tra- ditional statistical methods do, may be inefficient or inappropriate, but replac- ing the categories by successive integers or other arbitrary scores may distort the results. (For additional approaches to this question, see sections below on ordered structures.) Regression-like analysis of ordinal categorical variables is quite well developed, but their multivariate analysis needs further research. New log-bilinear models have been proposed, but to date they deal specifically

Methods of Data Collection, Representation, and Analysis / 18S with only two or three categorical variables. Additional research extending the new models, improving computational algorithms, and integrating the models with work on scaling promise to lead to valuable new knowledge. Models for Event Histories Event-history studies yield the sequence of events that respondents to a survey sample experience over a period of time; for example, the timing of marriage, childbearing, or labor force participation. Event-history data can be used to study educational progress, demographic processes (migration, fertility, and mortality), mergers of firms, labor market behavior? and even riots, strikes, and revolutions. As interest in such data has grown, many researchers have turned to models that pertain to changes in probabilities over time to describe when and how individuals move among a set of qualitative states. Much of the progress in models for event-history data builds on recent developments in statistics and biostatistics for life-time, failure-time, and haz- ard models. Such models permit the analysis of qualitative transitions in a population whose members are undergoing partially random organic deterio- ration, mechanical wear, or other risks over time. With the increased com- plexity of event-history data that are now being collected, and the extension of event-history data bases over very long periods of time, new problems arise that cannot be effectively handled by older types of analysis. Among the prob- lems are repeated transitions, such as between unemployment and employment or marriage and divorce; more than one time variable (such as biological age, calendar time, duration in a stage, and time exposed to some specified con- dition); latent variables (variables that are explicitly modeled even though not observed); gaps in the data; sample attrition that is not randomly distributed over the categories; and respondent difficulties in recalling the exact timing of events. Models for Multiple-Item Measurement For a variety of reasons, researchers typically use multiple measures (or multiple indicators) to represent theoretical concepts. Sociologists, for example, often rely on two or more variables (such as occupation and education) to measure an individual's socioeconomic position; educational psychologists or- dinarily measure a student's ability with multiple test items. Despite the fact that the basic observations are categorical, in a number of applications this is interpreted as a partitioning of something continuous. For example, in test theory one thinks of the measures of both item difficulty and respondent ability as continuous variables, possibly multidimensional in character. Classical test theory and newer item-response theories in psychometrics deal with the extraction of information from multiple measures. Testing, which is a major source of data in education and other areas, results in millions of test

186 / The Behavioral and Social Sciences items stored in archives each year for purposes ranging from college admissions to job-training programs for industry. One goal of research on such test data is to be able to make comparisons among persons or groups even when different test items are used. Although the information collected from each respondent is intentionally incomplete in order to keep the tests short and simple, item- response techniques permit researchers to reconstitute the fragments into an accurate picture of overall group proficiencies. These new methods provide a better theoretical handle on individual differences, and they are expected to be extremely important in developing and using tests. For example, they have been used in attempts to equate different forms of a test given in successive waves during a year, a procedure made necessary in large-scale testing programs by legislation requiring disclosure of test-scoring keys at the time results are given. An example of the use of item-response theory in a significant research effort is the National Assessment of Educational Progress (NAEP). The goal of this project is to provide accurate, nationally representative information on the average (rather than individual) proficiency of American children in a wide variety of academic subjects as they progress through elementary and secondary school. This approach is an improvement over the use of trend data on uni- versity entrance exams, because NAEP estimates of academic achievements (by broad characteristics such as age, grade, region, ethnic background, and so on) are not distorted by the self-selected character of those students who seek admission to college, graduate, and professional programs. Item-response theory also forms the basis of many new psychometric in- struments, known as computerized adaptive testing, currently being imple- mented by the U.S. military services and under additional development in many testing organizations. In adaptive tests, a computer program selects items for each examiner based upon the examinee's success with previous items. Gen- erally, each person gets a slightly different set of items and the equivalence of scale scores is established by using item-response theory. Adaptive testing can greatly reduce the number of items needed to achieve a given level of meas- urement accuracy. Nonlinear, Nonadditive Models Virtually all statistical models now in use impose a linearity or additivity assumption of some kind, sometimes after a nonlinear transformation of var- iables. Imposing these forms on relationships that do not, in fact, possess them may well result in false descriptions and spurious effects. Unwary users, es- pecially of computer software packages, can easily be misled. But more realistic nonlinear and nonadditive multivariate models are becoming available. Exten- sive use with empirical data is likely to force many changes and enhancements in such models and stimulate quite different approaches to nonlinear multi- variate analysis in the next decade.

Methods of Data Collection, Representation, and Analysis / 187 Geometric and Algebraic Models Geometric and algebraic models attempt to describe underlying structural relations among variables. In some cases they are part of a probabilistic ap- proach, such as the algebraic models underlying regression or the geometric representations of correlations between items in a technique called factor anal- ysis. In other cases, geometric and algebraic models are developed without explicitly modeling the element of randomness or uncertainty that is always present in the data. Although this latter approach to behavioral and social sciences problems has been less researched than the probabilistic one, there are some advantages in developing the structural aspects independent of the statistical ones. We begin the discussion with some inherently geometric rep- resentations and then turn to numerical representations for ordered data. Although geometry is a huge mathematical topic, little of it seems directly applicable to the kinds of data encountered in the behavioral and social sci- ences. A major reason is that the primitive concepts normally used in geome- try points, lines, coincidence—do not correspond naturally to the kinds of qualitative observations usually obtained in behavioral and social sciences con- texts. Nevertheless, since geometric representations are used to reduce bodies of data, there is a real need to develop a deeper understanding of when such representations of social or psychological data make sense. Moreover, there is a practical need to understand why geometric computer algorithms, such as those of multidimensional scaling, work as well as they apparently do. A better understanding of the algorithms will increase the efficiency and appropriate- ness of their use, which becomes increasingly important with the widespread availability of scaling programs for microcomputers. Scaling Over the past 50 years several kinds of well-understood scaling techniques have been developed and widely used to assist in the search for appropriate geometric representations of empirical data. The whole field of scaling is now entering a critical juncture in terms of unifying and synthesizing what earlier appeared to be disparate contributions. Within the past few years it has become apparent that several major methods of analysis, including some that are based on probabilistic assumptions, can be unified under the rubric of a single gen- eralized mathematical structure. For example, it has recently been demon- strated that such diverse approaches as nonmetric multidimensional scaling, principal-components analysis, factor analysis, correspondence analysis, and log-linear analysis have more in common in terms of underlying mathematical structure than had earlier been realized. Nonmetric multidimensional scaling is a method that begins with data about the ordering established by subjective similarity (or nearness) between pairs of stimuli. The idea is to embed the stimuli into a metric space (that is, a geometry

188 / The Behavioral and Social Sciences with a measure of distance between points) in such a way that distances between points corresponding to stimuli exhibit the same ordering as do the data. This method has been successfully applied to phenomena that, on other grounds, are known to be describable in terms of a specific geometric structure; such applications were used to validate the procedures. Such validation was done, for example, with respect to the perception of colors, which are known to be describable in terms of a particular three-dimensional structure known as the Euclidean color coordinates. Similar applications have been made with Morse code symbols and spoken phonemes. The technique is now used in some biological and engineering applications, as well as in some of the social sciences, as a method of data exploration and simplification. One question of interest is how to develop an axiomatic basis for various geometries using as a primitive concept an observable such as the subject's ordering of the relative similarity of one pair of stimuli to another, which is the typical starting point of such scaling. The general task is to discover properties of the qualitative data sufficient to ensure that a mapping into the geometric structure exists and, ideally, to discover an algorithm for finding it. Some work of this general type has been carried out: for example, there is an elegant set of axioms based on laws of color matching that yields the three-dimensional vectorial representation of color space. But the more general problem of un- derstanding the conditions under which the multidimensional scaling algo- rithms are suitable remains unsolved. In addition, work is needed on under- standing more general, non-Euclidean spatial models. Ordered Factorial Systems One type of structure common throughout the sciences arises when an ordered dependent variable is affected by two or more ordered independent variables. This is the situation to which regression and analysis-of-variance models are often applied; it is also the structure underlying the familiar physical identities, in which physical units are expressed as products of the powers of other units (for example, energy has the unit of mass times the square of the unit of distance divided by the square of the unit of time). There are many examples of these types of structures in the behavioral and social sciences. One example is the ordering of preference of commodity bun- dles collections of various amounts of commodities which may be revealed directly by expressions of preference or indirectly by choices among alternative sets of bundles. A related example is preferences among alternative courses of action that involve various outcomes with differing degrees of uncertainty; this is one of the more thoroughly investigated problems because of its potential importance in decision making. A psychological example is the trade-off be- tween delay and amount of reward, yielding those combinations that are equally reinforcing. In a common, applied kind of problem, a subject is given descrip- tions of people in terms of several factors, for example, intelligence, creativity,

Methods of Data Collection, Representation, and Analysis / 189 diligence, and honesty, and is asked to rate them according to a criterion such as suitability for a particular job. In all these cases and a myriad of others like them the question is whether the regularities of the data permit a numerical representation. Initially, three types of representations were studied quite fully: the dependent variable as a sum, a product, or a weighted average of the measures associated with the independent variables. The first two representations underlie some psycholog- ical and economic investigations, as well as a considerable portion of physical measurement and modeling in classical statistics. The third representation, averaging, has proved most useful in understanding preferences among un- certain outcomes and the amalgamation of verbally described traits, as well as some physical variables. For each of these three cases adding, multiplying, and averaging re- searchers know what properties or axioms of order the data must satisfy for such a numerical representation to be appropriate. On the assumption that one or another of these representations exists, and using numerical ratings by sub- jects instead of ordering, a scaling technique called functional measurement (referring to the function that describes how the dependent variable relates to the independent ones) has been developed and applied in a number of domains. What remains problematic is how to encompass at the ordinal level the fact that some random error intrudes into nearly all observations and then to show how that randomness is represented at the numerical level; this continues to be an unresolved and challenging research issue. During the past few years considerable progress has been made in under- standing certain representations inherently different from those just discussed. The work has involved three related thrusts. The first is a scheme of classifying structures according to how uniquely their representation is constrained. The three classical numerical representations are known as ordinal, interval, and ratio scale types. For systems with continuous numerical representations and of scale type at least as rich as the ratio one, it has been shown that only one additional type can exist. A second thrust is to accept structural assumptions, like factorial ones, and to derive for each scale the possible functional relations among the independent variables. And the third thrust is to develop axioms for the properties of an order relation that leads to the possible representations. Much is now known about the possible nonadditive representations of both the muItifactor case and the one where stimuli can be combined, such as . . . . . . . com fining sounc . intensities. Closely related to this classification of structures is the question: What state- ments, formulated in terms of the measures arising in such representations, can be viewed as meaningful in the sense of corresponding to something em- pirical? Statements here refer to any scientific assertions, including statistical ones, formulated in terms of the measures of the variables and logical and mathematical connectives. These are statements for which asserting truth or /

190 / The Behavioral and Social Sciences falsity makes sense. In particular, statements that remain invariant under certain symmetries of structure have played an important role in classical geometry, dimensional analysis in physics, and in relating measurement and statistical models applied to the same phenomenon. In addition, these ideas have been used to construct models in more formally developed areas of the behavioral and social sciences, such as psychophysics. Current research has emphasized the communality of these historically independent developments and is at- tempting both to uncover systematic, philosophically sound arguments as to why invariance under symmetries is as important as it appears to be and to understand what to do when structures lack symmetry, as, for example, when variables have an inherent upper bound. Clustering Many subjects do not seem to be correctly represented in terms of distances in continuous geometric space. Rather, in some cases, such as the relations among meanings of words which is of great interest in the study of memory representations a description in terms of tree-like, hierarchial structures ap- pears to be more illuminating. This kind of description appears appropriate both because of the categorical nature of the judgments and the hierarchial, rather than trade-off, nature of the structure. Individual items are represented as the terminal nodes of the tree, and groupings by different degrees of similarity are shown as intermediate nodes, with the more general groupings occurring nearer the root of the tree. Clustering techniques, requiring considerable com- putational power, have been and are being developed. Some successful appli- cations exist, but much more refinement is anticipated. Network Models Several other lines of advanced modeling have progressed in recent years, opening new possibilities for empirical specification and testing of a variety of theories. In social network data, relationships among units, rather than the units themselves, are the primary objects of study: friendships among persons, trade ties among nations, cocitation clusters among research scientists, inter- locking among corporate boards of directors. Special models for social network data have been developed in the past decade, and they give, among other things, precise new measures of the strengths of relational ties among units. A major challenge in social network data at present is to handle the statistical depend- ence that arises when the units sampled are related in complex ways. STATISTICAL INFERENCE AND ANALYSIS As was noted earlier, questions of design, representation, and analysis are intimately intertwined. Some issues of inference and analysis have been dis-

Methods of Data Collection, Representation, and Analysis / 191 cussed above as related to specific data collection and modeling approaches. This section discusses some more general issues of statistical inference and advances in several current approaches to them. Causal Inference Behavioral and social scientists use statistical methods primarily to infer the effects of treatments, interventions, or policy factors. Previous chapters in- cluded many instances of causal knowledge gained this way. As noted above, the large experimental study of alternative health care financing discussed in Chapter 2 relied heavily on statistical principles and techniques, including randomization, in the design of the experiment and the analysis of the resulting data. Sophisticated designs were necessary in order to answer a variety of questions in a single large study without confusing the effects of one program difference (such as prepayment or fee for service) with the effects of another (such as different levels of deductible costs), or with effects of unobserved variables (such as genetic differences). Statistical techniques were also used to ascertain which results applied across the whole enrolled population and which were confined to certain subgroups (such as individuals with high blood pres- sure) and to translate utilization rates across different programs and types of patients into comparable overall dollar costs and health outcomes for alternative financing options. A classical experiment, with systematic but randomly assigned variation of the variables of interest (or some reasonable approach to this), is usually con- sidered the most rigorous basis from which to draw such inferences. But ran- dom samples or randomized experimental manipulations are not always fea- sible or ethically acceptable. Then, causal inferences must be drawn from observational studies, which, however well designed, are less able to ensure that the observed (or inferred) relationships among variables provide clear evidence on the underlying mechanisms of cause and effect. Certain recurrent challenges have been identified in studying causal infer- ence. One challenge arises from the selection of background variables to be measured, such as the sex, nativity, or parental religion of individuals in a comparative study of how education affects occupational success. The adequacy of classical methods of matching groups in background variables and adjusting for covariates needs further investigation. Statistical adjustment of biases linked to measured background variables is possible, but it can become complicated. Current work in adjustment for selectivity bias is aimed at weakening implau- sible assumptions, such as normality, when carrying out these adjustments. Even after adjustment has been made for the measured background variables, other, unmeasured variables are almost always still affecting the results (such as family transfers of wealth or reading habits). Analyses of how the conclusions might change if such unmeasured variables could be taken into account is

192 / The Behavioral and Social Sciences essential in attempting to make causal inferences from an observational study, and systematic work on useful statistical models for such sensitivity analyses is just beginning. The third important issue arises from the necessity for distinguishing among competing hypotheses when the explanatory variables are measured with dif- ferent degrees of precision. Both the estimated size and significance of an effect are diminished when it has large measurement error, and the coefficients of other correlated variables are affected even when the other variables are meas- ured perfectly. Similar results arise from conceptual errors, when one measures only proxies for a theoretical construct (such as years of education to represent amount of learning). In some cases, there are procedures for simultaneously or iteratively estimating both the precision of complex measures and their effect . . On a particu tar criterion. Although complex models are often necessary to infer causes, once their output is available, it should be translated into understandable displays for evaluation Results that depend on the accuracy of a multivariate model and the associated software need to be subjected to appropriate checks, including the evaluation of graphical displays, group comparisons, and other analyses. New Statistical Techniques Internal Resampling One of the great contributions of twentieth-century statistics was to dem- onstrate how a properly drawn sample of sufficient size, even if it is only a tiny fraction of the population of interest, can yield very good estimates of most population characteristics. When enough is known at the outset about the characteristic in question for example, that its distribution is roughly nor- mal inference from the sample data to the population as a whole is straight- forward, and one can easily compute measures of the certainty of inference, a common example being the 9S percent confidence interval around an estimate. But population shapes are sometimes unknown or uncertain, and so inference procedures cannot be so simple. Furthermore, more often than not, it is difficult to assess even the degree of uncertainty associated with complex data and with the statistics needed to unravel complex social and behavioral phenomena. Internal resampling methods attempt to assess this uncertainty by generating a number of simulated data sets similar to the one actually observed. The definition of similar is crucial, and many methods that exploit different types of similarity have been devised. These methods provide researchers the freedom to choose scientifically appropriate procedures and to replace procedures that are valid under assumed distributional shapes with ones that are not so re- stricted. Flexible and imaginative computer simulation is. the key to these methods. For a simple random sample, the "bootstrap" method repeatedly resamples the obtained data (with replacement) to generate a distribution of

Methods of Data Collection, Representation, and Analysis / 193 possible data sets. The distribution of any estimator can thereby be simulated and measures of the certainty of inference be derived. The "jackknife" method repeatedly omits a fraction of the data and in this way generates a distribution of possible data sets that can also be used to estimate variability. These methods can also be used to remove or reduce bias. For example, the ratio-estimator, a statistic that is commonly used in analyzing sample surveys and censuses, is known to be biased, and the jackknife method can usually remedy this defect. The methods have been extended to other situations and types of analysis, such as multiple regression. There are indications that under relatively general conditions, these methods, and others related to them, allow more accurate estimates of the uncertainty of inferences than do the traditional ones that are based on assumed (usually, normal) distributions when that distributional assumption is unwarranted. For complex samples, such internal resampling or subsampling facilitates estimat- ing the sampling variances of complex statistics. An older and simpler, but equally important, idea is to use one independent subsample in searching the data to develop a model and at least one separate subsample for estimating and testing a selected model. Otherwise, it is next to impossible to make allowances for the excessively close fitting of the model that occurs as a result of the creative search for the exact characteristics of the sample data characteristics that are to some degree random and will not predict well to other samples. Robust Techniques Many technical assumptions underlie the analysis of data. Some, like the assumption that each item in a sample is drawn independently of other items, can be weakened when the data are sufficiently structured to admit simple alternative models, such as serial correlation. Usually, these models require that a few parameters be estimated. Assumptions about shapes of distributions, normality being the most common, have proved to be particularly important, and considerable progress has been made in dealing with the consequences of different assumptions. More recently, robust techniques have been designed that permit sharp, valid discriminations among possible values of parameters of central tendency for a wide variety of alternative distributions by reducing the weight given to oc- casional extreme deviations. It turns out that by giving up, say, 10 percent of the discrimination that could be provided under the rather unrealistic as- sumption of normality, one can greatly improve performance in more realistic situations, especially when unusually large deviations are relatively common. These valuable modifications of classical statistical techniques have been extended to multiple regression, in which procedures of iterative reweighting can now offer relatively good performance for a variety of underlying distri- butional shapes. They should be extended to more general schemes of analysis.

194 / The Behavioral and Social Sciences In some contexts notably the most classical uses of analysis of variance the use of adequate robust techniques should help to bring conventional statistical practice closer to the best standards that experts can now achieve. Many Interrelated Parameters In trying to give a more accurate representation of the real world than is possible with simple models, researchers sometimes use models with many parameters, all of which must be estimated from the data. Classical principles of estimation, such as straightforward maximum-likelihood, do not yield re- liable estimates unless either the number of observations is much larger than the number of parameters to be estimated or special designs are used in con- junction with strong assumptions. Bayesian methods do not draw a distinction between fixed and random parameters, and so may be especially appropriate for such problems. A variety of statistical methods have recently been developed that can be interpreted as treating many of the parameters as or similar to random quan- tities, even if they are regarded as representing fixed quantities to be estimated. Theory and practice demonstrate that such methods can improve the simpler fixed-parameter methods from which they evolved, especially when the num- ber of observations is not large relative to the number of parameters. Successful applications include college and graduate school admissions, where quality of previous school is treated as a random parameter when the data are insufficient to separately estimate it well. Efforts to create appropriate models using this general approach for small-area estimation and underc.ount adjustment in the census are important potential applications. Missing Data In data analysis, serious problems can arise when certain kinds of (quanti- tative or qualitative) information is partially or wholly missing. Various ap- proaches to dealing with these problems have been or are being developed. One of the methods developed recently for dealing with certain aspects of missing data is called multiple imputation: each missing value in a data set is replaced by several values representing a range of possibilities, with statistical dependence among missing values reflected by linkage among their replace- ments. It is currently being used to handle a major problem of incompatibility between the 1980 and previous Bureau of Census public-use tapes with respect to occupation codes. The extension of these techniques to address such prob- lems as nonresponse to income questions in the Current Population Survey has been examined in exploratory applications with great promise. Computing Computer Packages and Expert Systems The development of high-speed computing and data handling has funda- mentally changed statistical analysis. Methodologies for all kinds of situations

Methods of Data Collection, Representation, and Analysis / l9S are rapidly being developed and made available for use in computer packages that may be incorporated into interactive expert systems. This computing ca- pability offers the hope that much data analyses will be more carefully and more effectively done than previously and that better strategies for data analysis will move from the practice of expert statisticians, some of whom may not have tried to articulate their own strategies, to both wide discussion and general use. But powerful tools can be hazardous, as witnessed by occasional dire misuses of existing statistical packages. Until recently the only strategies available were to train more expert methodologists or to train substantive scientists in more methodology, but without the updating of their training it tends to become outmoded. Now there is the opportunity to capture in expert systems the current best methodological advice and practice. If that opportunity is ex- ploited, standard methodological training of social scientists will shift to em- phasizing strategies in using good expert systems - including understanding the nature and importance of the comments it provides rather than in how to patch together something on one's own. With expert systems, almost all behavioral and social scientists should become able to conduct any of the more common styles of data analysis more effectively and with more confidence than all but the most expert do today. However, the difficulties in developing expert systems that work as hoped for should not be underestimated. Human experts cannot readily explicate all of the complex cognitive network that constitutes an important part of their knowledge. As a result, the first attempts at expert systems were not especially successful (as discussed in Chapter 1~. Additional work is expected to overcome these limitations, but it is not clear how long it will take. Exploratory Analysis and Graphic Presentation The formal focus of much statistics research in the middle half of the twen- tieth century was on procedures to confirm or reject precise, a priori hypotheses developed in advance of collecting data—that is, procedures to determine statistical significance. There was relatively little systematic work on realistically rich strategies for the applied researcher to use when attacking real-world problems with their multiplicity of objectives and sources of evidence. More recently, a species of quantitative detective work, called exploratory data anal- ysis, has received increasing attention. In this approach, the researcher seeks out possible quantitative relations that may be present in the data. The tech- niques are flexible and include an important component of graphic represen- tations. While current techniques have evolved for single responses in situa- tions of modest complexity, extensions to multiple responses and to single responses in more complex situations are now possible. Graphic and tabular presentation is a research domain in active renaissance, stemming in part from suggestions for new kinds of graphics made possible by computer capabilities, for example, hanging histograms and easily assimi- lated representations of numerical vectors. Research on data presentation has

196 / The Behavioral and Social Sciences been carried out by statisticians, psychologists, cartographers, and other spe- cialists, and attempts are now being made to incorporate findings and concepts from linguistics, industrial and publishing design, aesthetics, and classification studies in library science. Another influence has been the rapidly increasing availability of powerful computational hardware and software, now available even on desktop computers. These ideas and capabilities are leading to an increasing number of behavioral experiments with substantial statistical input. Nonetheless, criteria of good graphic and tabular practice are still too much matters of tradition and dogma, without adequate empirical evidence or theo- retical coherence. To broaden the respective research outlooks and vigorously develop such evidence and coherence, extended collaborations between statis- tical and mathematical specialists and other scientists are needed, a major objective being to understand better the visual and cognitive processes (see Chapter 1) relevant to effective use of graphic or tabular approaches. Combining Evidence Combining evidence from separate sources is a recurrent scientific task, and formal statistical methods for doing so go back 30 years or more. These methods include the theory and practice of combining tests of individual hypotheses, sequential design and analysis of experiments, comparisons of laboratories, and Bayesian and likelihood paradigms. There is now growing interest in more ambitious analytical syntheses, which are often called meta-analyses. One stimulus has been the appearance of syntheses explicitly combining all existing investigations in particular fields, such as prison parole policy, classroom size in primary schools, cooperative studies of ther- apeutic treatments for coronary heart disease, early childhood education in- terventions, and weather modification experiments. In such fields, a serious approach to even the simplest question how to put together separate esti- mates of effect size from separate investigations leads quickly to difficult and interesting issues. One issue involves the lack of independence among the available studies, due, for example, to the effect of influential teachers on the research projects of their students. Another issue is selection bias, because only some of the studies carried out, usually those with "significant" findings, are available and because the literature search may not find out all relevant studies that are available. In addition, experts agree, although informally, that the quality of studies from different laboratories and facilities differ appreciably and that such information probably should be taken into account. Inevitably, the studies to be included used different designs and concepts and controlled or measured different variables, making it difficult to know how to combine them. Rich, informal syntheses, allowing for individual appraisal, may be better than catch-all formal modeling, but the literature on formal meta-analytic models

Methods of Data Collection, Representation, and Analysis / 197 is growing and may be an important area of discovery in the next decade, relevant both to statistical analysis per se and to improved syntheses in the behavioral and social and other sciences. OPPORTUNITIES AND NEEDS This chapter has cited a number of methodological topics associated with 1 1 · ~ 1 . ~ . ~ oenav~ora~ and social sciences research that appear to be particularly active and promising at the present time. As throughout the report, they constitute illus- trative examples of what the committee believes to be important areas of re- search in the coming decade. In this section we describe recommendations for an additional $16 million annually to facilitate both the development of meth- odologically oriented research and, equally important, its communication throughout the research community. Methodological studies, including early computer implementations, have for the most part been carried out by individual investigators with small teams of colleagues or students. Occasionally, such research has been associated with quite large substantive projects, and some of the current developments of computer packages, graphics, and expert systems clearly require large, orga- nized efforts, which often lie at the boundary between grant-supported work and commercial development. As such research is often a key to understanding complex bodies of behavioral and social sciences data, it is vital to the health of these sciences that research support continue on methods relevant to prob- lems of modeling, statistical analysis, representation, and related aspects of behavioral and social sciences data. Researchers and funding agencies should also be especially sympathetic to the inclusion of such basic methodological work in large experimental and longitudinal studies. Additional funding for work in this area, both in terms of individual research grants on methodological issues and in terms of augmentation of large projects to include additional methodological aspects, should be provided largely in the form of investigator- initiated project grants. Ethnographic and comparative studies also typically rely on project grants to individuals and small groups of investigators. While this type of support should continue, provision should also be made to facilitate the execution of studies using these methods by research teams and to provide appropriate methodological training through the mechanisms outlined below. Overall, we recommend an increase of $4 million in the level of investigator- initiated grant support for methodological work. An additional $1 million should be devoted to a program of centers for methodological research. Many of the new methods and models described in the chapter, if and when adopted to any large extent, will demand substantially greater amounts of research devoted to appropriate analysis and computer implementation. New

198 / The Behavioral and Social Sciences user interfaces and numerical algorithms will need to be designed and new computer programs written. And even when generally available methods (such as maximum-likelihood) are applicable, model application still requires skillful development in particular contexts. Many of the familiar general methods that are applied in the statistical analysis of data are known to provide good ap- proximations when sample sizes are sufficiently large, but their accuracy varies with the specific model and data used. To estimate the accuracy requires ex- tensive numerical exploration. Investigating the sensitivity of results to the assumptions of the models is important and requires still more creative, thoughtful research. It takes substantial efforts of these kinds to bring any new model on line, and the need becomes increasingly important and difficult as statistical models move toward greater realism, usefulness, complexity, and availability in computer form. More complexity in turn will increase the demand for com- putational power. Although most of this demand can be satisfied by increas- ingly powerful desktop computers, some access to mainframe and even su- percomputers will be needed in selected cases. We recommend an additional $4 million annually to cover the growth in computational demands for model development and testing. Interaction and cooperation between the developers and the users of statis- tical and mathematical methods need continual stimulation both ways. Ef- forts should be made to teach new methods to a wider variety of potential users than is now the case. Several ways appear effective for methodologists to com- municate to empirical scientists: running summer training programs for grad- uate students, faculty, and other researchers; encouraging graduate students, perhaps through degree requirements, to make greater use of the statistical, mathematical, and methodological resources at their own or affiliated univer- sities; associating statistical and mathematical research specialists with large- scale data collection projects; and developing statistical packages that incor- porate expert systems in applying the methods. Methodologists, in turn, need to become more familiar with the problems actually faced by empirical scientists in the laboratory and especially in the field. Several ways appear useful for communication in this direction: encour- aging graduate students in methodological specialties, perhaps through degree requirements, to work directly on empirical research; creating postdoctoral fellowships aimed at integrating such specialists into ongoing data collection projects; and providing for large data collection projects to engage relevant methodological specialists. In addition, research on and development of sta- tistical packages and expert systems should be encouraged to involve the mul- tidisciplinary collaboration of experts with experience in statistical, computer, . . . anc ~ cognitive sciences. A final point has to do with the promise held out by bringing different research methods to bear on the same problems. As our discussions of research methods in this and other chapters have emphasized, different methods have

Methods of Data Collection, Representation, and Analysis / 199 different powers and limitations, and each is designed especially to elucidate one or more particular facets of a subject. An important type of interdisciplinary work is the collaboration of specialists in different research methodologies on a substantive issue, examples of which have been noted throughout this report. If more such research were conducted cooperatively, the power of each method pursued separately would be increased. To encourage such multidisciplinary work, we recommend increased support for fellowships, research workshops, anc . tramlug institutes. Funding for fellowships, both pre- and postdoctoral, should be aimed at giving methodologists experience with substantive problems and at upgrading the methodological capabilities of substantive scientists. Such targeted fellow- ship support should be increased by $4 million annually, of which $3 million should be for predoctoral fellowships emphasizing the enrichment of meth- odological concentrations. The new support needed for research workshops is estimated to be $1 million annually. And new support needed for various kinds of advanced training institutes aimed at rapidly diffusing new methodological findings among substantive scientists is estimated to be $2 million annually.

This volume explores the scientific frontiers and leading edges of research across the fields of anthropology, economics, political science, psychology, sociology, history, business, education, geography, law, and psychiatry, as well as the newer, more specialized areas of artificial intelligence, child development, cognitive science, communications, demography, linguistics, and management and decision science. It includes recommendations concerning new resources, facilities, and programs that may be needed over the next several years to ensure rapid progress and provide a high level of returns to basic research.

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Multimodal Affective Computing pp 159–165 Cite as

Methods for Data Representation

• Ramón Zatarain Cabada   ORCID: orcid.org/0000-0002-4524-3511 4 ,
• Héctor Manuel Cárdenas López   ORCID: orcid.org/0000-0002-6823-4933 4 &
• Hugo Jair Escalante   ORCID: orcid.org/0000-0003-4603-3513 5  
• First Online: 21 April 2023

115 Accesses

This chapter provides an overview of the preprocessing techniques for preparing data for personality recognition. It begins with explaining adaptations required for handling large datasets that cannot be loaded into memory. The chapter then focuses on image preprocessing techniques in videos, including face delineation, obturation, and various techniques applied to video images. The chapter also discusses sound preprocessing, such as common sound representation techniques, spectral coefficients, prosody, and intonation. Finally, Mel spectral and delta Mel spectral coefficients are discussed as sound representation techniques for personality recognition. The primary aim of this chapter is to help readers understand different video processing techniques that can be used in data representation for personality recognition.

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

An, G., Levitan, S. I., Levitan, R., Rosenberg, A., Levine, M., & Hirschberg, J. (2016). Automatically classifying self-rated personality scores from speech. In Interspeech (pp. 1412–1416).

Google Scholar  

Arya, R., Kumar, A., Bhushan, M., & Samant, P. (2022). Big Five personality traits prediction using brain signals. International Journal of Fuzzy System Applications (IJFSA), 11 (2), 1–10.

Article   Google Scholar  

dos Santos, W. R., Ramos, R. M. S., & Paraboni, I. (2019). Computational personality recognition from facebook text: psycholinguistic features, words and facets. New Review of Hypermedia and Multimedia, 25 (4), 268–287.

Fan, X., Yan, Y., Wang, X., Yan, H., Li, Y., Xie, L., & Yin, E. (2020). Emotion recognition measurement based on physiological signals. In 2020 13th International Symposium on Computational Intelligence and Design (ISCID) (pp. 81–86). IEEE.

Fink, B., Neave, N., Manning, J. T., & Grammer, K. (2005). Facial symmetry and the ‘Big-Five’ personality factors. Personality and Individual Differences, 39 (3), 523–529.

Fung, P., Dey, A., Siddique, F. B., Lin, R., Yang, Y., Wan, Y., & Chan, H. Y. R. (2016). Zara the supergirl: an empathetic personality recognition system. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations (pp. 87–91).

Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42 , 419–427.

Mohammadi, G., & Vinciarelli, A. (2012). Automatic personality perception: prediction of trait attribution based on prosodic features. IEEE Transactions on Affective Computing, 3 (3), 273–284.

Polzehl, T., Möller, S., & Metze, F. (2010). Automatically assessing acoustic manifestations of personality in speech. In 2010 IEEE Spoken Language Technology Workshop (pp. 7–12). IEEE.

Poria, S., Gelbukh, A., Agarwal, B., Cambria, E., & Howard, N. (2013). Common sense knowledge based personality recognition from text. In Advances in Soft Computing and Its Applications: 12th Mexican International Conference on Artificial Intelligence, MICAI 2013, Mexico City, Mexico, November 24–30, 2013, Proceedings, Part II 12 (pp. 484–496). Springer.

Potapova, R., & Potapov, V. (2016). On individual polyinformativity of speech and voice regarding speakers auditive attribution (forensic phonetic aspect). In Speech and Computer: 18th International Conference, SPECOM 2016, Budapest, Hungary, August 23-27, 2016, Proceedings 18 (pp. 507–514). Springer.

Schuller, B., Steidl, S., Batliner, A., Nöth, E., Vinciarelli, A., Burkhardt, F., Van Son, R., Weninger, F., Eyben, F., Bocklet, T., et al. (2015). A survey on perceived speaker traits: personality, likability, pathology, and the first challenge. Computer Speech & Language, 29 (1), 100–131.

Tung, K., Liu, P.-K., Chuang, Y.-C., Wang, S.-H., & Wu, A.-Y. A. (2018). Entropy-assisted multi-modal emotion recognition framework based on physiological signals. In 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES) (pp. 22–26). IEEE.

Yu, J., & Markov, K. (2017). Deep learning based personality recognition from Facebook status updates. In 2017 IEEE 8th International Conference on Awareness Science and Technology (iCAST) (pp. 383–387). IEEE.

Zhang, Z., Wang, H., & Liu, S. (2018). Scene character recognition via bag-of-words model: A comprehensive study. In Communications, Signal Processing, and Systems: Proceedings of the 2016 International Conference on Communications, Signal Processing, and Systems (pp. 819–826). Springer.

Download references

Author information

Authors and affiliations.

Instituto Tecnológico de Culiacán, Culiacán, Sinaloa, Mexico

Ramón Zatarain Cabada & Héctor Manuel Cárdenas López

Instituto Nacional de Astrofísica, Puebla, Puebla, Mexico

Hugo Jair Escalante

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Cite this chapter.

Cabada, R.Z., López, H.M.C., Escalante, H.J. (2023). Methods for Data Representation. In: Multimodal Affective Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-32542-7_13

Download citation

DOI : https://doi.org/10.1007/978-3-031-32542-7_13

Published : 21 April 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-32541-0

Online ISBN : 978-3-031-32542-7

eBook Packages : Mathematics and Statistics Mathematics and Statistics (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Mastering the Art of Data Representation Statistics 

In today’s world, data is king. From businesses to healthcare to government, everyone relies on data to make informed decisions. But raw data can be overwhelming and difficult to make sense of. This is where data representation statistics come in. In this blog post, we will explore the importance of data representation statistics and how they can help you make sense of your data.

What are Data Representation Statistics?

Table of Contents

Data representation statistics is the process of converting raw data into a format that is easy to understand and interpret. This involves using various statistical methods to analyze and summarize the data. Data representation statistics can help you identify patterns, trends, and relationships in your data, which can help you make informed decisions.

Why are Data Representation Statistics Important?

Data representation statistics are important for several reasons:

Helps you make informed decisions

By converting raw data into a format that is easy to understand and interpret, it can help you make informed decisions.

Identifies patterns and trends 

It can help you identify patterns and trends in your data that may not be obvious when looking at raw data.

Communicate your findings 

It can help you communicate your findings to others in a clear and concise manner.

Provides insights 

It can provide insights into your data that you may not have considered.

Enables data-driven decision making 

By providing insights and identifying patterns and trends, data representation statistics can enable data-driven decision-making.

Methods of Data Representation Statistics

Tables – Tables are a simple and effective way to present data in rows and columns, allowing for easy comparison and summarization.

Bar charts – Bar charts are used to compare the frequency or distribution of data points in different categories, with each category represented by a separate bar.

Line charts – Line charts are used to show trends in data over time, with data points connected by a line.

Scatter plots – Scatter plots are used to show the relationship between two variables, with each data point represented by a dot on a two-dimensional graph.

Pie charts – Pie charts are used to show the distribution of data points in different categories as a percentage of the whole, with each category represented by a slice of a circular graph.

Box plots – Box plots are used to show the distribution of data points, with the box representing the interquartile range (IQR), the whiskers representing the range of the data, and outliers represented by dots or asterisks.

Heat maps – Heat maps are used to show the density of data points in a two-dimensional grid, with different colors representing different levels of density.

Histograms – Histograms are used to show the frequency distribution of a single variable, with the data grouped into intervals and represented as bars on a graph.

Frequency tables – Frequency tables are used to summarize the frequency distribution of a single variable, with the data grouped into intervals and displayed in a table.

Stacked bar charts – Stacked bar charts are used to compare the frequency or distribution of data points in different categories, with each bar divided into segments representing different subcategories.

Box and whisker plots – Box and whisker plots are used to show the distribution of data points, with the box representing the IQR and the whiskers representing the range of the data.

Stem and leaf plots – Stem and leaf plots are used to show the distribution of data points, with the stems representing the tens or hundreds digit and the leaves representing the ones or units digit.

Time series plots – Time series plots are used to show trends in data over time, with data points plotted on a graph with a time axis.

Polar plots – Polar plots are used to show the distribution of data points in a circular graph, with the distance from the center representing the value of a variable and the angle representing a category.

Waterfall charts – Waterfall charts are used to show the changes in a variable over time, with each change represented by a segment of a bar that rises or falls.

Dot plots – Dot plots are used to show the distribution of data points, with each data point represented by a dot on a horizontal axis.

Radial bar charts – Radial bar charts are used to show the distribution of data points in a circular graph, with each bar representing a category and the length of the bar representing the value of a variable.

Area charts – Area charts are used to show the trend of data over time, with data points connected by a line and the area between the line and the x-axis shaded.

Radar charts – Radar charts are used to show the distribution of data points in a circular graph, with each category represented by a spoke and the length of the spoke representing the value of a variable.

Violin plots – Violin plots are used to show the distribution of data points, with the shape of the plot representing the density of the data.

Gantt charts – Gantt charts are used to show the timeline of a project, with each task represented by a horizontal bar and the length of the bar representing the duration of the task.

Chord diagrams – Chord diagrams are used to show the relationships between different categories, with the size of the chords representing the strength of the relationships.

Word clouds – Word clouds are used to show the frequency of words in a text document, with more frequently used words displayed in larger fonts.

Sankey diagrams – Sankey diagrams are used to show the flow of data between different categories, with the width of the lines representing the volume of the data.

Spider charts – Spider charts are used to show the distribution of data points in a circular graph, with each variable represented by a spoke and the length of the spoke representing the value of the variable.

Map charts – Map charts are used to show the distribution of data points on a map, with each data point represented by a symbol or a color.

Tree maps – Tree maps are used to show the hierarchical structure of data, with each level represented by a rectangle and the size of the rectangle representing the value of the data.

Bullet charts – Bullet charts are used to show the progress towards a goal, with a vertical bar representing the actual value and a horizontal bar representing the target value.

Heat bars – Heat bars are used to show the density of data points in a one-dimensional graph, with different colors representing different levels of density.

Contour plots – Contour plots are used to show the three-dimensional shape of data, with lines representing points of equal value.

Motion charts – Motion charts are used to show changes in data over time, with data points moving on a graph.

Funnel charts – Funnel charts are used to show the conversion rates of a process, with each step of the process represented by a decreasing bar.

Marimekko charts – Marimekko charts are used to show the relationship between two categorical variables, with the width of the bars representing the relative size of the categories.

Sparklines – Sparklines are used to show the trends in data over time, with data points represented as a small line or bar within a larger text document or table.

Polar area charts – Polar area charts are used to show the distribution of data points in a circular graph, with the area of the segment representing the value of a variable.

Candlestick charts – Candlestick charts are used to show the daily changes in the price of a financial asset, with each candlestick representing the opening, closing, high, and low prices.

Radar area charts – Radar area charts are used to show the distribution of data points in a circular graph, with each variable represented by a spoke and the area of the shape representing the value of the variable.

Donut charts – Donut charts are similar to pie charts, but with a hole in the center, allowing for the display of additional information.

3D plots – 3D plots are used to show the shape of data in three dimensions, with different colors or shades representing different levels of the data.

How to Find the Methods of Data Representation Statistics

There are several ways to find the methods of data representation statistics, including:

Research online

There are a multitude of resources available online that can provide information on various methods of data representation statistics. These can include websites, academic journals, and forums.

Consult textbooks

Textbooks on statistics and data analysis often contain sections or chapters dedicated to data visualization techniques, which can provide information on various methods of data representation statistics.

Attend training courses

Many training courses on statistics and data analysis will cover various methods of data representation statistics. These courses may be offered online or in person and can be a great way to learn about different data visualization techniques.

Ask Experts

Experts in the field of statistics and data analysis can provide valuable insights into various methods of data representation statistics. This can include professors, researchers, and practitioners.

Use statistical software

Many statistical software packages come with built-in data visualization tools that can be used to explore different methods of data representation statistics. These software packages may also include tutorials and documentation that can provide information on various data visualization techniques.

By utilizing these methods, you can gain a better understanding of the different methods of data representation statistics and how to use them to effectively communicate insights and findings from your data.

Tips for Effective Data Representation

Here are some tips for effective data representation:

• Choose the right method – Choose the method that best suits your data and your audience.
• Keep it simple – Use simple language and avoid unnecessary jargon.
• Be clear and concise – Use clear and concise language to communicate your findings.
• Use colors and labels – Use colors and labels to make your data more visually appealing and easier to understand.
• Check your data – Make sure your data is accurate and up-to-date.

Data representation statistics are essential for making sense of raw data. By converting raw data into a format that is easy to understand and interpret, data representation statistics can help you identify patterns, trends, and relationships in your data. This, in turn, can help you make informed decisions and drive data-driven decision-making. With the right methods and tips, you can effectively represent your data and communicate your findings to others.

Related Posts

8 easiest programming language to learn for beginners.

There are so many programming languages you can learn. But if you’re looking to start with something easier. We bring to you a list of…

10 Online Tutoring Help Benefits

Do you need a computer science assignment help? Get the best quality assignment help from computer science tutors at affordable prices. They always presented to help…

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council; Division of Behavioral and Social Sciences and Education; Commission on Behavioral and Social Sciences and Education; Committee on Basic Research in the Behavioral and Social Sciences; Gerstein DR, Luce RD, Smelser NJ, et al., editors. The Behavioral and Social Sciences: Achievements and Opportunities. Washington (DC): National Academies Press (US); 1988.

The Behavioral and Social Sciences: Achievements and Opportunities.

• Hardcopy Version at National Academies Press

5 Methods of Data Collection, Representation, and Analysis

This chapter concerns research on collecting, representing, and analyzing the data that underlie behavioral and social sciences knowledge. Such research, methodological in character, includes ethnographic and historical approaches, scaling, axiomatic measurement, and statistics, with its important relatives, econometrics and psychometrics. The field can be described as including the self-conscious study of how scientists draw inferences and reach conclusions from observations. Since statistics is the largest and most prominent of methodological approaches and is used by researchers in virtually every discipline, statistical work draws the lion’s share of this chapter’s attention.

Problems of interpreting data arise whenever inherent variation or measurement fluctuations create challenges to understand data or to judge whether observed relationships are significant, durable, or general. Some examples: Is a sharp monthly (or yearly) increase in the rate of juvenile delinquency (or unemployment) in a particular area a matter for alarm, an ordinary periodic or random fluctuation, or the result of a change or quirk in reporting method? Do the temporal patterns seen in such repeated observations reflect a direct causal mechanism, a complex of indirect ones, or just imperfections in the data? Is a decrease in auto injuries an effect of a new seat-belt law? Are the disagreements among people describing some aspect of a subculture too great to draw valid inferences about that aspect of the culture?

Such issues of inference are often closely connected to substantive theory and specific data, and to some extent it is difficult and perhaps misleading to treat methods of data collection, representation, and analysis separately. This report does so, as do all sciences to some extent, because the methods developed often are far more general than the specific problems that originally gave rise to them. There is much transfer of new ideas from one substantive field to another—and to and from fields outside the behavioral and social sciences. Some of the classical methods of statistics arose in studies of astronomical observations, biological variability, and human diversity. The major growth of the classical methods occurred in the twentieth century, greatly stimulated by problems in agriculture and genetics. Some methods for uncovering geometric structures in data, such as multidimensional scaling and factor analysis, originated in research on psychological problems, but have been applied in many other sciences. Some time-series methods were developed originally to deal with economic data, but they are equally applicable to many other kinds of data.

• In economics: large-scale models of the U.S. economy; effects of taxation, money supply, and other government fiscal and monetary policies; theories of duopoly, oligopoly, and rational expectations; economic effects of slavery.
• In psychology: test calibration; the formation of subjective probabilities, their revision in the light of new information, and their use in decision making; psychiatric epidemiology and mental health program evaluation.
• In sociology and other fields: victimization and crime rates; effects of incarceration and sentencing policies; deployment of police and fire-fighting forces; discrimination, antitrust, and regulatory court cases; social networks; population growth and forecasting; and voting behavior.

Even such an abridged listing makes clear that improvements in methodology are valuable across the spectrum of empirical research in the behavioral and social sciences as well as in application to policy questions. Clearly, methodological research serves many different purposes, and there is a need to develop different approaches to serve those different purposes, including exploratory data analysis, scientific inference about hypotheses and population parameters, individual decision making, forecasting what will happen in the event or absence of intervention, and assessing causality from both randomized experiments and observational data.

This discussion of methodological research is divided into three areas: design, representation, and analysis. The efficient design of investigations must take place before data are collected because it involves how much, what kind of, and how data are to be collected. What type of study is feasible: experimental, sample survey, field observation, or other? What variables should be measured, controlled, and randomized? How extensive a subject pool or observational period is appropriate? How can study resources be allocated most effectively among various sites, instruments, and subsamples?

The construction of useful representations of the data involves deciding what kind of formal structure best expresses the underlying qualitative and quantitative concepts that are being used in a given study. For example, cost of living is a simple concept to quantify if it applies to a single individual with unchanging tastes in stable markets (that is, markets offering the same array of goods from year to year at varying prices), but as a national aggregate for millions of households and constantly changing consumer product markets, the cost of living is not easy to specify clearly or measure reliably. Statisticians, economists, sociologists, and other experts have long struggled to make the cost of living a precise yet practicable concept that is also efficient to measure, and they must continually modify it to reflect changing circumstances.

Data analysis covers the final step of characterizing and interpreting research findings: Can estimates of the relations between variables be made? Can some conclusion be drawn about correlation, cause and effect, or trends over time? How uncertain are the estimates and conclusions and can that uncertainty be reduced by analyzing the data in a different way? Can computers be used to display complex results graphically for quicker or better understanding or to suggest different ways of proceeding?

Advances in analysis, data representation, and research design feed into and reinforce one another in the course of actual scientific work. The intersections between methodological improvements and empirical advances are an important aspect of the multidisciplinary thrust of progress in the behavioral and social sciences.

• Designs for Data Collection

Four broad kinds of research designs are used in the behavioral and social sciences: experimental, survey, comparative, and ethnographic.

Experimental designs, in either the laboratory or field settings, systematically manipulate a few variables while others that may affect the outcome are held constant, randomized, or otherwise controlled. The purpose of randomized experiments is to ensure that only one or a few variables can systematically affect the results, so that causes can be attributed. Survey designs include the collection and analysis of data from censuses, sample surveys, and longitudinal studies and the examination of various relationships among the observed phenomena. Randomization plays a different role here than in experimental designs: it is used to select members of a sample so that the sample is as representative of the whole population as possible. Comparative designs involve the retrieval of evidence that is recorded in the flow of current or past events in different times or places and the interpretation and analysis of this evidence. Ethnographic designs, also known as participant-observation designs, involve a researcher in intensive and direct contact with a group, community, or population being studied, through participation, observation, and extended interviewing.

Experimental Designs

Laboratory experiments.

Laboratory experiments underlie most of the work reported in Chapter 1 , significant parts of Chapter 2 , and some of the newest lines of research in Chapter 3 . Laboratory experiments extend and adapt classical methods of design first developed, for the most part, in the physical and life sciences and agricultural research. Their main feature is the systematic and independent manipulation of a few variables and the strict control or randomization of all other variables that might affect the phenomenon under study. For example, some studies of animal motivation involve the systematic manipulation of amounts of food and feeding schedules while other factors that may also affect motivation, such as body weight, deprivation, and so on, are held constant. New designs are currently coming into play largely because of new analytic and computational methods (discussed below, in “Advances in Statistical Inference and Analysis”).

Two examples of empirically important issues that demonstrate the need for broadening classical experimental approaches are open-ended responses and lack of independence of successive experimental trials. The first concerns the design of research protocols that do not require the strict segregation of the events of an experiment into well-defined trials, but permit a subject to respond at will. These methods are needed when what is of interest is how the respondent chooses to allocate behavior in real time and across continuously available alternatives. Such empirical methods have long been used, but they can generate very subtle and difficult problems in experimental design and subsequent analysis. As theories of allocative behavior of all sorts become more sophisticated and precise, the experimental requirements become more demanding, so the need to better understand and solve this range of design issues is an outstanding challenge to methodological ingenuity.

The second issue arises in repeated-trial designs when the behavior on successive trials, even if it does not exhibit a secular trend (such as a learning curve), is markedly influenced by what has happened in the preceding trial or trials. The more naturalistic the experiment and the more sensitive the meas urements taken, the more likely it is that such effects will occur. But such sequential dependencies in observations cause a number of important conceptual and technical problems in summarizing the data and in testing analytical models, which are not yet completely understood. In the absence of clear solutions, such effects are sometimes ignored by investigators, simplifying the data analysis but leaving residues of skepticism about the reliability and significance of the experimental results. With continuing development of sensitive measures in repeated-trial designs, there is a growing need for more advanced concepts and methods for dealing with experimental results that may be influenced by sequential dependencies.

Randomized Field Experiments

The state of the art in randomized field experiments, in which different policies or procedures are tested in controlled trials under real conditions, has advanced dramatically over the past two decades. Problems that were once considered major methodological obstacles—such as implementing randomized field assignment to treatment and control groups and protecting the randomization procedure from corruption—have been largely overcome. While state-of-the-art standards are not achieved in every field experiment, the commitment to reaching them is rising steadily, not only among researchers but also among customer agencies and sponsors.

The health insurance experiment described in Chapter 2 is an example of a major randomized field experiment that has had and will continue to have important policy reverberations in the design of health care financing. Field experiments with the negative income tax (guaranteed minimum income) conducted in the 1970s were significant in policy debates, even before their completion, and provided the most solid evidence available on how tax-based income support programs and marginal tax rates can affect the work incentives and family structures of the poor. Important field experiments have also been carried out on alternative strategies for the prevention of delinquency and other criminal behavior, reform of court procedures, rehabilitative programs in mental health, family planning, and special educational programs, among other areas.

In planning field experiments, much hinges on the definition and design of the experimental cells, the particular combinations needed of treatment and control conditions for each set of demographic or other client sample characteristics, including specification of the minimum number of cases needed in each cell to test for the presence of effects. Considerations of statistical power, client availability, and the theoretical structure of the inquiry enter into such specifications. Current important methodological thresholds are to find better ways of predicting recruitment and attrition patterns in the sample, of designing experiments that will be statistically robust in the face of problematic sample recruitment or excessive attrition, and of ensuring appropriate acquisition and analysis of data on the attrition component of the sample.

Also of major significance are improvements in integrating detailed process and outcome measurements in field experiments. To conduct research on program effects under field conditions requires continual monitoring to determine exactly what is being done—the process—how it corresponds to what was projected at the outset. Relatively unintrusive, inexpensive, and effective implementation measures are of great interest. There is, in parallel, a growing emphasis on designing experiments to evaluate distinct program components in contrast to summary measures of net program effects.

Finally, there is an important opportunity now for further theoretical work to model organizational processes in social settings and to design and select outcome variables that, in the relatively short time of most field experiments, can predict longer-term effects: For example, in job-training programs, what are the effects on the community (role models, morale, referral networks) or on individual skills, motives, or knowledge levels that are likely to translate into sustained changes in career paths and income levels?

Survey Designs

Many people have opinions about how societal mores, economic conditions, and social programs shape lives and encourage or discourage various kinds of behavior. People generalize from their own cases, and from the groups to which they belong, about such matters as how much it costs to raise a child, the extent to which unemployment contributes to divorce, and so on. In fact, however, effects vary so much from one group to another that homespun generalizations are of little use. Fortunately, behavioral and social scientists have been able to bridge the gaps between personal perspectives and collective realities by means of survey research. In particular, governmental information systems include volumes of extremely valuable survey data, and the facility of modern computers to store, disseminate, and analyze such data has significantly improved empirical tests and led to new understandings of social processes.

Within this category of research designs, two major types are distinguished: repeated cross-sectional surveys and longitudinal panel surveys. In addition, and cross-cutting these types, there is a major effort under way to improve and refine the quality of survey data by investigating features of human memory and of question formation that affect survey response.

Repeated cross-sectional designs can either attempt to measure an entire population—as does the oldest U.S. example, the national decennial census—or they can rest on samples drawn from a population. The general principle is to take independent samples at two or more times, measuring the variables of interest, such as income levels, housing plans, or opinions about public affairs, in the same way. The General Social Survey, collected by the National Opinion Research Center with National Science Foundation support, is a repeated cross sectional data base that was begun in 1972. One methodological question of particular salience in such data is how to adjust for nonresponses and “don’t know” responses. Another is how to deal with self-selection bias. For example, to compare the earnings of women and men in the labor force, it would be mistaken to first assume that the two samples of labor-force participants are randomly selected from the larger populations of men and women; instead, one has to consider and incorporate in the analysis the factors that determine who is in the labor force.

In longitudinal panels, a sample is drawn at one point in time and the relevant variables are measured at this and subsequent times for the same people. In more complex versions, some fraction of each panel may be replaced or added to periodically, such as expanding the sample to include households formed by the children of the original sample. An example of panel data developed in this way is the Panel Study of Income Dynamics (PSID), conducted by the University of Michigan since 1968 (discussed in Chapter 3 ).

Comparing the fertility or income of different people in different circumstances at the same time to find correlations always leaves a large proportion of the variability unexplained, but common sense suggests that much of the unexplained variability is actually explicable. There are systematic reasons for individual outcomes in each person’s past achievements, in parental models, upbringing, and earlier sequences of experiences. Unfortunately, asking people about the past is not particularly helpful: people remake their views of the past to rationalize the present and so retrospective data are often of uncertain validity. In contrast, generation-long longitudinal data allow readings on the sequence of past circumstances uncolored by later outcomes. Such data are uniquely useful for studying the causes and consequences of naturally occurring decisions and transitions. Thus, as longitudinal studies continue, quantitative analysis is becoming feasible about such questions as: How are the decisions of individuals affected by parental experience? Which aspects of early decisions constrain later opportunities? And how does detailed background experience leave its imprint? Studies like the two-decade-long PSID are bringing within grasp a complete generational cycle of detailed data on fertility, work life, household structure, and income.

Advances in Longitudinal Designs

Large-scale longitudinal data collection projects are uniquely valuable as vehicles for testing and improving survey research methodology. In ways that lie beyond the scope of a cross-sectional survey, longitudinal studies can sometimes be designed—without significant detriment to their substantive interests—to facilitate the evaluation and upgrading of data quality; the analysis of relative costs and effectiveness of alternative techniques of inquiry; and the standardization or coordination of solutions to problems of method, concept, and measurement across different research domains.

Some areas of methodological improvement include discoveries about the impact of interview mode on response (mail, telephone, face-to-face); the effects of nonresponse on the representativeness of a sample (due to respondents’ refusal or interviewers’ failure to contact); the effects on behavior of continued participation over time in a sample survey; the value of alternative methods of adjusting for nonresponse and incomplete observations (such as imputation of missing data, variable case weighting); the impact on response of specifying different recall periods, varying the intervals between interviews, or changing the length of interviews; and the comparison and calibration of results obtained by longitudinal surveys, randomized field experiments, laboratory studies, onetime surveys, and administrative records.

It should be especially noted that incorporating improvements in methodology and data quality has been and will no doubt continue to be crucial to the growing success of longitudinal studies. Panel designs are intrinsically more vulnerable than other designs to statistical biases due to cumulative item non-response, sample attrition, time-in-sample effects, and error margins in repeated measures, all of which may produce exaggerated estimates of change. Over time, a panel that was initially representative may become much less representative of a population, not only because of attrition in the sample, but also because of changes in immigration patterns, age structure, and the like. Longitudinal studies are also subject to changes in scientific and societal contexts that may create uncontrolled drifts over time in the meaning of nominally stable questions or concepts as well as in the underlying behavior. Also, a natural tendency to expand over time the range of topics and thus the interview lengths, which increases the burdens on respondents, may lead to deterioration of data quality or relevance. Careful methodological research to understand and overcome these problems has been done, and continued work as a component of new longitudinal studies is certain to advance the overall state of the art.

Longitudinal studies are sometimes pressed for evidence they are not designed to produce: for example, in important public policy questions concerning the impact of government programs in such areas as health promotion, disease prevention, or criminal justice. By using research designs that combine field experiments (with randomized assignment to program and control conditions) and longitudinal surveys, one can capitalize on the strongest merits of each: the experimental component provides stronger evidence for casual statements that are critical for evaluating programs and for illuminating some fundamental theories; the longitudinal component helps in the estimation of long-term program effects and their attenuation. Coupling experiments to ongoing longitudinal studies is not often feasible, given the multiple constraints of not disrupting the survey, developing all the complicated arrangements that go into a large-scale field experiment, and having the populations of interest overlap in useful ways. Yet opportunities to join field experiments to surveys are of great importance. Coupled studies can produce vital knowledge about the empirical conditions under which the results of longitudinal surveys turn out to be similar to—or divergent from—those produced by randomized field experiments. A pattern of divergence and similarity has begun to emerge in coupled studies; additional cases are needed to understand why some naturally occurring social processes and longitudinal design features seem to approximate formal random allocation and others do not. The methodological implications of such new knowledge go well beyond program evaluation and survey research. These findings bear directly on the confidence scientists—and others—can have in conclusions from observational studies of complex behavioral and social processes, particularly ones that cannot be controlled or simulated within the confines of a laboratory environment.

Memory and the Framing of Questions

A very important opportunity to improve survey methods lies in the reduction of nonsampling error due to questionnaire context, phrasing of questions, and, generally, the semantic and social-psychological aspects of surveys. Survey data are particularly affected by the fallibility of human memory and the sensitivity of respondents to the framework in which a question is asked. This sensitivity is especially strong for certain types of attitudinal and opinion questions. Efforts are now being made to bring survey specialists into closer contact with researchers working on memory function, knowledge representation, and language in order to uncover and reduce this kind of error.

Memory for events is often inaccurate, biased toward what respondents believe to be true—or should be true—about the world. In many cases in which data are based on recollection, improvements can be achieved by shifting to techniques of structured interviewing and calibrated forms of memory elicitation, such as specifying recent, brief time periods (for example, in the last seven days) within which respondents recall certain types of events with acceptable accuracy.

• “Taking things altogether, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy?”
• “Taken altogether how would you say things are these days—would you say you are very happy, pretty happy, or not too happy?”

Presenting this sequence in both directions on different forms showed that the order affected answers to the general happiness question but did not change the marital happiness question: responses to the specific issue swayed subsequent responses to the general one, but not vice versa. The explanations for and implications of such order effects on the many kinds of questions and sequences that can be used are not simple matters. Further experimentation on the design of survey instruments promises not only to improve the accuracy and reliability of survey research, but also to advance understanding of how people think about and evaluate their behavior from day to day.

Comparative Designs

Both experiments and surveys involve interventions or questions by the scientist, who then records and analyzes the responses. In contrast, many bodies of social and behavioral data of considerable value are originally derived from records or collections that have accumulated for various nonscientific reasons, quite often administrative in nature, in firms, churches, military organizations, and governments at all levels. Data of this kind can sometimes be subjected to careful scrutiny, summary, and inquiry by historians and social scientists, and statistical methods have increasingly been used to develop and evaluate inferences drawn from such data. Some of the main comparative approaches are cross-national aggregate comparisons, selective comparison of a limited number of cases, and historical case studies.

Among the more striking problems facing the scientist using such data are the vast differences in what has been recorded by different agencies whose behavior is being compared (this is especially true for parallel agencies in different nations), the highly unrepresentative or idiosyncratic sampling that can occur in the collection of such data, and the selective preservation and destruction of records. Means to overcome these problems form a substantial methodological research agenda in comparative research. An example of the method of cross-national aggregative comparisons is found in investigations by political scientists and sociologists of the factors that underlie differences in the vitality of institutions of political democracy in different societies. Some investigators have stressed the existence of a large middle class, others the level of education of a population, and still others the development of systems of mass communication. In cross-national aggregate comparisons, a large number of nations are arrayed according to some measures of political democracy and then attempts are made to ascertain the strength of correlations between these and the other variables. In this line of analysis it is possible to use a variety of statistical cluster and regression techniques to isolate and assess the possible impact of certain variables on the institutions under study. While this kind of research is cross-sectional in character, statements about historical processes are often invoked to explain the correlations.

More limited selective comparisons, applied by many of the classic theorists, involve asking similar kinds of questions but over a smaller range of societies. Why did democracy develop in such different ways in America, France, and England? Why did northeastern Europe develop rational bourgeois capitalism, in contrast to the Mediterranean and Asian nations? Modern scholars have turned their attention to explaining, for example, differences among types of fascism between the two World Wars, and similarities and differences among modern state welfare systems, using these comparisons to unravel the salient causes. The questions asked in these instances are inevitably historical ones.

Historical case studies involve only one nation or region, and so they may not be geographically comparative. However, insofar as they involve tracing the transformation of a society’s major institutions and the role of its main shaping events, they involve a comparison of different periods of a nation’s or a region’s history. The goal of such comparisons is to give a systematic account of the relevant differences. Sometimes, particularly with respect to the ancient societies, the historical record is very sparse, and the methods of history and archaeology mesh in the reconstruction of complex social arrangements and patterns of change on the basis of few fragments.

Like all research designs, comparative ones have distinctive vulnerabilities and advantages: One of the main advantages of using comparative designs is that they greatly expand the range of data, as well as the amount of variation in those data, for study. Consequently, they allow for more encompassing explanations and theories that can relate highly divergent outcomes to one another in the same framework. They also contribute to reducing any cultural biases or tendencies toward parochialism among scientists studying common human phenomena.

One main vulnerability in such designs arises from the problem of achieving comparability. Because comparative study involves studying societies and other units that are dissimilar from one another, the phenomena under study usually occur in very different contexts—so different that in some cases what is called an event in one society cannot really be regarded as the same type of event in another. For example, a vote in a Western democracy is different from a vote in an Eastern bloc country, and a voluntary vote in the United States means something different from a compulsory vote in Australia. These circumstances make for interpretive difficulties in comparing aggregate rates of voter turnout in different countries.

The problem of achieving comparability appears in historical analysis as well. For example, changes in laws and enforcement and recording procedures over time change the definition of what is and what is not a crime, and for that reason it is difficult to compare the crime rates over time. Comparative researchers struggle with this problem continually, working to fashion equivalent measures; some have suggested the use of different measures (voting, letters to the editor, street demonstration) in different societies for common variables (political participation), to try to take contextual factors into account and to achieve truer comparability.

A second vulnerability is controlling variation. Traditional experiments make conscious and elaborate efforts to control the variation of some factors and thereby assess the causal significance of others. In surveys as well as experiments, statistical methods are used to control sources of variation and assess suspected causal significance. In comparative and historical designs, this kind of control is often difficult to attain because the sources of variation are many and the number of cases few. Scientists have made efforts to approximate such control in these cases of “many variables, small N.” One is the method of paired comparisons. If an investigator isolates 15 American cities in which racial violence has been recurrent in the past 30 years, for example, it is helpful to match them with 15 cities of similar population size, geographical region, and size of minorities—such characteristics are controls—and then search for systematic differences between the two sets of cities. Another method is to select, for comparative purposes, a sample of societies that resemble one another in certain critical ways, such as size, common language, and common level of development, thus attempting to hold these factors roughly constant, and then seeking explanations among other factors in which the sampled societies differ from one another.

Ethnographic Designs

Traditionally identified with anthropology, ethnographic research designs are playing increasingly significant roles in most of the behavioral and social sciences. The core of this methodology is participant-observation, in which a researcher spends an extended period of time with the group under study, ideally mastering the local language, dialect, or special vocabulary, and participating in as many activities of the group as possible. This kind of participant-observation is normally coupled with extensive open-ended interviewing, in which people are asked to explain in depth the rules, norms, practices, and beliefs through which (from their point of view) they conduct their lives. A principal aim of ethnographic study is to discover the premises on which those rules, norms, practices, and beliefs are built.

The use of ethnographic designs by anthropologists has contributed significantly to the building of knowledge about social and cultural variation. And while these designs continue to center on certain long-standing features—extensive face-to-face experience in the community, linguistic competence, participation, and open-ended interviewing—there are newer trends in ethnographic work. One major trend concerns its scale. Ethnographic methods were originally developed largely for studying small-scale groupings known variously as village, folk, primitive, preliterate, or simple societies. Over the decades, these methods have increasingly been applied to the study of small groups and networks within modern (urban, industrial, complex) society, including the contemporary United States. The typical subjects of ethnographic study in modern society are small groups or relatively small social networks, such as outpatient clinics, medical schools, religious cults and churches, ethnically distinctive urban neighborhoods, corporate offices and factories, and government bureaus and legislatures.

As anthropologists moved into the study of modern societies, researchers in other disciplines—particularly sociology, psychology, and political science—began using ethnographic methods to enrich and focus their own insights and findings. At the same time, studies of large-scale structures and processes have been aided by the use of ethnographic methods, since most large-scale changes work their way into the fabric of community, neighborhood, and family, affecting the daily lives of people. Ethnographers have studied, for example, the impact of new industry and new forms of labor in “backward” regions; the impact of state-level birth control policies on ethnic groups; and the impact on residents in a region of building a dam or establishing a nuclear waste dump. Ethnographic methods have also been used to study a number of social processes that lend themselves to its particular techniques of observation and interview—processes such as the formation of class and racial identities, bureaucratic behavior, legislative coalitions and outcomes, and the formation and shifting of consumer tastes.

Advances in structured interviewing (see above) have proven especially powerful in the study of culture. Techniques for understanding kinship systems, concepts of disease, color terminologies, ethnobotany, and ethnozoology have been radically transformed and strengthened by coupling new interviewing methods with modem measurement and scaling techniques (see below). These techniques have made possible more precise comparisons among cultures and identification of the most competent and expert persons within a culture. The next step is to extend these methods to study the ways in which networks of propositions (such as boys like sports, girls like babies) are organized to form belief systems. Much evidence suggests that people typically represent the world around them by means of relatively complex cognitive models that involve interlocking propositions. The techniques of scaling have been used to develop models of how people categorize objects, and they have great potential for further development, to analyze data pertaining to cultural propositions.

Ideological Systems

Perhaps the most fruitful area for the application of ethnographic methods in recent years has been the systematic study of ideologies in modern society. Earlier studies of ideology were in small-scale societies that were rather homogeneous. In these studies researchers could report on a single culture, a uniform system of beliefs and values for the society as a whole. Modern societies are much more diverse both in origins and number of subcultures, related to different regions, communities, occupations, or ethnic groups. Yet these subcultures and ideologies share certain underlying assumptions or at least must find some accommodation with the dominant value and belief systems in the society.

The challenge is to incorporate this greater complexity of structure and process into systematic descriptions and interpretations. One line of work carried out by researchers has tried to track the ways in which ideologies are created, transmitted, and shared among large populations that have traditionally lacked the social mobility and communications technologies of the West. This work has concentrated on large-scale civilizations such as China, India, and Central America. Gradually, the focus has generalized into a concern with the relationship between the great traditions—the central lines of cosmopolitan Confucian, Hindu, or Mayan culture, including aesthetic standards, irrigation technologies, medical systems, cosmologies and calendars, legal codes, poetic genres, and religious doctrines and rites—and the little traditions, those identified with rural, peasant communities. How are the ideological doctrines and cultural values of the urban elites, the great traditions, transmitted to local communities? How are the little traditions, the ideas from the more isolated, less literate, and politically weaker groups in society, transmitted to the elites?

India and southern Asia have been fruitful areas for ethnographic research on these questions. The great Hindu tradition was present in virtually all local contexts through the presence of high-caste individuals in every community. It operated as a pervasive standard of value for all members of society, even in the face of strong little traditions. The situation is surprisingly akin to that of modern, industrialized societies. The central research questions are the degree and the nature of penetration of dominant ideology, even in groups that appear marginal and subordinate and have no strong interest in sharing the dominant value system. In this connection the lowest and poorest occupational caste—the untouchables—serves as an ultimate test of the power of ideology and cultural beliefs to unify complex hierarchical social systems.

Historical Reconstruction

Another current trend in ethnographic methods is its convergence with archival methods. One joining point is the application of descriptive and interpretative procedures used by ethnographers to reconstruct the cultures that created historical documents, diaries, and other records, to interview history, so to speak. For example, a revealing study showed how the Inquisition in the Italian countryside between the 1570s and 1640s gradually worked subtle changes in an ancient fertility cult in peasant communities; the peasant beliefs and rituals assimilated many elements of witchcraft after learning them from their persecutors. A good deal of social history—particularly that of the family—has drawn on discoveries made in the ethnographic study of primitive societies. As described in Chapter 4 , this particular line of inquiry rests on a marriage of ethnographic, archival, and demographic approaches.

Other lines of ethnographic work have focused on the historical dimensions of nonliterate societies. A strikingly successful example in this kind of effort is a study of head-hunting. By combining an interpretation of local oral tradition with the fragmentary observations that were made by outside observers (such as missionaries, traders, colonial officials), historical fluctuations in the rate and significance of head-hunting were shown to be partly in response to such international forces as the great depression and World War II. Researchers are also investigating the ways in which various groups in contemporary societies invent versions of traditions that may or may not reflect the actual history of the group. This process has been observed among elites seeking political and cultural legitimation and among hard-pressed minorities (for example, the Basque in Spain, the Welsh in Great Britain) seeking roots and political mobilization in a larger society.

Ethnography is a powerful method to record, describe, and interpret the system of meanings held by groups and to discover how those meanings affect the lives of group members. It is a method well adapted to the study of situations in which people interact with one another and the researcher can interact with them as well, so that information about meanings can be evoked and observed. Ethnography is especially suited to exploration and elucidation of unsuspected connections; ideally, it is used in combination with other methods—experimental, survey, or comparative—to establish with precision the relative strengths and weaknesses of such connections. By the same token, experimental, survey, and comparative methods frequently yield connections, the meaning of which is unknown; ethnographic methods are a valuable way to determine them.

• Models for Representing Phenomena

The objective of any science is to uncover the structure and dynamics of the phenomena that are its subject, as they are exhibited in the data. Scientists continuously try to describe possible structures and ask whether the data can, with allowance for errors of measurement, be described adequately in terms of them. Over a long time, various families of structures have recurred throughout many fields of science; these structures have become objects of study in their own right, principally by statisticians, other methodological specialists, applied mathematicians, and philosophers of logic and science. Methods have evolved to evaluate the adequacy of particular structures to account for particular types of data. In the interest of clarity we discuss these structures in this section and the analytical methods used for estimation and evaluation of them in the next section, although in practice they are closely intertwined.

A good deal of mathematical and statistical modeling attempts to describe the relations, both structural and dynamic, that hold among variables that are presumed to be representable by numbers. Such models are applicable in the behavioral and social sciences only to the extent that appropriate numerical measurement can be devised for the relevant variables. In many studies the phenomena in question and the raw data obtained are not intrinsically numerical, but qualitative, such as ethnic group identifications. The identifying numbers used to code such questionnaire categories for computers are no more than labels, which could just as well be letters or colors. One key question is whether there is some natural way to move from the qualitative aspects of such data to a structural representation that involves one of the well-understood numerical or geometric models or whether such an attempt would be inherently inappropriate for the data in question. The decision as to whether or not particular empirical data can be represented in particular numerical or more complex structures is seldom simple, and strong intuitive biases or a priori assumptions about what can and cannot be done may be misleading.

Recent decades have seen rapid and extensive development and application of analytical methods attuned to the nature and complexity of social science data. Examples of nonnumerical modeling are increasing. Moreover, the widespread availability of powerful computers is probably leading to a qualitative revolution, it is affecting not only the ability to compute numerical solutions to numerical models, but also to work out the consequences of all sorts of structures that do not involve numbers at all. The following discussion gives some indication of the richness of past progress and of future prospects although it is by necessity far from exhaustive.

In describing some of the areas of new and continuing research, we have organized this section on the basis of whether the representations are fundamentally probabilistic or not. A further useful distinction is between representations of data that are highly discrete or categorical in nature (such as whether a person is male or female) and those that are continuous in nature (such as a person’s height). Of course, there are intermediate cases involving both types of variables, such as color stimuli that are characterized by discrete hues (red, green) and a continuous luminance measure. Probabilistic models lead very naturally to questions of estimation and statistical evaluation of the correspondence between data and model. Those that are not probabilistic involve additional problems of dealing with and representing sources of variability that are not explicitly modeled. At the present time, scientists understand some aspects of structure, such as geometries, and some aspects of randomness, as embodied in probability models, but do not yet adequately understand how to put the two together in a single unified model. Table 5-1 outlines the way we have organized this discussion and shows where the examples in this section lie.

A Classification of Structural Models.

Probability Models

Some behavioral and social sciences variables appear to be more or less continuous, for example, utility of goods, loudness of sounds, or risk associated with uncertain alternatives. Many other variables, however, are inherently categorical, often with only two or a few values possible: for example, whether a person is in or out of school, employed or not employed, identifies with a major political party or political ideology. And some variables, such as moral attitudes, are typically measured in research with survey questions that allow only categorical responses. Much of the early probability theory was formulated only for continuous variables; its use with categorical variables was not really justified, and in some cases it may have been misleading. Recently, very significant advances have been made in how to deal explicitly with categorical variables. This section first describes several contemporary approaches to models involving categorical variables, followed by ones involving continuous representations.

Log-Linear Models for Categorical Variables

Many recent models for analyzing categorical data of the kind usually displayed as counts (cell frequencies) in multidimensional contingency tables are subsumed under the general heading of log-linear models, that is, linear models in the natural logarithms of the expected counts in each cell in the table. These recently developed forms of statistical analysis allow one to partition variability due to various sources in the distribution of categorical attributes, and to isolate the effects of particular variables or combinations of them.

Present log-linear models were first developed and used by statisticians and sociologists and then found extensive application in other social and behavioral sciences disciplines. When applied, for instance, to the analysis of social mobility, such models separate factors of occupational supply and demand from other factors that impede or propel movement up and down the social hierarchy. With such models, for example, researchers discovered the surprising fact that occupational mobility patterns are strikingly similar in many nations of the world (even among disparate nations like the United States and most of the Eastern European socialist countries), and from one time period to another, once allowance is made for differences in the distributions of occupations. The log-linear and related kinds of models have also made it possible to identify and analyze systematic differences in mobility among nations and across time. As another example of applications, psychologists and others have used log-linear models to analyze attitudes and their determinants and to link attitudes to behavior. These methods have also diffused to and been used extensively in the medical and biological sciences.

Regression Models for Categorical Variables

Models that permit one variable to be explained or predicted by means of others, called regression models, are the workhorses of much applied statistics; this is especially true when the dependent (explained) variable is continuous. For a two-valued dependent variable, such as alive or dead, models and approximate theory and computational methods for one explanatory variable were developed in biometry about 50 years ago. Computer programs able to handle many explanatory variables, continuous or categorical, are readily available today. Even now, however, the accuracy of the approximate theory on given data is an open question.

Using classical utility theory, economists have developed discrete choice models that turn out to be somewhat related to the log-linear and categorical regression models. Models for limited dependent variables, especially those that cannot take on values above or below a certain level (such as weeks unemployed, number of children, and years of schooling) have been used profitably in economics and in some other areas. For example, censored normal variables (called tobits in economics), in which observed values outside certain limits are simply counted, have been used in studying decisions to go on in school. It will require further research and development to incorporate information about limited ranges of variables fully into the main multivariate methodologies. In addition, with respect to the assumptions about distribution and functional form conventionally made in discrete response models, some new methods are now being developed that show promise of yielding reliable inferences without making unrealistic assumptions; further research in this area promises significant progress.

One problem arises from the fact that many of the categorical variables collected by the major data bases are ordered. For example, attitude surveys frequently use a 3-, 5-, or 7-point scale (from high to low) without specifying numerical intervals between levels. Social class and educational levels are often described by ordered categories. Ignoring order information, which many traditional statistical methods do, may be inefficient or inappropriate, but replacing the categories by successive integers or other arbitrary scores may distort the results. (For additional approaches to this question, see sections below on ordered structures.) Regression-like analysis of ordinal categorical variables is quite well developed, but their multivariate analysis needs further research. New log-bilinear models have been proposed, but to date they deal specifically with only two or three categorical variables. Additional research extending the new models, improving computational algorithms, and integrating the models with work on scaling promise to lead to valuable new knowledge.

Models for Event Histories

Event-history studies yield the sequence of events that respondents to a survey sample experience over a period of time; for example, the timing of marriage, childbearing, or labor force participation. Event-history data can be used to study educational progress, demographic processes (migration, fertility, and mortality), mergers of firms, labor market behavior, and even riots, strikes, and revolutions. As interest in such data has grown, many researchers have turned to models that pertain to changes in probabilities over time to describe when and how individuals move among a set of qualitative states.

Much of the progress in models for event-history data builds on recent developments in statistics and biostatistics for life-time, failure-time, and hazard models. Such models permit the analysis of qualitative transitions in a population whose members are undergoing partially random organic deterioration, mechanical wear, or other risks over time. With the increased complexity of event-history data that are now being collected, and the extension of event-history data bases over very long periods of time, new problems arise that cannot be effectively handled by older types of analysis. Among the problems are repeated transitions, such as between unemployment and employment or marriage and divorce; more than one time variable (such as biological age, calendar time, duration in a stage, and time exposed to some specified condition); latent variables (variables that are explicitly modeled even though not observed); gaps in the data; sample attrition that is not randomly distributed over the categories; and respondent difficulties in recalling the exact timing of events.

Models for Multiple-Item Measurement

For a variety of reasons, researchers typically use multiple measures (or multiple indicators) to represent theoretical concepts. Sociologists, for example, often rely on two or more variables (such as occupation and education) to measure an individual’s socioeconomic position; educational psychologists ordinarily measure a student’s ability with multiple test items. Despite the fact that the basic observations are categorical, in a number of applications this is interpreted as a partitioning of something continuous. For example, in test theory one thinks of the measures of both item difficulty and respondent ability as continuous variables, possibly multidimensional in character.

Classical test theory and newer item-response theories in psychometrics deal with the extraction of information from multiple measures. Testing, which is a major source of data in education and other areas, results in millions of test items stored in archives each year for purposes ranging from college admissions to job-training programs for industry. One goal of research on such test data is to be able to make comparisons among persons or groups even when different test items are used. Although the information collected from each respondent is intentionally incomplete in order to keep the tests short and simple, item-response techniques permit researchers to reconstitute the fragments into an accurate picture of overall group proficiencies. These new methods provide a better theoretical handle on individual differences, and they are expected to be extremely important in developing and using tests. For example, they have been used in attempts to equate different forms of a test given in successive waves during a year, a procedure made necessary in large-scale testing programs by legislation requiring disclosure of test-scoring keys at the time results are given.

An example of the use of item-response theory in a significant research effort is the National Assessment of Educational Progress (NAEP). The goal of this project is to provide accurate, nationally representative information on the average (rather than individual) proficiency of American children in a wide variety of academic subjects as they progress through elementary and secondary school. This approach is an improvement over the use of trend data on university entrance exams, because NAEP estimates of academic achievements (by broad characteristics such as age, grade, region, ethnic background, and so on) are not distorted by the self-selected character of those students who seek admission to college, graduate, and professional programs.

Item-response theory also forms the basis of many new psychometric instruments, known as computerized adaptive testing, currently being implemented by the U.S. military services and under additional development in many testing organizations. In adaptive tests, a computer program selects items for each examinee based upon the examinee’s success with previous items. Generally, each person gets a slightly different set of items and the equivalence of scale scores is established by using item-response theory. Adaptive testing can greatly reduce the number of items needed to achieve a given level of measurement accuracy.

Nonlinear, Nonadditive Models

Virtually all statistical models now in use impose a linearity or additivity assumption of some kind, sometimes after a nonlinear transformation of variables. Imposing these forms on relationships that do not, in fact, possess them may well result in false descriptions and spurious effects. Unwary users, especially of computer software packages, can easily be misled. But more realistic nonlinear and nonadditive multivariate models are becoming available. Extensive use with empirical data is likely to force many changes and enhancements in such models and stimulate quite different approaches to nonlinear multivariate analysis in the next decade.

Geometric and Algebraic Models

Geometric and algebraic models attempt to describe underlying structural relations among variables. In some cases they are part of a probabilistic approach, such as the algebraic models underlying regression or the geometric representations of correlations between items in a technique called factor analysis. In other cases, geometric and algebraic models are developed without explicitly modeling the element of randomness or uncertainty that is always present in the data. Although this latter approach to behavioral and social sciences problems has been less researched than the probabilistic one, there are some advantages in developing the structural aspects independent of the statistical ones. We begin the discussion with some inherently geometric representations and then turn to numerical representations for ordered data.

Although geometry is a huge mathematical topic, little of it seems directly applicable to the kinds of data encountered in the behavioral and social sciences. A major reason is that the primitive concepts normally used in geometry—points, lines, coincidence—do not correspond naturally to the kinds of qualitative observations usually obtained in behavioral and social sciences contexts. Nevertheless, since geometric representations are used to reduce bodies of data, there is a real need to develop a deeper understanding of when such representations of social or psychological data make sense. Moreover, there is a practical need to understand why geometric computer algorithms, such as those of multidimensional scaling, work as well as they apparently do. A better understanding of the algorithms will increase the efficiency and appropriateness of their use, which becomes increasingly important with the widespread availability of scaling programs for microcomputers.

Over the past 50 years several kinds of well-understood scaling techniques have been developed and widely used to assist in the search for appropriate geometric representations of empirical data. The whole field of scaling is now entering a critical juncture in terms of unifying and synthesizing what earlier appeared to be disparate contributions. Within the past few years it has become apparent that several major methods of analysis, including some that are based on probabilistic assumptions, can be unified under the rubric of a single generalized mathematical structure. For example, it has recently been demonstrated that such diverse approaches as nonmetric multidimensional scaling, principal-components analysis, factor analysis, correspondence analysis, and log-linear analysis have more in common in terms of underlying mathematical structure than had earlier been realized.

Nonmetric multidimensional scaling is a method that begins with data about the ordering established by subjective similarity (or nearness) between pairs of stimuli. The idea is to embed the stimuli into a metric space (that is, a geometry with a measure of distance between points) in such a way that distances between points corresponding to stimuli exhibit the same ordering as do the data. This method has been successfully applied to phenomena that, on other grounds, are known to be describable in terms of a specific geometric structure; such applications were used to validate the procedures. Such validation was done, for example, with respect to the perception of colors, which are known to be describable in terms of a particular three-dimensional structure known as the Euclidean color coordinates. Similar applications have been made with Morse code symbols and spoken phonemes. The technique is now used in some biological and engineering applications, as well as in some of the social sciences, as a method of data exploration and simplification.

One question of interest is how to develop an axiomatic basis for various geometries using as a primitive concept an observable such as the subject’s ordering of the relative similarity of one pair of stimuli to another, which is the typical starting point of such scaling. The general task is to discover properties of the qualitative data sufficient to ensure that a mapping into the geometric structure exists and, ideally, to discover an algorithm for finding it. Some work of this general type has been carried out: for example, there is an elegant set of axioms based on laws of color matching that yields the three-dimensional vectorial representation of color space. But the more general problem of understanding the conditions under which the multidimensional scaling algorithms are suitable remains unsolved. In addition, work is needed on understanding more general, non-Euclidean spatial models.

Ordered Factorial Systems

One type of structure common throughout the sciences arises when an ordered dependent variable is affected by two or more ordered independent variables. This is the situation to which regression and analysis-of-variance models are often applied; it is also the structure underlying the familiar physical identities, in which physical units are expressed as products of the powers of other units (for example, energy has the unit of mass times the square of the unit of distance divided by the square of the unit of time).

There are many examples of these types of structures in the behavioral and social sciences. One example is the ordering of preference of commodity bundles—collections of various amounts of commodities—which may be revealed directly by expressions of preference or indirectly by choices among alternative sets of bundles. A related example is preferences among alternative courses of action that involve various outcomes with differing degrees of uncertainty; this is one of the more thoroughly investigated problems because of its potential importance in decision making. A psychological example is the trade-off between delay and amount of reward, yielding those combinations that are equally reinforcing. In a common, applied kind of problem, a subject is given descriptions of people in terms of several factors, for example, intelligence, creativity, diligence, and honesty, and is asked to rate them according to a criterion such as suitability for a particular job.

In all these cases and a myriad of others like them the question is whether the regularities of the data permit a numerical representation. Initially, three types of representations were studied quite fully: the dependent variable as a sum, a product, or a weighted average of the measures associated with the independent variables. The first two representations underlie some psychological and economic investigations, as well as a considerable portion of physical measurement and modeling in classical statistics. The third representation, averaging, has proved most useful in understanding preferences among uncertain outcomes and the amalgamation of verbally described traits, as well as some physical variables.

For each of these three cases—adding, multiplying, and averaging—researchers know what properties or axioms of order the data must satisfy for such a numerical representation to be appropriate. On the assumption that one or another of these representations exists, and using numerical ratings by subjects instead of ordering, a scaling technique called functional measurement (referring to the function that describes how the dependent variable relates to the independent ones) has been developed and applied in a number of domains. What remains problematic is how to encompass at the ordinal level the fact that some random error intrudes into nearly all observations and then to show how that randomness is represented at the numerical level; this continues to be an unresolved and challenging research issue.

During the past few years considerable progress has been made in understanding certain representations inherently different from those just discussed. The work has involved three related thrusts. The first is a scheme of classifying structures according to how uniquely their representation is constrained. The three classical numerical representations are known as ordinal, interval, and ratio scale types. For systems with continuous numerical representations and of scale type at least as rich as the ratio one, it has been shown that only one additional type can exist. A second thrust is to accept structural assumptions, like factorial ones, and to derive for each scale the possible functional relations among the independent variables. And the third thrust is to develop axioms for the properties of an order relation that leads to the possible representations. Much is now known about the possible nonadditive representations of both the multifactor case and the one where stimuli can be combined, such as combining sound intensities.

Closely related to this classification of structures is the question: What statements, formulated in terms of the measures arising in such representations, can be viewed as meaningful in the sense of corresponding to something empirical? Statements here refer to any scientific assertions, including statistical ones, formulated in terms of the measures of the variables and logical and mathematical connectives. These are statements for which asserting truth or falsity makes sense. In particular, statements that remain invariant under certain symmetries of structure have played an important role in classical geometry, dimensional analysis in physics, and in relating measurement and statistical models applied to the same phenomenon. In addition, these ideas have been used to construct models in more formally developed areas of the behavioral and social sciences, such as psychophysics. Current research has emphasized the communality of these historically independent developments and is attempting both to uncover systematic, philosophically sound arguments as to why invariance under symmetries is as important as it appears to be and to understand what to do when structures lack symmetry, as, for example, when variables have an inherent upper bound.

Many subjects do not seem to be correctly represented in terms of distances in continuous geometric space. Rather, in some cases, such as the relations among meanings of words—which is of great interest in the study of memory representations—a description in terms of tree-like, hierarchial structures appears to be more illuminating. This kind of description appears appropriate both because of the categorical nature of the judgments and the hierarchial, rather than trade-off, nature of the structure. Individual items are represented as the terminal nodes of the tree, and groupings by different degrees of similarity are shown as intermediate nodes, with the more general groupings occurring nearer the root of the tree. Clustering techniques, requiring considerable computational power, have been and are being developed. Some successful applications exist, but much more refinement is anticipated.

Network Models

Several other lines of advanced modeling have progressed in recent years, opening new possibilities for empirical specification and testing of a variety of theories. In social network data, relationships among units, rather than the units themselves, are the primary objects of study: friendships among persons, trade ties among nations, cocitation clusters among research scientists, interlocking among corporate boards of directors. Special models for social network data have been developed in the past decade, and they give, among other things, precise new measures of the strengths of relational ties among units. A major challenge in social network data at present is to handle the statistical dependence that arises when the units sampled are related in complex ways.

• Statistical Inference and Analysis

As was noted earlier, questions of design, representation, and analysis are intimately intertwined. Some issues of inference and analysis have been discussed above as related to specific data collection and modeling approaches. This section discusses some more general issues of statistical inference and advances in several current approaches to them.

Causal Inference

Behavioral and social scientists use statistical methods primarily to infer the effects of treatments, interventions, or policy factors. Previous chapters included many instances of causal knowledge gained this way. As noted above, the large experimental study of alternative health care financing discussed in Chapter 2 relied heavily on statistical principles and techniques, including randomization, in the design of the experiment and the analysis of the resulting data. Sophisticated designs were necessary in order to answer a variety of questions in a single large study without confusing the effects of one program difference (such as prepayment or fee for service) with the effects of another (such as different levels of deductible costs), or with effects of unobserved variables (such as genetic differences). Statistical techniques were also used to ascertain which results applied across the whole enrolled population and which were confined to certain subgroups (such as individuals with high blood pressure) and to translate utilization rates across different programs and types of patients into comparable overall dollar costs and health outcomes for alternative financing options.

A classical experiment, with systematic but randomly assigned variation of the variables of interest (or some reasonable approach to this), is usually considered the most rigorous basis from which to draw such inferences. But random samples or randomized experimental manipulations are not always feasible or ethically acceptable. Then, causal inferences must be drawn from observational studies, which, however well designed, are less able to ensure that the observed (or inferred) relationships among variables provide clear evidence on the underlying mechanisms of cause and effect.

Certain recurrent challenges have been identified in studying causal inference. One challenge arises from the selection of background variables to be measured, such as the sex, nativity, or parental religion of individuals in a comparative study of how education affects occupational success. The adequacy of classical methods of matching groups in background variables and adjusting for covariates needs further investigation. Statistical adjustment of biases linked to measured background variables is possible, but it can become complicated. Current work in adjustment for selectivity bias is aimed at weakening implausible assumptions, such as normality, when carrying out these adjustments. Even after adjustment has been made for the measured background variables, other, unmeasured variables are almost always still affecting the results (such as family transfers of wealth or reading habits). Analyses of how the conclusions might change if such unmeasured variables could be taken into account is essential in attempting to make causal inferences from an observational study, and systematic work on useful statistical models for such sensitivity analyses is just beginning.

The third important issue arises from the necessity for distinguishing among competing hypotheses when the explanatory variables are measured with different degrees of precision. Both the estimated size and significance of an effect are diminished when it has large measurement error, and the coefficients of other correlated variables are affected even when the other variables are measured perfectly. Similar results arise from conceptual errors, when one measures only proxies for a theoretical construct (such as years of education to represent amount of learning). In some cases, there are procedures for simultaneously or iteratively estimating both the precision of complex measures and their effect on a particular criterion.

Although complex models are often necessary to infer causes, once their output is available, it should be translated into understandable displays for evaluation. Results that depend on the accuracy of a multivariate model and the associated software need to be subjected to appropriate checks, including the evaluation of graphical displays, group comparisons, and other analyses.

New Statistical Techniques

Internal resampling.

One of the great contributions of twentieth-century statistics was to demonstrate how a properly drawn sample of sufficient size, even if it is only a tiny fraction of the population of interest, can yield very good estimates of most population characteristics. When enough is known at the outset about the characteristic in question—for example, that its distribution is roughly normal—inference from the sample data to the population as a whole is straightforward, and one can easily compute measures of the certainty of inference, a common example being the 95 percent confidence interval around an estimate. But population shapes are sometimes unknown or uncertain, and so inference procedures cannot be so simple. Furthermore, more often than not, it is difficult to assess even the degree of uncertainty associated with complex data and with the statistics needed to unravel complex social and behavioral phenomena.

Internal resampling methods attempt to assess this uncertainty by generating a number of simulated data sets similar to the one actually observed. The definition of similar is crucial, and many methods that exploit different types of similarity have been devised. These methods provide researchers the freedom to choose scientifically appropriate procedures and to replace procedures that are valid under assumed distributional shapes with ones that are not so restricted. Flexible and imaginative computer simulation is the key to these methods. For a simple random sample, the “bootstrap” method repeatedly resamples the obtained data (with replacement) to generate a distribution of possible data sets. The distribution of any estimator can thereby be simulated and measures of the certainty of inference be derived. The “jackknife” method repeatedly omits a fraction of the data and in this way generates a distribution of possible data sets that can also be used to estimate variability. These methods can also be used to remove or reduce bias. For example, the ratio-estimator, a statistic that is commonly used in analyzing sample surveys and censuses, is known to be biased, and the jackknife method can usually remedy this defect. The methods have been extended to other situations and types of analysis, such as multiple regression.

There are indications that under relatively general conditions, these methods, and others related to them, allow more accurate estimates of the uncertainty of inferences than do the traditional ones that are based on assumed (usually, normal) distributions when that distributional assumption is unwarranted. For complex samples, such internal resampling or subsampling facilitates estimating the sampling variances of complex statistics.

An older and simpler, but equally important, idea is to use one independent subsample in searching the data to develop a model and at least one separate subsample for estimating and testing a selected model. Otherwise, it is next to impossible to make allowances for the excessively close fitting of the model that occurs as a result of the creative search for the exact characteristics of the sample data—characteristics that are to some degree random and will not predict well to other samples.

Robust Techniques

Many technical assumptions underlie the analysis of data. Some, like the assumption that each item in a sample is drawn independently of other items, can be weakened when the data are sufficiently structured to admit simple alternative models, such as serial correlation. Usually, these models require that a few parameters be estimated. Assumptions about shapes of distributions, normality being the most common, have proved to be particularly important, and considerable progress has been made in dealing with the consequences of different assumptions.

More recently, robust techniques have been designed that permit sharp, valid discriminations among possible values of parameters of central tendency for a wide variety of alternative distributions by reducing the weight given to occasional extreme deviations. It turns out that by giving up, say, 10 percent of the discrimination that could be provided under the rather unrealistic assumption of normality, one can greatly improve performance in more realistic situations, especially when unusually large deviations are relatively common.

These valuable modifications of classical statistical techniques have been extended to multiple regression, in which procedures of iterative reweighting can now offer relatively good performance for a variety of underlying distributional shapes. They should be extended to more general schemes of analysis.

In some contexts—notably the most classical uses of analysis of variance—the use of adequate robust techniques should help to bring conventional statistical practice closer to the best standards that experts can now achieve.

Many Interrelated Parameters

In trying to give a more accurate representation of the real world than is possible with simple models, researchers sometimes use models with many parameters, all of which must be estimated from the data. Classical principles of estimation, such as straightforward maximum-likelihood, do not yield reliable estimates unless either the number of observations is much larger than the number of parameters to be estimated or special designs are used in conjunction with strong assumptions. Bayesian methods do not draw a distinction between fixed and random parameters, and so may be especially appropriate for such problems.

A variety of statistical methods have recently been developed that can be interpreted as treating many of the parameters as or similar to random quantities, even if they are regarded as representing fixed quantities to be estimated. Theory and practice demonstrate that such methods can improve the simpler fixed-parameter methods from which they evolved, especially when the number of observations is not large relative to the number of parameters. Successful applications include college and graduate school admissions, where quality of previous school is treated as a random parameter when the data are insufficient to separately estimate it well. Efforts to create appropriate models using this general approach for small-area estimation and undercount adjustment in the census are important potential applications.

Missing Data

In data analysis, serious problems can arise when certain kinds of (quantitative or qualitative) information is partially or wholly missing. Various approaches to dealing with these problems have been or are being developed. One of the methods developed recently for dealing with certain aspects of missing data is called multiple imputation: each missing value in a data set is replaced by several values representing a range of possibilities, with statistical dependence among missing values reflected by linkage among their replacements. It is currently being used to handle a major problem of incompatibility between the 1980 and previous Bureau of Census public-use tapes with respect to occupation codes. The extension of these techniques to address such problems as nonresponse to income questions in the Current Population Survey has been examined in exploratory applications with great promise.

Computer Packages and Expert Systems

The development of high-speed computing and data handling has fundamentally changed statistical analysis. Methodologies for all kinds of situations are rapidly being developed and made available for use in computer packages that may be incorporated into interactive expert systems. This computing capability offers the hope that much data analyses will be more carefully and more effectively done than previously and that better strategies for data analysis will move from the practice of expert statisticians, some of whom may not have tried to articulate their own strategies, to both wide discussion and general use.

But powerful tools can be hazardous, as witnessed by occasional dire misuses of existing statistical packages. Until recently the only strategies available were to train more expert methodologists or to train substantive scientists in more methodology, but without the updating of their training it tends to become outmoded. Now there is the opportunity to capture in expert systems the current best methodological advice and practice. If that opportunity is exploited, standard methodological training of social scientists will shift to emphasizing strategies in using good expert systems—including understanding the nature and importance of the comments it provides—rather than in how to patch together something on one’s own. With expert systems, almost all behavioral and social scientists should become able to conduct any of the more common styles of data analysis more effectively and with more confidence than all but the most expert do today. However, the difficulties in developing expert systems that work as hoped for should not be underestimated. Human experts cannot readily explicate all of the complex cognitive network that constitutes an important part of their knowledge. As a result, the first attempts at expert systems were not especially successful (as discussed in Chapter 1 ). Additional work is expected to overcome these limitations, but it is not clear how long it will take.

Exploratory Analysis and Graphic Presentation

The formal focus of much statistics research in the middle half of the twentieth century was on procedures to confirm or reject precise, a priori hypotheses developed in advance of collecting data—that is, procedures to determine statistical significance. There was relatively little systematic work on realistically rich strategies for the applied researcher to use when attacking real-world problems with their multiplicity of objectives and sources of evidence. More recently, a species of quantitative detective work, called exploratory data analysis, has received increasing attention. In this approach, the researcher seeks out possible quantitative relations that may be present in the data. The techniques are flexible and include an important component of graphic representations. While current techniques have evolved for single responses in situations of modest complexity, extensions to multiple responses and to single responses in more complex situations are now possible.

Graphic and tabular presentation is a research domain in active renaissance, stemming in part from suggestions for new kinds of graphics made possible by computer capabilities, for example, hanging histograms and easily assimilated representations of numerical vectors. Research on data presentation has been carried out by statisticians, psychologists, cartographers, and other specialists, and attempts are now being made to incorporate findings and concepts from linguistics, industrial and publishing design, aesthetics, and classification studies in library science. Another influence has been the rapidly increasing availability of powerful computational hardware and software, now available even on desktop computers. These ideas and capabilities are leading to an increasing number of behavioral experiments with substantial statistical input. Nonetheless, criteria of good graphic and tabular practice are still too much matters of tradition and dogma, without adequate empirical evidence or theoretical coherence. To broaden the respective research outlooks and vigorously develop such evidence and coherence, extended collaborations between statistical and mathematical specialists and other scientists are needed, a major objective being to understand better the visual and cognitive processes (see Chapter 1 ) relevant to effective use of graphic or tabular approaches.

Combining Evidence

Combining evidence from separate sources is a recurrent scientific task, and formal statistical methods for doing so go back 30 years or more. These methods include the theory and practice of combining tests of individual hypotheses, sequential design and analysis of experiments, comparisons of laboratories, and Bayesian and likelihood paradigms.

There is now growing interest in more ambitious analytical syntheses, which are often called meta-analyses. One stimulus has been the appearance of syntheses explicitly combining all existing investigations in particular fields, such as prison parole policy, classroom size in primary schools, cooperative studies of therapeutic treatments for coronary heart disease, early childhood education interventions, and weather modification experiments. In such fields, a serious approach to even the simplest question—how to put together separate estimates of effect size from separate investigations—leads quickly to difficult and interesting issues. One issue involves the lack of independence among the available studies, due, for example, to the effect of influential teachers on the research projects of their students. Another issue is selection bias, because only some of the studies carried out, usually those with “significant” findings, are available and because the literature search may not find out all relevant studies that are available. In addition, experts agree, although informally, that the quality of studies from different laboratories and facilities differ appreciably and that such information probably should be taken into account. Inevitably, the studies to be included used different designs and concepts and controlled or measured different variables, making it difficult to know how to combine them.

Rich, informal syntheses, allowing for individual appraisal, may be better than catch-all formal modeling, but the literature on formal meta-analytic models is growing and may be an important area of discovery in the next decade, relevant both to statistical analysis per se and to improved syntheses in the behavioral and social and other sciences.

• Opportunities and Needs

This chapter has cited a number of methodological topics associated with behavioral and social sciences research that appear to be particularly active and promising at the present time. As throughout the report, they constitute illustrative examples of what the committee believes to be important areas of research in the coming decade. In this section we describe recommendations for an additional $16 million annually to facilitate both the development of methodologically oriented research and, equally important, its communication throughout the research community.

Methodological studies, including early computer implementations, have for the most part been carried out by individual investigators with small teams of colleagues or students. Occasionally, such research has been associated with quite large substantive projects, and some of the current developments of computer packages, graphics, and expert systems clearly require large, organized efforts, which often lie at the boundary between grant-supported work and commercial development. As such research is often a key to understanding complex bodies of behavioral and social sciences data, it is vital to the health of these sciences that research support continue on methods relevant to problems of modeling, statistical analysis, representation, and related aspects of behavioral and social sciences data. Researchers and funding agencies should also be especially sympathetic to the inclusion of such basic methodological work in large experimental and longitudinal studies. Additional funding for work in this area, both in terms of individual research grants on methodological issues and in terms of augmentation of large projects to include additional methodological aspects, should be provided largely in the form of investigator-initiated project grants.

Ethnographic and comparative studies also typically rely on project grants to individuals and small groups of investigators. While this type of support should continue, provision should also be made to facilitate the execution of studies using these methods by research teams and to provide appropriate methodological training through the mechanisms outlined below.

Overall, we recommend an increase of $4 million in the level of investigator-initiated grant support for methodological work. An additional $1 million should be devoted to a program of centers for methodological research.

Many of the new methods and models described in the chapter, if and when adopted to any large extent, will demand substantially greater amounts of research devoted to appropriate analysis and computer implementation. New user interfaces and numerical algorithms will need to be designed and new computer programs written. And even when generally available methods (such as maximum-likelihood) are applicable, model application still requires skillful development in particular contexts. Many of the familiar general methods that are applied in the statistical analysis of data are known to provide good approximations when sample sizes are sufficiently large, but their accuracy varies with the specific model and data used. To estimate the accuracy requires extensive numerical exploration. Investigating the sensitivity of results to the assumptions of the models is important and requires still more creative, thoughtful research. It takes substantial efforts of these kinds to bring any new model on line, and the need becomes increasingly important and difficult as statistical models move toward greater realism, usefulness, complexity, and availability in computer form. More complexity in turn will increase the demand for computational power. Although most of this demand can be satisfied by increasingly powerful desktop computers, some access to mainframe and even supercomputers will be needed in selected cases. We recommend an additional $4 million annually to cover the growth in computational demands for model development and testing.

Interaction and cooperation between the developers and the users of statistical and mathematical methods need continual stimulation—both ways. Efforts should be made to teach new methods to a wider variety of potential users than is now the case. Several ways appear effective for methodologists to communicate to empirical scientists: running summer training programs for graduate students, faculty, and other researchers; encouraging graduate students, perhaps through degree requirements, to make greater use of the statistical, mathematical, and methodological resources at their own or affiliated universities; associating statistical and mathematical research specialists with large-scale data collection projects; and developing statistical packages that incorporate expert systems in applying the methods.

Methodologists, in turn, need to become more familiar with the problems actually faced by empirical scientists in the laboratory and especially in the field. Several ways appear useful for communication in this direction: encouraging graduate students in methodological specialties, perhaps through degree requirements, to work directly on empirical research; creating postdoctoral fellowships aimed at integrating such specialists into ongoing data collection projects; and providing for large data collection projects to engage relevant methodological specialists. In addition, research on and development of statistical packages and expert systems should be encouraged to involve the multidisciplinary collaboration of experts with experience in statistical, computer, and cognitive sciences.

A final point has to do with the promise held out by bringing different research methods to bear on the same problems. As our discussions of research methods in this and other chapters have emphasized, different methods have different powers and limitations, and each is designed especially to elucidate one or more particular facets of a subject. An important type of interdisciplinary work is the collaboration of specialists in different research methodologies on a substantive issue, examples of which have been noted throughout this report. If more such research were conducted cooperatively, the power of each method pursued separately would be increased. To encourage such multidisciplinary work, we recommend increased support for fellowships, research workshops, and training institutes.

Funding for fellowships, both pre-and postdoctoral, should be aimed at giving methodologists experience with substantive problems and at upgrading the methodological capabilities of substantive scientists. Such targeted fellowship support should be increased by $4 million annually, of which $3 million should be for predoctoral fellowships emphasizing the enrichment of methodological concentrations. The new support needed for research workshops is estimated to be $1 million annually. And new support needed for various kinds of advanced training institutes aimed at rapidly diffusing new methodological findings among substantive scientists is estimated to be $2 million annually.

• Cite this Page National Research Council; Division of Behavioral and Social Sciences and Education; Commission on Behavioral and Social Sciences and Education; Committee on Basic Research in the Behavioral and Social Sciences; Gerstein DR, Luce RD, Smelser NJ, et al., editors. The Behavioral and Social Sciences: Achievements and Opportunities. Washington (DC): National Academies Press (US); 1988. 5, Methods of Data Collection, Representation, and Analysis.
• PDF version of this title (16M)

In this Page

Other titles in this collection.

• The National Academies Collection: Reports funded by National Institutes of Health

Recent Activity

• Methods of Data Collection, Representation, and Analysis - The Behavioral and So... Methods of Data Collection, Representation, and Analysis - The Behavioral and Social Sciences: Achievements and Opportunities

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• School Guide
• Mathematics
• Number System and Arithmetic
• Trigonometry
• Probability
• Mensuration
• Maths Formulas
• Class 8 Maths Notes
• Class 9 Maths Notes
• Class 10 Maths Notes
• Class 11 Maths Notes
• Class 12 Maths Notes
• What are the rational numbers between 3 and 5?
• In how many ways a committee of 3 can be made from a total of 10 members?
• Which kind of angle is between the smallest and the largest?
• How many bit strings of length 9 have exactly 4 0's?
• What are non negative real numbers?
• Is 0.5 a whole number?
• What is 2i equal to?
• What are the six trigonometry functions?
• What is the magnitude of the complex number 3 - 2i?
• What are the uses of arithmetic mean?
• How to find the ratio in which a point divides a line?
• Evaluate sin 35° sin 55° - cos 35° cos 55°
• If tan (A + B) = √3 and tan (A – B) = 1/√3, 0° B, then find A and B
• How to find the vertex angle?
• What is the most likely score from throwing two dice?
• If two numbers a and b are even, then prove that their sum a + b is even
• State whether Every whole number is a natural number or not
• How to convert a complex number to exponential form?
• What happens when you subtract two negatives?

What are the different ways of Data Representation?

The process of collecting the data and analyzing that data in large quantity is known as statistics. It is a branch of mathematics trading with the collection, analysis, interpretation, and presentation of numeral facts and figures.

It is a numerical statement that helps us to collect and analyze the data in large quantity the statistics are based on two of its concepts:

• Statistical Data 
• Statistical Science

Statistics must be expressed numerically and should be collected systematically.

Data Representation

The word data refers to constituting people, things, events, ideas. It can be a title, an integer, or anycast.  After collecting data the investigator has to condense them in tabular form to study their salient features. Such an arrangement is known as the presentation of data.

It refers to the process of condensing the collected data in a tabular form or graphically. This arrangement of data is known as Data Representation.

The row can be placed in different orders like it can be presented in ascending orders, descending order, or can be presented in alphabetical order. 

Example: Let the marks obtained by 10 students of class V in a class test, out of 50 according to their roll numbers, be: 39, 44, 49, 40, 22, 10, 45, 38, 15, 50 The data in the given form is known as raw data. The above given data can be placed in the serial order as shown below: Roll No. Marks 1 39 2 44 3 49 4 40 5 22 6 10 7 45 8 38 9 14 10 50 Now, if you want to analyse the standard of achievement of the students. If you arrange them in ascending or descending order, it will give you a better picture. Ascending order: 10, 15, 22, 38, 39, 40, 44. 45, 49, 50 Descending order: 50, 49, 45, 44, 40, 39, 38, 22, 15, 10 When the row is placed in ascending or descending order is known as arrayed data.

Types of Graphical Data Representation

Bar chart helps us to represent the collected data visually. The collected data can be visualized horizontally or vertically in a bar chart like amounts and frequency. It can be grouped or single. It helps us in comparing different items. By looking at all the bars, it is easy to say which types in a group of data influence the other.

Now let us understand bar chart by taking this example  Let the marks obtained by 5 students of class V in a class test, out of 10 according to their names, be: 7,8,4,9,6 The data in the given form is known as raw data. The above given data can be placed in the bar chart as shown below: Name Marks Akshay 7 Maya 8 Dhanvi 4 Jaslen 9 Muskan 6

A histogram is the graphical representation of data. It is similar to the appearance of a bar graph but there is a lot of difference between histogram and bar graph because a bar graph helps to measure the frequency of categorical data. A categorical data means it is based on two or more categories like gender, months, etc. Whereas histogram is used for quantitative data.

For example:

The graph which uses lines and points to present the change in time is known as a line graph. Line graphs can be based on the number of animals left on earth, the increasing population of the world day by day, or the increasing or decreasing the number of bitcoins day by day, etc. The line graphs tell us about the changes occurring across the world over time. In a  line graph, we can tell about two or more types of changes occurring around the world.

For Example:

Pie chart is a type of graph that involves a structural graphic representation of numerical proportion. It can be replaced in most cases by other plots like a bar chart, box plot, dot plot, etc. As per the research, it is shown that it is difficult to compare the different sections of a given pie chart, or if it is to compare data across different pie charts.

Frequency Distribution Table

A frequency distribution table is a chart that helps us to summarise the value and the frequency of the chart. This frequency distribution table has two columns, The first column consist of the list of the various outcome in the data, While the second column list the frequency of each outcome of the data. By putting this kind of data into a table it helps us to make it easier to understand and analyze the data. 

For Example: To create a frequency distribution table, we would first need to list all the outcomes in the data. In this example, the results are 0 runs, 1 run, 2 runs, and 3 runs. We would list these numerals in numerical ranking in the foremost queue. Subsequently, we ought to calculate how many times per result happened. They scored 0 runs in the 1st, 4th, 7th, and 8th innings, 1 run in the 2nd, 5th, and the 9th innings, 2 runs in the 6th inning, and 3 runs in the 3rd inning. We set the frequency of each result in the double queue. You can notice that the table is a vastly more useful method to show this data.  Baseball Team Runs Per Inning Number of Runs Frequency           0       4           1        3            2        1            3        1

Sample Questions

Question 1: Considering the school fee submission of 10 students of class 10th is given below:

In order to draw the bar graph for the data above, we prepare the frequency table as given below. Fee submission No. of Students Paid   6 Not paid    4 Now we have to represent the data by using the bar graph. It can be drawn by following the steps given below: Step 1: firstly we have to draw the two axis of the graph X-axis and the Y-axis. The varieties of the data must be put on the X-axis (the horizontal line) and the frequencies of the data must be put on the Y-axis (the vertical line) of the graph. Step 2: After drawing both the axis now we have to give the numeric scale to the Y-axis (the vertical line) of the graph It should be started from zero and ends up with the highest value of the data. Step 3: After the decision of the range at the Y-axis now we have to give it a suitable difference of the numeric scale. Like it can be 0,1,2,3…….or 0,10,20,30 either we can give it a numeric scale like 0,20,40,60… Step 4: Now on the X-axis we have to label it appropriately. Step 5: Now we have to draw the bars according to the data but we have to keep in mind that all the bars should be of the same length and there should be the same distance between each graph

Question 2: Watch the subsequent pie chart that denotes the money spent by Megha at the funfair. The suggested colour indicates the quantity paid for each variety. The total value of the data is 15 and the amount paid on each variety is diagnosed as follows:

Chocolates – 3

Wafers – 3

Toys – 2

Rides – 7

To convert this into pie chart percentage, we apply the formula:  (Frequency/Total Frequency) × 100 Let us convert the above data into a percentage: Amount paid on rides: (7/15) × 100 = 47% Amount paid on toys: (2/15) × 100 = 13% Amount paid on wafers: (3/15) × 100 = 20% Amount paid on chocolates: (3/15) × 100 = 20 %

Question 3: The line graph given below shows how Devdas’s height changes as he grows.

Given below is a line graph showing the height changes in Devdas’s as he grows. Observe the graph and answer the questions below.

(i) What was the height of  Devdas’s at 8 years? Answer: 65 inches (ii) What was the height of  Devdas’s at 6 years? Answer:  50 inches (iii) What was the height of  Devdas’s at 2 years? Answer: 35 inches (iv) How much has  Devdas’s grown from 2 to 8 years? Answer: 30 inches (v) When was  Devdas’s 35 inches tall? Answer: 2 years.

Please Login to comment...

• School Learning
• WhatsApp To Launch New App Lock Feature
• Top Design Resources for Icons
• Node.js 21 is here: What’s new
• Zoom: World’s Most Innovative Companies of 2024
• 30 OOPs Interview Questions and Answers (2024)

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Data Representation in Computer: Number Systems, Characters, Audio, Image and Video

• Post author: Anuj Kumar
• Post published: 16 July 2021
• Post category: Computer Science
• Post comments: 0 Comments

Table of Contents

• 1 What is Data Representation in Computer?
• 2.1 Binary Number System
• 2.2 Octal Number System
• 2.3 Decimal Number System
• 2.4 Hexadecimal Number System
• 3.4 Unicode
• 4 Data Representation of Audio, Image and Video
• 5.1 What is number system with example?

What is Data Representation in Computer?

A computer uses a fixed number of bits to represent a piece of data which could be a number, a character, image, sound, video, etc. Data representation is the method used internally to represent data in a computer. Let us see how various types of data can be represented in computer memory.

Before discussing data representation of numbers, let us see what a number system is.

Number Systems

Number systems are the technique to represent numbers in the computer system architecture, every value that you are saving or getting into/from computer memory has a defined number system.

A number is a mathematical object used to count, label, and measure. A number system is a systematic way to represent numbers. The number system we use in our day-to-day life is the decimal number system that uses 10 symbols or digits.

The number 289 is pronounced as two hundred and eighty-nine and it consists of the symbols 2, 8, and 9. Similarly, there are other number systems. Each has its own symbols and method for constructing a number.

A number system has a unique base, which depends upon the number of symbols. The number of symbols used in a number system is called the base or radix of a number system.

Let us discuss some of the number systems. Computer architecture supports the following number of systems:

Binary Number System

Octal number system, decimal number system, hexadecimal number system.

A Binary number system has only two digits that are 0 and 1. Every number (value) represents 0 and 1 in this number system. The base of the binary number system is 2 because it has only two digits.

The octal number system has only eight (8) digits from 0 to 7. Every number (value) represents with 0,1,2,3,4,5,6 and 7 in this number system. The base of the octal number system is 8, because it has only 8 digits.

The decimal number system has only ten (10) digits from 0 to 9. Every number (value) represents with 0,1,2,3,4,5,6, 7,8 and 9 in this number system. The base of decimal number system is 10, because it has only 10 digits.

A Hexadecimal number system has sixteen (16) alphanumeric values from 0 to 9 and A to F. Every number (value) represents with 0,1,2,3,4,5,6, 7,8,9,A,B,C,D,E and F in this number system. The base of the hexadecimal number system is 16, because it has 16 alphanumeric values.

Here A is 10, B is 11, C is 12, D is 13, E is 14 and F is 15 .

Data Representation of Characters

There are different methods to represent characters . Some of them are discussed below:

The code called ASCII (pronounced ‘􀀏’.S-key”), which stands for American Standard Code for Information Interchange, uses 7 bits to represent each character in computer memory. The ASCII representation has been adopted as a standard by the U.S. government and is widely accepted.

A unique integer number is assigned to each character. This number called ASCII code of that character is converted into binary for storing in memory. For example, the ASCII code of A is 65, its binary equivalent in 7-bit is 1000001.

Since there are exactly 128 unique combinations of 7 bits, this 7-bit code can represent only128 characters. Another version is ASCII-8, also called extended ASCII, which uses 8 bits for each character, can represent 256 different characters.

For example, the letter A is represented by 01000001, B by 01000010 and so on. ASCII code is enough to represent all of the standard keyboard characters.

It stands for Extended Binary Coded Decimal Interchange Code. This is similar to ASCII and is an 8-bit code used in computers manufactured by International Business Machines (IBM). It is capable of encoding 256 characters.

If ASCII-coded data is to be used in a computer that uses EBCDIC representation, it is necessary to transform ASCII code to EBCDIC code. Similarly, if EBCDIC coded data is to be used in an ASCII computer, EBCDIC code has to be transformed to ASCII.

ISCII stands for Indian Standard Code for Information Interchange or Indian Script Code for Information Interchange. It is an encoding scheme for representing various writing systems of India. ISCII uses 8-bits for data representation.

It was evolved by a standardization committee under the Department of Electronics during 1986-88 and adopted by the Bureau of Indian Standards (BIS). Nowadays ISCII has been replaced by Unicode.

Using 8-bit ASCII we can represent only 256 characters. This cannot represent all characters of written languages of the world and other symbols. Unicode is developed to resolve this problem. It aims to provide a standard character encoding scheme, which is universal and efficient.

It provides a unique number for every character, no matter what the language and platform be. Unicode originally used 16 bits which can represent up to 65,536 characters. It is maintained by a non-profit organization called the Unicode Consortium.

The Consortium first published version 1.0.0 in 1991 and continues to develop standards based on that original work. Nowadays Unicode uses more than 16 bits and hence it can represent more characters. Unicode can represent characters in almost all written languages of the world.

Data Representation of Audio, Image and Video

In most cases, we may have to represent and process data other than numbers and characters. This may include audio data, images, and videos. We can see that like numbers and characters, the audio, image, and video data also carry information.

We will see different file formats for storing sound, image, and video .

Multimedia data such as audio, image, and video are stored in different types of files. The variety of file formats is due to the fact that there are quite a few approaches to compressing the data and a number of different ways of packaging the data.

For example, an image is most popularly stored in Joint Picture Experts Group (JPEG ) file format. An image file consists of two parts – header information and image data. Information such as the name of the file, size, modified data, file format, etc. is stored in the header part.

The intensity value of all pixels is stored in the data part of the file. The data can be stored uncompressed or compressed to reduce the file size. Normally, the image data is stored in compressed form. Let us understand what compression is.

Take a simple example of a pure black image of size 400X400 pixels. We can repeat the information black, black, …, black in all 16,0000 (400X400) pixels. This is the uncompressed form, while in the compressed form black is stored only once and information to repeat it 1,60,000 times is also stored.

Numerous such techniques are used to achieve compression. Depending on the application, images are stored in various file formats such as bitmap file format (BMP), Tagged Image File Format (TIFF), Graphics Interchange Format (GIF), Portable (Public) Network Graphic (PNG).

What we said about the header file information and compression is also applicable for audio and video files. Digital audio data can be stored in different file formats like WAV, MP3, MIDI, AIFF, etc. An audio file describes a format, sometimes referred to as the ‘container format’, for storing digital audio data.

For example, WAV file format typically contains uncompressed sound and MP3 files typically contain compressed audio data. The synthesized music data is stored in MIDI(Musical Instrument Digital Interface) files.

Similarly, video is also stored in different files such as AVI (Audio Video Interleave) – a file format designed to store both audio and video data in a standard package that allows synchronous audio with video playback, MP3, JPEG-2, WMV, etc.

FAQs About Data Representation in Computer

What is number system with example.

Let us discuss some of the number systems. Computer architecture supports the following number of systems: 1. Binary Number System 2. Octal Number System 3. Decimal Number System 4. Hexadecimal Number System

Related posts:

10 Types of Computers | History of Computers, Advantages

• What is Microprocessor? Evolution of Microprocessor, Types, Features

What is operating system? Functions, Types, Types of User Interface

What is cloud computing classification, characteristics, principles, types of cloud providers.

• What is Debugging? Types of Errors

What are Functions of Operating System? 6 Functions

What is flowchart in programming symbols, advantages, preparation.

• Advantages and Disadvantages of Flowcharts

What is C++ Programming Language? C++ Character Set, C++ Tokens

• What are C++ Keywords? Set of 59 keywords in C ++

What are Data Types in C++? Types

What are operators in c different types of operators in c, what are expressions in c types, what are decision making statements in c types, types of storage devices, advantages, examples, you might also like.

What is Problem Solving Algorithm?, Steps, Representation

Types of Computer Software: Systems Software, Application Software

What is Big Data? Characteristics, Tools, Types, Internet of Things (IOT)

Types of Computer Memory, Characteristics, Primary Memory, Secondary Memory

What is Computer System? Definition, Characteristics, Functional Units, Components

What is Artificial Intelligence? Functions, 6 Benefits, Applications of AI

Data and Information: Definition, Characteristics, Types, Channels, Approaches

Advantages and Disadvantages of Operating System

Generations of Computer First To Fifth, Classification, Characteristics, Features, Examples

• Entrepreneurship
• Organizational Behavior
• Financial Management
• Communication
• Human Resource Management
• Sales Management
• Marketing Management

PMCLounge.com

Simplifying Project Management

Data Representation Techniques

Data Representation is one of the tools and techniques of the Plan Resource Management process. A legit question here would be, what is Data Representation of any use as part of this process where we are planning resources?

Charts, be it hierarchical, matrix or text based are used to document and communicate the roles and responsibilities of the team members. Now regardless of the method used, the objective here is to ensure that every work package has a clear owner and everyone in the project team understand their roles and responsibilities.

Here are the most common chart types,

1. Hierarchical Charts

These are your classic chart structures that is used to show relationships in top-down format. Examples include,

Work Breakdown Structure (WBS)

WBS shows how project deliverables are broken down into work packages. Check out more on WBS here

Organization Breakdown Structure (OBS)

OBS lists the project work packages or activities according to the organization’s existing departments or teams. For example, the purchasing department can look into its portion to understand the expected work packages or activities from them as part of the project

Resource Breakdown Structure (RBS)

RBS represents team and physical resources related by category as well as the resource type. Every descending level represents an increasingly detailed description of the resource until the information is good enough to be used in conjunction with WBS. Check out more on RBS here

2. Responsibility Assignment Matrix

A Responsibility Assignment Matrix or RAM shows the connection between work packages or activities and the project team members. One of the biggest advantages of this is the fact that this ensures only one person is accountable for a task avoiding confusion. One example of RAM is a RACI Matrix or Responsible, Accountable, Consult and Inform matrix. This simple chart lists all the activities in the first left column and resources in the first row followed by RACI entries in the matrix

3. Text Oriented Formats

Sometimes detailed descriptions are required and a simple RACI chart won’t do. In that case documents are created clearly outlining descriptive information like responsibilities, competencies, qualifications, position descriptions etc. These documents can be easily reused as templates for future projects as well

Check more articles on  Resource Management

Share this:

Continue reading, related posts.

Which form of Power should you use?

August 27, 2019 May 22, 2020

Conflicts are Good! Here are the 4 Reasons Why

August 26, 2019 September 26, 2019

Vroom’s Expectancy Theory | McClelland’s Human Motivation Theory | McKinsey’s 7S Framework

August 23, 2019 September 26, 2019

3 Comments on “Data Representation Techniques”

Every videos I have watched , you always have someother video in reference to. Why don’t u line up ur videos in orderly manner according to the PMP processes. For ex. Quality management knowledge area has 3 processes. Plan QM, Manage Q, and Control Q) so when we play the first video plan QM, you say we already discussed this process in the previous video. Now what process comes before PlanQM. ?? Like wise all other videos u always say u discussed in previous videos, which is very confusing. Pls take ur videos in orderly manner. Thanks.

Have you checked this link yet? – https://www.pmclounge.com/pmp-training/

super clear and easy to understand. Thank you so much, it have me a lot

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Notify me of follow-up comments by email.

Notify me of new posts by email.

Pass your PMP Exam in 50 days!

Superposed Atomic Representation for Robust High-Dimensional Data Recovery of Multiple Low-Dimensional Structures

• Yulong Wang College of Informatics, Huazhong Agricultural University, China Engineering Research Center of Intelligent Technology for Agriculture, Ministry of Education, China Key Laboratory of Smart Farming Technology for Agricultural Animals, Ministry of Agriculture and Rural Affairs, China

• Video/Poster

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Information

• For Readers
• For Authors
• For Librarians

Developed By

Subscription.

Login to access subscriber-only resources.

Part of the PKP Publishing Services Network

Copyright © 2024, Association for the Advancement of Artificial Intelligence

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 21 March 2024

Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models

• Francisco Carrillo-Perez 1 ,
• Marija Pizurica 1 , 2 ,
• Yuanning Zheng   ORCID: orcid.org/0000-0002-0018-3252 1 ,
• Tarak Nath Nandi 3 ,
• Ravi Madduri   ORCID: orcid.org/0000-0003-2130-2887 3 ,
• Jeanne Shen 4 &
• Olivier Gevaert   ORCID: orcid.org/0000-0002-9965-5466 1 , 5  

Nature Biomedical Engineering ( 2024 ) Cite this article

765 Accesses

30 Altmetric

Metrics details

• Cancer imaging
• Computational models
• Machine learning

Training machine-learning models with synthetically generated data can alleviate the problem of data scarcity when acquiring diverse and sufficiently large datasets is costly and challenging. Here we show that cascaded diffusion models can be used to synthesize realistic whole-slide image tiles from latent representations of RNA-sequencing data from human tumours. Alterations in gene expression affected the composition of cell types in the generated synthetic image tiles, which accurately preserved the distribution of cell types and maintained the cell fraction observed in bulk RNA-sequencing data, as we show for lung adenocarcinoma, kidney renal papillary cell carcinoma, cervical squamous cell carcinoma, colon adenocarcinoma and glioblastoma. Machine-learning models pretrained with the generated synthetic data performed better than models trained from scratch. Synthetic data may accelerate the development of machine-learning models in scarce-data settings and allow for the imputation of missing data modalities.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

92,52 € per year

only 7,71 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

scGPT: toward building a foundation model for single-cell multi-omics using generative AI

Haotian Cui, Chloe Wang, … Bo Wang

Highly sensitive spatial transcriptomics using FISHnCHIPs of multiple co-expressed genes

Xinrui Zhou, Wan Yi Seow, … Kok Hao Chen

Towards a general-purpose foundation model for computational pathology

Richard J. Chen, Tong Ding, … Faisal Mahmood

Data availability

TCGA data can be downloaded from the GDC platform ( https://portal.gdc.cancer.gov/ ). The two GEO series used in this study can be downloaded from the GEO platform: GSE50760 and GSE226069 . The PBTA dataset can be downloaded from the Gabriella Miller Kids First Data Resource Portal (KF-DRC, https://kidsfirstdrc.org ). Microsatellite-instability-status data can be downloaded from the Kaggle platform: https://www.kaggle.com/datasets/joangibert/tcga_coad_msi_mss_jpg . Case IDs used for this work as well as the RNA-seq encodings obtained for all experiments are available under an academic-use-only licence at https://rna-cdm.stanford.edu . One million synthetic images are available in the Dryad platform at https://doi.org/10.5061/dryad.6djh9w174 (ref. 77 ).

Code availability

A demo for generating synthetic images and the code are available under an academic-use-only licence at https://rna-cdm.stanford.edu .

Sung, H. et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71 , 209–249 (2021).

Article   PubMed   Google Scholar  

Jones, P. A. & Baylin, S. B. The epigenomics of cancer. Cell 128 , 683–692 (2007).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lujambio, A. & Lowe, S. W. The microcosmos of cancer. Nature 482 , 347–355 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Frangioni, J. V. New technologies for human cancer imaging. J. Clin. Oncol. 26 , 4012–4021 (2008).

Article   PubMed   PubMed Central   Google Scholar  

Williams, B. J., Bottoms, D. & Treanor, D. Future-proofing pathology: the case for clinical adoption of digital pathology. J. Clin. Pathol. 70 , 1010–1018 (2017).

Heindl, A., Nawaz, S. & Yuan, Y. Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology. Lab. Invest. 95 , 377–384 (2015).

Cheng, J. et al. Identification of topological features in renal tumor microenvironment associated with patient survival. Bioinformatics 34 , 1024–1030 (2018).

Article   CAS   PubMed   Google Scholar  

Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24 , 1559–1567 (2018).

Castillo, D. et al. Integration of RNA-seq data with heterogeneous microarray data for breast cancer profiling. BMC Bioinformatics 18 , 506 (2017).

Yu, D. et al. Copy number variation in plasma as a tool for lung cancer prediction using Extreme Gradient Boosting (XGBoost) classifier. Thorac. Cancer 11 , 95–102 (2020).

Maros, M. E. et al. Machine learning workflows to estimate class probabilities for precision cancer diagnostics on DNA methylation microarray data. Nat. Protoc. 15 , 479–512 (2020).

Chen, R. J. et al. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 16144–16155 (IEEE, 2022).

Carrillo-Perez, F. et al. Machine-learning-based late fusion on multi-omics and multi-scale data for non-small-cell lung cancer diagnosis. J. Pers. Med. 12 , 601 (2022).

Lee, C. & van der Schaar, M. A variational information bottleneck approach to multi-omics data integration. In International Conference on Artificial Intelligence and Statistics 1513–1521 (PMLR, 2021).

Chen, R. J. et al. Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis. IEEE Trans. Med. Imaging 41 , 757–770 (2020).

Article   ADS   Google Scholar  

Cheerla, A. & Gevaert, O. Deep learning with multimodal representation for pancancer prognosis prediction. Bioinformatics 35 , i446–i454 (2019).

Chen, R. J. et al. Pan-cancer integrative histology–genomic analysis via multimodal deep learning. Cancer Cell 40 , 865–878 (2022).

Vanguri, R. S. et al. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L) 1 blockade in patients with non-small cell lung cancer. Nat. Cancer 3 , 1151–1164 (2022).

Lipkova, J. et al. Artificial intelligence for multimodal data integration in oncology. Cancer Cell 40 , 1095–1110 (2022).

Weinstein, J. N. et al. The Cancer Genome Atlas Pan-cancer analysis project. Nat. Genet. 45 , 1113–1120 (2013).

Jennings, C. N. et al. Bridging the gap with the UK Genomics Pathology Imaging Collection. Nat. Med. 28 , 1107–1108 (2022).

Barrett, T. et al. NCBI GEO: archive for functional genomics data sets—update. Nucleic Acids Res. 41 , D991–D995 (2012).

Quiros, A. C., Murray-Smith, R. & Yuan, K. PathologyGAN: learning deep representations of cancer tissue. In Proceedings of the Third Conference on Medical Imaging with Deep Learning 121 , 669–695 (PMLR, 2020).

Quiros, A. C., Murray-Smith, R. & Yuan, K. Learning a low dimensional manifold of real cancer tissue with PathologyGAN. Preprint at https://arxiv.org/abs/1907.02644v5 (2020).

Viñas, R., Andrés-Terré, H., Liò, P. & Bryson, K. Adversarial generation of gene expression data. Bioinformatics 38 , 730–737 (2022).

Mitra, R. & MacLean, A. L. RVAgene: generative modeling of gene expression time series data. Bioinformatics 37 , 3252–3262 (2021).

Qiu, Y. L., Zheng, H. & Gevaert, O. Genomic data imputation with variational auto-encoders. Gigascience 9 , giaa082 (2020).

Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 5769–5779 (Curran Associates, 2017).

Metz, L., Poole, B., Pfau, D. & Sohl-Dickstein, J. Unrolled generative adversarial networks. Preprint at https://doi.org/10.48550/arXiv.1611.02163 (2016).

Salimans, T. et al. Improved techniques for training gans. In Advances in Neural Information Processing Systems 29 (eds Lee, D. et al.) 2234–2242 (Curran Associates, 2016).

Zhao, S., Song, J. & Ermon, S. Infovae: balancing learning and inference in variational autoencoders. Proc. AAAI Conf. Artif. Intell. 33 , 5885–5892 (2019).

Google Scholar  

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C. & Chen, M. Hierarchical text-conditional image generation with CLIP latents. Preprint at https://doi.org/10.48550/arXiv.2204.06125 (2022).

Saharia, C. et al. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems 35 , 36479–36494 (PMLR, 2022).

Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. & Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning 2256–2265 (PMLR, 2015).

Radford, A. et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning 8748–8763 (PMLR, 2021).

Yu, K. H. et al. Association of omics features with histopathology patterns in lung adenocarcinoma. Cell Syst. 5 , 620–627 (2017).

Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1 , 800–810 (2020).

Schmauch, B. et al. A deep learning model to predict RNA-seq expression of tumours from whole slide images. Nat. Commun. 11 , 3877 (2020).

McInnes, L., Healy, J. & Melville, J. UMAP: uniform manifold approximation and projection for dimension reduction. Preprint at https://doi.org/10.48550/arXiv.1802.03426 (2018).

Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 6629–6640 (Curran Associates, 2017).

Binkowski, M., Sutherland, D. J., Arbel, M. & Gretton, A. Demystifying MMD GANS. Preprint at https://doi.org/10.48550/arXiv.1801.01401 (2018).

Kim, S. K. et al. A nineteen gene-based risk score classifier predicts prognosis of colorectal cancer patients. Mol. Oncol. 8 , 1653–1666 (2014).

Quintanal-Villalonga, A. et al. Comprehensive molecular characterization of lung tumors implicates AKT and MYC signaling in adenocarcinoma to squamous cell transdifferentiation. J. Hematol. Oncol. 14 , 170 (2021).

Graham, S. et al. Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58 , 101563 (2019).

Karimi, E. et al. Single-cell spatial immune landscapes of primary and metastatic brain tumours. Nature 614 , 555–563 (2023).

Han, S. et al. Rescuing defective tumor-infiltrating T-cell proliferation in glioblastoma patients. Oncol. Lett. 12 , 2924–2929 (2016).

Steyaert, S. et al. Multimodal deep learning to predict prognosis in adult and pediatric brain tumors. Commun. Med. 3 , 44 (2023).

Lehrer, M. et al. in Advances in Biology and Treatment of Glioblastoma (ed. Somasundaram, K.) 143–159 (Springer, 2017).

Yamashita, R. et al. Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol. 22 , 132–141 (2021).

Marisa, L. et al. Gene expression classification of colon cancer into molecular subtypes: characterization, validation, and prognostic value. PLoS Med. 10 , e1001453 (2013).

Li, W. et al. High resolution histopathology image generation and segmentation through adversarial training. Med. Image Anal. 75 , 102251 (2022).

Karras, T., Aittala, M., Aila, T. & Laine, S. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems , 35 , 26565–26577 (PMLR, 2022).

Chen, R. J., Lu, M. Y., Chen, T. Y., Williamson, D. F. & Mahmood, F. Synthetic data in machine learning for medicine and healthcare. Nat. Biomed. Eng. 5 , 493–497 (2021).

Azizi, S. et al. Robust and efficient medical imaging with self-supervision. Preprint at https://doi.org/10.48550/arXiv.2205.09723 (2022).

Dries, R. et al. Advances in spatial transcriptomic data analysis. Genome Res. 31 , 1706–1718 (2021).

Zheng, H., Brennan, K., Hernaez, M. & Gevaert, O. Benchmark of long non-coding RNA quantification for RNA sequencing of cancer samples. Gigascience 8 , giz145 (2019).

Bray, N. L., Pimentel, H., Melsted, P. & Pachter, L. Near-optimal probabilistic RNA-seq quantification. Nat. Biotechnol. 34 , 525–527 (2016).

Lu, M. Y. et al. AI-based pathology predicts origins for cancers of unknown primary. Nature 594 , 106–110 (2021).

Article   ADS   CAS   PubMed   Google Scholar  

Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5 , 555–570 (2021).

Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9 , 62–66 (1979).

Article   Google Scholar  

Goode, A., Gilbert, B., Harkes, J., Jukic, D. & Satyanarayanan, M. OpenSlide: a vendor-neutral software foundation for digital pathology. J. Pathol. Inform . 4 , 27 (2013).

Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25 , 1054–1056 (2019).

Ijaz, H. et al. Pediatric high-grade glioma resources from the Children’s Brain Tumor Tissue Consortium. Neuro Oncol. 22 , 163–165 (2020).

Higgins, I. et al. beta-VAE: learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations 1–13 (ICLR, 2017).

Hyvärinen, A. & Dayan, P. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res . 6 , 695−709 (2005).

Vincent, P. A connection between score matching and denoising autoencoders. Neural Comput. 23 , 1661–1674 (2011).

Article   ADS   MathSciNet   PubMed   Google Scholar  

Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33 , 6840–6851 (2020).

Ho, J. et al. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res. 23 , 1–33 (2022).

MathSciNet   Google Scholar  

Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention (eds Navab, N. et al.) 234–241 (Springer, 2015).

Grill, J. B. et al. Bootstrap your own latent–a new approach to self-supervised learning. Adv. Neural Inf. Process. Syst. 33 , 21271–21284 (2020).

Kaiser, L. et al. Fast decoding in sequence models using discrete latent variables. Proc. Mach. Learn. Res. 80 , 2390–2399 (2018).

Newman, A. M. et al. Robust enumeration of cell subsets from tissue expression profiles. Nat. Methods 12 , 453–457 (2015).

Newman, A. M. et al. Determining cell type abundance and expression from bulk tissues with digital cytometry. Nat. Biotechnol. 37 , 773–782 (2019).

Harrell, F. E., Califf, R. M., Pryor, D. B., Lee, K. L. & Rosati, R. A. Evaluating the yield of medical tests. JAMA 247 , 2543–2546 (1982).

Longato, E., Vettoretti, M. & Di Camillo, B. A practical perspective on the concordance index for the evaluation and selection of prognostic time-to-event models. J. Biomed. Inform. 108 , 103496 (2020).

Graf, E., Schmoor, C., Sauerbrei, W. & Schumacher, M. Assessment and comparison of prognostic classification schemes for survival data. Stat. Med. 18 , 2529–2545 (1999).

Carrillo-Perez, F. RNA-to-image multi-cancer synthesis using cascaded diffusion models, one million synthetic images. Dryad https://doi.org/10.5061/dryad.6djh9w174 (2023).

Download references

Acknowledgements

The results published here are in whole or in part based on data generated by the TCGA Research Network ( https://www.cancer.gov/tcga ). F.C.-P. was supported by MCIN/AEI/10.13039/501100011033 (grant number PID2021-128317OB-I00), Consejería de Universidad, Investigación e Innovación (grant number P20-00163), which are both funded by ‘ERDF A way of making Europe.’, and a Predoctoral scholarship from the Fulbright Spanish Commission. M.P. was supported by the Belgian American Educational Foundation and FWO (grant number 1161223N). Research reported here was further supported by the National Cancer Institute (NCI) (grant number R01 CA260271). This research used resources of the Argonne Leadership Computing Facility, a U.S. Department of Energy (DOE) Office of Science user facility at Argonne National Laboratory and is based on research supported by the U.S. DOE Office of Science-Advanced Scientific Computing Research Program, under Contract No. DE-AC02-06CH11357. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

Stanford Center for Biomedical Informatics Research (BMIR), Stanford University, School of Medicine, Stanford, CA, USA

Francisco Carrillo-Perez, Marija Pizurica, Yuanning Zheng & Olivier Gevaert

Internet technology and Data science Lab (IDLab), Ghent University, Ghent, Belgium

Marija Pizurica

Data Science and Learning Division, Argonne National Laboratory, Lemont, IL, USA

Tarak Nath Nandi & Ravi Madduri

Department of Pathology, Stanford University, School of Medicine, Palo Alto, CA, USA

Jeanne Shen

Department of Biomedical Data Science, Stanford University, School of Medicine, Stanford, CA, USA

Olivier Gevaert

You can also search for this author in PubMed   Google Scholar

Contributions

F.C.-P., M.P. and O.G. conceived and designed the study. F.C.-P., M.P. and Y.Z. performed data preprocessing. F.C.-P. developed the code. T.N.N. and R.M. contributed to code optimization and parallel training. R.M. and T.N.N. provided access to the Argonne National Laboratory platform. J.S. performed the analysis of the clinical impact and analysed the digital pathology quality. Y.Z. obtained the deconvolved RNA-seq data. F.C.-P. and M.P. generated the figures. O.G. supervised the work and obtained the funding. F.C.-P. and O.G. wrote the manuscript with contributions and/or revisions from all authors.

Corresponding author

Correspondence to Olivier Gevaert .

Ethics declarations

Competing interests.

Stanford has submitted a provisional patent application for this work with patent number 18/538,743, United States, 2023. The authors declare no competing interests.

Peer review

Peer review information.

Nature Biomedical Engineering thanks Moritz Gerstung, Ke Yuan and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 cell-percentage comparison between using bulk rna-seq and de-convolved expression..

a . Percentage of lymphocytes cells found by Hovernet in synthetic tiles generated using bulk RNA-Seq and haematopoietic de-convolved RNA-Seq. A significantly higher percentage of lymphocytes was found in all four out of five cancer types with a significantly p-value in four out of five of them (TCGA-CESC p-value = 0.15; TCGA-KIRP p-value = 6.08 × 10 −21 ; TCGA-LUAD p-value = 9.86 × 10 −16 ; TCGA-GBM p-value = 2.02 × 10 −7 ; TCGA-COAD p-value = 1.07 × 10 −22 ). The median difference is annotated in the plot per cancer type. b . UMAP projection of the bulk RNA-Seq expression (circles) and the counterpart deconvolved haematopoietic RNA-Seq (crosses). Clear differences can be observed in the expression, with a mean percentage difference of 7% across the cancer types, which corresponds to a similar increase in lymphocytes in the majority of the cancer types.

Extended Data Fig. 2 Microsatellite-instability-status prediction.

Comparison between a model trained from scratch and a model that have been pretrained using SimCLR on synthetic tiles, on a different number of real tiles sampled from the training set. Metrics are computed on a fivefold CV, and results correspond to those obtained on the different test sets. The model pretrained on the synthetic tiles always outperform the model trained from scratch, no matter the number of training samples that are used.

Supplementary information

Supplementary information.

Supplementary Figures and Tables.

Reporting Summary

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Carrillo-Perez, F., Pizurica, M., Zheng, Y. et al. Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models. Nat. Biomed. Eng (2024). https://doi.org/10.1038/s41551-024-01193-8

Download citation

Received : 27 December 2022

Accepted : 29 February 2024

Published : 21 March 2024

DOI : https://doi.org/10.1038/s41551-024-01193-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Help | Advanced Search

Computer Science > Robotics

Title: dhp-mapping: a dense panoptic mapping system with hierarchical world representation and label optimization techniques.

Abstract: Maps provide robots with crucial environmental knowledge, thereby enabling them to perform interactive tasks effectively. Easily accessing accurate abstract-to-detailed geometric and semantic concepts from maps is crucial for robots to make informed and efficient decisions. To comprehensively model the environment and effectively manage the map data structure, we propose DHP-Mapping, a dense mapping system that utilizes multiple Truncated Signed Distance Field (TSDF) submaps and panoptic labels to hierarchically model the environment. The output map is able to maintain both voxel- and submap-level metric and semantic information. Two modules are presented to enhance the mapping efficiency and label consistency: (1) an inter-submaps label fusion strategy to eliminate duplicate points across submaps and (2) a conditional random field (CRF) based approach to enhance panoptic labels through object label comprehension and contextual information. We conducted experiments with two public datasets including indoor and outdoor scenarios. Our system performs comparably to state-of-the-art (SOTA) methods across geometry and label accuracy evaluation metrics. The experiment results highlight the effectiveness and scalability of our system, as it is capable of constructing precise geometry and maintaining consistent panoptic labels. Our code is publicly available at this https URL .

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .