A Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
MEASURING WHAT MATTERS: EXERCISES IN DATA MANAGEMENT
EXERCISE 3: AGGREGATE AND ANALYZE
Acknowledgments The National Center on Parent, Family, and Community Engagement would like to acknowledge the
leadership of the Harvard Family Research Project, with support from the Brazelton Touchpoints Center, in developing this resource. These organizations represent diverse roles, expertise, and perspectives; their input and feedback were essential in creating this resource. We recognize and value the role of
parents and programs in making a difference for children, families, and communities.
This document was originally developed with funds from Grant #90HC0003 and modified with funds from Grant #90HC0014 for the U.S. Department of Health and Human Services, Administration for Children and Families,
Office of Head Start, and Office of Child Care, by the National Center on Parent, Family, and Community Engagement. This resource may be duplicated for noncommercial uses without permission.
For more information about this resource, please contact us: | 1-866-763-6481
Suggested citation: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Head Start, National Center on Parent, Family, and Community Engagement.
(Revised 2019). Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze.
Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Measuring What Matters Exercise 3: Aggregate and Analyze Exercise 3 is about aggregating, disaggregating, and analyzing data. Analyzing data means examining information you have collected and making sense of it.
This exercise introduces two ways to analyze your program’s data: 1) aggregation, and 2) disaggregation. “Aggregation” involves combining and presenting similar data from multiple sources. “Disaggregation” means taking a summary of data and breaking it into parts. Aggregating and disaggregating data can help you organize the data you have collected. Next, you can analyze and use the data.
This exercise presents a scenario about the fictional Hopeful Beginnings Head Start Program as it analyzes data from its seven sites. You can use this exercise to:
• Understand how aggregating data can give a whole picture of your program’s PFCE work.
• Understand how disaggregating data can provide information about how program sites or subgroups of families are making progress toward goals.
• Analyze data to help track family and program progress toward goals.
How to Use Exercise 3:
On Your Own • Read the scenario, Aggregating and Analyzing Data to Build Family
Connections. • Complete Table 4, using information from your own program.
With a Group • Share your answers to the prompts in Table 4. • Write any remaining questions you have about the data concepts in the
exercise. • Create a plan for applying the data concepts from the exercise to
your own work.
The Four Data Activities to Support Family Progress Toward Positive Family Outcomes
The exercises in this series are organized to follow the Four Data Activities. Each of these exercises focuses on a specific activity:
• Prepare: Get ready for data collection by thinking about the program goals, objectives, services, and expected outcomes that you need in order to show the reach and impact of your work.
• Collect: Identify how to gather data that are useful and easy to interpret.
• Aggregate and Analyze: Learn ways to look at data to examine progress for families and your program.
• Use and Share: Understand the importance of sharing data in accurate, appealing, and accessible ways and how data can inform various aspects of programming.
Begin with the Prepare exercise and follow with Collect, Aggregate and Analyze, and Use and Share. There may be times when it is useful to revisit one of the Four Data Activities as you learn more about your program’s data and progress.
2 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Parent and Family
In this resource, “parent” and
“family” refer to all adults who
interact with early childhood
systems in support of their child,
including biological, adoptive
and foster parents, pregnant
women and expectant families,
grandparents, legal and informal
guardians, and adult siblings.
Introduction The Head Start Parent, Family, and Community Engagement (PFCE) Framework is an organizational guide for collaboration among families and Head Start and Early Head Start programs, staff, and community service providers to promote positive, enduring outcomes for children and families. The Framework identifies equity, inclusiveness, cultural and linguistic responsiveness, and positive goal-oriented relationships as important drivers for these outcomes.
The PFCE Framework shows how family engagement strategies can be systemic, integrated, and comprehensive across services and systems in line with the Head Start Program Performance Standards.
Head Start Parent, Family, and
PROGRAM IMPACT AREAS
Continuous Learning and Quality Improvement
Teaching and Learning
Access and Continuity
Positive ParentChild Relationships
Families as Lifelong Educators
Families as Learners
Family Engagement in Transitions
Family Connections to Peers and Community
Families as Advocates and Leaders
Healthy and well
Learning and developing
Engaged in positive relationships with family members, caregivers, and other children
Ready for school
Successful in school and life
Equity, Inclusiveness, Cultural and Linguistic Responsiveness
Positive & GoalOriented Relationships
Community Engagement Framework
You can use data to engage families and support progress toward one or more of the seven PFCE Framework Family Outcomes. You can also use data to track progress as your program sets goals and develops and implements plans within the five-year project period.
3 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Aggregating and Disaggregating Data
What does it mean to aggregate data?
Aggregating data means summarizing or combining similar data. For example, an Early Head Start program could aggregate data about the total number of home visits that family services workers (FSWs) conduct in a single year (e.g., 214 home visits in year 2018). Or a Head Start program could aggregate data about the effectiveness of implementing a new parenting curriculum. Aggregate data is a summary of all of the data for a particular topic. By aggregating data, you may find stories and patterns in your program—about how services are delivered, for example, or about what is important to families, such as their home language.
What does it mean to disaggregate data?
Disaggregating data means breaking data into smaller parts. These smaller units of data are often called subsets or subgroups. Subgroups are usually made up of people or things that share certain characteristics.
Programs can use disaggregated data to compare subgroups. By comparing these data, you can determine whether there are specific subgroups that would benefit from additional training, services, or support.
For example, you can get a better understanding of families’ employment by creating subsets of data. These subsets might include information about neighborhoods, access to transportation, or education level.
Consider “access to transportation” to disaggregate employment data. A program may learn that 75 percent of family members who are employed have access to regular, safe transportation, while 25 percent of families do not have regular employment due to lack of access to safe and reliable transportation.
• Understand how to aggregate data across program sites and different service areas.
• Learn how to disaggregate data to identify patterns in different groups of families or sites.
• Analyze data to track family and program progress toward goals.
4 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
The Hopeful Beginnings Program: Aggregating and Analyzing Data to Build Family Connections Sylvia Wong is the Family Services Manager of the Hopeful Beginnings Program (HBP). HBP serves 450 families across seven Head Start sites. Sylvia was surprised by some of the results from a recent Family Strengths and Needs Assessment review. Families complete this assessment as part of the family partnership process. Sylvia noted that 50 percent of families reported they had very little time to socialize with family and friends. Fifty-five percent of families reported never taking part in community activities (see Figures 1 and 2).
Figure 1. Family Strengths and Needs Assessment: Figure 2. Family Strengths and Needs Assessment: Family time for socializing Participation in community activities
Do you and your children participate in community activities together (e.g., visit the library, playgrounds, and community organizations)?
Do you have time to socialize with family and friends?
Sylvia met with the family services team to discuss the results. The team consisted of FSWs and parents. The FSWs verified that during intake interviews many families seemed somewhat isolated. Sylvia was concerned. She knew from research that strong social support networks, including connections to peers and a larger community, are essential for positive family and child outcomes.
5 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Prepare The data led Sylvia and the team to revist a goal for the program’s five-year plan: support the development of Family Connections to Peers and Community, one of the seven Family Outcomes of the PFCE Framework. Table 1 shows the goal, objectives, and services (actions) related to this Family Outcome and related measures that the planning team proposed in its five-year application.
Table 1. HBP’s Baseline Application Information
Goal Objective Services (Actions) Expected Outcome Measures
What program What are we planning to do What actions are we What are our expected Measures of Effort: How much goal do we want to to reach our program goal? going to take to reach outcomes related to our programming are we offering? accomplish to support our objectives? goals, objectives, and services Are we carrying out services as family outcomes? (actions)?
Which PFCE Framework Family Outcome does this represent?
Measures of Effect: What difference is our program making? What are the changes in knowledge and behavior?
HBP will support the Over a five-year period, • Disseminate Families will have the Measures of effort: Number of development of Family HBP will: information about information, knowledge, and community resources parents are Connections to Peers 1) Increase families’ community resources access necessary to make use connected to and number of parents and Community. knowledge and awareness
of community resources they can use by providing them each month with information about such resources.
2) Increase opportunities for families to establish connections to one another by encouraging family participation in at least five family nights.
and programs on a monthly basis.
• Host family nights once a month so parents can get to know each other and discuss parenting topics. Provide child care and interpreters.
of community resources and each year will attend at least five parent networking meetings organized by HBP.
(PFCE Family Outcome: Family Connections to Peers and Community)
who attend the networking events.
Measures of effect: Parents report using more community resources than before and developing relationships with other families in the program and community.
6 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Collect In order to collect more specific data to track the program’s progress on its goal and objectives, Sylvia, the team, and a group of parents decided to create a survey about Family Connections to Peers and Community (see Figure 3). The survey included questions about how often families connected with friends and extended family members, and how often families took their children to local community sites, such as playgrounds or the public library. The survey also included an anonymous Family ID number for each family who filled out the survey, and a Site ID number for each family’s program site. The team translated the survey into the languages spoken by families in the program.
Before finalizing the survey, a group of families who were not part of the development team reviewed the survey and offered feedback. They suggested that families might increase their connections to peers and community at faith-based or cultural events. This new information was incorporated into the survey.
HOPEFUL BEGINNINGS PROGRAM FAMILY CONNECTIONS TO PEERS AND COMMUNITY SURVEY
We are interested in learning about some of the activities you do with your children and other families in the community. Please circle the number that corresponds to the best answer to the following questions.
How often do you… Never Rarely A few
times a month
A few times a week
Once a day
Spend time with your extended family? 1 2 3 4 5 Spend time with friends? 1 2 3 4 5 Set up playdates for your children? 1 2 3 4 5 Talk to other parents about parenting? 1 2 3 4 5 Take your child to the library? 1 2 3 4 5 Take your child to the park or playground? 1 2 3 4 5 Take your child to other local community sites? 1 2 3 4 5
Attend community events with your family (e.g., faith-based and cultural, town, or school events)?
1 2 3 4 5
7 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
The HBP team decided to integrate this survey into its program-wide family partnership process. In the fall and spring of every year, FSWs at the program were responsible for asking families questions about their goals, plans, and experiences in the community.
Sylvia trained the FSWs on how to ask families the new survey questions correctly. She also worked with the database software developer to create a new portal where FSWs could easily enter the Connections survey data. And she trained the FSWs to enter results accurately into the new system.
By December, the team had collected surveys from 317 families (70 percent of all families served). The team members followed specific steps to carefully enter the data.
ENTERING SURVEY DATA
Family services workers from HBP completed the Family Connections to Peers and Community survey, with families using paper surveys. They transferred the results into the online data management system by using the following steps:
Step 1. Locating each family in their database using the family’s unique ID.
Step 2. Entering the Connections survey portal.
Step 3. Selecting the response each family provided.
Step 4. Saving the information. photo courtesy of NCQTL
8 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Sylvia was excited to see the data from the surveys and would use a report from the program’s data management system to begin the review. She and the team needed to develop a data analysis plan–a roadmap for how to organize and analyze the data. Table 2 shows what Sylvia and the team planned for the analysis. They identified how they would analzye their data—and then left room to record their results.
Table 2. HBP Data Analysis Plan
Step Data Source Analysis Results
Identify how the program is doing overall in terms of the expected outcome, Family Connections to Peers and Community. This involves calculating the program-wide average connection score (see page 9).
Quantitative data analysis
Analyze families’ answers to each individual question in the survey. These answers will indicate the areas of family connections to peers and community that need the greatest improvement.
Qualitative data analysis
9 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Aggregating the Data The first step was to aggregate (summarize) the data across all sites. They did this to understand how all families, across all seven sites, were connecting to other families and community resources. The team wanted a score that represented an average of the data gathered from all families across the entire program. They called this the “program-wide average connection score.”
Calculating the Program-Wide Average Connection Score
To calculate the program-wide average connection score, Sylvia and the team took the following steps:
Step 1: Calculate the total score (or sum) for each family. The team did this by adding the numbers in each row together. Since the total number of questions in the survey was 8 and the score scale
Figure 4. HBP Program-Wide Average Connection Score
was 1–5, the minimum score a family could get was 8 (1 point x 8 questions), while the maximum was 40 (5 points x 8 questions).
Step 2: Calculate the program-wide average connection score. To do this, they added together the sum scores for each family (from Step 1). Then they divided that total by the number of families that had participated in the survey (317).
When HBP added the scores of all families together, the total sum was 5,706. They then divided this number by 317 (the number of families who participated in the survey). This resulted in a program-wide average connection score for HBP of 18 (5,706 / 317 = 18). See Figure 4.
HBP’s score was less than half of the maximum or ideal score. The data clearly showed that families were struggling to connect with their peers and the community. However, this program-wide average connection score did not give Sylvia and the team any specific information about family connections. The team members knew they needed to analyze their data further to find that information.
10 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
photo courtesy of NCQTL
Disaggregating the Data Sylvia and the FSWs needed to find more details about how families in HBP were connecting to their peers and community. In the data analysis plan, the team members had indicated they wanted to understand how families in the program had answered each of the survey questions. The answers would tell them about the activities families do with their children and other families in their communities. They needed to disaggregate their data by question.
Breaking the Data Down By Survey Question
Breaking the data down by how parents responded to each of the survey questions meant that Sylvia and the team needed to calculate each question’s average connection score.1 On average, how often are families going to the library? To the playground? How often are families spending time with friends?
Sylvia and the team calculated each question’s average score by using the following steps:
Step 1. Add up all the scores in each column that represents a survey question.
Step 2. Divide the total sum in each column by the number of families that answered that given question.
Figure 5. HBP Average Connection Scores by Question
Talking about Parenting
Community Events 2.60
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
By disaggregating the survey data by question, the team learned more about the families in the program. While parents were spending a good amount of time with their own families (at least once a week), they were rarely talking to other parents about parenting. The team also learned that parents were only rarely taking their children to the local library.
1 There are many ways to disaggregate data. The examples provided here involve disaggregating data by question, by program site, or by family subgroups. Data can also be disaggregated by gender, ethnicity, education level, number of children, and age.
Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Analyzing the Data After disaggregating the data by question, Monica, one of the FSWs, wanted to understand what was going on at her site. She asked about her site’s average connection score. That request gave the team a great idea. They decided to expand what they had included in the data analysis plan by disaggregating the data by site. The team disaggregated the data to compare the average connection scores across program sites. They could then see whether there were specific sites in greater need of support. Figure 6 displays what they found.
Figure 6. HBP Average Connection Scores by Site
Monica was surprised to discover that her site (Site 7) had the lowest average connection score. She knew that parents at her site had expressed feelings of isolation. But she didn’t expect the score for her site to be so low.
Monica had a number of questions, but she wasn’t sure what to do next. During the team discussion, Sylvia suggested that Monica check in with José, the FSW for Site 6, which had the highest average connection score of 23.
Monica and José met to discuss Monica’s questions. They decided that they needed to outline specific information about each of their sites. They worked together to list the major characteristics of the families and community represented at their sites.
12 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Suggestions for Disaggregating Data by Subgroups of Families
How do you determine whether it makes sense to analyze data about a subgroup in your program? It depends on how large that subgroup is relative to your overall program. As a general rule, if 25 percent or more of program families fall into a particular subgroup, you should include that subgroup in your disaggregation analysis.
For example, if 10 percent of program families have one child, that subgroup of families may be too small to focus on for your data analysis.
In contrast, if 30 percent of families in your program are unemployed, the subgroup may be large enough for you to include in a disaggregation analysis.
Site 6 Characteristics
• The community has seen an increase in the number of families experiencing homelessness due to rising rents in the neighborhood.
• A library, elementary school, and park are within walking distance of the site.
• The site has an active father-engagement group.
• Many families have been enrolled at the site for more than one year.
Site 7 Characteristics
• Families live in a housing complex outside of town.
• The community lacks reliable public transportation in the evenings and on weekends.
• Many parents are full-time students at a local community college.
• Families report having close connections with extended family members.
• A library satellite site is within walking distance of the site.
Monica and José shared the list with the team. The team noted that families and the community at both sites exhibited strengths and challenges. The program staff thought about what they could do to partner with families to build on strengths and address challenges. They decided to talk with families at their program to learn about parents’ perspectives.
13 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
As the HBP example shows, aggregating and disaggregating data can be very helpful in making sense of data. The next step is to think about what the program is learning from that analysis. What does it mean for the program’s goal, objective(s), and services (actions)? In other words, when you analyze data, you need to keep in mind two important questions:
1. What? What did we learn?
2. So what? What does this mean for our program?
Using these two questions, Sylvia and the team reflected on their experience.
What did we learn? So what does this mean for our program?
HBP’s program-wide average connection score A connection score of 18 is relatively low. It was 18 on a scale of 1–40. (1=lowest; 40=highest) confirms the need for the program to focus on its
primary goal: supporting family connections with peers and community.
HBP’s families, on average, were2:
1. Spending time with their extended families at least once a week
2. Spending time with friends at least once a month
3. Talking with other parents about parenting only rarely
4. Taking their children to the library only rarely
5. Taking their children to museums, zoos, or other educational community attractions almost never
6. Attending community events several times a month
To design services and activities to meet the goal of increasing family connections with peers and community, HBP needs to pay close attention to the need for:
1. Networking opportunities for families to talk about parenting
2. Increasing awareness of and facilitating access to:
• Public libraries
• Museums, zoos, and other educational community attractions
Site 7 had the lowest connection score (9), while Further analysis of strategies used at Site 6 may Site 6 had the highest (23). benefit Site 7 and other sites.
2 Keep in mind that these results are program-wide averages. These results do not capture the extent of isolation that might exist for families at individual sites. This can be explored through data disaggregation.
How can programs engage families in data analysis?
• Gather input from families about the questions they would like to be asked.
• Find out what questions they would like answers to.
• Develop tools that collect valuable information and that engage families in a relationship-building process.
• Invite families to analyze the data if the data is anonymous.
• Share the results of the analysis with families. Use graphics to illustrate the results.
• Ask families about what they think the data means. How do they make sense of the data?
• Discuss potential uses for data.
14 Measuring What Matters: Exercises in Data Management—Exercise 3: Aggregate and Analyze
Using and Sharing Data for Program Improvement After analyzing the data from the fall of Year 1, Sylvia and the team revisited their program’s five-year plan. They knew that to make progress toward HBP’s goal, they would need to intensify their services. Handing out information and holding monthly parenting nights was not enough. They decided to add three additional activities in the spring:
• Distribute library card applications at parent-teacher conferences. A mother on the planning team explained that many families did not go to the library because they do not have library cards. Families didn’t know how to get library cards and felt intimidated asking for them. HBP arranged for library card applications in both English and Spanish to be available at all spring parent-teacher conferences. Families could fill them out at the event, with the assistance of the teacher. Interpreters were available when necessary. Later in the month, the family services team coordinated visits to the local libraries, where families could hand in their applications and receive their library cards.
• Arrange a monthly field trip to a museum, library, or other organization. To support families’ use of community resources, each site’s family services team set up field trips to different community organizations. Some field trips were held during program hours
Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we’ll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.
Why Hire Collepals.com writers to do your paper?
Quality- We are experienced and have access to ample research materials.
We write plagiarism Free Content
Confidential- We never share or sell your personal information to third parties.
Support-Chat with us today! We are always waiting to answer all your questions.