Postsecondary Success Asset Map: A Toolkit for School Improvement

The PSAM survey is organized by the five asset areas. While the assets together form an interconnected whole, we recommend creating a data set for each asset area and bringing your PSAM team together either (a) monthly for five times to analyze the data for each asset or (b) a full-day or two half-day retreats. A 60-90 minute block is needed for each asset area.

The assets are equal in importance and do not need to be reviewed and discussed in any particular order. However, what is helpful is to align the analysis of the data with current school plans and priorities, such as the overall school improvement plan or professional development plan. For example, if a school-wide professional development priority is math, not only should you consider starting with the Content Knowledge asset, you may want to bring in the chair of the math department for that analysis meeting. Think strategically about what the best order is for your school and who will be most helpful in reviewing and reflecting on each data set.

The school will receive a survey report with raw data that needs to be exported into an Excel spreadsheet, which has the functionality to present your data in graphs, charts, and tables. Unless you have someone on your staff with the time and expertise to export, filter, and present your data in a visually accessible way, we recommend sending the raw data file to either a consultant or a trusted partner with data collection and analysis expertise–a university, CBO, researcher, or perhaps a graduate student – to process it for your school.

Resource

Agenda: Analyzing PSAM Data

Bring your PSAM team together

By the time you have organized your data, you should have a committed team of staff and administrators who have been helping to lead the PSAM process. This can be an existing data team or a specific PSAM committee. Of great importance is that the team reflects the diversity of roles within the school to capture varying perspectives. Ideally, the team will include:

• Principal
• One other senior administrator (e.g., AP, Dean)
• Instructional or advisory leadership in each of the core content areas (ELA and Math in particular)
• Guidance counselor, college counselor, and/or career advisor
• College support partner, organization (e.g., College Access Program coordinator or advisor)
• Other key community partners/stakeholders

As mentioned in the math professional development example above, consider inviting content or skill area experts to join when you analyze and discuss specific asset areas. For example, a higher education partner could be very helpful to any of the asset discussions, but perhaps most of all to Transition Knowledge and Skills.

Analyze PSAM survey data

Analyzing the data from your PSAM survey means organizing, reviewing, and discussing it in ways that reveal the relationships, patterns, and trends that can be found within each asset and across them. It means asking questions about the data that will help reveal patterns, surprises, and important findings. Some helpful questions for the team members to consider for each asset are:

Individually:

• What do I notice about the data? What resonates with me as—Surprising or familiar? Likely or unlikely? Clear or confusing?
• What specific questions does the data raise for me?
• What issues do I think are most important to discuss?

Collectively:

• What obvious and important findings and patterns does the data reveal?
• How reliable are the responses provided by our data? How do we know?
  • What other data do we have that can give more information about these findings?
• How does this data compare to other data that we have? For example, how does PSAM data compare to quantitative data on the same issue or topic, such as attendance data or 9th grade retention rates?
• What survey items reflect disagreement among the staff? What might account for the areas of disagreement or lack of consensus for any particular item among the faculty? Some responses might be:
  • We need more data or better data.
  • We need more specific data about the grade level/s in which students are being served.
  • We need to better share and communicate data and information to the school community.
  • There may be disagreement or uncertainty about the grade levels in which students are being served.
  • There may be disagreement about the numbers of students who are being reached. If so, what can we do about that?
  • There may be disagreement about the quality of the program. If so, what are the stated goals of the program and are they being met?

Most importantly, as outlined below, is: What is the evidence for determining the validity of the finding? What other data do we have that can give more information if needed? For example, how can we determine which students have a certain skill or are being taught it? Can we use classroom observations? Grades? Program enrollment and attendance data? Reports at department meetings?

Analyzing the data is the first opportunity for the team to begin to better understand the current state of the college and career readiness options being offered in order to plan how to increase the numbers of students being served by effective programs and/or improve the options and accessibility.

Identify and review your evidence

Many survey respondents will answer questions based on their perceptions and on the anecdotes they have heard and shared with their colleagues. For this reason, it is critical that the PSAM team identify the evidence or lack of for findings by asking:

• What is the evidence that a specific program or policy is effective?
• What is the evidence for determining how many students have a certain skill or are involved in a program, or taking advantage of the opportunities that the school is offering?

The following is a sample of the kinds of evidence to consider when discussing and analyzing the PSAM data.

• Student data by grade-levels
  • Course grades, enrollment, discipline, race, ethnicity, SES, credit accumulation, volunteer and extra-curricular activities, etc.
• Family engagement by student grade-levels
• School and district policies
  • Compliance data
• Ninth grade orientation
  • Enrollment, curriculum
• Graduation requirements
• Curricula and course offerings matched to student enrollment data
• Professional development curricula and opportunities
• Classroom practices
• Assessment calendars and guides
• Targeted guidance counseling/advisory
  • Participants, case loads
• School- led college prep programs/activities matched to student participation data
• Partner-led programs/activities
• Work-based learning opportunities (e.g., job shadowing, internships)
• After school, weekend, and summer programs

Digging deeper: more questions to ask about the data

Depending on how much time your team has to discuss the data, the team may want to go deeper with particular programs or policies that rise to the surface because they are perceived as, for example, effective but under-utilized. For example:

• Which students are most served by the college and career readiness programs made available by our school? Are we reaching all the students, especially those who need it most? How do we know?
• Or, do we have the same group of students participating in these programs? If so, which students or sub groups consistently do not take advantage of these opportunities?
• What do we think are the issues and barriers that prevent these students from participating in these programs?
• What do you think the school can do to ensure that students who need them most can participate in them?

Tips for Success for School Leaders

• Communicate the purpose and goals of doing this work. It's important for the team to have a clear sense of why they are doing this and what they are striving to achieve.
• Include another high-level school adminsitrator on the PSAM team to spread and build capacity for the work. It is still important for the principal to participate, but including other school leaders will further demonstrate the importance of the work and deepen buy-in.
• Designate a team member to document in writing the group’s goals, decisions, progress, and accomplishments. Keep a binder or folder of the important documents and worksheets, which can be celebrated and made accessible to all members of the school community once it’s completed.
• Work closely with this or another designated team member to maintain and organize the work of the team.
• Decide the order in which the team wants to review and analyze the PSAM data for each meeting. As mentioned, there may be a natural choice based on the school improvement plan, upcoming professional development, or other local issues.
• Plan ahead. If possible, give team members the data in advance of each meeting and ask them to look at it before everyone sits down together to analyze and discuss it. The team will need about 45-60 minutes for the discussion in each asset area. If they have had a chance to look at the data before the meeting, you can use valuable meeting time for discussions.
• Allow time in each meeting for team members to reflect on and discuss what the data means, and where there are areas of strength.
• Keep a “parking lot.” Document ideas you don't want to lose in a “parking lot” for when the team will be setting priorities and developing the plan of action.
• Use or modify the Activity: Reviewing and Analyzing Data from the PSAM Survey