The SEELS data collection plan is designed to collect in-depth longitudinal information on the school and nonschool experiences of 8- to 12-year-olds who are special education students at the beginning of the 1999-2000 school year. Because SEELS is intended to be far reaching in terms of conceptual domains that it will address, data will be collected from multiple sources by using several data collection methods. Instruments to be used in data collection are in the process of development. Whenever possible, items from existing data collection instruments will be used to maximize the extent to which SEELS data can be compared with other national databases.

Below, we describe the general approach to each data collection component designed for SEELS. We then outline the content of each data collection instrument in terms of the domains of the SEELS conceptual framework and the specific research questions that each will address. Discussion of approaches to the issues of obtaining parental consent for data collection and identifying schools attended by sample students conclude this section.

Parents/guardians will be interviewed during years 1, 3, and 5 of SEELS (see study timeline). The SEELS conceptual framework (see Figure 3-1) holds that a child’s nonschool experiences, such as extracurricular activities and friendships; historical information, such as age when disability was first identified; household characteristics, such as socioeconomic status; and a family’s level and type of involvement in school-related areas are crucial to student outcomes. Parents/guardians are the most knowledgeable about these aspects of students’ lives.

Given the size of the SEELS sample and the resources likely to be available for data collection, in-person interviews are not feasible. Computer-assisted telephone interviewing (CATI) will be used for parent interviews. To eliminate the need for respondents to answer items that are inappropriate to the age, disability, or other circumstances of the household or student, carefully constructed skip patterns will be programmed.

Parent/guardian interviews will begin with a screening section identifying the adult who is best able to talk about the student and his/her experiences in school, who will be referred to here as a parent. If a parent is not sufficiently fluent in English to be interviewed and is a Spanish speaker, a Spanish language version of the interview will be administered. Appropriate follow-up procedures will be included to produce a high response rate. In addition, aggressive efforts

will be made to minimize any potential bias resulting from the telephone interview approach by developing methods of obtaining information for families without telephones (e.g., a simplified mailed version of the questionnaire). Design assumptions are that the average telephone interview length is 40 minutes and that at least a 70% response rate is achieved.

Most of the questions from the wave 1 parent interview will be repeated in waves 2 and 3. However, items that do not change with time, such as when the child was first diagnosed with a disability, will not be repeated. Items that are appropriate for older students, such as questions about employment, pregnancy/parenting, arrest history, and having a driver’s license, will be included only in wave 3.

The academic performance of students in their elementary and middle school years is central to the range of outcomes addressed in the SEELS conceptual framework. To accurately assess academic performance in the key areas of reading, mathematics, and academic problem solving, direct assessments of a student’s abilities by a trained on-site professional (other than the student’s own teacher) will be conducted in years 2, 3, and 5. This assessment still is being designed and will be pretested in the fall of 1999.

One or more persons (depending on the number of students per community) to conduct direct assessments of students will be recruited through contacts with the school district special education director or, in schools with several students to be assessed, with the school principal. The study team will contract with these persons to conduct assessments during a specific time frame in the spring of the assessment years for a fixed amount of money per completed assessment (currently estimated to be $50). The names of students to be assessed and appropriate training materials will be supplied to each assessor by the study contractor. Assessments are expected to take an average of 30 to 45 minutes per student. Students who require significant accommodations or modifications in the assessment process may require more time to complete assessments. A 75% completion rate is estimated.

Brief student interviews will be conducted in years 2, 3, and 5 in conjunction with the direct assessments conducted in those years by a professional who is not the student’s teacher. Although parents and teachers will be asked about many aspects of the student’s life, the student is usually the best respondent regarding feelings about his or her social, school, and other experiences.

In year 5 of the study, questions about adolescent risk behaviors (e.g., smoking, drinking, drug use, and gang involvement) are planned to be included in the student interview. An approach to obtaining reliable and valid information from students will need to be developed and submitted for OMB approval by the study contractor prior to incorporation in wave 3 data collection.

Student interviews are expected to take 10 to 15 minutes, so that the entire assessment and interview experience can be completed in less than an hour per student, on average. Students who require substantial accommodations for the assessment or interview are likely to require additional time to complete the two components. As with the accompanying direct assessment, a 75% completion rate is estimated.

The SEELS conceptual framework suggests the importance of the classroom experience in the lives of students, as well as the importance of behaviors and performance in the classroom setting. To obtain information on these domains, the student’s primary language arts teacher (as identified by the school principal) will be asked to complete a mailed, self-administered questionnaire during years 2, 3, and 5. The language arts teacher was selected, rather than another teacher, because a key focus of the student assessment is reading and other language arts skills. Because one purpose of measuring classroom practices is to help explain variations in assessments of academic performance, language arts instruction is the appropriate content area in which to explore instructional practices and adaptations or accommodations made for students in special education. In the context of this instructional setting, the language arts teacher can provide information on student behaviors while engaged in a fairly demanding academic setting. It is also anticipated that virtually all students will be enrolled in some form of language arts instruction for a portion of the school day, whether in a special education or a general education setting.

The language arts teacher survey will focus on the instructional techniques and curriculum that the teacher uses with the student, accommodations or modifications in classroom practice used by the student, and the teacher’s training and perceived competence in general and special education. It also will include a variety of questions regarding the student’s classroom performance, both academic and behavioral.

Reimbursement in the range of $10 should be considered for completion of the survey. A 75% response rate is estimated.

The school program survey will be administered in years 2, 3, and 5. The principal of each school attended by a sample student will be asked to identify the student’s special education teacher. For students who have been declassified from special education and no longer have a special education teacher, the principal will be instructed to identify the person at the school best able to answer questions about the student’s overall school program. In the elementary grades, the language arts teacher and the person best able to report on the student’s overall school program may be the same respondent.

The purpose of this survey is to identify the student’s accommodations, supports and related services; content of IEP goals; and overall school program and performance (e.g., achievement test scores, days absent from school). This survey also will include questions about the special education teacher’s experience and background, because s/he may be providing a range of services for the special education student. For students in middle and high school, teacher respondents also will be asked to include a copy of the student’s transcript. In addition, respondents will be asked to identify all of the settings where the student receives some language arts instruction and the approximate amount of time spent weekly in each.

Reimbursement in the range of $10 should be considered for completion of the school program questionnaire. A 75% response rate is estimated.

The school background survey, focusing on schoolwide characteristics and policies, will be conducted in years 2, 3, and 5. The principal will be asked to complete a mail questionnaire that describes the general characteristics of the school and local school district, including, for example, the demographics of the school, student enrollment, grade levels served, and policies regarding special education services. Other school-related factors, such as the orientation toward special education students and organizational structure and availability of specialized services, may provide important contextual information for understanding the placement, provision of services, and outcomes for students in special education. A 75% response rate is estimated.

A focus on telephone interviewing of parents will result in an underrepresentation of households that have no telephones or have unlisted telephone numbers. To assess the extent to which nonresponse to the telephone interviews introduces sample bias, OSEP may wish to commission a nonresponse study as part of SEELS. Such a nonresponse study could involve selecting LEAs with particularly high nonresponse and conducting personal interviews in households that did not respond to the telephone survey. Data from these in-person interviews would be used to identify any biases in the telephone interview sample and correct for them through weighting procedures.

The conceptual framework and research questions determine the development of the data collection plan and the content of the data collection instruments. Table 3-1 demonstrates the content areas and research questions that are included in each data collection instrument.

^† OSEP is commissioning a separate large-scale "Study of Personnel Needs in Special Education," which will provide much more extensive information on these personnel.

All SEELS data collection protocols will be pretested as part of the design process. The pretest of all data collection protocols, with the exception of the direct assessment, will take place in the spring of 1999. The direct assessment is on a different timeline and will have an extensive pretest in the fall of 1999. The purpose of the pretest is to ensure that the protocols and instruments function according to their design and to provide an opportunity to address problems that may arise during the pretest. In accordance with OMB requirements, the parent interview, language arts teacher survey, school program survey, record review, and school background survey will be conducted with no more than nine participants each. In addition, several different administrations of the direct assessment/student interview are currently under consideration for various groups of students in special education, including at least those with: (1) mild disabilities, (2) deafness/hearing impairments, (3) low vision/blindness, (4) cognitive disabilities, and (5) physical/health disabilities. The general intention of the pretests is to examine all aspects of the data collection, including the contact/follow-up procedures and the completion of the instruments, and conduct an interview with the respondents. The following activities are included in the pretest:

• Timing. An important feature of the pretest will be to measure the amount of time that is required to complete each subsection of the surveys/assessments, as well as the entire time required. This information will indicate which sections need to be shortened or revised in the final product.
• Comprehension. Respondents will be interviewed to establish how easily they were able to understand instruments, both in terms of their directions and in the content and format of specific items.
• Item characteristics. For each item, an analysis will be conducted to ascertain whether: (1) the responses obtained are believable, (2) exhibit variations in responses, and (3) are appropriate for the students or settings at which it is directed.
• Logical flow/skip patterns. Respondents will be interviewed to establish whether the instruments flow logically from one section to another, or whether some revision could improve the flow.
• Logistics. The pretest will also test the logistics in terms of the sequence of activities that will be required for respondents to either respond or participate appropriately.
• Accommodations. In the case of the direct assessment, significant effort will be directed at providing a variety of accommodations so that students can participate. The pretest will thoroughly examine the degree to which the accommodations were implemented and the likely effects of their large-scale implementation.

A concerted effort should be made to obtain consent of parents to have their student included in SEELS. For example, when students are selected for the study sample by the study contractor, a letter can be mailed to parents/guardians informing them of the student’s selection and what participation means. A brief form can be included for the parent to return to the study contractor in a postage-paid envelope indicating agreement or refusal to have the student in the study. Students for whom no consent form is returned should be retained in the study.

In addition, parents can be informed before each student data collection that their child will be involved in a direct assessment and interview. A brief form can be provided for them to report to the study contractor if they decline to have their student involved.

Finally, the study contractor will be supplied with a letter from the U.S. Family Educational Rights and Privacy Act (FERPA) Office asserting that for purposes of SEELS, the study contractor is an agent of the U.S. Department of Education; therefore, schools and school districts are permitted under FERPA to share information about students and from students’ school records with the study contractor. This authorization will be sufficient in the majority of schools and LEAs to obtain information from teachers and school records about students.

Parents will be asked during the wave 1 parent interview whether their child is in school and, if so, the name and location of the school the child attends currently and is expected to attend in Fall 2000-2001 (year 2). Schools will be contacted at the beginning of year 2 to confirm the student’s attendance. If a student has moved, schools will be asked the reason the child left the school (e.g., transferred, dropped out) and the name and location of the subsequent school or district if the student transferred to another school. Newly identified schools will receive a letter describing the study, along with the student tracking questions asked of the initial school.

Because parent interviews will be occurring concurrently with school-level data collection in waves 2 and 3, parent interviews cannot be used to learn the name of the child’s current school in advance of school-level data collection. For waves 2 and 3, therefore, parents will be mailed a form in the summer asking for the name and location of the school they expect their child to attend in the fall. As with all data collection efforts, appropriate follow-up procedures should be used to produce high response rates, including telephone follow-up with nonrespondents.

Having identified the schools attended by sample children, the study contractor can distribute the language arts teacher questionnaire, the school program questionnaire, and the school background questionnaire to the principals at the appropriate schools, with a request that they pass along the language arts and school program questionnaires to the appropriate staff members for each student and complete the school background questionnaire themselves. The study contractor also can contact district-level personnel in the LEAs of schools attended by sample students to arrange to contract for persons to administer the direct student assessment to sample students.

BACK

NEXT