ASSESSMENT
OF BASIC COMPETENCIES
A review by Prof. Barry P. Frost, University of Alberta
(Reprinted from The Alberta School Psychologist)
The Other ABC
Barry P. Frost, Ph.D.
With much to-do and hype the Kaufman ABC (Assessment Battery for Children) has been marketed both in the U.S.A. and Canada though, in several respects, particularly the Faces and Places Sub-test, it is unsuitable for use in Canada (or other countries).
Much less publicity has surrounded the birth of another instrument that is more suitable for use in Canada, having been "normed" in Alberta and other provinces, as well as representative areas of the U.S.A. This instrument is the "Assessment of Basic Competencies: A Functional Assessment of Children's Abilities", developed by Jwalla P. Somwaru. This test battery measures a student's level of development in areas that are relevant to learning in school and provides diagnostic information for the design of instruction.
Of major importance in the rationale of this battery is the fact that?'the assessment focuses on skills that are learned, instead of on presumed underlying abilities and processes" (Somwaru, 1981, p. 1). The tenets on which the model is based may be stated as follows: "simple associative learnings lead to other learnings, which in turn lead to generalizations, such as concepts and rules, Learning in any one area is a continuum. A process of task analysis enables one to determine where a student is located in the continuum, and what must be learned in order to achieve a particular goal." (Somwaru, 1981, p. 1). This rationale is based upon the Cumulative Learning Model of Gagne (1968).
Also of major importance is the fact that the battery is based on a latent trait model which allows of criterion referenced assessment as well as norm-referenced assessment. There are two ways in which criterion-referenced assessment may be carried out: (a) in a diagnostic mode in which performance on skills and clusters is evaluated, or (b) a developmental mode in which performance is evaluated against a developmental scale or sequence.
The primary "score" on each test in this battery is a criterion-referenced "Level of Competence". In turn, this may be converted into a Developmental Age, Grade Equivalent, or a Percentile Rank for an age or grade group. The percentile rank can also be converted into a standard score (normal curve equivalent).
The combination of criterion-referenced and norm-referenced information is useful in the assessment of children who are described as slow learners, learning-disabled, behavior problems, as well as other handicapped children.
The rationale for the battery, following the lead of P. E. Vernon with respect to the overlap of intelligence and achievement functions, avoids the need for an ability- achievement discrepancy, and suggests, instead, a generalized-specific learning discrepancy.
Somwaru writes that "the constructs of intelligence, aptitude, and achievement are best used to categorize the uses to which tests are put. Figure I shows the relationship among the three dimensions of the Assessment. While the location of the Information Processing domain appears to be basic, or central to the acquisition of Language, Mathematics, and other school subjects, it does not represent a different kind of learning. Its component skills are more generalized than others, while those of Language or Mathematics are more specific. Moreover, they overlap and interact very strongly." (Somwaru, 1981, p. 2).
Another element of major importance in this approach is the fact that the Assessment provides a single integrated system in the p}ace of using several tests that differ in purpose, content, norms and scales. As Somwaru puts it, "it provides an integrated battery with a single purpose, in which the parts are logically and statistically related, and which has been scaled on a common metric and normed at the same time on similar and overlapping populations. No oblique inferences are necessary, and a diagnostician moves directly from assessment to recommendations for instruction". (Somwaru, 1981, p. 2).
In addition, the battery's equal interval scaling allows a user to assess growth over time by taking repeated measures. This is not possible with the usual ordinal-scale based normative tests.
The system is appropriate for a wide range of children who are not coping well with school, whether they be labelled as learning-disabled, emotionally disturbed, language-disordered, educable-mentally-handicapped, dyslexic, behavior-disordered, or disadvantaged. It is not designed for those with sensory (visual or hearing) or speech impairments. According to Somwaru, the battery may be used with bilingual students, or those using English as a second language, provided that the purpose of the assessment is to determine their capability to cope with school learning which is conducted in English. "In general", he says, the battery "is nonbiased if the purpose of the assessment is to determine educational needs of children in the context of our current organization and methods of schooling, as the skills assessed are those which are necessary for successful functioning in school." (Somwaru, 1981, p.7).
Somwaru states that the decision to give the battery should be based on prior information from other tests or from teachers' assessments. If such prior information is not available, a preliminary test such as the Survey of Basic Competencies (Somwaru, 1979a) may be administered. This survey consists of four short scales (Information Processing, Language, Reading, and Mathematics) which assess the same content domains as the full ABC.
In addition to this Survey, there is available a Test of Early Learning Skills (Somwaru, 1979b) made up of items from the ABC domains, and consisting of three short scales (Thinking, Language and Number) which is appropriate for children aged 3½ through 5½ years. The ABC proper may be used from pre-school through grade 8.
All tests, in the three batteries, are power tests, i.e., un-timed. Somwaru claims that the Survey of Basic Competencies and the Test Of Early Learning Skills should each take about one-half hour for administration but that the Assessment of Basic Competencies would take about two to three hours for the administration of the whole battery, and about another half-hour for scoring and conversion of the scores. Diagnostic interpretation and the writing of a prescription, or suggestions for instruction, would take additional time.
In the writer's experience (with learning disabled children) the above is very much an underestimate. Four and one half hours were taken to administer the full battery to some children of this type, and the scoring and conversion of scores, if both diagnostic and developmental forms were used, would be close to one and one-half hours. It seems to the writer that, at least with LD children, one would want to use both forms.
The diagnostic form has "items grouped in clusters and arranged in parallel strands in ascending order of difficulty. In the diagnostic mode, the focus is on which skills and clusters appear to be mastered and which ones are not. Provision is made for recording the number of items passed in each cluster in relation to the number of items administered in the cluster." (Somwaru, 1981, p. 13).
The developmental form has items arranged "in ascending order of difficulty, irrespective of the clusters to which they belong. In the developmental mode, the focus is on the level of competence of the student in the developmental scale, and provision is made for recording the basal item, the ceiling item, and the operating range" (ibid, p. 13). The diagnostic form can be completed on the basis of data entered onto the developmental form.
Unlike most tests, the ABC allows of assessment of growth or change over time. Such an assessment requires that two conditions be met:
1. The scale should be independent of time, place and the individuals
tested.
2.The measurement should be in equal intervals along the scale.
These requirements are met by the Latent Trait basis of the ABC. The Scales of the ABC, which were derived independently of each other, can be used for assessing growth in each test area.
As Somwaru notes, :"I.Q.'s cannot be used for assessing growth, as they were not designed for this purpose. They were designed to report individual differences in relatien to the average of successive age groups. Other standard scores have the same deficiency. Percentile ranks are tied to the distribution of norm groups and do not have equal intervals along the scale." (Somwaru, 1981, p. 114).
Somwaru's manual also includes a very useful section on diagnostic teaching entitled "From Diagnosis to Instruction". He notes the deficiencies of the ability training model (see also, Arter & Jenkins, 1979), and gives reasons for preferring the alternative task analysis approach which he says emphasizes "component skills and their integration into complex terminal behaviors. The Assessment of Basic Competencies is designed to operationalize the task analysis model for diagnostic-prescriptive teaching." (ibid, p. 115).
The statistical basis of the ABC is impressive. Assessment calibration and norming data were obtained on large Samples from four regions of the U.S.A. and three areas of Canada (see tables 58 through 60, reprinted by kind permission of Dr. Somwaru). Space does not permit a full description of the latest trait approach, which is detailed in the manual, but it is important to note that the basic measure, the "Level of Competence" is a competency based measure. It is possible, as Somwaru points out (p. 161), "to convert the Level of Competence into a Developmental Age (or age equivalent) or a Grade Equivalent; it is also possible to determine a percentile rank based on age or grade, and to convert the percentile rank into a standard score or Normal Curve Equivalent."
The test-retest reliability coefficients reported in table 65 (q.v.) are impressive). Construct validity was approached via the Smallest Space Analysis procedure (Schlesinger & Guttman, 1969) and led to the structure specified in Figures I and 2.
Content validity was checked by several consultants (listed in the Acknowledgements section) who "reviewed the items for their content and ability to assess the clusters of skills listed in the model (Figure 2). It was considered important that the items test the identified skills, and that the clusters adequately assess the defined test areas. items were dropped or modified if they did not fit the content design; in some cases new items were written." In addition, "several professionals in Special Education were asked to review the content design of the test for its appropriateness for · ..students (with special needs)". (ibid, p. 168).
To assess criterion-related Validity, several investigations were carried out and these are specified in Table 68. These figures deomonstrate the usually found increase in magnitude with increasing chronological age but the criteria themselves leave a lot to be desired, especially the WRAT (see Snart et al., 1983). Much more work needs to be done in this area.
Somwaru has also developed a computer program (Apple II+ currently but programs for other microcomputers are said to be in a state of development). This program, the ABC Report Writer, is such that "after scoring the protocol, the diagnostician can spend about 50-60 minutes at the microcomputer and produce a 14-page report." (Somwaru, 1983). This computer program is very disappointing and requires much further work. In particular, for a program of this sort to be really useful, it must be possible for raw scores to be entered and the program do the rest of the analysis. At present it is quicker for a psychologist to use the manual for report development.
In conclusion, it is very appropriate to note Somwaru's words on the subject of professional usage:
"The Assessment is a specia]ist's instrument. Its use requires adequate academic background and training. Psychologists, psychometrists, diagnosticians, consultants, or other specialists who have had training in the use of individualized tests would be able to use the battery after a brief preparation... It cannot be overemphasized that interpretation of the test results should be carried out by persons who are professionally trained, and not delegated to test administrators who may not have the background necessary to do so." (Somwaru, 1981, p. 7).
The ABC and its associated tests, together with much other material,
including training manuals and apparatus, may be obtained from the author
and publisher:
Dr. J.P. Somwaru, Imnard Ltd., 139 Bernard Avenue, Richmond Hill,
Ontario L4C 9Z6, Canada.
e-mail: jsomwaru@idirect.com
The writer wishes to thank Dr. Somwaru for his cooperation and, in
particular, permission to reprint sections of the manual. It is the writer's
belief that Dr. Somwaru's instruments will prove to be
of much greater use to the school psychologist than the majority
of current instruments, most of them much better publicized. As with much
in current North American life, one needs to look at the content rather
than the packaging.
Back to ABCReview (beginning of the article)