WRITINGS
|
The new marking system The new scheme of
marking answer scripts of the G.C.E. Ordinary Level Examination was
designed by the Commissioner of Examinations in July 1973. The
objectives of the new scheme were to increase the public confidence
in the marking and assessment of answer scripts, enhance the
reliability of marking and to expedite the release of results. scheme of marking, since answer scripts were to be marked by the examiners between 2nd and 14th January, 1974 in a few specially selected marking centres, marks of all the candidates were to be available in the department by 15th January 1974. Since there were 420,000 candidates, most of them offering as many as eight subjects, there were to be 3 million marks for the different subjects. Had we followed the system adopted hitherto, it would have taken us three months to key-punch and verify the three million marks for the different subjects. Such a delay would have not only been unjustifiable, but also unfair by the candidates and the examiners. The public would never have pardoned us for such an undue delay. A suggestion was made that we arrange for the examiners to enter the grades manually in results sheets and post them to schools. Apart from the inherent weakness of this method, it was inadequate since it did not provide for the release of results of private candidates. It was during this time that Mr G. Santhiran, Manager IBM (Sri Lanka) suggested to me the use of Mark Sense Cards. It is now an obsolete system in countries like the U S, while in the developing countries its fast becoming a popular method. With the help of Mr M.A. Kandiah, IBM Computer Engineer who had just returned after a study of the system in New Delhi, we examined the feasibility of using Mark Sense Cards. We studied some of the cards used at Meerut University in India. Of course, since our system of examination and data processing was completely different and because of the large number of candidates and examiners involved, we had to develop an entirely new system. Thus the new system has the following distinct features. i. Improve processing schedule - By eliminating the need to key punch, the data could be processed in a very short time. ii. Maintain security and secrecy. The cards will be marked by the examiners themselves and there will be no need for any other person to handle the mark sheets. iii. Ensure greater security. Errors that occur during key-punching due to bad hand writing etc. will be eliminated as the examiners who prepare the mark sheets will themselves be marking the cards. We designed the new card and sent the copy to the Mark Sense Cards manufacturers in Australia and requested an urgent proof. As the time grew nearer, IBM (Sri Lanka) had to send some 15 telex messages and a cable calling for a proof copy of the Mark Sense cards. But on account of a postal strike in Sydney at this time, we were unable to obtain a proof copy of the card we designed. On September 19th, IBM (Sri Lanka) received a telex message from Sydney Australia suggesting that if the manufacture of cards could be approved, the cards could be manufactured and put on a vessel leaving Sydney on September 22. To us the situation was challenging. We had to approve the production of a million cards without seeing the proof copy. (Believe me, I still haven’t seen the proof copy of this card). We had to obtain a foreign exchange allocation and an import permit within 24 hours. Had we missed this ship, we would have had to abandon the new system, since the next ship leaving Sydney would be arriving in Colombo in January 1974. We took the risk and approved the immediate production of the Mark Sense cards and requested they be put on the ship arriving in Colombo on November 20th, and proceeded to obtain necessary foreign exchange and the import license. Unfortunately, the cable approving production was delayed and by the time the cards were manufactured, the ship had left Sydney. IBM (Australia), the card manufacturers transported the million cards overland to Melbourne and put it on the ship in Melbourne on September 25th 1973. The new system also required mark sensing reproduction equipment. The base machine was available in Sri Lanka and only the mark sensing device had to be imported. Had the mark sensing device been ordered from the manufacturers in Italy and imported by sea freight it would take over 4 months. IBM (Sri Lanka) obtained a full list of spares for the Mark Sensing device and ordered it on emergency. Director of Foreign Resources and the Controller of Imports allowed us air freight for these spares. The spares arrived here early December, and with three Computer Engineers working round the clock, we were able to get the machine in working order by late December. The cards arrived in Colombo in November. Four hundred and fifty thousand cards had to be pre-punched and interpreted containing the following information, Center Number, Subject Number, Prefix and Index Number of the candidates. Dr Arthur Perera, Data Processing Manager, Central Bank, arranged for the necessary programs to be written and agreed to punch the cards off the computer. We had originally estimated the time required for computer punching to be 40 hours but when the first part of the cards was done, we realized that the total time required was over one hundred hours. Central Bank computer could not be utilized for so many hours and we had to seek the assistance of the Census Department computer to do part of the file. We used the computer system at the Census Department for a 30-hour non-stop run. Since the four hundred and fifty thousand pre-punched cards had to be sorted out and separated for dispatch to 10,000 examiners, the pre-punching and interpreting had to be completed by December 20th the latest. The interpreter which we had ordered had not yet arrived and we had to use the Interpreter at the Insurance Corporation. We worked the Insurance Corporation Interpreter for almost 22 hours a day for nearly seven days and were able to get the cards ready by December 20th. After the commencement of marking, the Mark Sense cards were delivered to the Marking Centers. Commencing 14th January, 1974, the completed cards were received in the Data Processing Branch of the Department of Examinations. On the morning of 15th January, we started feeding the cards to the Mark Sensing Machine installed in the Data Processing Branch of the Examinations Department, which automatically punches the marks on the data processing cards at the speed of 100 cards per minute. We had originally planned to run the machine from 8.00 A.M. till 8.00 P.M. but after the first day, since the machine was behaving so well, we ran the machine from 8.00 A.M. until 6.00 A.M. the next day with a few half hour breaks in between for five consecutive days and were able to complete the work ahead of schedule. Reflecting back on the system adopted, at times I feel that we have taken many risks at several times. Its a system which we had never tried before. The machine installed in the Examinations Department is the only Mark Sensing machine in Sri Lanka. The nearest place with a similar machine is New Delhi. But since three IBM engineers stood by and an expert IBM engineer in New Delhi had been alerted to be in readiness to fly over in case of emergency, we were prepared to take the risks. I am glad that now we are in a position to report that the New Data Processing System has proved a success and has enabled us to release the results so early. It is with a sense of justifiable pride I say that this indeed is a unique achievement in the field of educational testing. Considering the number of candidates, the variety of subjects offered and the examination being in three language media, it is an achievement unparalleled anywhere else in the world.
|
An Editorial comment on my article 'The
New marking System'
|
Ceylon Daily News
D.R.Wijewardene Mawatha, Colombo. February 18, 1974 Mark WellOn Friday our centre page carried an article by the Superintendent of Examinations under the pedestrian heading “The New Marking system” which may have put off some readers. But if they had read on they would have found that it was in fact a little bit like those urgent operations undertaken in wartime to launch a submarine on a secret mission , get a clutch of desperadoes off the ground or those highly computerized capers of a film thriller. The Examinations Department, IBM (Sri Lanka and Australia) got mixed up in what is called the Mark Sense Cards – almost wholly incomprehensible to the lay mind, but totally approved by it because of its laudable results. An assortment of persons, departments and computers from various institutions were drawn in, not to mention setting up of Mark Sensing Device from spares flown here in a hurry. The cards missed their ship in Sydney, but managed to get on board in Melbourne: they ended a race in time to give over 420,000 GCE ‘O’ level candidates their results in six weeks. Truly a giant operation for which the Commissioner and the Department of Examinations should be roundly applauded and warmly congratulated. It is a feat of tremendous proportions which any little snag could delay and snarl up the whole. And that hectic undertaking with computers goes to prove that, initiative, the courage to make quick decisions, the elimination of red tape and the stretching of discretionary authority to the maximum could achieve the soundest results from the most haunting situation. Every mousy bureaucrat chewing his red tape in a corner should take a lesson. |
|
THE CEYLON DAILY NEWS, WEDNESDAY, DECEMBER
4, 1974 Several important matters were raised by Mr. E.H.de Alwis, former Director of Education, in his article that appeared in these columns on the same subject on November 21, 1974, which need clarification not only for his own benefit but also for the benefit of others who may entertain similar misgivings and misconceptions. The University of London makes its selections purely on the basis of the grades obtained at the G.C.E. (AL) Examination although of course there are several British universities and other institutes of higher learning which are not satisfied with the grades alone and insist on the detailed marks obtained by the candidates. Since the number of students seeking admission to London University is not greatly in excess of the number of vacancies in the university, the need to look for any other eligibility beyond the grades does not arise. The situation in Sri Lanka is somewhat different. For the next academic year (1975) , for the 3700 places in all the faculties, in all the six campuses, there are nearly 15,000 candidates with the required grades. It should be obvious that the selections could not be made based on the grades alone. We need some finer discrimination index in order o select the more able ones out of these. Hence the inability to follow the London University method and the need for standardization. Standardization of marks for university admissions should in no way be interpreted as some sort of tampering with grades. The G.C.E. ‘A’ Level examination is essentially an attainment test and candidates are awarded grades in accordance with their level performance. These grades are never tampered with. For reasons stated above these grades provide only the eligibility for the standardization of marks. Prior to 1970 the grades obtained at the G.C.E. ‘A’ Level Examination provided the eligibility and since this alone was inadequate actual selections were made on the basis of the aggregate of the raw scores. The only difference now is the selections are based upon the aggregate of standardized marks. That the standardization in this context is not only prudent but also the more refined and fairer thing to do cannot be debated. That the raw scores by themselves are not additive and that as they are, they are not comparable is now universally accepted in educational measurement. It is unfortunate that Mr. de Alwis has drawn examples from other universities. There are thousands of universities in the developed countries like the U.S and Canada and also in the Asian countries some of which are as prestigious as the London University (if not more) where the selections are made not on grades but on standardized marks. The need to standardize media-wise has been adequately explained in a brochure published by the Ministry of Education. Suffice it to mention here that the difficulty of a question is partly dependent on its language and that the intrinsic differences in the various languages make the rendering of a question in another language medium and maintaining the same difficulty level almost impossible. There is no known realistic means to measure whether the original version of a test in one medium are of the same difficulty level. This factor alone is sufficient grounds for media-wise standardization and it is mostly for this reason, I venture to think, that media-wise standardization is done in respect of raw scores which forms the basis of selection for almost all the universities in Canada. Mr. de Alwis states that since the number of Sinhala candidates is more their mean is bound to be low. He gives the example that 25 for Chemistry in Sinhala would be raised to 50 as against 40 in the English medium which would also be raised to 50. This is both factually and statistically incorrect. Besides it gives the impression that on account of standardization Sinhala medium candidates are at a distinct advantage over the English and Tamil medium candidates. In keeping with the principles of the Law of Probability, the size of the population has no bearing on its mean. This is true whether the total population or a scientifically selected representative sample is selected for one’s own purpose. Mr. deAlwis’ statement is factually incorrect is amply demonstrated by the following: G.C.E. (A/L) – 1972 Subject Medium Total number of candidates Mean
Physics
Sinhala
7225 31 G.C.E. (A/L) – 1974
Pure Math
Sinhala
4646 40 Mr. de Alwis believes that the underlying assumption in standardization of marks is that marks should be normally distributed. Nothing is further from the truth. There’s no such assumption whatsoever underlying this standardization of marks. Since it is only a linear transformation the original distribution remains unchanged after standardization. If the original distribution was skewed or normal the standardized score distribution will be skewed or normal in exactly the same fashion and the original rank order of the candidates will remain unchanged. It was argued that “the A/L students are a highly selected lot of comparable intelligence and scholarstic attainment and that when they start their “A” Levels they are almost on the same footing.” This is incorrect. The requirement for ‘A’ Level is G.C.E. O/L with three credit passes two of which are in the A/Level subjects and the selection to A Level class is based on grades. This does not mean that all the candidates have the same grades. There may be one who has obtained six distinctions and another from the same school and same class who has obtained 3 credit passes and 3 ordinary passes. To say that the total A/L student population when they start their “A” Level classes is of the same level is absolute nonsense. Likewise their performance at the ‘A’ Level examination will have a high average and a small standard deviation. Almost every one of the students who offered Pure Math for the April 1974 examination must have obtained at least a credit or a distinction in Pure Math at the O/L examination. But their performance at the A/Level was as follows: PURE MATH - 1974 G.C.E. A/L Examination Number of Candidates – 6855 Lowest Marks - 00% (39 Candidates scored zero marks) Highest Marks - 96% Mean - 39 Standard deviation – 12.97 These parameters show that the true picture is nowhere close to the ideal imagined. Since the district based selections have been introduced as a corrective for any inadequacies, the need to have another corrective is questioned. This I presume is in reference to standardization. It must be emphatically stated that standardization is not resorted to as a corrective for any inadequacies. It is done purely and simply to enable the marks of all four subjects to be added together. Whether the examination is conducted in one medium or several media, as long as the marks of more than one subject are aggregated the need for some kind of standardization remains. I earnestly hope these facts would help to dissipate some of the fears and misgivings many have regarding standardization. |
