Back with you Zhihau.... Besides mathematics and technical skills PISA and TIMSS also measure reading proficiency. With TIMSS though reading skills are assessed separately in its Program in International Reading Study (PIRLS). But while PISA/TIMSS only measure core mathematical, technical and reading skills their rankings are deemed to reflect the quality of an education system generally.
On the basis of Singapore's exalted rankings it's regarded as having a first-class education system by educators and educationists around the world who put great faith in PISA/TIMSS rankings. The fact that PISA/TIMSS only measure numeracy, literacy and technical skills is ignored by such people. But education is about far more than the imparting of such basic skills. It's also about imparting concepts about the society they live in, about their country's history and that of mankind's generally, about other peoples and cultures, about the great themes of the human condition and about human relationships. Only the humanities and literature can provide such cultural and psychological sustenance. You won't get this from basic numeracy and literacy education where the aim is to simply prepare people to be robotic cogs in the system. While this may suit the PAP state to reduce people to docile digits, making them much easier to control, such thinking violates the basic principles of education which is meant to develop "the whole person". The buzz word for this is "holistic education" which assorted PAP Ministers and Directors of Education rave on about but have failed to significantly implement in Singapore.
Apart from imparting basic numeracy, literacy and technical skills the quality of an education system depends on:
a) The degree to which it nurtures students' creativity, intellectual curiosity and desire to learn and develops their innovative and critical thinking capacities - attributes which the PAP leadership has been constantly banging on about since the late 1980s but which they glumly concede Singaporean students so often lack.
b) How well it practices and implements meritocracy and equality of opportunity - two more principles which PAP leaders also loudly and often proclaim. But as any thinking Singaporean knows Singapore operates of racist pro-Chinese education system which discriminates against Malays and Indians as well as one which favours class over equality. Both policies directly violate meritocracy. Regarding the class factor, the high cost of tuition ensures that the wealthy can pay for the best tutoring for their children giving them a massive head-start in the Singapore's highly competitive exam-driven education system compared to children from much poorer households.
c) The extent to which it provides a stimulating yet low-stress learning experience environment for students, teachers and parents. As almost every Singaporean knows such is not the case in Singapore's pressure-cooker education system where students and teachers are forced to operate in highly stressful school systems where causes much student and teacher burn-out and where student suicide is constant concern for schools and teachers, especially around exam times.
Regarding your question about biased sampling and its enforcement with respect to PISA/TIMSS tests: One first needs to examine how both organisations conduct their tests in the countries being surveyed to address this question.
As I already mentioned, PISA and TIMSS allow in each country they survey to supervise the tests with a locally chosen manager for the task. With the PISA such a person is a National Project Manager while with TIMSS it's a National Research Co-ordinator. Moreover, PISA actually states in its 2009 Technical Report that their NPMs "contributed to the verification and evaluation of survey results, analyses and reports."
Giving locally-chosen managers such power to administer and conduct tests on behalf of a foreign organisation leaves much room for nationalist education agendas to be pursued to the detriment of objective measurement of student performance in TIMSS and PISA tests.
In their critique of TIMSS tests, US educationists William and Berchie Holliday noted: "... there is good reason to believe that some government officials in their respective countries placed in charge of administrating TIMSS tests unfairly selected students for testing and did not act in accordance with TIMSS student-sampling methods". Moreover, they say that biased selection is likely to be most prevalent in countries where students attend specialised elite schools. And Singapore is one such country with its extensive system of elite schools.
With regard to TIMSS tests, the Hollidays, note that in 16 of 21 countries participating countries in TIMSS tests, students attending 'high performance' schools elite-type schools that focus on mathematics and science. The Hollidays rightly ask whether all participating countries have "honestly and completely reported their administrative procedures and follow TIMSS rules - or did some nations 'modify' the sampling rules and procedures to their countries clear advantage?"
The likelihood that countries such as Singapore may have engaged in such practices when one considers the considerable leeway they seem to enjoy when conducting TIMSS tests. For example, TIMSS has permitted Singapore to have a third sampling stage with its tests, according to its 2007 Sample Design and Technical Reports. While schools are first sampled and then classes within the participating schools, Singapore was allowed a further level of sampling within the classes being tested. Two classes per school were sampled in Singapore and 19 students were sampled within each class. With up to 40 students per class in Singaporean schools, this would represent a significant reduction in students taking PISA tests. No reason can be found in the TIMSS 2007 Technical Report on why Singapore was permitted to perform this sampling procedure while none of the other 58 countries in the TIMSS survey were allowed to.
Similar sampling manipulation would seem to be occurring with PISA tests by Singapore education authorities, according to Singaporean teacher Sutharsan John Isles. He disputed MOE claims that the students chosen for PISA tests accurately represented the 15-year-old cohort that PISA uses to test a country's educational achievement. He said MOE did not say how it selected students from public and private schools for testing.
From his own experience as a teacher in Singapore Isles said that students from the brightest stream (the Express stream) were overwhelmingly chosen for the PISA tests. It was "very unlikely" that any students from the Normal Academic stream - to which the bulk of students belonged - were ever selected. And it was even less likely that students from the Normal Technical stream, where the weakest students were, would ever be chosen, he said.
Isles comments are quite credible, especially when one considers the PAP states propensity for using statistics to pursue its various political and social engineering agendas. And woe betide any Singaporean who dares question official statistics, whether on education or anything else the government deems important. Many on this forum would know of Singaporeans and foreign commentators who've been viciously vilified by the PAP government when they've done so.
Certainly no MOE official is going to blow the whistle on dubious MOE practices regarding PISA/TIMSS testing, if he wants to keep his job and avoid government persecution and perhaps legal prosecution.
