質疑中國在 PISA 測試中名列第一

中國在 PISA 測試中名列第一——但這就是為什麽它的測試成績令人難以置信

https://www.washingtonpost.com/education/2019/12/04/china-is-no-pisa-heres-why-its-test-scores-are-hard- believe/
作者：瓦萊麗·施特勞斯 2019 年 12 月 4 日

在新發布的國際學生評估項目分數中，中國大陸是最大贏家，該項目每三年對數十個國家的 15 歲學生進行數學、閱讀和科學測試。 2018 年，來自 79 個國家和學校係統的 60 萬名學生參加了考試，中國的四個省份（在 PISA 中構成中國大陸）在所有三個科目中均排名第一。

但我們有充分的理由對中國大陸的分數持懷疑態度，這就是學生成績、考試、教育政策和 K-12 學校改革專家 Tom Loveless 這篇文章的主題。

洛夫萊斯曾是六年級教師和哈佛大學政策教授，曾是治理研究高級研究員，也是華盛頓非營利性布魯金斯學會布朗教育政策中心主任。 他撰寫了 16 卷《布朗中心美國教育報告》，這是一份分析教育重要趨勢的年度報告。

這並不是 Loveless 第一次對中國的 PISA 分數發表評論。 2013年，我曾寫過一篇文章質疑上海在2012年PISA中的第一名。 在那次考試中，美國學生的表現並不高於 65 個國家和教育係統的平均水平（與往常一樣）。

2013年底公布2012年成績時，PISA的讚助者經濟合作與發展組織表示，上海樣本中使用的學校代表了該市15歲的人口。 當時在布魯金斯學會任職的洛夫利斯和一些中國問題專家表示，流動兒童經常被排除在上海的學校之外，而上海比中國其他地區更富裕。 經合組織堅持這一結果。

順便說一句，在 2018 年 PISA 成績中，新加坡在所有三個科目上均排名第二。 美國學生在閱讀方麵排名第八，在科學方麵排名第11，在數學方麵排名第30，自幾十年前PISA開始以來，這些分數沒有發生顯著變化。

作者：湯姆·洛夫萊斯

2018 年 PISA 成績已出爐。 一般來說，根據過去的記錄，各國的得分都在預期範圍內。 除了一個。 B-S-J-Z 的得分令人震驚，B-S-J-Z 是參賽的中國四個省份的縮寫：北京、上海、江蘇和浙江。 在 77 個國際體係中，B-S-J-Z 在閱讀、數學和科學這三個科目中均排名第一。

2015年至2018年，參加PISA的中國四個省份發生變化，浙江取代了廣東。 2018 年組的分數顯著高於 2015 年組（恰當地稱為 B-S-J-G）。 事實上，差異如此之大，必然會引起人們的注意。

B-S-J-Z 的閱讀成績高出 61 分（494 比 555），數學高出 60 分（531 比 591），科學高出 72 分（518 比 590）。 像這樣的差異有多罕見？ 為了回答這個問題，我檢查了 2006 年至 2015 年的 PISA 數據。

對於每個三年的測試間隔，我計算了每個國家/地區在三項 PISA 測試中的變化，並將其轉換為絕對值。 結果產生了 497 個觀測值，平均值為 9.5 分，標準差為 8.6。

因此，一個國家得分的典型變化約為 10 分。 2015 年和 2018 年中國參與者之間的差異至少是這個數字的六倍。 這些差異也至少是所有間隔變化的標準差的七倍。 非常不尋常。

一個合理的假設是，改變參與 PISA 的省份，即使隻是四個省份中的一個，也會影響測試成績。 事實上，當我最初在 Twitter 上就此主題撰寫帖子時，我忽略了這一變化，並將 2015-2018 年的分數差異視為參與省份相同。 我對這個錯誤表示歉意。 然而，我的錯誤確實凸顯了一個更大的問題：中國的 PISA 分數應該受到懷疑。

為什麽中國人口第一大省廣東被剔除，而加入浙江？ 改變後分數飆升隻是巧合嗎？

由於戶口對 15 歲人口的剔除效應，以及經合組織允許中國批準哪些省份可以進行測試，中國各省份過去的 PISA 分數受到了質疑（我和其他人）。 2009年，PISA測試在中國12個省份進行，其中包括幾個農村地區，但僅公布了上海的分數。

三年後，英國廣播公司報道稱，“中國政府迄今不允許經合組織公布實際數據。” 截至目前，該數據尚未公布。

經合組織通過攻擊批評者和閉門進行數據審查來回應過去的批評。 中國各省份的 PISA 分數籠罩在陰雲之上。 我敦促經合組織盡快公布對 2018 年數據進行的質量檢查結果，以及 2015 年和 2018 年參與者按省分列的分數。

國際評估的可信度取決於測試程序的透明度，包括如何選擇參與者以及報告測試結果的規則。 經合組織不公開其在中國進行的評估，可能會損害 PISA 的可信度。

瓦萊麗·施特勞斯 (Valerie Strauss) 是一位教育作家，運營著“答卷”博客。 1987年，她加入《華盛頓郵報》，擔任亞洲地區的助理外國編輯，此前曾在路透社擔任國家安全編輯和國會山軍事/外交事務記者。 她此前還曾在 UPI 和《洛杉磯時報》工作過。

China is No. 1 on PISA — but here’s why its test scores are hard to believe
https://www.washingtonpost.com/education/2019/12/04/china-is-no-pisa-heres-why-its-test-scores-are-hard-believe/

By Valerie Strauss December 4, 2019 

Mainland China was the big winner in the newly released scores on the Program for International Student Assessment, which tests 15-year-old students in dozens of countries in math, reading and science every three years. With 600,000 students from 79 countries and school systems taking the exam in 2018, four provinces in China — which for PISA constitutes mainland China — were collectively ranked No. 1 in all three subjects.

But there is good reason to view the scores from mainland China with skepticism, and that’s the subject of this post by Tom Loveless, an expert on student achievement, testing, education policy and K-12 school reform.

A former sixth-grade teacher and Harvard policy professor, Loveless was a senior fellow in governance studies and director of the Brown Center on Education Policy at the Washington-based nonprofit Brookings Institution. He wrote 16 volumes of “The Brown Center Report on American Education,” an annual report analyzing important trends in education.

This isn’t the first time Loveless has commented on PISA scores from China. In 2013, I wrote a post questioning the No. 1 ranking of Shanghai in the 2012 PISA. In that test administration, U.S. students performed no better than average among 65 countries and education systems (like usual).

When the 2012 scores were released in late 2013, the Organization for Economic Cooperation and Development, which sponsors PISA, said the schools that were used in the Shanghai sample represent the city’s 15-year-old population. Loveless, then at the Brookings Institution, and some China experts said that migrant children were routinely excluded from schools in Shanghai, which is wealthier than the rest of China. The OECD has stood by the results.

Incidentally, in the 2018 PISA results, Singapore was second in all three subjects. U.S. students ranked eighth in reading, 11th in science and 30th in math, with scores that have not significantly changed since PISA began a few decades ago.

By Tom Loveless

The 2018 PISA results are out. Generally, countries scored within an expected range given their past records. Except one. The scores are astonishing for B-S-J-Z, an acronym for the four Chinese provinces that participated: Beijing, Shanghai, Jiangsu and Zhejiang. Out of 77 international systems, B-S-J-Z scored No. 1 in all three subjects: reading, math, and science.

The four Chinese provinces taking PISA changed from 2015 to 2018, with Zhejiang taking the place of Guangdong. The 2018 group’s scores are dramatically higher than those of the 2015 group (which appropriately is called B-S-J-G). In fact, the differences are so large that they are bound to raise eyebrows.

B-S-J-Z’s scores are 61 scale score points higher (494 versus 555) in reading, 60 points higher (531 versus 591) in math, and a whopping 72 points higher (518 versus 590) in science. How uncommon are differences like these? To answer that question, I examined PISA data from 2006-2015.

For each three-year test interval, I computed the changes for each country on the three PISA tests and converted them to absolute values. That produced 497 observations, with a mean of 9.5 points and standard deviation of 8.6.

So the typical change in a nation’s scores is about 10 points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount. The differences are also at least seven times the standard deviation of all interval changes. Highly unusual.

A reasonable hypothesis is that changing the provinces participating in PISA, even if it was just one out of a group of four, influenced the test scores. Indeed, when I originally composed a thread for Twitter on this topic, I overlooked the change and treated the 2015-2018 score differences as if the participating provinces were the same. I apologize for the error. My mistake does underscore, however, the larger issue: that PISA scores from China should be viewed skeptically.

Why was Guangdong, China’s most populous province, dropped from participating and Zhejiang added? Is it only a coincidence that scores soared after the change?

The past PISA scores of Chinese provinces have been called into question (by me and others) because of the culling effect of hukou on the population of 15-year-olds — and for the OECD allowing China to approve which provinces can be tested. In 2009, PISA tests were administered in 12 Chinese provinces, including several rural areas, but only scores from Shanghai were released.

Three years later, the BBC reported, “The Chinese government has so far not allowed the OECD to publish the actual data.” To this day, the data have not been released.

The OECD responded to past criticism by attacking critics and conducting data reviews behind closed doors. A cloud hangs over PISA scores from Chinese provinces. I urge the OECD to release, as soon as possible, the results of any quality checks of 2018 data that have been conducted, along with scores, disaggregated by province, from both the 2015 and 2018 participants.

The credibility of international assessments rests on the transparency of test procedures, including how participants are selected and the rules for reporting test results. The OECD risks undermining the credibility of PISA by not being open on its conduct of the assessment in China.

By Valerie Strauss
Valerie Strauss is an education writer who runs the Answer Sheet blog. She came to The Washington Post as an assistant foreign editor for Asia in 1987 and weekend foreign desk editor after working for Reuters as national security editor and a military/foreign affairs reporter on Capitol Hill. She also previously worked at UPI and the L.A. Times.