Search "pawread dot com" for the original.
Generally speaking, there are two types of mature competition rankings: public rankings and final private rankings. The corresponding data is also different, divided into validation set and test set. The true answers of these two parts of data will not be disclosed, but after the participants submit their results, the public rankings will only publish the results of the validation set for everyone's reference, and will not disclose the results and content of the final test set. This is because the real-time ranking is only to help everyone understand the approximate level of their algorithm and how much it differs from the strongest algorithm. Although it does not directly provide answers, if participants adjust their settings and submit results repeatedly, they can still analyze the content and distribution of this part of the data to some extent. Therefore, this part of the data is only for reference, and the final ranking is determined only by the test set. Therefore, the public rankings of the competition can only reflect the problem to a certain extent and cannot represent the final ranking completely.
"Although some interim rankings are not used for the final ranking, people are easily attracted by the magic of these rankings." Meng Fanqi remembered that he and Tang Huang had participated in two small competitions in the future, and he didn't know why he kept staring at the rankings.
His mood would fluctuate greatly whether his submission score was higher or lower by one place.
"This is the case in any industry. Look at the novel industry, film and television industry, and the star industry. They all create anxiety." Tang Huang disagreed, "If there is no anxiety, create a ranking list to create anxiety. Everyone wants to be the best. When this ranking list comes out, it's like sprinkling bait into a calm pool. The fish that used to lie motionless suddenly become active."
"The sports industry is the same. They love to talk about who is the GOAT (Greatest of All Time), like LeBron James vs. Kobe Bryant, Messi vs. Ronaldo." Tang Huang continued to complain. The sports industry is now suffering from this trend and is almost becoming a fan circle. "There are more and more ridiculous statistics. In the past, they only counted goals, but now they even count which part of the body the goal was scored with. A few days ago, I saw someone say that Ronaldo's brother scored a goal."
Meng Fanqi was checking the information of the submitted results and was stunned for a moment when he heard this. "Ronaldo has a brother?"
After thinking carefully, he realized that it was his second brother.
In fact, the submission website had already been announced on November 11th. The submission window for this year was very short, and it was not like many later competitions that divided the validation set and opened the public rankings during the competition.
On November 13th, the submission of results would be closed.
Unconsciously, forty to fifty days had passed, and Meng Fanqi had polished these papers several times.
Not only that, when he found that the experiments in the papers had been completed, he connected the detection algorithm to the classification model that had been trained for a long time and started running the detection competition data.
The detection task is an advanced version of the classification task. After your program distinguishes the category of this image, the next step is to use a rectangular box to encircle the position of the object in the image. This is the familiar box on people's faces.
The next step is segmentation, which does not use such large and regular shapes like rectangular boxes, but represents the detailed contour of a certain object at the pixel level on the image. It is a kind of operation similar to automatic image segmentation.
Of course, whether it is detection or segmentation, it requires manual annotation of the original answers of the training set.
The IMAGENET-2013 detection track dataset is not too large, with nearly 400,000 images in total, divided into 200 categories. This advanced type of data is much more laborious to annotate compared to the number of data and classification.
However, compared to the 5,717 images in 2012, in just one year, it has made a huge leap of a hundred times.
"I didn't expect there to be so much time." Meng Fanqi remembered that at this time, most of the detections were still based on traditional methods like HOG and LBP, and the highest mAP on this dataset in 2013 was only about 0.225.
Since he had enough time to complete the experiments in the papers, he naturally took the time to strike a blow to these old-fashioned methods.
Each participating team often trains several versions of models and then integrates them through some permutations and combinations, submitting multiple times to ensure that their results will not be affected by some unstable factors.
This is also a way to pursue higher performance because no one can guarantee which result of theirs performs the best on the positional data.
Sometimes the difference between first place and second place is just a hair's breadth, maybe just two or three decimal places.
But Meng Fanqi didn't have the need to do this at all.There wasn't much time left to do anything else. Meng Fanqi had planned to submit the results early on the 11th, believing that less is more.
However, Tang Huang stopped him, saying that a hero always makes a dramatic entrance at the last minute.
"The submission results aren't displayed in real time, they're announced all at once on the 14th," Meng Fanqi pointed out this awkward issue.
"Uh..." Tang Huang had no choice but to explain, "Even though others can't see it, the organizers can. At the last moment, let's give them a little shock from China!"
-------------------
On the other side of the ocean, Stanford University's AI lab, SAIL, was established in 1963 during the first wave of neural networks. It has witnessed two booms and two declines in AI.
Now, it is led by young Chinese-American scientist Li Feifei, the organizer of IMAGENET.
When Li Feifei started the IMAGENET project in 2009, she was at Princeton. She later moved to Stanford, became a tenured professor, and began leading Stanford's AI lab this year.
Taking over such a historic lab is not an easy task, especially with this year's IMAGENET competition just ending, Li Feifei is quite busy.
She glanced at the results for the new year yesterday, and it was as expected.
There were no particularly groundbreaking papers this year, everyone was still learning from AlexNet and exploring new tracks.
Deep neural networks stood out last year, but who can be 100% sure that this is the right path?
Even the best-performing model still has a Top-5 error rate of more than 11 points, and usually, this result might be an ensemble prediction of multiple networks. It's good for ranking, but it doesn't have practical application value. Li Feifei didn't want her IMAGENET to become a playground for brushers.
The road ahead is long.
Just then, her phone rang. Li Feifei picked it up and saw it was Deng Jia.
"Holy shit, professor, you have to see the competition's validation results."
Deng Jia sounded very excited, swearing right off the bat.
"Results? What results?" Li Feifei didn't know what had happened. She had already seen the leaderboard yesterday, everyone was at a similar level, what could be calculated today?
It wasn't convenient to link to the server again, so Li Feifei said, "Just screenshot it and send it to me."
"Beep..."
The call was abruptly ended. Li Feifei frowned slightly, wondering what was wrong with him today, he wasn't usually this restless.
Soon, two images were sent over.
Li Feifei opened them one by one, her pupils dilating slightly, her breathing unconsciously quickening.
The top line of both leaderboards was the same team.
Team name: Dream.
The submission descriptions only differed by one letter: "AsingleDreamNet." and "AsingleDreamDet."
Among a group of submissions that integrated multiple models, the word 'single' stood out conspicuously.