UPDATE (February 4, 2024): This is the discussion about this project on HN: here. Please specifically read @dang's comment regarding the core assumption of this project: here. On a personal note, the number of Stories removed yesterday (Saturday, February 3, 2024) was the lowest ever recorded by the service. This includes 2 duplicate Stories. As a side note, in the list always check whether a Story is a duplicate or not: this is a very reasonable reason for removal and unfortunately I have no way of automatically determining it in the service!
The purpose of this project is to try to understand the type and scale of the moderation of the Hacker News Front Page.
NOTE: I love Hacker News. I try to read it every day. In the case of OnnxStream (here for example), 95% of the comments were helpful and intelligent. I also understand that moderating a site with huge traffic and where users are basically anonymous must be a very difficult task.
Returning to the purpose of this project, from what I have been able to see, the "public" (i.e. observable from the outside) moderation of the Front Page consists of two main tools: modification of the title of a Story (voluntarily or involuntarily influencing its growth in terms of rank) or directly its removal.
Regarding the first type of moderation, an excellent site is already available that tracks changes to Story titles. Here instead I will focus on the second type.
For the reasons explained in the "Why?" section below, I have developed a small application that logs all the Stories that are removed from the Front Page, for personal use. I later discovered that there is no tool/website that provides this type of information and I decided to make it public here. It was a difficult decision but my rationale is: is it better to have more transparency or less transparency?
If you know of a tool/website similar to this, please let me know: I will archive this repo or set it to private.
A possible very positive outcome for this project could be to have a list similar to this, but available directly among the HN lists. Or even to notify a user when a Story is penalized on the Front Page, perhaps indicating the number of flags and/or the reason, for example.
Feel free to skip this part or click to expand
A friend of mine posted two Stories on Hacker News related to OnnxStream (31 days apart), the first related to SDXL Turbo support and the second related to TinyLlama and Mistral 7B support.
In the case of the first, the Story was among the first on the Front Page, until its title was changed from "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes" to "OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2". This effectively "killed" the Story. One user pointed out that the new title didn't reflect the spirit of the Story (thanks @practice9).
In the case of the second, the Story was in third place on the Front Page, less than an hour after the submission. In this case it was simply removed from the Front Page.
Having discovered this, perplexed, I sent an email to the moderator. @dang, who was very kind and quick in his response, explained to me that the Story had been flagged by users even without being explicitly [flagged], and that he could therefore only hypothesize the causes of the flag. His hypothesis was that (some?) users might be fed up with news related to LLMs.
While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
So I decided to develop a small console application to determine the frequency of this phenomenon (actually I was also motivated by the prospect of writing some C# code, after more than 2 years of complete abstinence). I subsequently discovered that there were no tools/websites that monitored this specific phenomenon and I therefore decided to make it public here.
Using the official HN API, the service fetches 90 Top Stories every minute and makes a comparison with the first 30 Top Stories (i.e. the Front Page) fetched the previous minute. It logs all missing Stories here. The assumption is that a Story cannot go from the top 30 to a position greater than 90 in a single minute, without having been explicitly removed. If a Story reappears on the Front Page, it is removed from this log. All Stories present in the second-chance pool are excluded from the log. Title and URL are those from when the Story first appeared in the top 30. The number of points and comments and the rank are those from when the Story was removed from the Front Page. The ID points to the news.social-protocols.org page for that Story, which provides a graph of the Story's position on the Front Page over time.
NOTE: always check whether a Story is a duplicate or not: this is a very reasonable reason for removal and unfortunately I have no way of automatically determining it in the service!
- 42773311 #5 98 points 39 comments -> Matt Mullenweg, Automattic's CEO, Seems Bound and Determined to Wreck WordPress
- 42775545 #5 113 points 103 comments -> It sure looks like Meta stole a lot of books to build its AI
- 42776131 #22 9 points 4 comments -> Protecting the Meaning and Value of American Citizenship
- 42776364 #9 9 points 1 comments -> White House Executive Orders
- 42776378 #21 8 points 1 comments -> Bambu denies their own documented restrictions [video]
- 42750976 #20 9 points 1 comments -> Rolls-Royce Merlin alternative uses
- 42778045 #20 9 points 0 comments -> Meta making users automatically follow Trump and Vance on Facebook
- 42777938 #13 77 points 29 comments -> Meta Censoring '#Democrat' on Instagram
- 42778075 #15 10 points 0 comments -> Ancient Celtic tribe had women at its social center
- 42777716 #17 26 points 43 comments -> X, Facebook, Instagram, and YouTube sign EU code to tackle hate speech
- 42778678 #20 -> Why is this "Elon is Naziscum" token hidden?
- 42778472 #6 43 points 23 comments -> Open Source Alternative to Vercel, Netlify and Heroku
- 42778872 #19 8 points 6 comments -> The Gilded Age novel that helps explain our fascination with Luigi Mangione
- 42777874 #25 25 points 7 comments -> The QUIC API OpenSSL will not provide (2021)
- 42780144 #16 4 points 0 comments -> Microsoft Bing Now Hides Google Search Results
- 42780074 #22 4 points 0 comments -> Promoting Beautiful Federal Civic Architecture
- 42772983 #26 209 points 72 comments -> Official DeepSeek R1 Now on Ollama
- 42782168 #27 7 points 4 comments -> Show HN: Hold yourself accountable for gym visits with a $10 stake
- 42789020 #10 5 points 0 comments -> The Arduous Luxembourg Visa Process
- 42790729 #3 65 points 28 comments -> The Day Instagram Blocked Democracy
- 42791337 #30 10 points 3 comments -> FOSDEM '25 Protest
- 42790962 #29 5 points 1 comments -> Usenet is a worldwide distributed electronic bulletin board system
- 42770351 #13 13 points 7 comments -> Calling Strangers Uncle and Auntie
- 42770341 #23 47 points 32 comments -> The testing pyramid is an outdated economic model
- 42783438 #27 13 points 4 comments -> Show HN: Fixa – an open source Python package for testing voice agents
- 42793629 #28 4 points 0 comments -> Netflix price increase leaves fans furious
- 42757076 #11 21 points 7 comments -> Examples of quick hash tables and dynamic arrays in C
- 42797974 #28 9 points 7 comments -> The war over a private island in the San Francisco Bay
- 42799328 #15 9 points 0 comments -> I built an active community of trans people online
- 42799974 #11 23 points 4 comments -> Kidnapping of Ledger Co-Founder Sparks Major Investigation in France
- 42801839 #11 30 points 40 comments -> Jürgen Schmidhuber – Can All-Purpose Robots Fuel a Comeback?
- 42803201 #25 15 points 16 comments -> Mass deportations would deliver a catastrophic blow to the U.S. Economy
- 42803562 #16 35 points 5 comments -> Hackers exploit 16 zero-days on first day of Pwn2Own Automotive 2025
- 42803559 #12 32 points 52 comments -> Liberux: The Linux Phone You Have Been Waiting For
- 42802842 #8 29 points 40 comments -> How to Inspect React Server Component Activity with Next.js and OpenTelemetry
- 42803483 #17 5 points 1 comments -> Lessons from building a small-scale AI application
- 42805324 #4 9 points 0 comments -> Replit, the Figma Replacement
- 42805585 #6 4 points 1 comments -> Space Ex High School Probe Designs (2024)
- 42806390 #29 10 points 2 comments -> OpenAI Computer-Using Agent
- 42805546 #18 6 points 1 comments -> Humanity's Last Exam
- 42807043 #21 39 points 12 comments -> Finnish Wartime Photograph Archive (1939-1945)
- 42810115 #8 13 points 15 comments -> New Tesla Model Y
- 42810898 #12 5 points 3 comments -> Mississippi Senate Bill 2319 – The "Contraception Begins at Erection" Act
- 42810496 #28 4 points 1 comments -> Where to get an anticapitalist/anti-surveillance email address
- 42810646 #23 7 points 1 comments -> New Rust Jobs Site
- 42811704 #5 5 points 4 comments -> L33T coding is DEAD code
- 42777563 #28 5 points 0 comments -> Plausible Fiction – David Spivak
- 42810865 #28 22 points 40 comments -> Modular PC Design: Sustainable Approach Enhanced Repairability Reduced E-Waste
- 42813345 #12 28 points 45 comments -> AI Is Creating a Generation of Illiterate Programmers
- 42810737 #22 39 points 63 comments -> New Electrical Code Could Doom Most Common EV Charging
- 42812617 #10 10 points 41 comments -> Could Self-Driving Buses Bring Vehicle Autonomy Home?
- 42816028 #7 7 points 1 comments -> Kubestatus: Open source tool to easily add status page to your K8s cluster
- 42812859 #14 149 points 1 comments -> Lightpanda: The headless browser designed for AI and automation
- 42817124 #23 43 points 36 comments -> Looking for work is a full time job
- 42817738 #6 6 points 1 comments -> Wikipedia 'Arbitrators' Tearfully Sanction Pro-Hamas Editors
- 42786740 #2 15 points 7 comments -> The Bizzare Story of the Three Identical Strangers
- 42819834 #3 63 points 34 comments -> Feeling Targeted: Executive Order Ending Wasteful DEIA Efforts
- 42821943 #7 11 points 4 comments -> Why Your AI Product Team Needs an AI Quality Lead
- 42822204 #4 7 points 1 comments -> TikTok's 'cute winter boots' trend explained
- 42822430 #30 6 points 5 comments -> ICE Raids Are an Escalation of Our Long-Simmering De Facto Cold Civil War
- 42822780 #15 14 points 4 comments -> Do Russians Support the War in Ukraine?
- 42822873 #4 33 points 11 comments -> Bloomberg Announces Effort to Ensure U.S. Honors Paris Agreement Commitments
- 42794691 #9 18 points 2 comments -> Giant, Mysterious Spires Ruled the Earth Long Before Trees Did
- 42824966 #6 17 points 6 comments -> US Department of Labor to Cease and Desist All Investigation and Enforcement
- 42826120 #21 4 points 1 comments -> GitHub Copilot web interface
- 42827266 #6 27 points 4 comments -> Asahi Linux Silicon M3 Series Feature Support
- 42803946 #21 7 points 0 comments -> Xbox 360 security in details: the long way to RGH3
- 42832360 #15 6 points 0 comments -> DeepSeek v3 and r1 free unlimited
- 42830923 #10 32 points 40 comments -> Are Americans' perceptions of the economy and crime broken?
- 42831384 #11 16 points 14 comments -> Two F-14 fighter jets were diverted to USS Midway
- 42832838 #16 91 points 2 comments -> Qwen2.5-7B-Instruct-1M and Qwen2.5-14B-Instruct-1M
- 42832132 #28 46 points 50 comments -> Austin rents have fallen for nearly two years
- 42834911 #1 102 points 12 comments -> Mark Zuckerberg: This Man Is a Coward
- 42835932 #3 9 points 2 comments -> Eye Contact
- 42836675 #6 30 points 15 comments -> The race for "AI Supremacy" is over – at least for now
- 42836934 #6 6 points 3 comments -> Show HN: Voice Cloning and Multilingual TTS in One Click (Windows)
- 42838720 #17 17 points 0 comments -> Tim Cook Is Failing Us
- 42839502 #1 31 points 6 comments -> Facebook Ban on Linux
- 42831927 #16 142 points 18 comments -> OpenJazz is a free, open-source version of the classic Jazz Jackrabbit games
- 42813683 #21 53 points 3 comments -> Wikenigma – An Encyclopedia of Unknowns
- 42838779 #23 4 points 0 comments -> Announcing support for DeepSeek-R1 in our IDE plugin, self-hosted by Qodo
- 42806413 #12 36 points 4 comments -> Why Liquidity Matters
- 42839880 #25 3 points 0 comments -> Verifying FPGA using Free OS tools and frameworks
- 42817713 #15 40 points 41 comments -> GenAI Art Is the Least Imaginative Use of AI Imaginable
- 42838774 #16 90 points 31 comments -> Microplastics block blood flow in the brain, mouse study reveals
- 42841461 #28 21 points 0 comments -> How DeepSeek-R1 Was Built, for Dummies