AI companies Scorecard

src/lib/components/Doomers.svelte

@@ -145,6 +145,13 @@
 			number: 0.8,
 			source: 'https://twitter.com/DanHendrycks/status/1642394635657162753'
 		},
+		{
+			name: 'Roman Yampolskiy',
+			title: 'AI Safety Researcher',
+			probability: '98%',
+			number: 0.02,
+			source: 'https://www.scientificamerican.com/article/ai-safety-research-only-enables-the-dangers-of-runaway-superintelligence/#:~:text=1%20percent%20chance%20to%20maybe%202%20percent.'
+		},
 		{
 			name: 'Eliezer Yudkowsky',
 			title: 'Founder of MIRI',

src/posts/learn.md

@@ -47,6 +47,7 @@ Here are some resources to get you started.
 - [Human Compatible: Artificial Intelligence and the Problem of Control](https://www.goodreads.com/en/book/show/44767248) (Stuart Russell)
 - [Our Final Invention: Artificial Intelligence and the End of the Human Era](https://www.goodreads.com/en/book/show/17286699) (James Barrat)
 - [The Precipice: Existential Risk and the Future of Humanity](https://www.goodreads.com/en/book/show/50963653) (Toby Ord)
+- [Uncontrollable](https://www.goodreads.com/book/show/202416160-uncontrollable) (Darren McKee)
 
 ## Courses
 

src/posts/polls-and-surveys.md

@@ -8,7 +8,7 @@ How much do regular people and experts worry about AI risks?
 ## Catastrophic risks from AI
 
 - **[AI researchers, AIImpacts](https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/)**: give "really bad outcomes (such as human extinction)" a 14% probability, with a median of 5%. Only 18% believe the control problem is not important.
-- **[AI engineers / startup founders, State of AI Engineering](https://elemental-croissant-32a.notion.site/State-of-AI-Engineering-2023-20c09dc1767f45988ee1f479b4a84135#694f89e86f9148cb855220ec05e9c631)**: over 60% have a [p(doom)](/pdoom) > 25%. Only 12% have a p(doom) = 0.
+- **[AI engineers/startup founders, State of AI Engineering](https://elemental-croissant-32a.notion.site/State-of-AI-Engineering-2023-20c09dc1767f45988ee1f479b4a84135#694f89e86f9148cb855220ec05e9c631)**: over 60% have a [p(doom)](/pdoom) > 25%. Only 12% have a p(doom) = 0.
 - **[AI safety researchers, AlignmentForum](https://web.archive.org/web/20221013014859/https://www.alignmentforum.org/posts/QvwSr5LsxyDeaPK5s/existential-risk-from-ai-survey-results)**: respondents assigned a median probability of 20% to x-risk caused due to a lack of enough technical research, and 30% to x-risk caused due to a failure of AI systems to do what the people deploying them intended, with huge variation (for example, there are data points at both ~1% and ~99%).
 - **[UK citizens, PublicFirst](https://publicfirst.co.uk/ai/)**: think there's a 9% probability humans will go extinct because of AI. About 50% of say they're very or somewhat worried about this.
 - **[German citizens, Kira](https://www.zeit.de/digital/2023-04/ki-risiken-angst-umfrage-forschung-kira)**: Only 14% believe AI will have a positive influence on the world, 40% mixed, 40% negative.
@@ -20,6 +20,7 @@ How much do regular people and experts worry about AI risks?
 - [**US citizens, YouGov**](https://www.vox.com/future-perfect/2023/8/18/23836362/ai-slow-down-poll-regulation): 72% want AI to slow down, 8% want to speed up. 83% of voters believe AI could accidentally cause a catastrophic event
 - [**US citizens, YouGov**](https://theaipi.org/poll-shows-voters-oppose-open-sourcing-ai-models-support-regulatory-representation-on-boards-and-say-ai-risks-outweigh-benefits-2/): 73% believe AI companies should be held liable for harms from technology they create, 67% think the AI models’ power should be restricted, and 65% believe keeping AI out of the hands of bad actors is more important than providing AI’s benefits to everyone.
 - [**US citizens, AIPI**](https://www.politico.com/newsletters/digital-future-daily/2023/11/29/exclusive-what-people-actually-think-about-ai-00129147): 49:20 support "an international treaty to ban any ‘smarter-than-human’ artificial intelligence (AI)?", 70:14 support "Preventing AI from quickly reaching superhuman capabilities"
+- [**US citizens, AIPI**](https://acrobat.adobe.com/id/urn:aaid:sc:VA6C2:181573f9-b3ce-43c0-b106-1d06558008e6): 77% In favor of going slow with AI development, 8% want to speed up. 56% Agree to stop/slow down vs 27% disagree. 50% Feel afraid of AI.
 - [**US CS professors, Axios Generation Lab**](https://www.axios.com/2023/09/05/ai-regulations-expert-survey): About 1 in 5 predicted AI will "definitely" stay in human control. The rest were split between those saying AI will "probably" or "definitely" get out of human control and those saying "probably not".
 Just 1 in 6 said AI shouldn't or can't be regulated. Only a handful trust the private sector to self-regulate.
 - [**US citizens, Sentience Institute**](https://www.sentienceinstitute.org/aims-survey-supplement-2023): There was broad support for steps that could be taken to slow down development. People supported public campaigns to slow down AI development (71.3%), government regulation that slows down development (71.0%), and a six-month pause on some kinds of AI developments (69.1%). Support for a ban on artificial general intelligence (AGI) that is smarter than humans was 62.9%.

src/routes/api/posts/+server.ts

@@ -5,9 +5,17 @@ import { communitiesMeta } from '../../communities/communities'
 import { meta as pdoomMeta } from '../../pdoom/meta'
 import { meta as quotesMeta } from '../../quotes/meta'
 import { meta as emailBuilderMeta } from '../../email-builder/meta'
+import { meta as scorecardMeta } from '../../scorecard/meta'
 
 /** When adding an extra route, make sure to add the metadata here for SEO purposes */
-const hardCodedPages: Post[] = [outcomesMeta, communitiesMeta, pdoomMeta, quotesMeta, emailBuilderMeta]
+const hardCodedPages: Post[] = [
+	outcomesMeta,
+	communitiesMeta,
+	pdoomMeta,
+	quotesMeta,
+	emailBuilderMeta,
+	scorecardMeta
+]
 
 async function getPosts() {
 	let posts: Post[] = []

src/routes/scorecard/+page.svelte

@@ -0,0 +1,26 @@
+<script lang="ts">
+	import PostMeta from '$lib/components/PostMeta.svelte'
+	import Scorecard from './Scorecard.svelte'
+	import { meta } from './meta'
+
+	let { title, description, date } = meta
+</script>
+
+<PostMeta {title} {description} {date} />
+
+<h1>{title}</h1>
+<p>
+	AI Companies are building some of the most impactful technology in the world. PauseAI keeps track
+	of how responsible these companies are acting, and what they are doing to prevent various <a
+		href="/risks">risks</a
+	>.
+</p>
+<p>
+	<em
+		>This scorecard is a work in progress. The scores and explanations in this PR are temporary
+		values, and do not yet represent PauseAI's or any jury's opinions. Please <a
+			href="https://github.com/joepio/pauseai/pull/32">contribute suggestion on github</a
+		>!</em
+	>
+</p>
+<Scorecard />

src/routes/scorecard/Scorecard.svelte

@@ -0,0 +1,73 @@
+<script lang="ts">
+	import Button from '$lib/components/Button.svelte'
+	import Cell from './TableCell.svelte'
+	import { categories } from './categories'
+	import { companies } from './companies'
+
+	let showExplanation = false
+</script>
+
+<Button on:click={() => (showExplanation = !showExplanation)}>Toggle explanations</Button>
+
+<table class={showExplanation ? 'table--big' : ''}>
+	<thead>
+		<tr>
+			<th />
+			{#each categories as category (category.name)}
+				<Cell title={category.name} explanation={category.explanation} {showExplanation} />
+			{/each}
+			<Cell title="Total" explanation="Average of the scores" {showExplanation} />
+		</tr>
+	</thead>
+	<tbody>
+		{#each companies as company (company.name)}
+			<tr class="company">
+				<td class="name">{company.name}</td>
+				{#each categories as category (category.name)}
+					{#if company[category.key] === undefined}
+						<td />
+					{:else}
+						<Cell
+							score={company[category.key].score}
+							explanation={company[category.key].explanation}
+							{showExplanation}
+						/>
+					{/if}
+				{/each}
+				<Cell score={company.totalScore} explanation="Average of all scores." {showExplanation} />
+				<!-- <td class="total">{company.totalScore}</td> -->
+			</tr>
+		{/each}
+	</tbody>
+</table>
+
+<style>
+	table {
+		max-width: var(--page-width);
+		overflow-x: auto;
+		overflow: visible;
+		display: block;
+		/* Too much margin  */
+		/* margin-left: calc(50% - var(--page-width) / 2); */
+	}
+
+	.table--big {
+		margin-left: max(calc(50% - 50vw), -10rem);
+	}
+	:global(td),
+	:global(th) {
+		padding: 0.5rem;
+		text-align: left;
+		vertical-align: top;
+	}
+
+	.total {
+		font-weight: bold;
+		color: var(--brand);
+		font-size: 1.2rem;
+	}
+
+	.name {
+		font-weight: bold;
+	}
+</style>

src/routes/scorecard/TableCell.svelte

@@ -0,0 +1,96 @@
+<script lang="ts">
+	export let score: number | undefined = undefined
+	export let explanation: string | undefined = undefined
+	export let showExplanation: boolean
+	export let title: string | undefined = undefined
+	let showTooltip = false
+	let color: string | undefined
+
+	$: {
+		if (score !== undefined) {
+			if (score >= 9) {
+				color = 'green'
+			} else if (score <= 1) {
+				color = 'red'
+			} else {
+				const percentage = (score - 1) / 8
+				const red = Math.round(255 * (1 - percentage))
+				const green = Math.round(255 * percentage)
+				color = `rgb(${red}, ${green}, 0)`
+			}
+		}
+	}
+
+	function maybeShowTooltip() {
+		if (!showExplanation) {
+			showTooltip = true
+		}
+	}
+
+	function handleClick() {
+		!showExplanation && (showExplanation = !showExplanation)
+	}
+</script>
+
+<td
+	on:mouseover={maybeShowTooltip}
+	on:mouseout={() => (showTooltip = false)}
+	on:focus={maybeShowTooltip}
+	on:blur={() => (showTooltip = false)}
+	on:click={handleClick}
+	class={showExplanation ? '' : 'enable-tooltips'}
+>
+	{#if title !== undefined}
+		<div class="title">
+			{title}
+		</div>
+	{/if}
+	{#if score !== undefined}
+		<div class="score" style="color: {color}">
+			{score}
+		</div>
+	{/if}
+	{#if showExplanation}
+		<div class="explanation">
+			{explanation}
+		</div>
+	{/if}
+	{#if !showExplanation && showTooltip}
+		<div class="tooltip">
+			{explanation}
+		</div>
+	{/if}
+</td>
+
+<style>
+	td.enable-tooltips:hover {
+		cursor: help;
+		background-color: var(--bg-subtle);
+	}
+	.title {
+		font-weight: bold;
+	}
+	.score {
+		font-size: 1.2rem;
+		font-weight: bold;
+	}
+
+	.explanation {
+		font-size: 0.8rem;
+		opacity: 0.7;
+		min-width: 10rem;
+	}
+
+	.tooltip {
+		position: absolute;
+		background: var(--bg);
+		border: 1px solid var(--text);
+		border-radius: 0.5rem;
+		padding: 0.5rem;
+		font-size: 0.8rem;
+		opacity: 0.95;
+		min-width: 10rem;
+		max-width: 15rem;
+		z-index: 1;
+	}
+</style>

src/routes/scorecard/categories.ts

@@ -0,0 +1,62 @@
+import type { Category } from './types'
+
+export const categories: Category[] = [
+	{
+		name: 'Acknowledge',
+		key: 'acknowledge',
+		explanation:
+			// 10: Pushing the conversation forward, making it more mainstream
+			// 8: Has devoted substatial resources to public communication of AI risks
+			// 7: Has made an attemt to communicate concerns to the public
+			// 6: Has acknowledged xrisk, but no attempt to highlight these concerns
+			// 5: E.g.no acknowledgement of risks, but also no denial.
+			// 4: Active downlplaying of concerns, but in reasonable way
+			// 3: Minimization of risk, dismissal, but not explicitly denying or attempt to move the conversation
+			// 0: Actively denying the risks of AI, not discussing in good faith, attacking advocates on a personal level
+			// How likely are they saying the risks are to happen?
+			'How well does the company acknowledge the risks of AI? Do they acknowledge all the risks? How long did it take them to acknowledge the risks?'
+	},
+	{
+		name: 'Lobby',
+		key: 'lobby',
+		// 10: Actively introducing draft legislations / concrete proposals that harm their internal business model but that are good for AI safety
+		// 5: No lobbying, no attempt to influence regulation in any direction
+		// 1: Using personal connections, money or financial incentives to influence regulation in a way that is very harmful to AI safety.
+		// 0: Criminal lobbying, using illegal means to prevent meaningful AI safety regulation.
+		explanation:
+			'How much does the company lobby for AI regulation? Do they lobby for deregulation or for stricter regulation?'
+	},
+	{
+		name: 'Deployment',
+		key: 'deployment',
+		explanation:
+			// What principles drive your decisions about what (not) to deploy?
+			// To what degree do you control access to your technology?
+			// And do you monitor usage?
+			// Are you willing to introduce new capabilities that others have not?
+			// What are you open sourcing?
+			// What are your plans
+			// 0: Releases model weights of frontier models regardless of the risks.
+			// 2: Releases frontier model weights, but only after testing for safety.
+			// 3: Open model weights for models that are behind the state of the art, where comparable models have demonstrated to be (catastrophically) safe.
+			// 5: No open source,
+			'How long does the company wait before deploying a new model? Do they wait until they have a good understanding of the risks of the model? Are their models tested thoroughly?'
+	},
+	{
+		name: 'Research',
+		key: 'research',
+		explanation:
+			// 10:
+			// 5:
+			// 0:
+			'Is the company doing research on AI safety or alignment? Are they doing research on how to mitigate the risks of AI?'
+	}
+	// Frontier column
+	// Are you pushing capabilities forward or are you following others to provide services?
+	// This is where OpenAI scores 0 points.
+	// To what extent do their future plans advance the state of the art?
+
+	// Culture column
+	// To what extent do you have a culture of safety?
+	// Do the employees have a good understanding of the risks of AI?
+]