From 7295aa1cf85cb04687db772801799493d4eca103 Mon Sep 17 00:00:00 2001 From: swyx Date: Tue, 5 Mar 2024 15:21:07 -0800 Subject: [PATCH] vault backup: 2024-03-05 - 1 files Affected files: Monthly Notes/Mar 2024 notes.md --- Monthly Notes/Mar 2024 notes.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Monthly Notes/Mar 2024 notes.md b/Monthly Notes/Mar 2024 notes.md index a8d9e696..ababec09 100644 --- a/Monthly Notes/Mar 2024 notes.md +++ b/Monthly Notes/Mar 2024 notes.md @@ -19,7 +19,7 @@ - Lower refusal rate - very good to combat anthropic safetyist image and topical vs gemini issues from feb - "Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities." [from Anthropic prompt engineer](https://twitter.com/alexalbert__/status/1764722513014329620) - criticized by [MMitchell](https://x.com/mmitchell_ai/status/1764739357112713267?s=20) and [Connor Leahy](https://x.com/NPCollapse/status/1764740710731837516?s=20) and [Delip Rao](https://x.com/deliprao/status/1764675843542995026?s=20) - - could be overrated - [GPT3 also does it because the needle is so out of context](https://x.com/zggyplaydguitar/status/1764791981782262103?s=46&t=90xQ8sGy63D2OtiaoGJuww) + - could be overrated - [GPT3 also does it because the needle is so out of context](https://x.com/zggyplaydguitar/status/1764791981782262103?s=46&t=90xQ8sGy63D2OtiaoGJuww). [Jim fan](https://twitter.com/DrJimFan/status/1765076396404363435) and [yannic kilcher agree]([https://youtu.be/GBOE9fVVVSM?si=IBMCYkmSiVg-MrFr](https://youtu.be/GBOE9fVVVSM?si=IBMCYkmSiVg-MrFr "https://youtu.be/GBOE9fVVVSM?si=IBMCYkmSiVg-MrFr")) - Evals - [choosing to highlight Finance, Medicine, Philosophy domain evals rather than MMLU/HumanEval is good](https://twitter.com/DrJimFan/status/1764719012678897738) - [59.5% on GPQA](https://x.com/idavidrein/status/1764675668175094169?s=20) is much better than generalist PhDs and GPT4 - GPQA author is impressed. [paper]([arxiv.org/abs/2311.12022](https://t.co/hb4u4xXzkw)). @@ -55,4 +55,5 @@ ## reading and learning -- both [thebloke](https://twitter.com/osanseviero/status/1765103307352055991) and teknium got their start 1 yr ago \ No newline at end of file +- both [thebloke](https://twitter.com/osanseviero/status/1765103307352055991) and teknium got their start 1 yr ago +- [Training great LLMs from ground zero](https://www.yitay.net/blog/training-great-llms-entirely-from-ground-zero-in-the-wilderness) - Yi Tay of Reka \ No newline at end of file