Update november-2024.md

research_and_future_trends/november-2024.md

@@ -46,5 +46,5 @@
 | [Adapter-based Approaches to Knowledge-enhanced Language Models - A Survey](https://arxiv.org/pdf/2411.16403) | This new research surveys "Adapter-based approaches to Knowledge-Enhanced Language Models (KELMs)" - a paradigm that enriches LLMs with structured knowledge from sources like Knowledge Graphs (KGs).  <br><br> This not only improves factual accuracy but also makes them more robust in knowledge-intensive tasks. 🤖📚 <br><br> Adapters are lightweight neural modules that integrate seamlessly into LLM architectures.  <br><br> Instead of fine-tuning massive models, adapters enable task-specific training with fewer resources and without catastrophic forgetting.  <br><br> 🔑 Key adapter types explored in this survey: <br> &nbsp; 1️⃣ Houlsby Adapter - Pioneer in modular enhancement. <br> &nbsp; 2️⃣ Pfeiffer Adapter - Enables multitask learning with efficient fusion techniques. <br> &nbsp; 3️⃣ K-Adapter - Excels in domain-specific knowledge injection, especially in biomedicine. <br><br> 📊 Trends and Insights <br> &nbsp; 🔹 Open-domain tasks dominate research, but biomedical KELMs show immense promise, leveraging rich KGs like UMLS and DBpedia. 🏥 <br> &nbsp; 🔹 Adapter-based KELMs have shown remarkable improvements in tasks like question answering (+8%) and sentiment analysis. <br> &nbsp; 🔹 While task-specific adapters are well-established, generative applications and low-resource domains remain ripe for innovation. | LLM Survey |
 | [A Survey on LLM-as-a-Judge](https://arxiv.org/pdf/2411.15594) | The idea of LLM-as-a-Judge is becoming essential to how we evaluate complex tasks. <br><br> From grading Olympiad-level problems to conducting research peer reviews, LLMs offer scalable, cost-effective, and consistent evaluation solutions - something human experts often struggle with due to fatigue, subjectivity, biases, and time constraints. <br><br> Even Agentic solutions are becoming LLM-as-a-Judge heavy. <br><br> 📜 This recent survey dives into the potential of these systems, exploring: <br> &nbsp; ✅ Strategies to enhance reliability, such as reducing biases and improving consistency.  <br> &nbsp; 📊 Frameworks for rigorous evaluation to ensure alignment with human judgment.  <br> &nbsp; 🌍 Practical applications spanning education, law, finance, and beyond.  <br><br> The paper also introduces a novel benchmark to test LLM-as-a-Judge systems and outlines strategies for improvement, such as fine-tuning models for task-specific reliability. | LLM-as-a-judge |
 | [STAR ATTENTION: EFFICIENT LLM INFERENCE OVER LONG SEQUENCES](https://arxiv.org/pdf/2411.17116) | This new research introduces "Star Attention" - NVIDIA's novel algorithm that speeds up LLM inference by up to 11x while preserving 95-100% accuracy! 🚀 <br><br> 🔑 Key Innovation: <br> &nbsp; 🔹 Splits long contexts into blocks with an anchor block <br> &nbsp; 🔹 Processes context in parallel across multiple hosts <br> &nbsp; 🔹 Maintains global attention capabilities <br> &nbsp; 🔹 Reduces memory requirements dramatically <br><br> 🍀 Technical Breakthrough:  <br> &nbsp; 1️⃣ Blockwise-local attention processing  <br> &nbsp; 2️⃣ Sequence-global attention for queries  <br> &nbsp; 3️⃣ Seamless integration with existing transformer models <br><br> 📈 Performance Impact: <br> &nbsp; 🔹 Up to 11x faster inference <br> &nbsp; 🔹 Works on sequences up to 1M tokens <br> &nbsp; 🔹 Minimal accuracy loss <br> &nbsp; 🔹 Increasingly impressive with larger models | Efficient LLM Inference |
-| []() |  |  |
+| [Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS](https://arxiv.org/pdf/2411.18478) | This new research introduces HiAR-ICL: a novel paradigm that takes reasoning to the next level! 🧠✨ <br><br> 🤔 What is HiAR-ICL? <br><br> HiAR-ICL shifts from example-driven learning to abstract reasoning patterns, incorporating: <br> &nbsp; 🔹 Five atomic reasoning actions that mimic human-like thinking, including decomposition and self-reflection. <br> &nbsp; 🔹 Monte Carlo Tree Search (MCTS) to explore reasoning paths and generate "thought cards" as cognitive templates. <br> &nbsp; 🔹 A cognitive complexity framework to match problems dynamically with optimal reasoning strategies.<br><br> 📊 Results: <br> &nbsp; HiAR-ICL achieves state-of-the-art accuracy in mathematical reasoning, outperforming leading models like GPT-4o and Claude 3.5 on benchmarks such as MATH and GSM8K. For instance, it boosted Qwen2.5-7B-Instruct to an accuracy of 79.6% (compared to GPT-4o's 76.6%). <br><br> 💡This teaches LLMs how to think rather than what to think. By automating reasoning processes, it eliminates the need for human intervention in crafting examples, making AI-driven problem-solving more adaptable and efficient. | In-Context Learning |
 | []() |  |  |